Eric Dumazet
567e4b7973
net: rfs: add hash collision detection ...
Receive Flow Steering is a nice solution but suffers from
hash collisions when a mix of connected and unconnected traffic
is received on the host, when flow hash table is populated.
Also, clearing flow in inet_release() makes RFS not very good
for short lived flows, as many packets can follow close().
(FIN , ACK packets, ...)
This patch extends the information stored into global hash table
to not only include cpu number, but upper part of the hash value.
I use a 32bit value, and dynamically split it in two parts.
For host with less than 64 possible cpus, this gives 6 bits for the
cpu number, and 26 (32-6) bits for the upper part of the hash.
Since hash bucket selection use low order bits of the hash, we have
a full hash match, if /proc/sys/net/core/rps_sock_flow_entries is big
enough.
If the hash found in flow table does not match, we fallback to RPS (if
it is enabled for the rxqueue).
This means that a packet for an non connected flow can avoid the
IPI through a unrelated/victim CPU.
This also means we no longer have to clear the table at socket
close time, and this helps short lived flows performance.
Signed-off-by: Eric Dumazet <edumazet@google.com>
Acked-by: Tom Herbert <therbert@google.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2015-02-08 16:53:57 -08:00
2014-11-06 14:59:19 -05:00
2015-02-02 18:27:56 +01:00
2013-09-23 16:29:41 -04:00
2014-12-10 15:33:16 -05:00
2013-04-08 17:16:57 -04:00
2015-02-05 14:33:28 -08:00
2015-02-05 14:33:28 -08:00
2015-02-04 09:14:09 +01:00
2012-06-17 16:29:38 -07:00
2014-11-24 05:16:40 -05:00
2015-01-19 16:02:06 -05:00
2014-10-27 15:51:16 +01:00
2014-02-12 19:23:32 -05:00
2014-09-23 12:09:27 -04:00
2014-10-25 08:07:30 +02:00
2013-07-31 17:50:01 -07:00
2013-08-10 01:16:44 -07:00
2014-11-24 05:16:42 -05:00
2014-09-18 10:54:36 +02:00
2013-12-28 17:02:46 -05:00
2011-11-30 18:51:03 -05:00
2013-07-31 17:50:02 -07:00
2009-03-24 23:32:03 -07:00
2015-01-27 17:09:04 -08:00
2014-11-10 13:27:49 -05:00
2014-11-10 13:27:49 -05:00
2015-02-04 16:14:24 -08:00
2014-01-13 17:39:11 +01:00
2011-08-08 14:26:29 -04:00
2015-01-23 10:30:47 +01:00
2014-12-19 00:19:23 +01:00
2014-11-26 12:25:43 -05:00
2013-12-10 17:56:54 -05:00
2014-02-08 10:36:58 -05:00
2014-08-22 19:57:23 -07:00
2014-11-19 16:23:49 -05:00
2014-02-09 17:32:50 -08:00
2013-12-06 12:37:56 -05:00
2014-07-17 16:02:29 -07:00
2013-12-10 21:50:00 -05:00
2013-09-20 14:49:32 -04:00
2013-09-20 14:49:32 -04:00
2013-09-20 14:49:32 -04:00
2014-02-09 17:32:49 -08:00
2014-02-09 17:32:49 -08:00
2014-10-30 14:54:11 -04:00
2013-01-09 23:59:53 -08:00
2014-09-08 09:51:29 +09:00
2014-09-16 10:08:49 +02:00
2013-11-07 19:28:58 -05:00
2014-02-04 20:19:51 -08:00
2013-09-20 14:49:33 -04:00
2013-03-26 12:32:13 -04:00
2015-02-05 00:40:22 -08:00
2014-04-16 15:05:11 -04:00
2014-02-12 07:02:11 +01:00
2014-11-12 15:01:35 -05:00
2013-09-20 14:49:33 -04:00
2014-09-30 01:02:26 -04:00
2015-01-27 16:59:56 -08:00
2015-01-28 23:04:15 -08:00
2014-06-04 22:46:38 -07:00
2015-01-18 01:56:32 -05:00
2014-11-05 16:30:03 -05:00
2013-09-20 14:49:33 -04:00
2014-02-06 09:34:58 +01:00
2014-12-19 00:19:23 +01:00
2014-10-14 13:15:15 -04:00
2014-04-15 12:58:34 -04:00
2014-11-05 16:59:04 -05:00
2014-11-26 15:45:04 -05:00
2015-01-05 22:55:24 -05:00
2014-05-07 15:30:52 -04:00
2014-09-08 09:51:29 +09:00
2014-05-14 16:07:23 -04:00
2015-01-05 22:44:46 -05:00
2014-07-01 23:46:21 -07:00
2014-09-08 16:42:42 -07:00
2014-08-24 18:09:23 -07:00
2015-01-05 22:55:24 -05:00
2014-05-24 00:32:30 -04:00
2015-01-19 14:32:03 -05:00
2014-12-31 18:25:54 -05:00
2015-01-19 14:32:03 -05:00
2014-10-02 18:30:58 +02:00
2015-02-05 14:33:28 -08:00
2010-02-16 23:05:38 -08:00
2015-02-05 14:33:28 -08:00
2014-11-24 04:28:49 -05:00
2013-09-21 14:01:39 -04:00
2013-09-21 14:01:38 -04:00
2014-10-14 02:18:27 +02:00
2013-09-21 14:01:38 -04:00
2013-09-21 14:01:38 -04:00
2014-12-10 15:21:24 -05:00
2013-09-21 14:01:38 -04:00
2013-09-21 14:01:38 -04:00
2014-01-01 22:22:43 -05:00
2013-09-21 14:01:38 -04:00
2013-09-21 14:01:38 -04:00
2014-12-10 15:21:24 -05:00
2013-09-21 14:01:38 -04:00
2014-01-03 20:56:48 -05:00
2015-01-27 11:10:13 +01:00
2014-12-19 00:19:23 +01:00
2013-12-06 12:37:56 -05:00
2014-09-22 16:23:15 -04:00
2014-11-05 23:52:33 -08:00
2013-10-01 17:06:14 -04:00
2013-09-21 14:01:39 -04:00
2014-12-26 17:07:08 -05:00
2015-01-19 14:21:18 -05:00
2011-05-27 13:41:33 -04:00
2013-09-21 14:01:39 -04:00
2014-08-07 20:55:21 -04:00
2015-01-18 01:03:45 -05:00
2014-02-08 10:36:58 -05:00
2013-09-21 14:01:39 -04:00
2014-12-19 00:19:23 +01:00
2013-09-21 14:01:39 -04:00
2015-02-04 01:34:14 -05:00
2014-10-06 18:02:32 -04:00
2015-01-13 17:51:08 -05:00
2014-05-23 16:28:53 -04:00
2013-09-23 01:51:08 -04:00
2013-09-23 01:51:08 -04:00
2013-09-23 01:51:08 -04:00
2014-01-21 23:17:20 -08:00
2014-12-17 11:49:55 +01:00
2013-10-10 00:08:07 -04:00
2013-09-23 01:51:08 -04:00
2015-01-15 18:26:16 -05:00
2015-01-19 14:21:26 -05:00
2015-01-31 17:49:37 -08:00
2013-09-23 01:51:09 -04:00
2014-06-02 11:00:41 -07:00
2014-10-15 07:48:18 +02:00
2015-02-08 16:53:57 -08:00
2013-12-19 18:51:20 -05:00
2013-09-23 01:51:09 -04:00
2015-02-01 23:16:34 -08:00
2013-10-21 18:43:02 -04:00
2015-02-08 01:03:12 -08:00
2012-06-09 14:56:12 -07:00
2014-01-19 19:53:18 -08:00
2014-05-22 14:57:15 -04:00
2015-01-24 23:15:40 -08:00
2014-10-01 21:35:51 -04:00
2015-02-04 01:34:15 -05:00
2013-07-27 22:14:06 -07:00
2015-01-24 23:15:40 -08:00
2013-09-23 16:29:40 -04:00
2014-09-05 17:35:28 -07:00
2013-09-23 16:29:41 -04:00
2010-04-22 16:12:36 -07:00
2014-09-02 13:37:56 +02:00
567e4b7973