Eric Dumazet 93f154b594 net: release dst entry in dev_hard_start_xmit()
One point of contention in high network loads is the dst_release() performed
when a transmited skb is freed. This is because NIC tx completion calls
dev_kree_skb() long after original call to dev_queue_xmit(skb).

CPU cache is cold and the atomic op in dst_release() stalls. On SMP, this is
quite visible if one CPU is 100% handling softirqs for a network device,
since dst_clone() is done by other cpus, involving cache line ping pongs.

It seems right place to release dst is in dev_hard_start_xmit(), for most
devices but ones that are virtual, and some exceptions.

David Miller suggested to define a new device flag, set in alloc_netdev_mq()
(so that most devices set it at init time), and carefuly unset in devices
which dont want a NULL skb->dst in their ndo_start_xmit().

List of devices that must clear this flag is :

- loopback device, because it calls netif_rx() and quoting Patrick :
    "ip_route_input() doesn't accept loopback addresses, so loopback packets
     already need to have a dst_entry attached."
- appletalk/ipddp.c : needs skb->dst in its xmit function

- And all devices that call again dev_queue_xmit() from their xmit function
(as some classifiers need skb->dst) : bonding, vlan, macvlan, eql, ifb, hdlc_fr

Signed-off-by: Eric Dumazet <dada1@cosmosbay.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
2009-05-18 22:19:19 -07:00
..
2009-04-13 15:16:33 -07:00
2009-04-28 02:05:21 -07:00
2009-04-21 18:12:11 -07:00
2009-04-17 15:45:15 -07:00
2009-05-18 21:46:40 -07:00
2009-01-21 14:02:21 -08:00
2008-12-08 01:14:16 -08:00
2009-03-27 00:46:38 -07:00
2009-01-21 14:02:23 -08:00
2009-01-21 14:02:24 -08:00
2009-01-21 14:02:24 -08:00
2009-01-21 14:02:25 -08:00
2008-11-25 18:12:49 -08:00
2009-04-24 04:45:33 -07:00
2009-05-17 11:59:51 -07:00
2009-04-14 21:57:00 -07:00
2008-11-25 18:25:32 -08:00
2009-04-11 02:53:46 -07:00
2009-01-07 18:10:24 -08:00
2009-04-14 21:57:02 -07:00
2009-04-14 21:57:03 -07:00
2009-04-16 02:20:06 -07:00
2009-04-04 16:51:14 -07:00
2009-05-17 20:47:44 -07:00
2009-05-17 20:47:44 -07:00
2009-03-02 22:32:48 -08:00
2009-02-15 23:32:01 -08:00
2009-02-15 23:32:01 -08:00
2009-02-15 23:32:01 -08:00
2009-03-02 22:32:43 -08:00
2009-04-27 03:27:43 -07:00
2008-09-22 19:27:10 -07:00
2009-04-16 02:20:07 -07:00
2009-05-18 15:19:01 -07:00
2009-05-17 21:03:45 -07:00
2009-04-14 21:57:04 -07:00
2008-02-01 20:54:03 +11:00
2009-04-16 02:20:08 -07:00
2008-11-20 20:28:00 -08:00
2009-01-07 17:30:09 -08:00
2009-01-21 14:33:50 -08:00
2008-11-25 18:23:59 -08:00
2009-04-16 02:34:48 -07:00
2009-02-04 16:43:44 -08:00
2009-02-26 21:02:19 -08:00
2008-11-25 18:24:57 -08:00
2009-04-16 02:20:08 -07:00
2009-05-17 15:19:36 -07:00
2009-01-06 10:47:44 -08:00
2009-04-13 15:16:31 -07:00
2008-08-14 04:26:38 -04:00
2009-05-18 15:19:01 -07:00
2009-04-16 02:20:15 -07:00
2009-03-27 00:46:48 -07:00
2009-04-16 02:20:15 -07:00
2008-11-25 18:24:38 -08:00
2009-05-17 21:03:46 -07:00
2009-05-04 11:08:41 -07:00
2009-04-16 02:20:16 -07:00
2009-04-16 02:20:16 -07:00
2009-04-13 15:16:26 -07:00
2009-05-18 15:19:01 -07:00
2007-10-19 11:53:42 -07:00
2009-04-16 02:20:18 -07:00
2009-04-16 02:20:21 -07:00
2009-04-04 16:38:42 -07:00
2009-04-16 02:20:22 -07:00
2009-03-27 00:46:53 -07:00
2008-03-26 00:18:00 -04:00
2009-03-27 00:46:53 -07:00
2009-03-28 23:37:27 -07:00
2009-04-08 15:52:16 -07:00
2009-01-07 17:26:48 -08:00
2009-03-13 16:09:12 -07:00
2008-01-28 15:11:27 -08:00
2009-03-13 16:09:12 -07:00
2009-03-13 16:09:12 -07:00
2008-11-10 15:11:56 -05:00
2009-04-13 16:21:03 -07:00
2009-04-16 02:20:24 -07:00
2009-04-29 17:32:34 -07:00
2009-04-29 17:32:34 -07:00
2009-01-07 17:56:54 -08:00
2009-04-16 02:20:25 -07:00
2009-04-13 15:16:25 -07:00
2007-10-16 21:10:29 -04:00
2008-08-07 01:55:50 -04:00
2008-01-28 15:11:27 -08:00
2009-01-07 18:09:36 -08:00
2009-04-16 02:20:25 -07:00
2009-03-27 00:46:48 -07:00
2008-04-16 20:06:50 -04:00
2008-11-25 18:25:49 -08:00
2008-08-29 02:13:33 -07:00
2008-01-28 15:07:10 -08:00
2008-08-29 02:14:29 -07:00
2008-08-29 02:14:43 -07:00
2009-03-20 00:51:22 -07:00
2009-04-21 01:41:03 -07:00
2008-05-30 22:18:02 -04:00
2009-01-11 00:06:36 -08:00
2009-04-13 15:16:21 -07:00
2009-01-21 14:02:31 -08:00