Virtualization - Apr 2015 - [PATCH net-next] virtio: document queue state logic

commit d631b94e7a15277858ec5f88d674d93080506999
    virtio: change comment in transmit

started clarifying the logic behind queue state management,
but introduced an inaccuracy: TX_BUSY does not cause
a BUG message.

Clean this up some more, explaining the tradeoffs in detail.

Signed-off-by: Michael S. Tsirkin <mst at redhat.com>
---

diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index a829930..63c7810 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -939,11 +939,15 @@ static netdev_tx_t start_xmit(struct sk_buff *skb, struct
net_device *dev)
 	skb_orphan(skb);
 	nf_reset(skb);
 
-	/* It is better to stop queue if running out of space
-	 * instead of forcing queuing layer to requeue the skb
-	 * by returning TX_BUSY (and cause a BUG message).
-	 * Since most packets only take 1 or 2 ring slots
-	 * this means 16 slots are typically wasted.
+	/* If running out of space, stop queue to avoid getting packets that we
+	 * are then unable to transmit.
+	 * An alternative would be to force queuing layer to requeue the skb by
+	 * returning NETDEV_TX_BUSY. However, NETDEV_TX_BUSY should not be
+	 * returned in a normal path of operation: it means that driver is not
+	 * maintaining the TX queue stop/start state properly, and causes
+	 * the stack to do a non-trivial amount of useless work.
+	 * Since most packets only take 1 or 2 ring slots, stopping the queue
+	 * early means 16 slots are typically wasted.
 	 */
 	if (sq->vq->num_free < 2+MAX_SKB_FRAGS) {
 		netif_stop_subqueue(dev, qnum);

This paraphrases DaveM (and steals some of his words) explaining why
a device shouldn't return NETDEV_TX_BUSY, even though it looks so inviting
to driver authors.

See http://www.spinics.net/lists/netdev/msg322350.html

Inspired-by: David Miller <davem at davemloft.net>
Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>

diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index dcf6ec27739b..a2cad44b8630 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -795,7 +795,10 @@ typedef u16 (*select_queue_fallback_t)(struct net_device
*dev,
  * netdev_tx_t (*ndo_start_xmit)(struct sk_buff *skb,
  *                               struct net_device *dev);
  *	Called when a packet needs to be transmitted.
- *	Must return NETDEV_TX_OK , NETDEV_TX_BUSY.
+ *	Returns NETDEV_TX_OK.  Can return NETDEV_TX_BUSY, but you should stop
+ *	the queue before that can happen; it's for obsolete devices and weird
+ *	corner cases, but the stack really does a non-trivial amount
+ *	of useless work if you return NETDEV_TX_BUSY.
  *        (can also return NETDEV_TX_LOCKED iff NETIF_F_LLTX)
  *	Required can not be NULL.
  *

From: Rusty Russell <rusty at rustcorp.com.au>
Date: Fri, 03 Apr 2015 22:17:17 +1030
> This paraphrases DaveM (and steals some of his words) explaining why
> a device shouldn't return NETDEV_TX_BUSY, even though it looks so
inviting
> to driver authors.
> 
> See http://www.spinics.net/lists/netdev/msg322350.html
> 
> Inspired-by: David Miller <davem at davemloft.net>
> Signed-off-by: Rusty Russell <rusty at rustcorp.com.au>
Applied.

From: "Michael S. Tsirkin" <mst at redhat.com>
Date: Thu, 2 Apr 2015 13:05:47 +0200
> commit d631b94e7a15277858ec5f88d674d93080506999
>     virtio: change comment in transmit
> 
> started clarifying the logic behind queue state management,
> but introduced an inaccuracy: TX_BUSY does not cause
> a BUG message.
> 
> Clean this up some more, explaining the tradeoffs in detail.
> 
> Signed-off-by: Michael S. Tsirkin <mst at redhat.com>
Applied, thanks Michael.