Michael S. Tsirkin
2022-Aug-17 06:54 UTC
[PATCH 0/6] virtio/vsock: introduce dgrams, sk_buff, and qdisc
On Mon, Aug 15, 2022 at 10:56:03AM -0700, Bobby Eshleman wrote:> Hey everybody, > > This series introduces datagrams, packet scheduling, and sk_buff usage > to virtio vsock. > > The usage of struct sk_buff benefits users by a) preparing vsock to use > other related systems that require sk_buff, such as sockmap and qdisc, > b) supporting basic congestion control via sock_alloc_send_skb, and c) > reducing copying when delivering packets to TAP. > > The socket layer no longer forces errors to be -ENOMEM, as typically > userspace expects -EAGAIN when the sk_sndbuf threshold is reached and > messages are being sent with option MSG_DONTWAIT. > > The datagram work is based off previous patches by Jiang Wang[1]. > > The introduction of datagrams creates a transport layer fairness issue > where datagrams may freely starve streams of queue access. This happens > because, unlike streams, datagrams lack the transactions necessary for > calculating credits and throttling. > > Previous proposals introduce changes to the spec to add an additional > virtqueue pair for datagrams[1]. Although this solution works, using > Linux's qdisc for packet scheduling leverages already existing systems, > avoids the need to change the virtio specification, and gives additional > capabilities. The usage of SFQ or fq_codel, for example, may solve the > transport layer starvation problem. It is easy to imagine other use > cases as well. For example, services of varying importance may be > assigned different priorities, and qdisc will apply appropriate > priority-based scheduling. By default, the system default pfifo qdisc is > used. The qdisc may be bypassed and legacy queuing is resumed by simply > setting the virtio-vsock%d network device to state DOWN. This technique > still allows vsock to work with zero-configuration.The basic question to answer then is this: with a net device qdisc etc in the picture, how is this different from virtio net then? Why do you still want to use vsock?> In summary, this series introduces these major changes to vsock: > > - virtio vsock supports datagrams > - virtio vsock uses struct sk_buff instead of virtio_vsock_pkt > - Because virtio vsock uses sk_buff, it also uses sock_alloc_send_skb, > which applies the throttling threshold sk_sndbuf. > - The vsock socket layer supports returning errors other than -ENOMEM. > - This is used to return -EAGAIN when the sk_sndbuf threshold is > reached. > - virtio vsock uses a net_device, through which qdisc may be used. > - qdisc allows scheduling policies to be applied to vsock flows. > - Some qdiscs, like SFQ, may allow vsock to avoid transport layer congestion. That is, > it may avoid datagrams from flooding out stream flows. The benefit > to this is that additional virtqueues are not needed for datagrams. > - The net_device and qdisc is bypassed by simply setting the > net_device state to DOWN. > > [1]: https://lore.kernel.org/all/20210914055440.3121004-1-jiang.wang at bytedance.com/ > > Bobby Eshleman (5): > vsock: replace virtio_vsock_pkt with sk_buff > vsock: return errors other than -ENOMEM to socket > vsock: add netdev to vhost/virtio vsock > virtio/vsock: add VIRTIO_VSOCK_F_DGRAM feature bit > virtio/vsock: add support for dgram > > Jiang Wang (1): > vsock_test: add tests for vsock dgram > > drivers/vhost/vsock.c | 238 ++++---- > include/linux/virtio_vsock.h | 73 ++- > include/net/af_vsock.h | 2 + > include/uapi/linux/virtio_vsock.h | 2 + > net/vmw_vsock/af_vsock.c | 30 +- > net/vmw_vsock/hyperv_transport.c | 2 +- > net/vmw_vsock/virtio_transport.c | 237 +++++--- > net/vmw_vsock/virtio_transport_common.c | 771 ++++++++++++++++-------- > net/vmw_vsock/vmci_transport.c | 9 +- > net/vmw_vsock/vsock_loopback.c | 51 +- > tools/testing/vsock/util.c | 105 ++++ > tools/testing/vsock/util.h | 4 + > tools/testing/vsock/vsock_test.c | 195 ++++++ > 13 files changed, 1176 insertions(+), 543 deletions(-) > > -- > 2.35.1
Jason Wang
2022-Aug-18 04:28 UTC
[PATCH 0/6] virtio/vsock: introduce dgrams, sk_buff, and qdisc
? 2022/8/17 14:54, Michael S. Tsirkin ??:> On Mon, Aug 15, 2022 at 10:56:03AM -0700, Bobby Eshleman wrote: >> Hey everybody, >> >> This series introduces datagrams, packet scheduling, and sk_buff usage >> to virtio vsock. >> >> The usage of struct sk_buff benefits users by a) preparing vsock to use >> other related systems that require sk_buff, such as sockmap and qdisc, >> b) supporting basic congestion control via sock_alloc_send_skb, and c) >> reducing copying when delivering packets to TAP. >> >> The socket layer no longer forces errors to be -ENOMEM, as typically >> userspace expects -EAGAIN when the sk_sndbuf threshold is reached and >> messages are being sent with option MSG_DONTWAIT. >> >> The datagram work is based off previous patches by Jiang Wang[1]. >> >> The introduction of datagrams creates a transport layer fairness issue >> where datagrams may freely starve streams of queue access. This happens >> because, unlike streams, datagrams lack the transactions necessary for >> calculating credits and throttling. >> >> Previous proposals introduce changes to the spec to add an additional >> virtqueue pair for datagrams[1]. Although this solution works, using >> Linux's qdisc for packet scheduling leverages already existing systems, >> avoids the need to change the virtio specification, and gives additional >> capabilities. The usage of SFQ or fq_codel, for example, may solve the >> transport layer starvation problem. It is easy to imagine other use >> cases as well. For example, services of varying importance may be >> assigned different priorities, and qdisc will apply appropriate >> priority-based scheduling. By default, the system default pfifo qdisc is >> used. The qdisc may be bypassed and legacy queuing is resumed by simply >> setting the virtio-vsock%d network device to state DOWN. This technique >> still allows vsock to work with zero-configuration. > The basic question to answer then is this: with a net device qdisc > etc in the picture, how is this different from virtio net then? > Why do you still want to use vsock?Or maybe it's time to revisit an old idea[1] to unify at least the driver part (e.g using virtio-net driver for vsock then we can all features that vsock is lacking now)? Thanks [1] https://lists.linuxfoundation.org/pipermail/virtualization/2018-November/039783.html> >> In summary, this series introduces these major changes to vsock: >> >> - virtio vsock supports datagrams >> - virtio vsock uses struct sk_buff instead of virtio_vsock_pkt >> - Because virtio vsock uses sk_buff, it also uses sock_alloc_send_skb, >> which applies the throttling threshold sk_sndbuf. >> - The vsock socket layer supports returning errors other than -ENOMEM. >> - This is used to return -EAGAIN when the sk_sndbuf threshold is >> reached. >> - virtio vsock uses a net_device, through which qdisc may be used. >> - qdisc allows scheduling policies to be applied to vsock flows. >> - Some qdiscs, like SFQ, may allow vsock to avoid transport layer congestion. That is, >> it may avoid datagrams from flooding out stream flows. The benefit >> to this is that additional virtqueues are not needed for datagrams. >> - The net_device and qdisc is bypassed by simply setting the >> net_device state to DOWN. >> >> [1]: https://lore.kernel.org/all/20210914055440.3121004-1-jiang.wang at bytedance.com/ >> >> Bobby Eshleman (5): >> vsock: replace virtio_vsock_pkt with sk_buff >> vsock: return errors other than -ENOMEM to socket >> vsock: add netdev to vhost/virtio vsock >> virtio/vsock: add VIRTIO_VSOCK_F_DGRAM feature bit >> virtio/vsock: add support for dgram >> >> Jiang Wang (1): >> vsock_test: add tests for vsock dgram >> >> drivers/vhost/vsock.c | 238 ++++---- >> include/linux/virtio_vsock.h | 73 ++- >> include/net/af_vsock.h | 2 + >> include/uapi/linux/virtio_vsock.h | 2 + >> net/vmw_vsock/af_vsock.c | 30 +- >> net/vmw_vsock/hyperv_transport.c | 2 +- >> net/vmw_vsock/virtio_transport.c | 237 +++++--- >> net/vmw_vsock/virtio_transport_common.c | 771 ++++++++++++++++-------- >> net/vmw_vsock/vmci_transport.c | 9 +- >> net/vmw_vsock/vsock_loopback.c | 51 +- >> tools/testing/vsock/util.c | 105 ++++ >> tools/testing/vsock/util.h | 4 + >> tools/testing/vsock/vsock_test.c | 195 ++++++ >> 13 files changed, 1176 insertions(+), 543 deletions(-) >> >> -- >> 2.35.1
Stefano Garzarella
2022-Sep-06 09:00 UTC
[PATCH 0/6] virtio/vsock: introduce dgrams, sk_buff, and qdisc
On Thu, Aug 18, 2022 at 12:28:48PM +0800, Jason Wang wrote:> >? 2022/8/17 14:54, Michael S. Tsirkin ??: >>On Mon, Aug 15, 2022 at 10:56:03AM -0700, Bobby Eshleman wrote: >>>Hey everybody, >>> >>>This series introduces datagrams, packet scheduling, and sk_buff usage >>>to virtio vsock. >>> >>>The usage of struct sk_buff benefits users by a) preparing vsock to use >>>other related systems that require sk_buff, such as sockmap and qdisc, >>>b) supporting basic congestion control via sock_alloc_send_skb, and c) >>>reducing copying when delivering packets to TAP. >>> >>>The socket layer no longer forces errors to be -ENOMEM, as typically >>>userspace expects -EAGAIN when the sk_sndbuf threshold is reached and >>>messages are being sent with option MSG_DONTWAIT. >>> >>>The datagram work is based off previous patches by Jiang Wang[1]. >>> >>>The introduction of datagrams creates a transport layer fairness issue >>>where datagrams may freely starve streams of queue access. This happens >>>because, unlike streams, datagrams lack the transactions necessary for >>>calculating credits and throttling. >>> >>>Previous proposals introduce changes to the spec to add an additional >>>virtqueue pair for datagrams[1]. Although this solution works, using >>>Linux's qdisc for packet scheduling leverages already existing systems, >>>avoids the need to change the virtio specification, and gives additional >>>capabilities. The usage of SFQ or fq_codel, for example, may solve the >>>transport layer starvation problem. It is easy to imagine other use >>>cases as well. For example, services of varying importance may be >>>assigned different priorities, and qdisc will apply appropriate >>>priority-based scheduling. By default, the system default pfifo qdisc is >>>used. The qdisc may be bypassed and legacy queuing is resumed by simply >>>setting the virtio-vsock%d network device to state DOWN. This technique >>>still allows vsock to work with zero-configuration. >>The basic question to answer then is this: with a net device qdisc >>etc in the picture, how is this different from virtio net then? >>Why do you still want to use vsock? > > >Or maybe it's time to revisit an old idea[1] to unify at least the >driver part (e.g using virtio-net driver for vsock then we can all >features that vsock is lacking now)?Sorry for coming late to the discussion! This would be great, though, last time I had looked at it, I had found it quite complicated. The main problem is trying to avoid all the net-specific stuff (MTU, ethernet header, HW offloading, etc.). Maybe we could start thinking about this idea by adding a new transport to vsock (e.g. virtio-net-vsock) completely separate from what we have now. Thanks, Stefano