Michael S. Tsirkin
2018-Dec-04 01:31 UTC
[RFC] Discuss about an new idea "Vsock over Virtio-net"
On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote:> On 2018/11/30 21:40, Michael S. Tsirkin wrote: > > On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: > >> > >> On 2018/11/30 ??8:55, Jason Wang wrote: > >>> > >>> On 2018/11/30 ??8:52, Michael S. Tsirkin wrote: > >>>>>> If you want to compare it with > >>>>>> something that would be TCP or QUIC. The fundamental > >>>>>> difference between > >>>>>> virtio-vsock and e.g. TCP is that TCP operates in a packet > >>>>>> loss environment. > >>>>>> So they are using timers for reliability, and receiver is > >>>>>> always free to > >>>>>> discard any unacked data. > >>>>> Virtio-net knows nothing above L2, so they are totally > >>>>> transparent to device > >>>>> itself. I still don't get why not using virtio-net instead. > >>>>> > >>>>> > >>>>> Thanks > >>>> Is your question why is virtio-vsock used instead of TCP on top of IP > >>>> on top of virtio-net? > >>>> > >>>> > >>> > >>> No, my question is why not do vsock through virtio-net. > >>> > >>> Thanks > >>> > >> > >> Just to clarify, it's not about vosck over ethernet, and it's not about > >> inventing new features or APIs. It's probably something like: > >> > >> - Let virtio-net driver probe vsock device and do vosck specific things if > >> needed to share as much codes. > >> > >> - A new kind of sockfd (which is vsock based) for vhost-net for it to do > >> vsock specific things (hopefully it can be transparent). > >> > >> The change should be totally transparent to userspace applications. > >> > >> Thanks > > > > Which code is duplicated between virtio vsock and virtio net right now? > > > > Hi Michael, > > AFAIK, there is almost no duplicate code between virtio vsock and virtio net now. > > But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, > it has some duplicate codes from virtio net. Based on it, we both think vsock > may use virtio net as a transport channel, in this way, vsock can use some of > virtio net great features. > > Thanks, > Yiwen.What I would do is just copy some code and show a performance benefit. If that works out it will be clearer which code should be shared. -- MST
On 2018/12/4 9:31, Michael S. Tsirkin wrote:> On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote: >> On 2018/11/30 21:40, Michael S. Tsirkin wrote: >>> On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: >>>> >>>> On 2018/11/30 ??8:55, Jason Wang wrote: >>>>> >>>>> On 2018/11/30 ??8:52, Michael S. Tsirkin wrote: >>>>>>>> If you want to compare it with >>>>>>>> something that would be TCP or QUIC. The fundamental >>>>>>>> difference between >>>>>>>> virtio-vsock and e.g. TCP is that TCP operates in a packet >>>>>>>> loss environment. >>>>>>>> So they are using timers for reliability, and receiver is >>>>>>>> always free to >>>>>>>> discard any unacked data. >>>>>>> Virtio-net knows nothing above L2, so they are totally >>>>>>> transparent to device >>>>>>> itself. I still don't get why not using virtio-net instead. >>>>>>> >>>>>>> >>>>>>> Thanks >>>>>> Is your question why is virtio-vsock used instead of TCP on top of IP >>>>>> on top of virtio-net? >>>>>> >>>>>> >>>>> >>>>> No, my question is why not do vsock through virtio-net. >>>>> >>>>> Thanks >>>>> >>>> >>>> Just to clarify, it's not about vosck over ethernet, and it's not about >>>> inventing new features or APIs. It's probably something like: >>>> >>>> - Let virtio-net driver probe vsock device and do vosck specific things if >>>> needed to share as much codes. >>>> >>>> - A new kind of sockfd (which is vsock based) for vhost-net for it to do >>>> vsock specific things (hopefully it can be transparent). >>>> >>>> The change should be totally transparent to userspace applications. >>>> >>>> Thanks >>> >>> Which code is duplicated between virtio vsock and virtio net right now? >>> >> >> Hi Michael, >> >> AFAIK, there is almost no duplicate code between virtio vsock and virtio net now. >> >> But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, >> it has some duplicate codes from virtio net. Based on it, we both think vsock >> may use virtio net as a transport channel, in this way, vsock can use some of >> virtio net great features. >> >> Thanks, >> Yiwen. > > What I would do is just copy some code and show a performance > benefit. If that works out it will be clearer which code > should be shared. >Hi Michael, I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) a month ago, and the performance as follows: I write a tool to test the vhost-vsock performance, mainly send big packet(64K) included guest->Host and Host->Guest. The result as follows: Before performance: Single socket Multiple sockets(Max Bandwidth) Guest->Host ~400MB/s ~480MB/s Host->Guest ~1450MB/s ~1600MB/s After performance: Single socket Multiple sockets(Max Bandwidth) Guest->Host ~1700MB/s ~2900MB/s Host->Guest ~1700MB/s ~2900MB/s>From the test results, the performance is improved obviously, and guestmemory will not be wasted. In addition, multiqueue feature I have not implemented it yet. Thanks, Yiwen.
Michael S. Tsirkin
2018-Dec-04 04:08 UTC
[RFC] Discuss about an new idea "Vsock over Virtio-net"
On Tue, Dec 04, 2018 at 10:21:40AM +0800, jiangyiwen wrote:> On 2018/12/4 9:31, Michael S. Tsirkin wrote: > > On Mon, Dec 03, 2018 at 11:10:58AM +0800, jiangyiwen wrote: > >> On 2018/11/30 21:40, Michael S. Tsirkin wrote: > >>> On Fri, Nov 30, 2018 at 09:10:03PM +0800, Jason Wang wrote: > >>>> > >>>> On 2018/11/30 ??8:55, Jason Wang wrote: > >>>>> > >>>>> On 2018/11/30 ??8:52, Michael S. Tsirkin wrote: > >>>>>>>> If you want to compare it with > >>>>>>>> something that would be TCP or QUIC. The fundamental > >>>>>>>> difference between > >>>>>>>> virtio-vsock and e.g. TCP is that TCP operates in a packet > >>>>>>>> loss environment. > >>>>>>>> So they are using timers for reliability, and receiver is > >>>>>>>> always free to > >>>>>>>> discard any unacked data. > >>>>>>> Virtio-net knows nothing above L2, so they are totally > >>>>>>> transparent to device > >>>>>>> itself. I still don't get why not using virtio-net instead. > >>>>>>> > >>>>>>> > >>>>>>> Thanks > >>>>>> Is your question why is virtio-vsock used instead of TCP on top of IP > >>>>>> on top of virtio-net? > >>>>>> > >>>>>> > >>>>> > >>>>> No, my question is why not do vsock through virtio-net. > >>>>> > >>>>> Thanks > >>>>> > >>>> > >>>> Just to clarify, it's not about vosck over ethernet, and it's not about > >>>> inventing new features or APIs. It's probably something like: > >>>> > >>>> - Let virtio-net driver probe vsock device and do vosck specific things if > >>>> needed to share as much codes. > >>>> > >>>> - A new kind of sockfd (which is vsock based) for vhost-net for it to do > >>>> vsock specific things (hopefully it can be transparent). > >>>> > >>>> The change should be totally transparent to userspace applications. > >>>> > >>>> Thanks > >>> > >>> Which code is duplicated between virtio vsock and virtio net right now? > >>> > >> > >> Hi Michael, > >> > >> AFAIK, there is almost no duplicate code between virtio vsock and virtio net now. > >> > >> But, if virtio vsock wants to support mergeable rx buffer and multiqueue feature, > >> it has some duplicate codes from virtio net. Based on it, we both think vsock > >> may use virtio net as a transport channel, in this way, vsock can use some of > >> virtio net great features. > >> > >> Thanks, > >> Yiwen. > > > > What I would do is just copy some code and show a performance > > benefit. If that works out it will be clearer which code > > should be shared. > > > > Hi Michael, > > I have already sent a series of patches (VSOCK: support mergeable rx buffer in vhost-vsock) > a month ago, and the performance as follows: > > I write a tool to test the vhost-vsock performance, mainly send big > packet(64K) included guest->Host and Host->Guest. The result as > follows: > > Before performance: > Single socket Multiple sockets(Max Bandwidth) > Guest->Host ~400MB/s ~480MB/s > Host->Guest ~1450MB/s ~1600MB/s > > After performance: > Single socket Multiple sockets(Max Bandwidth) > Guest->Host ~1700MB/s ~2900MB/s > Host->Guest ~1700MB/s ~2900MB/s > > >From the test results, the performance is improved obviously, and guest > memory will not be wasted.Oh I didn't see that one. Pls CC me in the future. Looking at it I agree zero page allocation looks like an issue but besides that, I think we can merge something similar and look at refactoring and future extensions later. However, any interface change (e.g. a new feature) must be CC'd to one of virtio lists (subscriber-only).> In addition, multiqueue feature I have not implemented it yet. > > Thanks, > Yiwen. >