Matej Zary
2010-Apr-30 21:43 UTC
[Xen-users] DomU - network performance and possibilities (HVM, PVM, pci-passthru, stubdom, PV on HVM...)
Hi, I want to (need to) do a network performance analysis/benchmark of various Xen networking solutions (analyzing the pros and cons of network infrastructure virtualisation) on top of Xen 4.0. So far I have Xen 4.0 running with various Dom0 kernels (git stable 2.6.32.11, git stable 2.6.31.13, mercurial 2.6.18.8, Debian distribution 2.6.26-2xen...want try some newer forward ported Xen kernel too). AFAIK, there are these possibilities for DomU "architecture". 1. DomU as classic emulated HVM. 2. DomU as classic PVM (pv_ops kernel and Xen kernel) 3. DomU as classic emulated HVM with Xen PV drivers (don't know whether these work with pv_ops Dom0, but for that case I have the 2.6.18.8 Xen linux) 4. DomU as classic PVM with pass-thru PCI NICs 5. DomU as classic HVM with pass-thru PCI NICs (don't have IOMMU capable HW :( ) 6. DomU as HVM with stubdom (according to various papers, this should be a bit faster than classic HVM DomU) 7. DomU as HVM with virtio drivers (is this possible in xen4.0?) I'm interested in the latency and bandwidth performance in phys-DomU and DomU-DomU scenarios (I have own tool for network latency benchmarking and for bandwidth there's the Iperf :) ). Also I will try to experiment a bit with the CPU pinning and schedulers in these scenarios. Also would like to test and testbench OpenSwitch as linux-bridge replacement (but I'm afraid I will not have time for this). So my questions are: 1. Is possible to use VirtIO with XEN 4.0 (as another approach on the PV-on-HVM)? 2. Is possible to use the Xen HVM DomU PV drivers (unmodified drivers?) with pvops Dom0? 3. If you have some ideas or hints, I will be very thankful. :) Regards Matej _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-May-13 13:12 UTC
[Xen-users] DomU - network performance and possibilities (HVM, PVM, pci-passthru, stubdom, PV on HVM...)
Hello, Matej: I didn''t get your initial post to xen-users because I got (automatically) unsubscribed from the list, due to my email provider getting blacklisted because of some spam problems, but now I subscribed again.. To answer your questions: 1) No, it''s not possible to use VirtIO with Xen. 2) Yes, it''s possible to use PV-on-HVM drivers in the guest with pvops dom0. 3) Take a look at: http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers Did you already make some benchmarks? If you need good latency then I think PCI passthrough will help a lot there.. or Xen netchannel2. -- Pasi Original email from Matej: Hi, I want to (need to) do a network performance analysis/benchmark of various Xen networking solutions (analyzing the pros and cons of network infrastructure virtualisation) on top of Xen 4.0. So far I have Xen 4.0 running with various Dom0 kernels (git stable 2.6.32.11, git stable 2.6.31.13, mercurial 2.6.18.8, Debian distribution 2.6.26-2xen...want try some newer forward ported Xen kernel too). AFAIK, there are these possibilities for DomU "architecture". 1. DomU as classic emulated HVM. 2. DomU as classic PVM (pv_ops kernel and Xen kernel) 3. DomU as classic emulated HVM with Xen PV drivers (don''t know whether these work with pv_ops Dom0, but for that case I have the 2.6.18.8 Xen linux) 4. DomU as classic PVM with pass-thru PCI NICs 5. DomU as classic HVM with pass-thru PCI NICs (don''t have IOMMU capable HW :( ) 6. DomU as HVM with stubdom (according to various papers, this should be a bit faster than classic HVM DomU) 7. DomU as HVM with virtio drivers (is this possible in xen4.0?) I''m interested in the latency and bandwidth performance in phys-DomU and DomU-DomU scenarios (I have own tool for network latency benchmarking and for bandwidth there''s the Iperf :) ). Also I will try to experiment a bit with the CPU pinning and schedulers in these scenarios. Also would like to test and testbench OpenSwitch as linux-bridge replacement (but I''m afraid I will not have time for this). So my questions are: 1. Is possible to use VirtIO with XEN 4.0 (as another approach on the PV-on-HVM)? 2. Is possible to use the Xen HVM DomU PV drivers (unmodified drivers?) with pvops Dom0? 3. If you have some ideas or hints, I will be very thankful. :) Regards Matej _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
lei yang
2010-May-18 09:42 UTC
Re: [Xen-users] DomU - network performance and possibilities (HVM, PVM, pci-passthru, stubdom, PV on HVM...)
On Thu, May 13, 2010 at 9:12 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:> Hello, > > Matej: I didn''t get your initial post to xen-users because I got > (automatically) unsubscribed > from the list, due to my email provider getting blacklisted because of some > spam problems, > but now I subscribed again.. > > To answer your questions: > > 1) No, it''s not possible to use VirtIO with Xen. > 2) Yes, it''s possible to use PV-on-HVM drivers in the guest with pvops > dom0. > > 3) Take a look at: > http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > > Did you already make some benchmarks? If you need good latency then I think > PCI passthrough will help a lot there.. or Xen netchannel2. > >Which one will be better for thoughput, pci passthrogh or netchanel2? Thanks Lei> -- Pasi > > > Original email from Matej: > > Hi, > > > I want to (need to) do a network performance analysis/benchmark of > various Xen networking solutions (analyzing the pros and cons of network > infrastructure virtualisation) on top of Xen 4.0. > > So far I have Xen 4.0 running with various Dom0 kernels (git stable > 2.6.32.11, git stable 2.6.31.13, mercurial 2.6.18.8, Debian distribution > 2.6.26-2xen...want try some newer forward ported Xen kernel too). > > AFAIK, there are these possibilities for DomU "architecture". > > 1. DomU as classic emulated HVM. > 2. DomU as classic PVM (pv_ops kernel and Xen kernel) > 3. DomU as classic emulated HVM with Xen PV drivers (don''t know whether > these work with pv_ops Dom0, but for that case I have the 2.6.18.8 Xen > linux) > 4. DomU as classic PVM with pass-thru PCI NICs > 5. DomU as classic HVM with pass-thru PCI NICs (don''t have IOMMU capable > HW :( ) > 6. DomU as HVM with stubdom (according to various papers, this should be > a bit faster than classic HVM DomU) > 7. DomU as HVM with virtio drivers (is this possible in xen4.0?) > > > > I''m interested in the latency and bandwidth performance in phys-DomU and > DomU-DomU scenarios (I have own tool for network latency benchmarking > and for bandwidth there''s the Iperf :) ). Also I will try to experiment > a bit with the CPU pinning and schedulers in these scenarios. > > Also would like to test and testbench OpenSwitch as linux-bridge > replacement (but I''m afraid I will not have time for this). > > So my questions are: > > 1. Is possible to use VirtIO with XEN 4.0 (as another approach on the > PV-on-HVM)? > 2. Is possible to use the Xen HVM DomU PV drivers (unmodified drivers?) > with pvops Dom0? > 3. If you have some ideas or hints, I will be very thankful. :) > > > Regards > > Matej > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >-- "We learn from failure, not from success!" _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-May-18 10:19 UTC
Re: [Xen-users] DomU - network performance and possibilities (HVM, PVM, pci-passthru, stubdom, PV on HVM...)
On Tue, May 18, 2010 at 05:42:15PM +0800, lei yang wrote:> On Thu, May 13, 2010 at 9:12 PM, Pasi Kärkkäinen <[1]pasik@iki.fi> wrote: > > Hello, > > Matej: I didn''t get your initial post to xen-users because I got > (automatically) unsubscribed > from the list, due to my email provider getting blacklisted because of > some spam problems, > but now I subscribed again.. > > To answer your questions: > > 1) No, it''s not possible to use VirtIO with Xen. > 2) Yes, it''s possible to use PV-on-HVM drivers in the guest with pvops > dom0. > > 3) Take a look at: > [2]http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > > Did you already make some benchmarks? If you need good latency then I > think > PCI passthrough will help a lot there.. or Xen netchannel2. > > > > Which one will be better for thoughput, pci passthrogh or netchanel2? >pci passthrough. -- Pasi> Thanks > Lei > > > > -- Pasi > > Original email from Matej: > > Hi, > > I want to (need to) do a network performance analysis/benchmark of > various Xen networking solutions (analyzing the pros and cons of network > infrastructure virtualisation) on top of Xen 4.0. > > So far I have Xen 4.0 running with various Dom0 kernels (git stable > 2.6.32.11, git stable 2.6.31.13, mercurial 2.6.18.8, Debian distribution > 2.6.26-2xen...want try some newer forward ported Xen kernel too). > > AFAIK, there are these possibilities for DomU "architecture". > > 1. DomU as classic emulated HVM. > 2. DomU as classic PVM (pv_ops kernel and Xen kernel) > 3. DomU as classic emulated HVM with Xen PV drivers (don''t know whether > these work with pv_ops Dom0, but for that case I have the 2.6.18.8 Xen > linux) > 4. DomU as classic PVM with pass-thru PCI NICs > 5. DomU as classic HVM with pass-thru PCI NICs (don''t have IOMMU capable > HW :( ) > 6. DomU as HVM with stubdom (according to various papers, this should be > a bit faster than classic HVM DomU) > 7. DomU as HVM with virtio drivers (is this possible in xen4.0?) > > I''m interested in the latency and bandwidth performance in phys-DomU and > DomU-DomU scenarios (I have own tool for network latency benchmarking > and for bandwidth there''s the Iperf :) ). Also I will try to experiment > a bit with the CPU pinning and schedulers in these scenarios. > > Also would like to test and testbench OpenSwitch as linux-bridge > replacement (but I''m afraid I will not have time for this). > > So my questions are: > > 1. Is possible to use VirtIO with XEN 4.0 (as another approach on the > PV-on-HVM)? > 2. Is possible to use the Xen HVM DomU PV drivers (unmodified drivers?) > with pvops Dom0? > 3. If you have some ideas or hints, I will be very thankful. :) > > Regards > > Matej > > _______________________________________________ > Xen-users mailing list > [3]Xen-users@lists.xensource.com > [4]http://lists.xensource.com/xen-users > > -- > "We learn from failure, not from success!" > > References > > Visible links > 1. mailto:pasik@iki.fi > 2. http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > 3. mailto:Xen-users@lists.xensource.com > 4. http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
lei yang
2010-May-18 13:45 UTC
Re: [Xen-users] DomU - network performance and possibilities (HVM, PVM, pci-passthru, stubdom, PV on HVM...)
On Tue, May 18, 2010 at 6:19 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:> On Tue, May 18, 2010 at 05:42:15PM +0800, lei yang wrote: > > On Thu, May 13, 2010 at 9:12 PM, Pasi Kärkkäinen <[1]pasik@iki.fi> > wrote: > > > > Hello, > > > > Matej: I didn''t get your initial post to xen-users because I got > > (automatically) unsubscribed > > from the list, due to my email provider getting blacklisted because > of > > some spam problems, > > but now I subscribed again.. > > > > To answer your questions: > > > > 1) No, it''s not possible to use VirtIO with Xen. > > 2) Yes, it''s possible to use PV-on-HVM drivers in the guest with > pvops > > dom0. > > > > 3) Take a look at: > > [2]http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > > > > Did you already make some benchmarks? If you need good latency then > I > > think > > PCI passthrough will help a lot there.. or Xen netchannel2. > > > > > > > > Which one will be better for thoughput, pci passthrogh or > netchanel2? > > > > pci passthrough. >Can you tell me why [?]? Thanks Lei> -- Pasi > > > Thanks > > Lei > > > > > > > > -- Pasi > > > > Original email from Matej: > > > > Hi, > > > > I want to (need to) do a network performance analysis/benchmark of > > various Xen networking solutions (analyzing the pros and cons of > network > > infrastructure virtualisation) on top of Xen 4.0. > > > > So far I have Xen 4.0 running with various Dom0 kernels (git stable > > 2.6.32.11, git stable 2.6.31.13, mercurial 2.6.18.8, Debian > distribution > > 2.6.26-2xen...want try some newer forward ported Xen kernel too). > > > > AFAIK, there are these possibilities for DomU "architecture". > > > > 1. DomU as classic emulated HVM. > > 2. DomU as classic PVM (pv_ops kernel and Xen kernel) > > 3. DomU as classic emulated HVM with Xen PV drivers (don''t know > whether > > these work with pv_ops Dom0, but for that case I have the 2.6.18.8 > Xen > > linux) > > 4. DomU as classic PVM with pass-thru PCI NICs > > 5. DomU as classic HVM with pass-thru PCI NICs (don''t have IOMMU > capable > > HW :( ) > > 6. DomU as HVM with stubdom (according to various papers, this > should be > > a bit faster than classic HVM DomU) > > 7. DomU as HVM with virtio drivers (is this possible in xen4.0?) > > > > I''m interested in the latency and bandwidth performance in phys-DomU > and > > DomU-DomU scenarios (I have own tool for network latency > benchmarking > > and for bandwidth there''s the Iperf :) ). Also I will try to > experiment > > a bit with the CPU pinning and schedulers in these scenarios. > > > > Also would like to test and testbench OpenSwitch as linux-bridge > > replacement (but I''m afraid I will not have time for this). > > > > So my questions are: > > > > 1. Is possible to use VirtIO with XEN 4.0 (as another approach on > the > > PV-on-HVM)? > > 2. Is possible to use the Xen HVM DomU PV drivers (unmodified > drivers?) > > with pvops Dom0? > > 3. If you have some ideas or hints, I will be very thankful. :) > > > > Regards > > > > Matej > > > > _______________________________________________ > > Xen-users mailing list > > [3]Xen-users@lists.xensource.com > > [4]http://lists.xensource.com/xen-users > > > > -- > > "We learn from failure, not from success!" > > > > References > > > > Visible links > > 1. mailto:pasik@iki.fi > > 2. http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > > 3. mailto:Xen-users@lists.xensource.com > > 4. http://lists.xensource.com/xen-users >-- "We learn from failure, not from success!" _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-May-18 14:47 UTC
Re: [Xen-users] DomU - network performance and possibilities (HVM, PVM, pci-passthru, stubdom, PV on HVM...)
On Tue, May 18, 2010 at 09:45:27PM +0800, lei yang wrote:> On Tue, May 18, 2010 at 6:19 PM, Pasi Kärkkäinen <[1]pasik@iki.fi> wrote: > > On Tue, May 18, 2010 at 05:42:15PM +0800, lei yang wrote: > > On Thu, May 13, 2010 at 9:12 PM, Pasi Kärkkäinen > <[1][2]pasik@iki.fi> wrote: > > > > Hello, > > > > Matej: I didn''t get your initial post to xen-users because I got > > (automatically) unsubscribed > > from the list, due to my email provider getting blacklisted > because of > > some spam problems, > > but now I subscribed again.. > > > > To answer your questions: > > > > 1) No, it''s not possible to use VirtIO with Xen. > > 2) Yes, it''s possible to use PV-on-HVM drivers in the guest with > pvops > > dom0. > > > > 3) Take a look at: > > [2][3]http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > > > > Did you already make some benchmarks? If you need good latency > then I > > think > > PCI passthrough will help a lot there.. or Xen netchannel2. > > > > > > > > Which one will be better for thoughput, pci passthrogh or > netchanel2? > > > > pci passthrough. > > > Can you tell me why ? >Because with pci passthrough the guest has direct access to the NIC. Netchannel and netchannel2 (bridged networking) goes through dom0. -- Pasi> Thanks > Lei > > > > -- Pasi > > Thanks > > Lei > > > > > > > > -- Pasi > > > > Original email from Matej: > > > > Hi, > > > > I want to (need to) do a network performance analysis/benchmark > of > > various Xen networking solutions (analyzing the pros and cons of > network > > infrastructure virtualisation) on top of Xen 4.0. > > > > So far I have Xen 4.0 running with various Dom0 kernels (git > stable > > 2.6.32.11, git stable 2.6.31.13, mercurial 2.6.18.8, Debian > distribution > > 2.6.26-2xen...want try some newer forward ported Xen kernel too). > > > > AFAIK, there are these possibilities for DomU "architecture". > > > > 1. DomU as classic emulated HVM. > > 2. DomU as classic PVM (pv_ops kernel and Xen kernel) > > 3. DomU as classic emulated HVM with Xen PV drivers (don''t know > whether > > these work with pv_ops Dom0, but for that case I have the > 2.6.18.8 Xen > > linux) > > 4. DomU as classic PVM with pass-thru PCI NICs > > 5. DomU as classic HVM with pass-thru PCI NICs (don''t have IOMMU > capable > > HW :( ) > > 6. DomU as HVM with stubdom (according to various papers, this > should be > > a bit faster than classic HVM DomU) > > 7. DomU as HVM with virtio drivers (is this possible in xen4.0?) > > > > I''m interested in the latency and bandwidth performance in > phys-DomU and > > DomU-DomU scenarios (I have own tool for network latency > benchmarking > > and for bandwidth there''s the Iperf :) ). Also I will try to > experiment > > a bit with the CPU pinning and schedulers in these scenarios. > > > > Also would like to test and testbench OpenSwitch as linux-bridge > > replacement (but I''m afraid I will not have time for this). > > > > So my questions are: > > > > 1. Is possible to use VirtIO with XEN 4.0 (as another approach on > the > > PV-on-HVM)? > > 2. Is possible to use the Xen HVM DomU PV drivers (unmodified > drivers?) > > with pvops Dom0? > > 3. If you have some ideas or hints, I will be very thankful. :) > > > > Regards > > > > Matej > > > > _______________________________________________ > > Xen-users mailing list > > [3][4]Xen-users@lists.xensource.com > > [4][5]http://lists.xensource.com/xen-users > > > > -- > > "We learn from failure, not from success!" > > > > References > > > > Visible links > > 1. mailto:[6]pasik@iki.fi > > 2. [7]http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > > 3. mailto:[8]Xen-users@lists.xensource.com > > 4. [9]http://lists.xensource.com/xen-users > > -- > "We learn from failure, not from success!" > > References > > Visible links > 1. mailto:pasik@iki.fi > 2. mailto:pasik@iki.fi > 3. http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > 4. mailto:Xen-users@lists.xensource.com > 5. http://lists.xensource.com/xen-users > 6. mailto:pasik@iki.fi > 7. http://wiki.xensource.com/xenwiki/XenLinuxPVonHVMdrivers > 8. mailto:Xen-users@lists.xensource.com > 9. http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users