Hi James, After Opensolaris update to snv_109, GPLPV is finally working (tested on Win 2003 sp2, GPLPV 0.9.12-pre13) :) Last time the network driver does not work. Now it works with improved performance. Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps This is still much lower compared to Linux dom0 btw: dom0->domU 781 Mbps, domU->dom0 834 Mbps Any ideas what cause this performance difference? Regards, Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Just want to let you guys know, after Opensolaris update to snv_109, GPLPV is finally working (tested on Win 2003 sp2, GPLPV 0.9.12-pre13) :) Last time the network driver does not work. Now it works with improved performance. Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps This is still much lower compared to Linux dom0 btw: dom0->domU 781 Mbps, domU->dom0 834 Mbps Regards, Fajar
> Hi James, > > After Opensolaris update to snv_109, GPLPV is finally working (tested > on Win 2003 sp2, GPLPV 0.9.12-pre13) :) > > Last time the network driver does not work. Now it works with improved > performance. > Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, > With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps > > This is still much lower compared to Linux dom0 btw: > dom0->domU 781 Mbps, domU->dom0 834 Mbps > > Any ideas what cause this performance difference?How does the PV performance differ? I don''t know enough about the opensolaris network stack, but filtering isn''t getting in the way or something is it? James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Thu, Mar 12, 2009 at 3:28 PM, James Harper <james.harper@bendigoit.com.au> wrote:>> Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, >> With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps >> >> This is still much lower compared to Linux dom0 btw: >> dom0->domU 781 Mbps, domU->dom0 834 Mbps > > How does the PV performance differ? I don''t know enough about theI''m beginning to see a pattern to your answers :D So let me get this straight. GPLPV performance should be somewhere in the range on Linux HVM with PV drivers, and both of them should be lower than Linux PV right?> opensolaris network stack, but filtering isn''t getting in the way or > something is it?On that opensolaris dom0, Linux PV guest has this performance: dom0->domU 556 Mbps, domU->dom0 631Mbps which is lower compared to typical Linux PV on Linux dom0 performance. Could it be because dom0 <-> domU communication on Opensolaris still involves physical NIC as the dom0 IP is located on dom0 NIC (which is different from Linux, where dom0 IP can be on the bridge without involving any physical NIC)? I''ll try HVM Linux later (takes somewhat longer to setup). Regards, Fajar _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Fajar A. Nugraha wrote:> Just want to let you guys know, after Opensolaris update to snv_109, > GPLPV is finally working (tested on Win 2003 sp2, GPLPV 0.9.12-pre13) > :) > > Last time the network driver does not work. Now it works with improved > performance. > Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, > With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps > > This is still much lower compared to Linux dom0 btw: > dom0->domU 781 Mbps, domU->dom0 834 MbpsDo you have any numbers comparing disk performance? MRJ
On Thu, Mar 12, 2009 at 8:32 PM, Mark Johnson <Mark.Johnson@sun.com> wrote:> Do you have any numbers comparing disk performance?I think I may have to redo my setup for I/O test. I''m currently using zvol-backed storage with compression=on, and the results are kinda weird. On the same opensolaris dom0, tested with "dd if=/dev/zero of=test1G bs=1M count=2048" Windows without GPLPV: 48.3 MB/s Windows with GPLPV: 52.8 MB/s Linux PV: 86.5 MB/s dom0 (zfs, compression=on): 57.7 MB/s So ext3 on zvol is faster than zfs? Is it the effect of compression=on and if=/dev/zero? Windows domU and Linux PV has 1000MB memory, on Windows dd was from cygwin. dom0 was left with 984 MB memory. Regards, Fajar
Fajar A. Nugraha wrote:> On Thu, Mar 12, 2009 at 8:32 PM, Mark Johnson <Mark.Johnson@sun.com> wrote: >> Do you have any numbers comparing disk performance? > > I think I may have to redo my setup for I/O test. I''m currently using > zvol-backed storage with compression=on, and the results are kinda > weird. > > On the same opensolaris dom0, tested with "dd if=/dev/zero of=test1G > bs=1M count=2048" > Windows without GPLPV: 48.3 MB/s > Windows with GPLPV: 52.8 MB/s > Linux PV: 86.5 MB/s > dom0 (zfs, compression=on): 57.7 MB/s > > So ext3 on zvol is faster than zfs? Is it the effect of compression=on > and if=/dev/zero? > > Windows domU and Linux PV has 1000MB memory, on Windows dd was from > cygwin. dom0 was left with 984 MB memory.zero is easy to compress, so probably not a good example :-) I''m not sure what effect compression might have other than that, but if your using zfs in dom0, you should limit the size of the arc. echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/system If your using a disk file (vs a zvol) on a zfs filesystem, you should set the recordsize for that fs to 8k. zvols already default to 8k. zfs set recordsize=8k rpool/guests MRJ
Note that these drivers only work on domains that do not have vmdk-backed disks. I''m investigating and hopefully will have it sorted out soon. Fajar A. Nugraha wrote:> Just want to let you guys know, after Opensolaris update to snv_109, > GPLPV is finally working (tested on Win 2003 sp2, GPLPV 0.9.12-pre13) > :) > > Last time the network driver does not work. Now it works with improved > performance. > Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, > With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps > > This is still much lower compared to Linux dom0 btw: > dom0->domU 781 Mbps, domU->dom0 834 Mbps > > Regards, > > Fajar > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org-- ----------------------------------------------------- Russ Blaine | Solaris Kernel | russell.blaine@sun.com
> > On Thu, Mar 12, 2009 at 3:28 PM, James Harper > <james.harper@bendigoit.com.au> wrote: > >> Without GPLPV : dom0->domU 38.8 Mbps, domU->dom0 24.4 Mbps, > >> With GPLPV : dom0->domU 148 Mbps, domU->dom0 446 Mbps > >> > >> This is still much lower compared to Linux dom0 btw: > >> dom0->domU 781 Mbps, domU->dom0 834 Mbps > > > > How does the PV performance differ? I don''t know enough about the > > I''m beginning to see a pattern to your answers :D > So let me get this straight. GPLPV performance should be somewhere in > the range on Linux HVM with PV drivers, and both of them should be > lower than Linux PV right?PV on HVM could in theory be as fast as PV Linux, but it shouldn''t be faster. If PV performance was also low then GPLPV is probably the long place to be looking for a problem...> > opensolaris network stack, but filtering isn''t getting in the way or > > something is it? > > On that opensolaris dom0, Linux PV guest has this performance: > dom0->domU 556 Mbps, domU->dom0 631Mbps > > which is lower compared to typical Linux PV on Linux dom0 performance.That is the sort of target I''d expect to be aiming for with windows PV...> > Could it be because dom0 <-> domU communication on Opensolaris still > involves physical NIC as the dom0 IP is located on dom0 NIC (which is > different from Linux, where dom0 IP can be on the bridge without > involving any physical NIC)?I don''t know. I''m familiar with the Linux network stack but not so much the solaris one.> > I''ll try HVM Linux later (takes somewhat longer to setup). >If PV on Linux HVM is also slow then there could be a problem unrelated to GPLPV... James _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
On Fri, Mar 13, 2009 at 5:41 AM, Mark Johnson <Mark.Johnson@sun.com> wrote:>> So ext3 on zvol is faster than zfs? Is it the effect of compression=on >> and if=/dev/zero? > > zero is easy to compress, so probably not a good example :-)Yeah, but zfs and zvol should handle the compression (and also checksum) the same way right? Which makes me wonder why on earth did ext3 on zvol get such good performance.> > I''m not sure what effect compression might have other than > that, but if your using zfs in dom0, you should limit the > size of the arc. > > echo "set zfs:zfs_arc_max = 0x10000000" >> /etc/systemWill do. I''ll try again with uncompressed zvol as well.> > If your using a disk file (vs a zvol) on a zfs filesystem, > you should set the recordsize for that fs to 8k. zvols > already default to 8k. > > zfs set recordsize=8k rpool/guests > >I''m only using zvols. Why 8k, not 4k? AFAIK the default cluster size for NTFS is 8k, as is ext3''s default block size. Either way, since I''m testing using dd with 1M blocksize, shouldn''t the block size be less relevant? Regards, Fajar
On Fri, Mar 13, 2009 at 6:49 AM, Russ Blaine <russell.blaine@sun.com> wrote:> Note that these drivers only work on domains that do not have vmdk-backed > disks. I''m investigating and hopefully will have it sorted out soon.Might be upstream issue. On xen-users list there was a discussion about tap:qcow not working with PV drivers (both GPLPV and Linux PV drivers on HMV), only tap:aio works. I haven''t tried tap:vmdk (since it only use synchronous I/O, while tap:qcow uses asynchronous), but perhaps it''s the same issue. Regards, Fajar
On Fri, Mar 13, 2009 at 9:44 AM, Fajar A. Nugraha <fajar@fajar.net> wrote:> Will do. I''ll try again with uncompressed zvol as well.So here goes again, with compression=off, volblocksize=4k (for zvol), recordsize=4k (dom0 zfs comparison), zfs arc limited to 256M Command line : dd if=/dev/zero of=test2G bs=1M count=2048 without GPLPV : 6.6 MB/s with GPLPV : 12.1 MB/s Linux PV : 24.6 MB/s dom0 zfs : 21.8 MB/s Generally this makes more sense, GPLPV is doing a very good job. I''m still wondering how on earth ext3 on zvol is still FASTER compared to zfs on dom0. Regards, Fajar
Fajar A. Nugraha schrieb:> On Fri, Mar 13, 2009 at 9:44 AM, Fajar A. Nugraha <fajar@fajar.net> wrote: >> Will do. I''ll try again with uncompressed zvol as well. > > So here goes again, with compression=off, volblocksize=4k (for zvol), > recordsize=4k (dom0 zfs comparison), zfs arc limited to 256M > > Command line : dd if=/dev/zero of=test2G bs=1M count=2048 > without GPLPV : 6.6 MB/s > with GPLPV : 12.1 MB/s > Linux PV : 24.6 MB/s > dom0 zfs : 21.8 MB/s > > Generally this makes more sense, GPLPV is doing a very good job. I''m > still wondering how on earth ext3 on zvol is still FASTER compared to > zfs on dom0.Less and smaller checksums, ext3 is really less complicated than zfs, try ext2 if you really like to wonder. Regards, Florian> > Regards, > > Fajar > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org >
On Fri, Mar 13, 2009 at 1:33 PM, Florian Manschwetus <florianmanschwetus@gmx.de> wrote:>> I''m >> still wondering how on earth ext3 on zvol is still FASTER compared to >> zfs on dom0.> Less and smaller checksums, ext3 is really less complicated than zfs, > try ext2 if you really like to wonder.Doesn''t zvol also do checksums (and raid, and compression, and whatever other extra efforts done by zfs on the same pool with the same settings) ? So from that perspective shoudln''t ext3 on zvol performance be slower, because it adds another layer of operation on top zvols?