Xen users - Sep 2008 - performance of pv drivers for windows

Hello everybody,

I tried to measure the performance of the available drivers for windows as a 
HVM guest.
I used the gplpv drivers 0.9.11-pre17, the PV drivers from Novell, and the 
drivers 
from Citrix XenSource with the XenServer 5.

The Novell and gplpv drivers were more or less at the same speed, for both, 
network and disk performance.
The disk performance was about 10MB/s reading and writing sequentially, and 
about 1-1.5MB/s for reading and writing randomly.
The network speed was about 10-12MB/s, via a GigaBit line.

The Xensource drivers made at least about 30MB/s reading and writing 
sequentially, but for reading and writing randomly, it was also only lousy 
1.5MB/s.
Via network, over the GigaBit line, with the xensource drivers, the speed was 
about 78 MB/s.

The Windows system was a XP SP2.
hdparm on the dom0 gives about 60MB/s.
The network test was an ftp transfer, just downloading a 500MB file, without 
writing it to disk, writing to nul. The same in the dom0, writing the file 
to /dev/null gave me 112MB/s.

So I am wondering, what are the expected speed gains for the gplpv drivers?
Is the performance of the drivers bettter with different windows versions, 
e.g. windows server 2003?

kind regards
Sebastian

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sebastian Reitenbach wrote:
> Hello everybody,
>
> I tried to measure the performance of the available drivers for windows as
a
> HVM guest.
> I used the gplpv drivers 0.9.11-pre17, the PV drivers from Novell, and the 
> drivers 
> from Citrix XenSource with the XenServer 5.
>
> The Novell and gplpv drivers were more or less at the same speed, for both,
> network and disk performance.
> The disk performance was about 10MB/s reading and writing sequentially, and
> about 1-1.5MB/s for reading and writing randomly.
> The network speed was about 10-12MB/s, via a GigaBit line.
>
> The Xensource drivers made at least about 30MB/s reading and writing 
> sequentially, but for reading and writing randomly, it was also only lousy 
> 1.5MB/s.
>   
 1.5MB/s for random write access is not lousy , if you perform random 
write test in dom0 or on native non xen kernel
you''ll also get  1.5MB/s
> Via network, over the GigaBit line, with the xensource drivers, the speed
was
> about 78 MB/s.
>
> The Windows system was a XP SP2.
> hdparm on the dom0 gives about 60MB/s.
> The network test was an ftp transfer, just downloading a 500MB file,
without
> writing it to disk, writing to nul. The same in the dom0, writing the file 
> to /dev/null gave me 112MB/s.
>
> So I am wondering, what are the expected speed gains for the gplpv drivers?
> Is the performance of the drivers bettter with different windows versions, 
> e.g. windows server 2003?
>
> kind regards
> Sebastian
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>   

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi

I think there is something wrong on your system.
I have Windows 2003 Server installed and I get much better results even
without any PV drivers...
See the attachment and my previous mail below....

Regards,
  Steffen



-----------------------------

Hi

Something I found testing the lastest gplpv drivers just got my attention
again:

Without gplpv drivers, disk throughput is at 27mb/s (r+w).
With gplpv drivers, it reaches 56/45mb/s (r/w).
No surprise, I expected gplpv to be faster than qemu.

These values were measured using PassMark Performance Test 6.0.

What surprises me is the "Disk - Zufallssuche + LS" result:
Witout gplpv it reaches 31mb/s while gplpv drivers only reach 5mb/s.

I''ve attached the result.
Can someone explain this?

Regards,
  Steffen





-----Ursprüngliche Nachricht-----
Von: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von Sebastian
Reitenbach
Gesendet: Montag, 22. September 2008 11:19
An: xen-users@lists.xensource.com
Betreff: [Xen-users] performance of pv drivers for windows

Hello everybody,

I tried to measure the performance of the available drivers for windows as a

HVM guest.
I used the gplpv drivers 0.9.11-pre17, the PV drivers from Novell, and the 
drivers 
from Citrix XenSource with the XenServer 5.

The Novell and gplpv drivers were more or less at the same speed, for both, 
network and disk performance.
The disk performance was about 10MB/s reading and writing sequentially, and 
about 1-1.5MB/s for reading and writing randomly.
The network speed was about 10-12MB/s, via a GigaBit line.

The Xensource drivers made at least about 30MB/s reading and writing 
sequentially, but for reading and writing randomly, it was also only lousy 
1.5MB/s.
Via network, over the GigaBit line, with the xensource drivers, the speed
was 
about 78 MB/s.

The Windows system was a XP SP2.
hdparm on the dom0 gives about 60MB/s.
The network test was an ftp transfer, just downloading a 500MB file, without

writing it to disk, writing to nul. The same in the dom0, writing the file 
to /dev/null gave me 112MB/s.

So I am wondering, what are the expected speed gains for the gplpv drivers?
Is the performance of the drivers bettter with different windows versions, 
e.g. windows server 2003?

kind regards
Sebastian

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Hello everybody,
> 
> I tried to measure the performance of the available drivers for
windows as
> a HVM guest.
> I used the gplpv drivers 0.9.11-pre17, the PV drivers from Novell, and
the
> drivers from Citrix XenSource with the XenServer 5.
> 
> The Novell and gplpv drivers were more or less at the same speed, for
> both, network and disk performance.
> The disk performance was about 10MB/s reading and writing
sequentially,
> and about 1-1.5MB/s for reading and writing randomly.
> The network speed was about 10-12MB/s, via a GigaBit line.
XPsp2 has a known problem with LSO (which gplpv drivers support... I
assume the xensource drivers will too). Please try stopping the firewall
service (that is _not_ the same as turning off the firewall in network
settings - you actually have to go into services and stop the service).
> The Xensource drivers made at least about 30MB/s reading and writing
> sequentially, but for reading and writing randomly, it was also only
lousy
> 1.5MB/s.
> Via network, over the GigaBit line, with the xensource drivers, the
speed
> was about 78 MB/s.
My testing using iperf has shown that I can reach gigabit speeds on the
network, so it is possible.
> The Windows system was a XP SP2.
> hdparm on the dom0 gives about 60MB/s.
> The network test was an ftp transfer, just downloading a 500MB file,
> without writing it to disk, writing to nul. The same in the dom0,
writing
> the file to /dev/null gave me 112MB/s.
Please use iperf to test network speeds. It''s a bit more comparable to
the testing that myself and others have done.
> So I am wondering, what are the expected speed gains for the gplpv
> drivers?
> Is the performance of the drivers bettter with different windows
versions,
> e.g. windows server 2003?
XPsp2 with the firewall service enabled behaves badly when LSO is
enabled.
Windows 2003 sp1 has also performed badly in some testing I have done
(not as badly as XPsp2 - something like 50% worse performance instead of
90% worse performance)

My best results have been under Windows 2003 sp2. SP2 introduced some
pretty hefty improvements to NDIS which is the windows network driver
layer.

The advantages I''d expect to see in performance are due to increased
throughput and lower CPU usage.

Also, depending on the test tool you are using, the gplpv disk drivers
may perform quite poorly. This happens if the tools gives the windows
kernel buffers that are not aligned to a 512 byte boundary.

Please run debugview from sysinternals.com while you are running your
disk performance testing. It will periodically output some stats like:

XenVbd     stat_interrupts = 2408914
XenVbd     stat_interrupts_for_me = 314368
XenVbd     stat_reads = 131507
XenVbd     stat_writes = 208567
XenVbd     stat_unaligned_le_4096 = 7
XenVbd     stat_unaligned_le_8192 = 0
XenVbd     stat_unaligned_le_16384 = 0
XenVbd     stat_unaligned_le_32768 = 0
XenVbd     stat_unaligned_le_65536 = 0
XenVbd     stat_unaligned_gt_65536 = 0
XenVbd     stat_no_shadows = 0
XenVbd     stat_no_grants = 0
XenVbd     stat_outstanding_requests = 1

The things I''m interested in are that stat_unaligned_xxx figures. The
only unaligned requests I see during day to day operations are somewhere
between 5 and 10 that occur very very early during boot
(stat_unaligned_le_4096 = 7). However I have seen chkdsk, defrag, and at
least one testing tool issue requests not aligned on 512 byte
boundaries. When that happens, the gplpv drivers have to break the
request into 4096 byte chunks and submit each chunk, one at a time, to
blkback, which really slows things down.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Monday September 22 2008 11:57:39 Alexander Piavlo
wrote:
> Sebastian Reitenbach wrote:
> > Hello everybody,
> >
> > I tried to measure the performance of the available drivers for
windows
> > as a HVM guest.
> > I used the gplpv drivers 0.9.11-pre17, the PV drivers from Novell, and
> > the drivers
> > from Citrix XenSource with the XenServer 5.
> >
> > The Novell and gplpv drivers were more or less at the same speed, for
> > both, network and disk performance.
> > The disk performance was about 10MB/s reading and writing
sequentially,
> > and about 1-1.5MB/s for reading and writing randomly.
> > The network speed was about 10-12MB/s, via a GigaBit line.
> >
> > The Xensource drivers made at least about 30MB/s reading and writing
> > sequentially, but for reading and writing randomly, it was also only
> > lousy 1.5MB/s.
>
>  1.5MB/s for random write access is not lousy , if you perform random
> write test in dom0 or on native non xen kernel
> you''ll also get  1.5MB/s
Hm, ok, but without any PV drivers, I got about 7MB/sec. at least for writing 
to the disk.
Therefore I was wondering.
https://www.l00-bugdead-prods.de/XenBench/random_write_without_pv_driver.htm


kind regards
Sebastian

>
> > Via network, over the GigaBit line, with the xensource drivers, the
speed
> > was about 78 MB/s.
> >
> > The Windows system was a XP SP2.
> > hdparm on the dom0 gives about 60MB/s.
> > The network test was an ftp transfer, just downloading a 500MB file,
> > without writing it to disk, writing to nul. The same in the dom0,
writing
> > the file to /dev/null gave me 112MB/s.
> >
> > So I am wondering, what are the expected speed gains for the gplpv
> > drivers? Is the performance of the drivers bettter with different
windows
> > versions, e.g. windows server 2003?
> >
> > kind regards
> > Sebastian
> >
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@lists.xensource.com
> > http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,


On Monday September 22 2008 12:44:31 James Harper wrote:
> > Hello everybody,
> >
> > I tried to measure the performance of the available drivers for
>
> windows as
>
> > a HVM guest.
> > I used the gplpv drivers 0.9.11-pre17, the PV drivers from Novell, and
>
> the
>
> > drivers from Citrix XenSource with the XenServer 5.
> >
> > The Novell and gplpv drivers were more or less at the same speed, for
> > both, network and disk performance.
> > The disk performance was about 10MB/s reading and writing
>
> sequentially,
>
> > and about 1-1.5MB/s for reading and writing randomly.
> > The network speed was about 10-12MB/s, via a GigaBit line.
>
> XPsp2 has a known problem with LSO (which gplpv drivers support... I
> assume the xensource drivers will too). Please try stopping the firewall
> service (that is _not_ the same as turning off the firewall in network
> settings - you actually have to go into services and stop the service).
I reinstalled the dom0, and also created a fresh windows XP SP2 image.
Disabled the firewall and security center from the services, and ran the tests 
again.
With the firewall disabled, the network performance was much better, however, 
the overall throughput over the time was not constant.
Now I took iperf to measure the performance.
I measured the speeds of the system without any PV drivers, and then the GPLPV 
drivers, and the Novell drivers. The results you can find on the link below.
I used iperf, to measure bandwith, PassMark Performance Test 6.1 to measure 
the disk speed. You will also find the times of the FTP file transfer of the 
large file.

Without any of the PV drivers, I got about 7MB/s random write speed, but with 
PV drivers, it dropped to 1.5 MB/s.
Also with the PV drivers from novell, I got a much better network throughput 
result.

You can see the results here:
https://www.l00-bugdead-prods.de/XenBench/
>
> > The Xensource drivers made at least about 30MB/s reading and writing
> > sequentially, but for reading and writing randomly, it was also only
>
> lousy
>
> > 1.5MB/s.
> > Via network, over the GigaBit line, with the xensource drivers, the
>
> speed
>
> > was about 78 MB/s.
>
> My testing using iperf has shown that I can reach gigabit speeds on the
> network, so it is possible.
>
> > The Windows system was a XP SP2.
> > hdparm on the dom0 gives about 60MB/s.
> > The network test was an ftp transfer, just downloading a 500MB file,
> > without writing it to disk, writing to nul. The same in the dom0,
>
> writing
>
> > the file to /dev/null gave me 112MB/s.
>
> Please use iperf to test network speeds. It''s a bit more
comparable to
> the testing that myself and others have done.
>
> > So I am wondering, what are the expected speed gains for the gplpv
> > drivers?
> > Is the performance of the drivers bettter with different windows
>
> versions,
>
> > e.g. windows server 2003?
>
> XPsp2 with the firewall service enabled behaves badly when LSO is
> enabled.
> Windows 2003 sp1 has also performed badly in some testing I have done
> (not as badly as XPsp2 - something like 50% worse performance instead of
> 90% worse performance)
>
> My best results have been under Windows 2003 sp2. SP2 introduced some
> pretty hefty improvements to NDIS which is the windows network driver
> layer.
>
> The advantages I''d expect to see in performance are due to
increased
> throughput and lower CPU usage.
>
> Also, depending on the test tool you are using, the gplpv disk drivers
> may perform quite poorly. This happens if the tools gives the windows
> kernel buffers that are not aligned to a 512 byte boundary.
>
> Please run debugview from sysinternals.com while you are running your
> disk performance testing. It will periodically output some stats like:
>
> XenVbd     stat_interrupts = 2408914
> XenVbd     stat_interrupts_for_me = 314368
> XenVbd     stat_reads = 131507
> XenVbd     stat_writes = 208567
> XenVbd     stat_unaligned_le_4096 = 7
> XenVbd     stat_unaligned_le_8192 = 0
> XenVbd     stat_unaligned_le_16384 = 0
> XenVbd     stat_unaligned_le_32768 = 0
> XenVbd     stat_unaligned_le_65536 = 0
> XenVbd     stat_unaligned_gt_65536 = 0
> XenVbd     stat_no_shadows = 0
> XenVbd     stat_no_grants = 0
> XenVbd     stat_outstanding_requests = 1
>
> The things I''m interested in are that stat_unaligned_xxx figures.
The
> only unaligned requests I see during day to day operations are somewhere
> between 5 and 10 that occur very very early during boot
> (stat_unaligned_le_4096 = 7). However I have seen chkdsk, defrag, and at
> least one testing tool issue requests not aligned on 512 byte
> boundaries. When that happens, the gplpv drivers have to break the
> request into 4096 byte chunks and submit each chunk, one at a time, to
> blkback, which really slows things down.
I had that tool running, but it did not produced any output, I guess I used it 
in a wrong manner ;). 

kind regards
Sebastian
>
> James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > Please run debugview from sysinternals.com while you are running
your
> > disk performance testing. It will periodically output some stats
like:
> >
> > XenVbd     stat_interrupts = 2408914
> > XenVbd     stat_interrupts_for_me = 314368
> > XenVbd     stat_reads = 131507
> > XenVbd     stat_writes = 208567
> > XenVbd     stat_unaligned_le_4096 = 7
> > XenVbd     stat_unaligned_le_8192 = 0
> > XenVbd     stat_unaligned_le_16384 = 0
> > XenVbd     stat_unaligned_le_32768 = 0
> > XenVbd     stat_unaligned_le_65536 = 0
> > XenVbd     stat_unaligned_gt_65536 = 0
> > XenVbd     stat_no_shadows = 0
> > XenVbd     stat_no_grants = 0
> > XenVbd     stat_outstanding_requests = 1
> >
> > The things I''m interested in are that stat_unaligned_xxx
figures.
The
> > only unaligned requests I see during day to day operations are
somewhere
> > between 5 and 10 that occur very very early during boot
> > (stat_unaligned_le_4096 = 7). However I have seen chkdsk, defrag,
and at
> > least one testing tool issue requests not aligned on 512 byte
> > boundaries. When that happens, the gplpv drivers have to break the
> > request into 4096 byte chunks and submit each chunk, one at a time,
to
> > blkback, which really slows things down.
> I had that tool running, but it did not produced any output, I guess I
> used it
> in a wrong manner ;).
> 
You need to make sure that kernel debug message capture is on. In the
''Capture'' menu, make sure that ''Capture
Kernel'' is ticked.

Thanks

james

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Sebastian Reitenbach wrote:
>>  1.5MB/s for random write access is not lousy , if you perform random
>> write test in dom0 or on native non xen kernel
>> you''ll also get  1.5MB/s
>>     
> Hm, ok, but without any PV drivers, I got about 7MB/sec. at least for
writing
> to the disk.
>   
 This does not look realistic, especially since with random read you got 
only 1.6MB/sec. Maybe this is some QEMU caching issue. Also the graph 
shows 2  strange large time slots with zero write activity.

 Would you mind running tiobench benchmark 
http://tiobench.sourceforge.net/ inside dom0 with:
          tiotest -t 4 -f 1024 -r 5000 -b 4096 -d /test
where /test is a mountpoint of a newly created ext3 filesystem large 
enough to hold 1024*4 MB
or better yet:
         tiotest -t THREADS -f SIZE -r 5000 -b 4096 -d /test
where THREADS*SIZE > 2*(Available RAM)

 Thanks
 Alex
> Therefore I was wondering.
>
https://www.l00-bugdead-prods.de/XenBench/random_write_without_pv_driver.htm
>
>   

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users