Xen users - Aug 2008 - GPL PV drivers for Windows 0.9.11-pre12

I''ve just uploaded 0.9.11-pre12 of the GPL PV drivers for Windows.

Since -pre10 (and -pre11) I''ve fixed a heap of crashes that were
plaguing xennet under load, and also rewritten the interrupt/event
distribution logic to improve performance.

Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
600Gbit/second RX, which is considerably better than I was getting
before.

Please download it and give it a go.
http://www.meadowcourt.org/downloads

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Aug 20, 2008 at 09:41:37PM +1000, James Harper
wrote:
> I''ve just uploaded 0.9.11-pre12 of the GPL PV drivers for Windows.
> 
> Since -pre10 (and -pre11) I''ve fixed a heap of crashes that were
> plaguing xennet under load, and also rewritten the interrupt/event
> distribution logic to improve performance.
> 
> Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
> 600Gbit/second RX, which is considerably better than I was getting
> before.
> 
600 Gbit/sec is pretty damn fast ;) 

But yeah.. hopefully people can run some iometer benchmarks again for this
driver version! Would be nice know how it behaves on other peoples boxes.

-- Pasi

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> > Under windows 2003 I can now get network speeds of 1-2Gbit/second TX
and
> > 600Gbit/second RX, which is considerably better than I was getting
> > before.
> >
> 
> 600 Gbit/sec is pretty damn fast ;)
Oops. Yes, my units were a bit wrong there. 600Mbit/sec :)
> But yeah.. hopefully people can run some iometer benchmarks again for this
> driver version! Would be nice know how it behaves on other peoples boxes.
I just ran some tests under XP and they were pitiful, although the tests
weren''t run on the same box. Something like 12Mbit/second TX and
50Mbit/second RX. XP doesn''t seem to do large send offloads but that
doesn''t explain enough of the difference...

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

howdy,

this drivers are compatible with xenenterprise?

Regards,
---
Eduardo Silvestre
nfsi telecom, lda.

eduardo.silvestre@nfsi.pt
Tel. (+351) 21 949 2300 - Fax (+351) 21 949 2301
http://www.nfsi.pt/

----- Original Message -----
From: "James Harper" <james.harper@bendigoit.com.au>
To: "Pasi Kärkkäinen" <pasik@iki.fi>
Cc: xen-devel@lists.xensource.com, xen-users@lists.xensource.com
Sent: Wednesday, 20 August, 2008 1:03:48 PM GMT +00:00 GMT Britain, Ireland,
Portugal
Subject: [Xen-users] RE: [Xen-devel] GPL PV drivers for Windows 0.9.11-pre12
> > Under windows 2003 I can now get network speeds of 1-2Gbit/second TX
and
> > 600Gbit/second RX, which is considerably better than I was getting
> > before.
> >
> 
> 600 Gbit/sec is pretty damn fast ;)
Oops. Yes, my units were a bit wrong there. 600Mbit/sec :)
> But yeah.. hopefully people can run some iometer benchmarks again for this
> driver version! Would be nice know how it behaves on other peoples boxes.
I just ran some tests under XP and they were pitiful, although the tests
weren''t run on the same box. Something like 12Mbit/second TX and
50Mbit/second RX. XP doesn''t seem to do large send offloads but that
doesn''t explain enough of the difference...

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 20 August 2008, James Harper wrote:
> I''ve just uploaded 0.9.11-pre12 of the GPL PV drivers for Windows.
>
> Since -pre10 (and -pre11) I''ve fixed a heap of crashes that were
> plaguing xennet under load, and also rewritten the interrupt/event
> distribution logic to improve performance.
>
> Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
> 600Gbit/second RX, which is considerably better than I was getting
> before.
I presume that you mean 600Mbit/second... (not Gbit/second)

Still impressive !

Geert
>
> Please download it and give it a go.
> http://www.meadowcourt.org/downloads
>
> James
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users


-- 
Kobalt W.I.T.
Web & Information Technology
Brusselsesteenweg 152
1850 Grimbergen

Tel  : +32 479 339 655
Email: info@kobaltwit.be

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> howdy,
> 
> this drivers are compatible with xenenterprise?
> 
Probably not. Xen Enterprise has it''s own PV drivers I believe.

If someone has access to both I''d like to see some performance
comparisons :)

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> I just ran some tests under XP and they were pitiful, although the tests
> weren''t run on the same box. Something like 12Mbit/second TX and
> 50Mbit/second RX. XP doesn''t seem to do large send offloads but
that
> doesn''t explain enough of the difference...
> 
A bit google-ing suggested stopping the ''Windows Firewall/Internet
Connection Sharing (ICS)'' service, which gets windows using the
''large send offload'' feature and puts the TX up to around
50Mbit/second, which is a bit of an improvement.

With just the qemu devices on that server I get around 28Mbit/second TX and
about 25Mbit/second RX.

James 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Win2003 32bit live migrations don''t work for me. They were mostly
working with 0.9.11-pre9 but the xm migrate command hangs with
0.9.11-pre12. The domU is paused on the destination machine and the
migrating-domU is still running on the source machine.

Live migrations succeed when I revert to non-GPLPV mode so I don''t
think my setup is the problem.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> Win2003 32bit live migrations don''t work for me. They were mostly
> working with 0.9.11-pre9 but the xm migrate command hangs with
> 0.9.11-pre12. The domU is paused on the destination machine and the
> migrating-domU is still running on the source machine.
> 
> Live migrations succeed when I revert to non-GPLPV mode so I don''t
> think my setup is the problem.
> 
Yes, I probably broke it when I fixed the bugs in the network code.
I''ll
look at it next.

Thanks

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wednesday 20 August 2008 04:41:37 James Harper wrote:
> Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
> 600Gbit/second RX, which is considerably better than I was getting
> before.
Is there a particular "toolkit" you are using to test, or are you just
doing
file transfer tests?

If the latter, how are you doing the setup for the tests?

-Alan

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

''iperf -w1m -P2'' is what i''m using. I''ll
elaborate when I can get to a pc

-----Original Message-----
From: "Alan Murrell"<lists@murrell.ca>
Sent: 25/08/08 2:27:37 PM
To:
"xen-users@lists.xensource.com"<xen-users@lists.xensource.com>
Subject: Re: [Xen-users] GPL PV drivers for Windows 0.9.11-pre12

On Wednesday 20 August 2008 04:41:37 James Harper wrote:
> Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
> 600Gbit/second RX, which is considerably better than I was getting
> before.
Is there a particular "toolkit" you are using to test, or are you just
doing
file transfer tests?

If the latter, how are you doing the setup for the tests?

-Alan

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi

After having problems with earlier versions of the gplpv drivers (not
receiving any network packets), I just tried the current -pre12 drivers.
Still the same problem:
- the xennet device does not receive any packets.

Therefore I rebooted to stay without gplpv drivers, as the speed  is mostly
fine for me.
However, now I have some strange effects in the device manager (booted
without /gplpv):
- The xen net device is shown but marked.
- There are two Xen block device Driver entries shown but marked.
- Xen stub device drivers (Console & Virtual frame Buffer) shown but marked.
By marked, I mean this yellow exclamation mark.

Is this normal?
I am fearing to loose my data partition because of the duplicate block
device drivers.

Regards,
  Steffen


-----Ursprüngliche Nachricht-----
Von: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von James Harper
Gesendet: Mittwoch, 20. August 2008 13:42
An: xen-users@lists.xensource.com; xen-devel@lists.xensource.com
Betreff: [Xen-users] GPL PV drivers for Windows 0.9.11-pre12

I''ve just uploaded 0.9.11-pre12 of the GPL PV drivers for Windows.

Since -pre10 (and -pre11) I''ve fixed a heap of crashes that were
plaguing xennet under load, and also rewritten the interrupt/event
distribution logic to improve performance.

Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
600Gbit/second RX, which is considerably better than I was getting
before.

Please download it and give it a go.
http://www.meadowcourt.org/downloads

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Steffen,
	This is normal.  The drivers are installed, but the PV devices
aren''t surfaced to Windows when you don''t run the /GPLPV
switch.  This is
the same behavior exhibited by Windows as when one installs a driver for a
non-pnp device that doesn''t exist (or when one removes a non-pnp device
that
has a driver installed).  Booting with /GPLPV surfaces the PV drivers and
hides the others.  Prior to recent versions, I believe the HVM devices
showed up that way when booted with /GPLPV as well (mine does this in
0.9.10), but regardless you should only get corruption if you have both
block devices showing up as functional (a situation might be possible where
one or both are marked because they not working properly but are being used,
but if you check the properties, you should see "this device could not be
started," or "this device is missing," and both of those messages
indicate
that the marked device isn''t being used). I believe that there have
been
situations where both devices were functional in the past, but that has only
happened on rare occasions with specific circumstances (and even then, only
when booting with /GPLPV to the best of my knowledge).
	Dustin

-----Original Message-----
From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Steffen Heil
Sent: Monday, August 25, 2008 08:23
To: ''James Harper''
Cc: xen-users@lists.xensource.com
Subject: AW: [Xen-users] GPL PV drivers for Windows 0.9.11-pre12

Hi

After having problems with earlier versions of the gplpv drivers (not
receiving any network packets), I just tried the current -pre12 drivers.
Still the same problem:
- the xennet device does not receive any packets.

Therefore I rebooted to stay without gplpv drivers, as the speed  is mostly
fine for me.
However, now I have some strange effects in the device manager (booted
without /gplpv):
- The xen net device is shown but marked.
- There are two Xen block device Driver entries shown but marked.
- Xen stub device drivers (Console & Virtual frame Buffer) shown but marked.
By marked, I mean this yellow exclamation mark.

Is this normal?
I am fearing to loose my data partition because of the duplicate block
device drivers.

Regards,
  Steffen


-----Ursprüngliche Nachricht-----
Von: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] Im Auftrag von James Harper
Gesendet: Mittwoch, 20. August 2008 13:42
An: xen-users@lists.xensource.com; xen-devel@lists.xensource.com
Betreff: [Xen-users] GPL PV drivers for Windows 0.9.11-pre12

I''ve just uploaded 0.9.11-pre12 of the GPL PV drivers for Windows.

Since -pre10 (and -pre11) I''ve fixed a heap of crashes that were
plaguing
xennet under load, and also rewritten the interrupt/event distribution logic
to improve performance.

Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
600Gbit/second RX, which is considerably better than I was getting before.

Please download it and give it a go.
http://www.meadowcourt.org/downloads

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Hi
> 
> After having problems with earlier versions of the gplpv drivers (not
> receiving any network packets), I just tried the current -pre12 drivers.
> Still the same problem:
> - the xennet device does not receive any packets.
I''m at a loss to explain this still. Once I get this version stabilised
(just migration working after the change to irq handling) I''ll start
looking
at bugs like yours which seem to affect a minority of configurations.
> 
> Therefore I rebooted to stay without gplpv drivers, as the speed  is
> mostly
> fine for me.
> However, now I have some strange effects in the device manager (booted
> without /gplpv):
> - The xen net device is shown but marked.
> - There are two Xen block device Driver entries shown but marked.
> - Xen stub device drivers (Console & Virtual frame Buffer) shown but
> marked.
> By marked, I mean this yellow exclamation mark.
> 
> Is this normal?
> I am fearing to loose my data partition because of the duplicate block
> device drivers.
Yes, this should be the case. I can hide them, but then it makes it really
hard to upgrade or change them when booted without /GPLPV, so I decided to
leave them there but disabled (they deliberately fail the START irp).

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> On Wednesday 20 August 2008 04:41:37 James Harper wrote:
> > Under windows 2003 I can now get network speeds of 1-2Gbit/second TX
and
> > 600Gbit/second RX, which is considerably better than I was getting
> > before.
> 
> Is there a particular "toolkit" you are using to test, or are you
just
> doing
> file transfer tests?
> 
> If the latter, how are you doing the setup for the tests?
> 
In Dom0:
iperf -s -w1M

in DomU:
iperf -c dom0ip -w1M -P2 -r

the -P2 says do two TCP streams concurrently, the -r says do a tx test
then an rx test.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 26, 2008 at 11:37:59AM +1000, James Harper
wrote:
> > 
> > On Wednesday 20 August 2008 04:41:37 James Harper wrote:
> > > Under windows 2003 I can now get network speeds of 1-2Gbit/second
TX
> and
> > > 600Gbit/second RX, which is considerably better than I was
getting
> > > before.
> > 
> > Is there a particular "toolkit" you are using to test, or
are you just
> > doing
> > file transfer tests?
> > 
> > If the latter, how are you doing the setup for the tests?
> > 
> 
> In Dom0:
> iperf -s -w1M
> 
> in DomU:
> iperf -c dom0ip -w1M -P2 -r
> 
> the -P2 says do two TCP streams concurrently, the -r says do a tx test
> then an rx test.
> 
Do you also run bidirectional tests?

-d, --dualtest
       Do a bidirectional test simultaneously

-- Pasi

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > In Dom0:
> > iperf -s -w1M
> >
> > in DomU:
> > iperf -c dom0ip -w1M -P2 -r
> >
> > the -P2 says do two TCP streams concurrently, the -r says do a tx test
> > then an rx test.
> >
> 
> Do you also run bidirectional tests?
> 
> -d, --dualtest
>        Do a bidirectional test simultaneously
> 
I have done when testing for stability. -r does the bidirection test
consecutively instead of simultaneously.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Harper schreef:
> I''ve just uploaded 0.9.11-pre12 of the GPL PV drivers for Windows.
>
> Since -pre10 (and -pre11) I''ve fixed a heap of crashes that were
> plaguing xennet under load, and also rewritten the interrupt/event
> distribution logic to improve performance.
>
> Under windows 2003 I can now get network speeds of 1-2Gbit/second TX and
> 600Gbit/second RX, which is considerably better than I was getting
> before.
>
> Please download it and give it a go.
> http://www.meadowcourt.org/downloads
>
> James
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>   
While these numbers seem to be true for the RX/TX tests between domU <-> 
dom0, they don''t apply to the domU <-> other physical node on
lan. The
RX is fine, on my 100mbit lan I get about 94 / 95mbit. But the TX is 
very poor: only 18-20mbit. This has been since all versions I''ve tested
before, but of course hoped this one would make the difference. I''ve 
compared the results with a Linux domU PV, which does perform as 
expected (94-95mbit for both RX/TX).

Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> While these numbers seem to be true for the RX/TX tests between domU
<->
> dom0, they don''t apply to the domU <-> other physical node
on lan. The
> RX is fine, on my 100mbit lan I get about 94 / 95mbit. But the TX is
> very poor: only 18-20mbit. This has been since all versions I''ve
tested
> before, but of course hoped this one would make the difference.
I''ve
> compared the results with a Linux domU PV, which does perform as
> expected (94-95mbit for both RX/TX).
> 
Thanks for the feedback. I wonder if Large Send Offload is causing
problems somewhere... Can you try turning off Large Send Offload and
report the results?

I am looking at why RX performance is about 1/3 the speed of TX, but am
surprised that TX is so poor when sending to a physical network...

Maybe wait about 20 minutes before testing as I''ll upload a -pre13, as
-pre12 would sometimes (always?) crash when changing those network
settings.

In the meantime, can you please send me the output of a xenstore-ls
/local/domain/<id>/device/vif? Mine looks like this:

0 = ""
 state = "4"
 backend-id = "0"
 backend = "/local/domain/0/backend/vif/94/0"
 tx-ring-ref = "16093"
 rx-ring-ref = "16092"
 event-channel = "10"
 feature-no-csum-offload = "0"
 feature-sg = "1"
 feature-gso-tcpv4 = "1"
 request-rx-copy = "1"
 feature-rx-notify = "1"

Note the ''backend ='' line above? Also send me the output of a
xenstore-ls of that, eg in my case ''xenstore-ls
/local/domain/0/backend/vif/94/0''

Thanks

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> Maybe wait about 20 minutes before testing as I''ll upload a
-pre13, as
> -pre12 would sometimes (always?) crash when changing those network
> settings.
> 
Okay that''s uploaded now. http://www.meadowcourt.org/downloads

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> While these numbers seem to be true for the RX/TX tests between domU
<->
> dom0, they don''t apply to the domU <-> other physical node
on lan. The
> RX is fine, on my 100mbit lan I get about 94 / 95mbit. But the TX is
> very poor: only 18-20mbit. This has been since all versions I''ve
tested
> before, but of course hoped this one would make the difference.
I''ve
> compared the results with a Linux domU PV, which does perform as
> expected (94-95mbit for both RX/TX).
> 
I''ve just done some testing myself over a 1GB network... here are the
results.

Dom0 = Xen 3.2.1 Dom0 with a Broadcom gigabit network adapter (not the
same machine as my previous Dom0<->DomU testing was done on - this
system is not as highly spec''d).
DomU = Windows DomU running windows 2003 x32 + 0.9.11-pre13 of the GPLPV
drivers
Phys = Windows 2003 physical host with a Broadcom gigabit network
adapter
A Linksys managed gigabit switch separates Dom0 and Phys.

Dom0->Phys = 900mbit/second
Dom0->DomU = 975mbit/second
Phys->Dom0 = 850mbit/second
Phys->DomU = 700mbit/second
DomU->Dom0 = 1700mbit/second
DomU->Phys = 800mbit/second

Additionally, at the time of testing the nightly backup was occurring
which would have been moving data from Phys to Dom0. A bacula file
daemon and Exchange agent runs on Phys and the bacula director and
storage daemons run on Dom0.

So I guess we need to find out what is going on with your setup... what
network adapter are you using in Dom0 and does it support large send and
checksum offload? Or maybe something has changed in -pre13? I doubt it
though.

Thanks

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Harper schreef:
>> While these numbers seem to be true for the RX/TX tests between domU
>>     
> <->
>   
>> dom0, they don''t apply to the domU <-> other physical
node on lan. The
>> RX is fine, on my 100mbit lan I get about 94 / 95mbit. But the TX is
>> very poor: only 18-20mbit. This has been since all versions
I''ve
>>     
> tested
>   
>> before, but of course hoped this one would make the difference.
I''ve
>> compared the results with a Linux domU PV, which does perform as
>> expected (94-95mbit for both RX/TX).
>>
>>     
>
> Thanks for the feedback. I wonder if Large Send Offload is causing
> problems somewhere... Can you try turning off Large Send Offload and
> report the results?
>
> I am looking at why RX performance is about 1/3 the speed of TX, but am
> surprised that TX is so poor when sending to a physical network...
>
> Maybe wait about 20 minutes before testing as I''ll upload a
-pre13, as
> -pre12 would sometimes (always?) crash when changing those network
> settings.
>
> In the meantime, can you please send me the output of a xenstore-ls
> /local/domain/<id>/device/vif? Mine looks like this:
>
> 0 = ""
>  state = "4"
>  backend-id = "0"
>  backend = "/local/domain/0/backend/vif/94/0"
>  tx-ring-ref = "16093"
>  rx-ring-ref = "16092"
>  event-channel = "10"
>  feature-no-csum-offload = "0"
>  feature-sg = "1"
>  feature-gso-tcpv4 = "1"
>  request-rx-copy = "1"
>  feature-rx-notify = "1"
>
> Note the ''backend ='' line above? Also send me the output
of a
> xenstore-ls of that, eg in my case ''xenstore-ls
> /local/domain/0/backend/vif/94/0''
>
> Thanks
>
> James
>
>   
With Large Send Offload disabled, it indeed utilizes it''s max
bandwidth!
Both RX/TX are about 94mbit when testing at 100mbit lan, so that''s a 
perfect result! I first updated to pre13 before I tried this, so can''t 
tell if it had to do with your last version. However, I know I did test 
it with large send offload disabled before, but not with one of the 
latest pre''s. Do you still need the output requested?

Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> >
> With Large Send Offload disabled, it indeed utilizes it''s max
bandwidth!
> Both RX/TX are about 94mbit when testing at 100mbit lan, so that''s
a
> perfect result! I first updated to pre13 before I tried this, so
can''t
> tell if it had to do with your last version. However, I know I did
test
> it with large send offload disabled before, but not with one of the
> latest pre''s. Do you still need the output requested?
> 
If you could please re-test with -pre13 and Large Send enabled again it
would be useful. If that goes at full speed then I don''t really need
the
info from xenstore-ls, but if it goes slowly again I would appreciate
the xenstore-ls info.

It may be that your netback is telling the drivers that large send is
unsupported but that I''m activating it anyway, which may well result in
a slowdown.

Thanks

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Harper schreef:
> If you could please re-test with -pre13 and Large Send enabled again it
> would be useful. If that goes at full speed then I don''t really
need the
> info from xenstore-ls, but if it goes slowly again I would appreciate
> the xenstore-ls info.
>
> It may be that your netback is telling the drivers that large send is
> unsupported but that I''m activating it anyway, which may well
result in
> a slowdown.
>
> Thanks
>
> James
>   
Ah well, I tested both Large Send enabled and disabled on pre13, but 
didn''t disable it on pre12. So the answer would be that there is indeed
a problem with the Large Send Offload on my system. Iirc I''m using the 
same system as yours, a ML110 with Broadcom NIC on-board. Based on your 
test results, this one should work well.

Strange thing is when I do a xenstore-ls, I get this message back:
/usr/sbin/xenstore-ls: line 6: /usr/lib/xen-3.2-1/bin/xenstore-ls: No 
such file or directory
/usr/sbin/xenstore-ls: line 6: exec: /usr/lib/xen-3.2-1/bin/xenstore-ls: 
cannot execute: No such file or directory

Looking in /usr/lib/xen-3.2-1/bin, I find many files, including seven 
beginning with xenstore-... But no xenstore-ls. How could that be? I''m 
using Xen 3.2.1 on Debian Etch 4.0 with kernel  2.6.18-6-xen-686. This 
is the Etch Xen stock kernel, maybe it needs to be upgraded? What kernel 
are you running?

I just tried ''ethtool -k peth0'', which says the following:
"Cannot get
device udp large send offload settings: Operation not supported". So I 
suppose that confirms your assumption about the unsupported large send 
offload, but yours, thus the same card, supports it?

Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Ah well, I tested both Large Send enabled and disabled on pre13, but
> didn''t disable it on pre12. So the answer would be that there is
indeed
> a problem with the Large Send Offload on my system. Iirc I''m using
the
> same system as yours, a ML110 with Broadcom NIC on-board. Based on
your
> test results, this one should work well.
Well my system is an ML115 but probably otherwise identical. I haven''t
disabled any large send options at all anywhere to get
my
>500mbit/second results on the 1Gbit/second network...
> 
> Strange thing is when I do a xenstore-ls, I get this message back:
> /usr/sbin/xenstore-ls: line 6: /usr/lib/xen-3.2-1/bin/xenstore-ls: No
> such file or directory
> /usr/sbin/xenstore-ls: line 6: exec:
/usr/lib/xen-3.2-1/bin/xenstore-ls:
> cannot execute: No such file or directory
> 
> Looking in /usr/lib/xen-3.2-1/bin, I find many files, including seven
> beginning with xenstore-... But no xenstore-ls. How could that be?
Maybe you need to install ''xenstore-utils''?
# dpkg -S xenstore-ls
xenstore-utils: /usr/bin/xenstore-ls
xen-utils-3.1-1: /usr/lib/xen-3.1-1/bin/xenstore-ls
xen-utils-unstable: /usr/lib/xen-unstable/bin/xenstore-ls
> I''m
> using Xen 3.2.1 on Debian Etch 4.0 with kernel  2.6.18-6-xen-686. This
> is the Etch Xen stock kernel, maybe it needs to be upgraded? What
kernel
> are you running?
I''m using Xen 3.2.something from etch-backports. My kernel is
2.6.18-xen-amd64 which I assume is built from the same source as yours.
> I just tried ''ethtool -k peth0'', which says the
following: "Cannot get
> device udp large send offload settings: Operation not supported". So I
> suppose that confirms your assumption about the unsupported large send
> offload, but yours, thus the same card, supports it?
That is only udp large send offload. The one you want to notice is ''tcp
segmentation offload'', which is ''on'' on my system.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 26, 2008 at 09:04:23PM +1000, James Harper
wrote:
> > While these numbers seem to be true for the RX/TX tests between domU
> <->
> > dom0, they don''t apply to the domU <-> other physical
node on lan. The
> > RX is fine, on my 100mbit lan I get about 94 / 95mbit. But the TX is
> > very poor: only 18-20mbit. This has been since all versions
I''ve
> tested
> > before, but of course hoped this one would make the difference.
I''ve
> > compared the results with a Linux domU PV, which does perform as
> > expected (94-95mbit for both RX/TX).
> > 
> 
> Thanks for the feedback. I wonder if Large Send Offload is causing
> problems somewhere... Can you try turning off Large Send Offload and
> report the results?
> 
> I am looking at why RX performance is about 1/3 the speed of TX, but am
> surprised that TX is so poor when sending to a physical network...
> 
> Maybe wait about 20 minutes before testing as I''ll upload a
-pre13, as
> -pre12 would sometimes (always?) crash when changing those network
> settings.
> 
That reminded me of jumbo frames support.. do you know if anyone has tested
changing the MTU on Windows and actually using jumbo frames with gplpv drivers? 

Discussion about jumbo frames earlier:
http://lists.xensource.com/archives/html/xen-devel/2008-06/msg00519.html

-- Pasi

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> That reminded me of jumbo frames support.. do you know if anyone has
> tested
> changing the MTU on Windows and actually using jumbo frames with gplpv
> drivers?
> 
> Discussion about jumbo frames earlier:
> http://lists.xensource.com/archives/html/xen-devel/2008-06/msg00519.html
> 
I''m pretty sure it won''t work under windows if you set the MTU
over PAGE_SIZE (4096). I may have made some assumptions that a packet will
always fit in a single page.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 26, 2008 at 11:20:27PM +1000, James Harper
wrote:
> > That reminded me of jumbo frames support.. do you know if anyone has
> > tested
> > changing the MTU on Windows and actually using jumbo frames with gplpv
> > drivers?
> > 
> > Discussion about jumbo frames earlier:
> >
http://lists.xensource.com/archives/html/xen-devel/2008-06/msg00519.html
> > 
> 
> I''m pretty sure it won''t work under windows if you set
the MTU over PAGE_SIZE (4096).
> I may have made some assumptions that a packet will always fit in a single
page.
> 
Ok. that''s good to know before actually trying it.. :)

-- Pasi

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Same behavior in 0.9.11-pre13.

Keith

On Thu, Aug 21, 2008 at 8:59 PM, James Harper
<james.harper@bendigoit.com.au> wrote:
>>
>> Win2003 32bit live migrations don''t work for me. They were
mostly
>> working with 0.9.11-pre9 but the xm migrate command hangs with
>> 0.9.11-pre12. The domU is paused on the destination machine and the
>> migrating-domU is still running on the source machine.
>>
>> Live migrations succeed when I revert to non-GPLPV mode so I
don''t
>> think my setup is the problem.
>>
>
> Yes, I probably broke it when I fixed the bugs in the network code.
I''ll
> look at it next.
>
> Thanks
>
> James
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Harper schreef:
>
> Well my system is an ML115 but probably otherwise identical. I
haven''t
> disabled any large send options at all anywhere to get my
>   
>> 500mbit/second results on the 1Gbit/second network...
>>     
Oops, I meant the ML115, not 110. My mistake, so we do have identical 
systems..
> Maybe you need to install ''xenstore-utils''?
> # dpkg -S xenstore-ls
> xenstore-utils: /usr/bin/xenstore-ls
> xen-utils-3.1-1: /usr/lib/xen-3.1-1/bin/xenstore-ls
> xen-utils-unstable: /usr/lib/xen-unstable/bin/xenstore-ls
>
>   
You are correct. But that removed xen-utils, and now I no longer have 
''xm'' and ''xentop''? Re-installing xen-utils
will, of course, remove
xenstore-utils. Anyway, I pasted the output of xenstore-ls
below.
>
> I''m using Xen 3.2.something from etch-backports. My kernel is
> 2.6.18-xen-amd64 which I assume is built from the same source as yours.
>   
Mine is also from etch-backports
> That is only udp large send offload. The one you want to notice is
''tcp
> segmentation offload'', which is ''on'' on my
system.
>   
My mistake again, tso is off and can''t be enabled manually with
ethtool.

Here is the output of xenstore-ls:
0 = ""
 backend-id = "0"
 mac = "00:04:fc:40:e4:22"
 handle = "0"
 state = "4"
 backend = "/local/domain/0/backend/vif/12/0"
 tx-ring-ref = "15992"
 rx-ring-ref = "15960"
 event-channel = "7"
 feature-no-csum-offload = "0"
 feature-sg = "1"
 feature-gso-tcpv4 = "0"
 request-rx-copy = "1"
 feature-rx-notify = "1"

Any idea why my tso is disabled and yours not? Btw, I''m happy with the 
results so far, so if it''s too much trouble to get this working, we can
leave this one alone. I guess it''s my problem the system / kernel 
doesn''t support tso, it''s just strange that we both test on
the same
system with different results.

Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> On Thu, Aug 21, 2008 at 8:59 PM, James Harper
> <james.harper@bendigoit.com.au> wrote:
> >>
> >> Win2003 32bit live migrations don''t work for me. They
were mostly
> >> working with 0.9.11-pre9 but the xm migrate command hangs with
> >> 0.9.11-pre12. The domU is paused on the destination machine and
the
> >> migrating-domU is still running on the source machine.
> >>
> >> Live migrations succeed when I revert to non-GPLPV mode so I
don''t
> >> think my setup is the problem.
> >>
> >
> > Yes, I probably broke it when I fixed the bugs in the network code.
I''ll
> > look at it next.
> >
> 
> Same behavior in 0.9.11-pre13.
Hmmm... as I don''t have the hardware to test live migrations the best I
can do to approximate it is to do a save then a restore, which I have
tested pretty thoroughly on a 4 way SMP system.

When you say ''live migration'', is this with ''xm
migrate -l''? Did you try
it without the -l? the non-live migration is probably closer to the
save+restore that I''ve been testing.

If you can run the debugger on the source machine you might be able to
spot something, but there is some hang detection code in there that
fails when running under the debugger - for some reason when the
debugger is attached and code is executing at HIGH_LEVEL, it slows to a
crawl. If you are willing to test it I could send you a version of
xenpci.sys with that code removed.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> > That is only udp large send offload. The one you want to notice is
''tcp
> > segmentation offload'', which is ''on'' on my
system.
> >
> My mistake again, tso is off and can''t be enabled manually with
ethtool.

No idea. What does lspci -v tell you? Mine says:

08:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
Gigabit Ethernet PCI Express (rev 21)
        Subsystem: Hewlett-Packard Company Device 7032
        Flags: bus master, fast devsel, latency 0, IRQ 18
        Memory at fdff0000 (64-bit, non-prefetchable) [size=64K]
        Capabilities: [48] Power Management version 2
        Capabilities: [50] Vital Product Data <?>
        Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+
Queue=0/3 Enable-
        Capabilities: [d0] Express Endpoint, MSI 00
        Kernel driver in use: tg3
        Kernel modules: tg3, ipmi_si
> 
> Here is the output of xenstore-ls:
> 0 = ""
>  backend-id = "0"
>  mac = "00:04:fc:40:e4:22"
>  handle = "0"
>  state = "4"
>  backend = "/local/domain/0/backend/vif/12/0"
>  tx-ring-ref = "15992"
>  rx-ring-ref = "15960"
>  event-channel = "7"
>  feature-no-csum-offload = "0"
>  feature-sg = "1"
>  feature-gso-tcpv4 = "0"
>  request-rx-copy = "1"
>  feature-rx-notify = "1"
> 
> Any idea why my tso is disabled and yours not? Btw, I''m happy with
the
> results so far, so if it''s too much trouble to get this working,
we
can
> leave this one alone. I guess it''s my problem the system / kernel
> doesn''t support tso, it''s just strange that we both test
on the same
> system with different results.
Can you also do the xenstore-ls on the backend? Eg from your line above:
backend = "/local/domain/0/backend/vif/12/0"
do ''xenstore-ls /local/domain/0/backend/vif/12/0''

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 26, 2008 at 11:53:33PM +0200, Joost van den Broek
wrote:
> James Harper schreef:
> >
> >Well my system is an ML115 but probably otherwise identical. I
haven''t
> >disabled any large send options at all anywhere to get my
> >  
> >>500mbit/second results on the 1Gbit/second network...
> >>    
> Oops, I meant the ML115, not 110. My mistake, so we do have identical 
> systems..
> >Maybe you need to install ''xenstore-utils''?
> ># dpkg -S xenstore-ls
> >xenstore-utils: /usr/bin/xenstore-ls
> >xen-utils-3.1-1: /usr/lib/xen-3.1-1/bin/xenstore-ls
> >xen-utils-unstable: /usr/lib/xen-unstable/bin/xenstore-ls
> >
> >  
> You are correct. But that removed xen-utils, and now I no longer have 
> ''xm'' and ''xentop''? Re-installing
xen-utils will, of course, remove
> xenstore-utils. Anyway, I pasted the output of xenstore-ls below.
> >
> >I''m using Xen 3.2.something from etch-backports. My kernel is
> >2.6.18-xen-amd64 which I assume is built from the same source as yours.
> >  
> Mine is also from etch-backports
> >That is only udp large send offload. The one you want to notice is
''tcp
> >segmentation offload'', which is ''on'' on my
system.
> >  
> My mistake again, tso is off and can''t be enabled manually with
ethtool.
> 
> Here is the output of xenstore-ls:
> 0 = ""
> backend-id = "0"
> mac = "00:04:fc:40:e4:22"
> handle = "0"
> state = "4"
> backend = "/local/domain/0/backend/vif/12/0"
> tx-ring-ref = "15992"
> rx-ring-ref = "15960"
> event-channel = "7"
> feature-no-csum-offload = "0"
> feature-sg = "1"
> feature-gso-tcpv4 = "0"
> request-rx-copy = "1"
> feature-rx-notify = "1"
> 
> Any idea why my tso is disabled and yours not? 
Try running "uname -a", "xm info" and "ethtool -i
peth0" and paste them
here.. Just for comparison.

-- Pasi

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Wed, Aug 27, 2008 at 12:24 AM, James Harper
<james.harper@bendigoit.com.au> wrote:
>> On Thu, Aug 21, 2008 at 8:59 PM, James Harper
>> <james.harper@bendigoit.com.au> wrote:
>> >>
>> >> Win2003 32bit live migrations don''t work for me. They
were mostly
>> >> working with 0.9.11-pre9 but the xm migrate command hangs with
>> >> 0.9.11-pre12. The domU is paused on the destination machine
and the
>> >> migrating-domU is still running on the source machine.
>> >>
>> >> Live migrations succeed when I revert to non-GPLPV mode so I
don''t
>> >> think my setup is the problem.
>> >>
>> >
>> > Yes, I probably broke it when I fixed the bugs in the network
code.
> I''ll
>> > look at it next.
>> >
>>
>> Same behavior in 0.9.11-pre13.
>
> Hmmm... as I don''t have the hardware to test live migrations the
best I
> can do to approximate it is to do a save then a restore, which I have
> tested pretty thoroughly on a 4 way SMP system.
I just tried a regular save and was not successful. The domU changes
to migrating-domU but continues to run. The save file only grows to
1473 bytes. Here''s the output of xend.log:
[2008-08-27 05:59:26 19958] INFO (XendDomain:1165) Domain winserver
(24) unpaused.
[2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:89) [xc_save]:
/usr/lib64/xen/bin/xc_save 5 24 0 0 4
[2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:336) suspend
[2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:92) In
saveInputHandler suspend
[2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:94) Suspending 24 ...
[2008-08-27 06:01:17 19958] DEBUG (XendDomainInfo:467)
XendDomainInfo.shutdown(suspend)
[2008-08-27 06:01:17 19958] DEBUG (XendDomainInfo:1111)
XendDomainInfo.handleShutdownWatch
>
> When you say ''live migration'', is this with ''xm
migrate -l''? Did you try
> it without the -l? the non-live migration is probably closer to the
> save+restore that I''ve been testing.
Yes, I''ve been trying "--live" migration. Here''s the
output of xend.log:
[2008-08-27 06:33:16 19958] DEBUG (DevController:150) Waiting for devices vtpm.
[2008-08-27 06:33:16 19958] INFO (XendDomain:1165) Domain winserver
(30) unpaused.
[2008-08-27 06:34:13 19958] DEBUG (XendCheckpoint:89) [xc_save]:
/usr/lib64/xen/bin/xc_save 5 30 0 0 5
[2008-08-27 06:34:24 19958] INFO (XendCheckpoint:365) Saving memory
pages: iter 1  95%^M 1: sent 982159, skipped 849, delta 10959ms, dom0
84%, target 83%, sent 2936Mb/s, dirtied 3Mb/s 1041 pages
[2008-08-27 06:34:24 19958] INFO (XendCheckpoint:365) Saving memory
pages: iter 2   0%^M 2: sent 1006, skipped 35, delta 78ms, dom0 93%,
target 51%, sent 422Mb/s, dirtied 52Mb/s 124 pages
[2008-08-27 06:34:24 19958] INFO (XendCheckpoint:365) Saving memory
pages: iter 3   0%^M 3: sent 112, skipped 11, delta 32ms, dom0 31%,
target 71%, sent 114Mb/s, dirtied 17Mb/s 17 pages
[2008-08-27 06:34:25 19958] INFO (XendCheckpoint:365) Saving memory
pages: iter 4   0%^M 4: sent 15, skipped 2, Start last iteration
[2008-08-27 06:34:25 19958] DEBUG (XendCheckpoint:336) suspend
[2008-08-27 06:34:25 19958] DEBUG (XendCheckpoint:92) In
saveInputHandler suspend
[2008-08-27 06:34:25 19958] DEBUG (XendCheckpoint:94) Suspending 30 ...
[2008-08-27 06:34:25 19958] DEBUG (XendDomainInfo:467)
XendDomainInfo.shutdown(suspend)
[2008-08-27 06:34:25 19958] DEBUG (XendDomainInfo:1111)
XendDomainInfo.handleShutdownWatch


Non-live migration also failed. The domU continues to run and xend.log shows:
[2008-08-27 06:20:56 19958] DEBUG (DevController:150) Waiting for devices vtpm.
[2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:89) [xc_save]:
/usr/lib64/xen/bin/xc_save 5 27 0 0 4
[2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:336) suspend
[2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:92) In
saveInputHandler suspend
[2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:94) Suspending 27 ...
[2008-08-27 06:22:12 19958] DEBUG (XendDomainInfo:467)
XendDomainInfo.shutdown(suspend)
[2008-08-27 06:22:12 19958] DEBUG (XendDomainInfo:1111)
XendDomainInfo.handleShutdownWatch


When I reboot the domU into non-gplpv mode things work properly.
>
> If you can run the debugger on the source machine you might be able to
> spot something, but there is some hang detection code in there that
> fails when running under the debugger - for some reason when the
> debugger is attached and code is executing at HIGH_LEVEL, it slows to a
> crawl. If you are willing to test it I could send you a version of
> xenpci.sys with that code removed.
Sure.
>
> James
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Harper schreef:
> No idea. What does lspci -v tell you? Mine says:
>
> 08:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
> Gigabit Ethernet PCI Express (rev 21)
>         Subsystem: Hewlett-Packard Company Device 7032
>         Flags: bus master, fast devsel, latency 0, IRQ 18
>         Memory at fdff0000 (64-bit, non-prefetchable) [size=64K]
>         Capabilities: [48] Power Management version 2
>         Capabilities: [50] Vital Product Data <?>
>         Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+
> Queue=0/3 Enable-
>         Capabilities: [d0] Express Endpoint, MSI 00
>         Kernel driver in use: tg3
>         Kernel modules: tg3, ipmi_si
>
>   
06:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721 
Gigabit Ethernet PCI Express (rev 21)
        Subsystem: Hewlett-Packard Company NC320i PCIe Gigabit Server 
Adapter
        Flags: bus master, fast devsel, latency 0, IRQ 17
        Memory at fdff0000 (64-bit, non-prefetchable) [size=64K]
        Capabilities: [48] Power Management version 2
        Capabilities: [50] Vital Product Data
        Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+ 
Queue=0/3 Enable-
        Capabilities: [d0] Express Endpoint IRQ 0

That''s all, no more lines like yours..? The system (Debian) is
up-to-date.
>
> Can you also do the xenstore-ls on the backend? Eg from your line above:
> backend = "/local/domain/0/backend/vif/12/0"
> do ''xenstore-ls /local/domain/0/backend/vif/12/0''
>
> James
>   
bridge = "xenbr0"
domain = "win2003-2"
handle = "0"
uuid = "6a71e350-ed24-f7e6-0ff1-02395dcd62f8"
script = "/etc/xen/scripts/vif-bridge"
state = "4"
frontend = "/local/domain/12/device/vif/0"
mac = "00:04:fc:40:e4:22"
online = "1"
frontend-id = "12"
type = "netfront"
feature-sg = "1"
feature-gso-tcpv4 = "1"
feature-rx-copy = "1"
hotplug-status = "connected"

Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> James Harper schreef:
> > No idea. What does lspci -v tell you? Mine says:
> >
> > 08:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
> > Gigabit Ethernet PCI Express (rev 21)
> >         Subsystem: Hewlett-Packard Company Device 7032
> >         Flags: bus master, fast devsel, latency 0, IRQ 18
> >         Memory at fdff0000 (64-bit, non-prefetchable) [size=64K]
> >         Capabilities: [48] Power Management version 2
> >         Capabilities: [50] Vital Product Data <?>
> >         Capabilities: [58] Message Signalled Interrupts: Mask-
64bit+
> > Queue=0/3 Enable-
> >         Capabilities: [d0] Express Endpoint, MSI 00
> >         Kernel driver in use: tg3
> >         Kernel modules: tg3, ipmi_si
> >
> >
> 06:00.0 Ethernet controller: Broadcom Corporation NetXtreme BCM5721
> Gigabit Ethernet PCI Express (rev 21)
>         Subsystem: Hewlett-Packard Company NC320i PCIe Gigabit Server
> Adapter
>         Flags: bus master, fast devsel, latency 0, IRQ 17
>         Memory at fdff0000 (64-bit, non-prefetchable) [size=64K]
>         Capabilities: [48] Power Management version 2
>         Capabilities: [50] Vital Product Data
>         Capabilities: [58] Message Signalled Interrupts: Mask- 64bit+
> Queue=0/3 Enable-
>         Capabilities: [d0] Express Endpoint IRQ 0
> 
> That''s all, no more lines like yours..? The system (Debian) is
up-to-date.

Hmmm... I am more-or-less running Lenny now, not Etch as I may have
previously mentioned. Maybe I have a later version of lspci than you? It
will still be the same kernel though.

What does ''uname -a'' tell you?

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James Harper schreef:
> Hmmm... I am more-or-less running Lenny now, not Etch as I may have
> previously mentioned. Maybe I have a later version of lspci than you? It
> will still be the same kernel though.
>
> What does ''uname -a'' tell you?
>   
Ah sorry, didn''t notice, that explains things yes.

# Linux xenserver 2.6.18-6-xen-686 #1 SMP Wed Jun 18 01:17:40 UTC 2008 
i686 GNU/Linux

Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Good news! I''ve reinstalled windows 2003 ent. 32bit, .net2 and gplpv
0.9.11-pre13 and live migration is working. I''ve migrated 20 times
without issue. Save/restore is also working. ''xm shutdown
domU'' still
results in a "system recovered from unexpected shutdown" on the next
power up even though it appears to be shutting down properly.

I''m confused about the errors I was seeing last night. The only
difference is that I had installed an earlier version of GPLPV
(~0.9.9) then upgraded each time a new release came out. Live
migrations often succeeded in earlier 0.9.11-pre releases so I didn''t
think there was a problem with upgrading but now I''m not so sure.

Keith

On Wed, Aug 27, 2008 at 2:41 AM, keith coleman <keith@scaltro.com>
wrote:
> On Wed, Aug 27, 2008 at 12:24 AM, James Harper
> <james.harper@bendigoit.com.au> wrote:
>>> On Thu, Aug 21, 2008 at 8:59 PM, James Harper
>>> <james.harper@bendigoit.com.au> wrote:
>>> >>
>>> >> Win2003 32bit live migrations don''t work for me.
They were mostly
>>> >> working with 0.9.11-pre9 but the xm migrate command hangs
with
>>> >> 0.9.11-pre12. The domU is paused on the destination
machine and the
>>> >> migrating-domU is still running on the source machine.
>>> >>
>>> >> Live migrations succeed when I revert to non-GPLPV mode so
I don''t
>>> >> think my setup is the problem.
>>> >>
>>> >
>>> > Yes, I probably broke it when I fixed the bugs in the network
code.
>> I''ll
>>> > look at it next.
>>> >
>>>
>>> Same behavior in 0.9.11-pre13.
>>
>> Hmmm... as I don''t have the hardware to test live migrations
the best I
>> can do to approximate it is to do a save then a restore, which I have
>> tested pretty thoroughly on a 4 way SMP system.
>
> I just tried a regular save and was not successful. The domU changes
> to migrating-domU but continues to run. The save file only grows to
> 1473 bytes. Here''s the output of xend.log:
> [2008-08-27 05:59:26 19958] INFO (XendDomain:1165) Domain winserver
> (24) unpaused.
> [2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:89) [xc_save]:
> /usr/lib64/xen/bin/xc_save 5 24 0 0 4
> [2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:336) suspend
> [2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:92) In
> saveInputHandler suspend
> [2008-08-27 06:01:17 19958] DEBUG (XendCheckpoint:94) Suspending 24 ...
> [2008-08-27 06:01:17 19958] DEBUG (XendDomainInfo:467)
> XendDomainInfo.shutdown(suspend)
> [2008-08-27 06:01:17 19958] DEBUG (XendDomainInfo:1111)
> XendDomainInfo.handleShutdownWatch
>
>>
>> When you say ''live migration'', is this with
''xm migrate -l''? Did you try
>> it without the -l? the non-live migration is probably closer to the
>> save+restore that I''ve been testing.
>
> Yes, I''ve been trying "--live" migration.
Here''s the output of xend.log:
> [2008-08-27 06:33:16 19958] DEBUG (DevController:150) Waiting for devices
vtpm.
> [2008-08-27 06:33:16 19958] INFO (XendDomain:1165) Domain winserver
> (30) unpaused.
> [2008-08-27 06:34:13 19958] DEBUG (XendCheckpoint:89) [xc_save]:
> /usr/lib64/xen/bin/xc_save 5 30 0 0 5
> [2008-08-27 06:34:24 19958] INFO (XendCheckpoint:365) Saving memory
> pages: iter 1  95%^M 1: sent 982159, skipped 849, delta 10959ms, dom0
> 84%, target 83%, sent 2936Mb/s, dirtied 3Mb/s 1041 pages
> [2008-08-27 06:34:24 19958] INFO (XendCheckpoint:365) Saving memory
> pages: iter 2   0%^M 2: sent 1006, skipped 35, delta 78ms, dom0 93%,
> target 51%, sent 422Mb/s, dirtied 52Mb/s 124 pages
> [2008-08-27 06:34:24 19958] INFO (XendCheckpoint:365) Saving memory
> pages: iter 3   0%^M 3: sent 112, skipped 11, delta 32ms, dom0 31%,
> target 71%, sent 114Mb/s, dirtied 17Mb/s 17 pages
> [2008-08-27 06:34:25 19958] INFO (XendCheckpoint:365) Saving memory
> pages: iter 4   0%^M 4: sent 15, skipped 2, Start last iteration
> [2008-08-27 06:34:25 19958] DEBUG (XendCheckpoint:336) suspend
> [2008-08-27 06:34:25 19958] DEBUG (XendCheckpoint:92) In
> saveInputHandler suspend
> [2008-08-27 06:34:25 19958] DEBUG (XendCheckpoint:94) Suspending 30 ...
> [2008-08-27 06:34:25 19958] DEBUG (XendDomainInfo:467)
> XendDomainInfo.shutdown(suspend)
> [2008-08-27 06:34:25 19958] DEBUG (XendDomainInfo:1111)
> XendDomainInfo.handleShutdownWatch
>
>
> Non-live migration also failed. The domU continues to run and xend.log
shows:
> [2008-08-27 06:20:56 19958] DEBUG (DevController:150) Waiting for devices
vtpm.
> [2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:89) [xc_save]:
> /usr/lib64/xen/bin/xc_save 5 27 0 0 4
> [2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:336) suspend
> [2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:92) In
> saveInputHandler suspend
> [2008-08-27 06:22:12 19958] DEBUG (XendCheckpoint:94) Suspending 27 ...
> [2008-08-27 06:22:12 19958] DEBUG (XendDomainInfo:467)
> XendDomainInfo.shutdown(suspend)
> [2008-08-27 06:22:12 19958] DEBUG (XendDomainInfo:1111)
> XendDomainInfo.handleShutdownWatch
>
>
> When I reboot the domU into non-gplpv mode things work properly.
>
>>
>> If you can run the debugger on the source machine you might be able to
>> spot something, but there is some hang detection code in there that
>> fails when running under the debugger - for some reason when the
>> debugger is attached and code is executing at HIGH_LEVEL, it slows to a
>> crawl. If you are willing to test it I could send you a version of
>> xenpci.sys with that code removed.
>
> Sure.
>
>>
>> James
>>
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> Good news! I''ve reinstalled windows 2003 ent. 32bit, .net2 and
gplpv
> 0.9.11-pre13 and live migration is working. I''ve migrated 20 times
> without issue. Save/restore is also working. ''xm shutdown
domU'' still
> results in a "system recovered from unexpected shutdown" on the
next
> power up even though it appears to be shutting down properly.
> 
> I''m confused about the errors I was seeing last night. The only
> difference is that I had installed an earlier version of GPLPV
> (~0.9.9) then upgraded each time a new release came out. Live
> migrations often succeeded in earlier 0.9.11-pre releases so I
didn''t
> think there was a problem with upgrading but now I''m not so sure.
> 
If there were any problems with the upgrade then I would have expected
to have them manifest themselves is more obvious ways than just
migration not working.

One thing I have noticed though is that windows will try and make sure
that a given device gets assigned to the same IRQ, so there could be
something different about the IRQ or the IRQL that was assigned...

I don''t suppose you have a copy of the previous install still?

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> After installation of that drivers in Windows XP localized version
> (polish)
> i cannot boot with /gplpv switch (bluescreen, etc, i will attach one
> tommorow and provide futher details on request). Are gplpv drivers ready
> to
> be used with localized versions of windows*?
> 
> Apart from that they are working very well in win2003!
> 
> Btw, when are you going to make support for XP x64 version? It''s
basically
> the same kernel as w2k3, but so far they aren''t working in x64 xp.
There is no build environment provided in the windows DDK specifically for XP
x64. I think using w2k3 is the correct thing to do. In what way isn''t
it working?

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I do not have a copy of the previous install.

I have done more testing of the freshly installed domU with
0.9.11-pre13 and it is working well!

Keith

On Wed, Aug 27, 2008 at 8:12 PM, James Harper
<james.harper@bendigoit.com.au> wrote:
>>
>> Good news! I''ve reinstalled windows 2003 ent. 32bit, .net2 and
gplpv
>> 0.9.11-pre13 and live migration is working. I''ve migrated 20
times
>> without issue. Save/restore is also working. ''xm shutdown
domU'' still
>> results in a "system recovered from unexpected shutdown" on
the next
>> power up even though it appears to be shutting down properly.
>>
>> I''m confused about the errors I was seeing last night. The
only
>> difference is that I had installed an earlier version of GPLPV
>> (~0.9.9) then upgraded each time a new release came out. Live
>> migrations often succeeded in earlier 0.9.11-pre releases so I
didn''t
>> think there was a problem with upgrading but now I''m not so
sure.
>>
>
> If there were any problems with the upgrade then I would have expected
> to have them manifest themselves is more obvious ways than just
> migration not working.
>
> One thing I have noticed though is that windows will try and make sure
> that a given device gets assigned to the same IRQ, so there could be
> something different about the IRQ or the IRQL that was assigned...
>
> I don''t suppose you have a copy of the previous install still?
>
> James
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I''ve installed GPL PV drivers in XP64 using 2k3x64 check of
0.9.11-pre13
and everything went fine (functionally)... Appart from that, the 
performance was actually worst than using the default qemu devices...

I''ve already sent a message with the config i used but, briefly: dom 0 
3x500 - raid5 - lvm partitions for VM disks. 40Mb/s in dom0, 15Mb/s in 
xp64 domU with qemu and 4Mb/s enabling /gplpv...

Any clue?

Thanks in advance,

Juan Pablo

Ps: the domU config i use:  
http://lists.xensource.com/archives/html/xen-users/2008-08/msg00828.html


James Harper escribió:
>> After installation of that drivers in Windows XP localized version
>> (polish)
>> i cannot boot with /gplpv switch (bluescreen, etc, i will attach one
>> tommorow and provide futher details on request). Are gplpv drivers
ready
>> to
>> be used with localized versions of windows*?
>>
>> Apart from that they are working very well in win2003!
>>
>> Btw, when are you going to make support for XP x64 version?
It''s basically
>> the same kernel as w2k3, but so far they aren''t working in x64
xp.
>>     
>
> There is no build environment provided in the windows DDK specifically for
XP x64. I think using w2k3 is the correct thing to do. In what way
isn''t it working?
>
> James
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>   


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> I''ve installed GPL PV drivers in XP64 using 2k3x64 check of
0.9.11-pre13
> and everything went fine (functionally)...
Thanks for the report.
> Appart from that, the
> performance was actually worst than using the default qemu devices...
> 
> I''ve already sent a message with the config i used but, briefly:
dom 0
> 3x500 - raid5 - lvm partitions for VM disks. 40Mb/s in dom0, 15Mb/s in
> xp64 domU with qemu and 4Mb/s enabling /gplpv...
> 
> Any clue?
> 
No, but there are enough of these reports around that I''m making it a
priority to look at next.

Can you tell me what tools you are using for testing?

I know that there is a performance related problem with buffer alignment,
although I''m not sure how it relates to real world performance, or if
it relates at all to the poor performance that people have been seeing while
benchmarking.

Basically, Linux always aligns block device request buffers to a 512 byte
(sector) boundary in memory, eg the address of the buffer will always be
divisible by 512. The Dom0 xen block device driver backend enforces this too, so
I have to comply.

Windows, on the other hand, doesn''t care, and only mostly aligns
request buffers to a 512 byte boundary, so I have to double buffer the requests
(eg copy to (for a write) or from (for a read) a secondary buffer that is
aligned to a 512 byte boundary. There are small overheads with this but the main
problem is that the secondary buffer is only small and so a large read or write
request gets broken down into several requests that must be executed serially,
eg part 2 of the request can only be submitted once part 1 has been completed.
Fortunately I have only seen Windows issue large numbers of non-aligned requests
during format, chkdsk (online and boot) and certain benchmarking tools.

A disk driver under Windows using the scsiport framework (eg xenvbd) executes at
a very high IRQ Level, and it is impossible to allocate memory at that IRQL so
I''m limited as to what buffers I can make use of.

Sorry you asked? :)

I really would like to know of a simple, easy, and fast tool (eg command line
based) to do testing under a Linux DomU and a Windows DomU. It doesn''t
have to be incredibly accurate, as long as I can make comparisons between the
two environments... I find iometer painful :(

Thanks

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

In the windows DomU i use a tool called DiskTT... Basically it creates a 
file of the size you configure and writes and reads it in the number of 
chunks you say.

For linux you could use DD exactly the same... for example dd 
if=/dev/zero of=testFile bs=1024 count=1000 (1000 blocks of 1Kb)... It 
will tell you the write speed...
To test the read speed you could use hdparm  -t|-T  <device> (use 
--direct to NOT use caches :)

Something i forgot to tell you... read speeds were quite good actually 
in the xp64 , although also a bit slower than with qemu...

Thanks for the good job, and your response! :)

Juan pablo

James Harper escribió:
>> I''ve installed GPL PV drivers in XP64 using 2k3x64 check of
0.9.11-pre13
>> and everything went fine (functionally)...
>>     
>
> Thanks for the report.
>
>   
>> Appart from that, the
>> performance was actually worst than using the default qemu devices...
>>
>> I''ve already sent a message with the config i used but,
briefly: dom 0
>> 3x500 - raid5 - lvm partitions for VM disks. 40Mb/s in dom0, 15Mb/s in
>> xp64 domU with qemu and 4Mb/s enabling /gplpv...
>>
>> Any clue?
>>
>>     
>
> No, but there are enough of these reports around that I''m making
it a priority to look at next.
>
> Can you tell me what tools you are using for testing?
>
> I know that there is a performance related problem with buffer alignment,
although I''m not sure how it relates to real world performance, or if
it relates at all to the poor performance that people have been seeing while
benchmarking.
>
> Basically, Linux always aligns block device request buffers to a 512 byte
(sector) boundary in memory, eg the address of the buffer will always be
divisible by 512. The Dom0 xen block device driver backend enforces this too, so
I have to comply.
>
> Windows, on the other hand, doesn''t care, and only mostly aligns
request buffers to a 512 byte boundary, so I have to double buffer the requests
(eg copy to (for a write) or from (for a read) a secondary buffer that is
aligned to a 512 byte boundary. There are small overheads with this but the main
problem is that the secondary buffer is only small and so a large read or write
request gets broken down into several requests that must be executed serially,
eg part 2 of the request can only be submitted once part 1 has been completed.
Fortunately I have only seen Windows issue large numbers of non-aligned requests
during format, chkdsk (online and boot) and certain benchmarking tools.
>
> A disk driver under Windows using the scsiport framework (eg xenvbd)
executes at a very high IRQ Level, and it is impossible to allocate memory at
that IRQL so I''m limited as to what buffers I can make use of.
>
> Sorry you asked? :)
>
> I really would like to know of a simple, easy, and fast tool (eg command
line based) to do testing under a Linux DomU and a Windows DomU. It
doesn''t have to be incredibly accurate, as long as I can make
comparisons between the two environments... I find iometer painful :(
>
> Thanks
>
> James
>
>   


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Tue, Aug 26, 2008 at 7:04 AM, James Harper
<james.harper@bendigoit.com.au> wrote:
>> While these numbers seem to be true for the RX/TX tests between domU
> <->
>> dom0, they don''t apply to the domU <-> other physical
node on lan. The
>> RX is fine, on my 100mbit lan I get about 94 / 95mbit. But the TX is
>> very poor: only 18-20mbit. This has been since all versions
I''ve
> tested
>> before, but of course hoped this one would make the difference.
I''ve
>> compared the results with a Linux domU PV, which does perform as
>> expected (94-95mbit for both RX/TX).
>>
>
> Thanks for the feedback. I wonder if Large Send Offload is causing
> problems somewhere... Can you try turning off Large Send Offload and
> report the results?
Hello,

I have 2 systems (same hardware) and in each one of its, I have 2
windows 2003 (all updates).
witch gplpv -pre10 the system work, but the performance of network is
very, very slow
with gplpv -pre13 same problem.
and disabling the "Large Send Offload" network work fine:

=================iperf tests
in server with this command: iperf -s
in client with this command: iperf -c IP_REMOTE_SERVER -d -l 1M -w 1M

VM1 - before change
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-10.1 sec   113 MBytes  93.9 Mbits/sec
[  4]  0.0-12.4 sec  2.00 MBytes  1.36 Mbits/sec
-----------------------------------------------------------
VM1 - after change
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[  4]  0.0-10.1 sec  58.0 MBytes  48.4 Mbits/sec
[  6]  0.0-10.1 sec   104 MBytes  86.6 Mbits/sec
------------------------------------------------------------

VM2 - before change
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[  6]  0.0-10.0 sec   101 MBytes  84.7 Mbits/sec
[  4]  0.0-10.1 sec   768 KBytes   624 Kbits/sec
------------------------------------------------------------
VM2 - after change
------------------------------------------------------------
[ ID] Interval       Transfer     Bandwidth
[  5]  0.0-10.0 sec  37.3 MBytes  31.2 Mbits/sec
[  6]  0.0-10.0 sec   111 MBytes  93.0 Mbits/sec
------------------------------------------------------------
=================
but in Dom0 I see this errors (look dropped packets) in the servers

Server1
=======peth2     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:31459415 errors:0 dropped:4058498 overruns:0 frame:0
          TX packets:10061928 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:1933464944 (1.8 GiB)  TX bytes:2618272070 (2.4 GiB)
          Memory:d8420000-d8440000
=======
Server2
=======peth2     Link encap:Ethernet  HWaddr FE:FF:FF:FF:FF:FF
          inet6 addr: fe80::fcff:ffff:feff:ffff/64 Scope:Link
          UP BROADCAST RUNNING NOARP  MTU:1500  Metric:1
          RX packets:28573933 errors:0 dropped:50760 overruns:0 frame:0
          TX packets:1347689 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:100
          RX bytes:4134924775 (3.8 GiB)  TX bytes:1336000345 (1.2 GiB)
          Base address:0x3080 Memory:d8540000-d8560000
=======
with Large Send Offload "active", no dropped packets for long time
(never??)
after the change.. the systems reports various dropped packets.

exists any problem (now or in future) with droppers packets ??
Is possible set Large Send Offload disabled for default ?? consequences ??

thanks

-- 
-- 
Victor Hugo dos Santos
Linux Counter #224399

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users