Xen devel - Nov 2008 - theoretical network rx performance of Windows with PV drivers

I''m finding some odd things during development of the GPLPV and am
wondering if I''m just expecting too much of a HVM Windows DomU.

I''m using iperf for testing, and the most I can get on a 1.8GHz AMD1210
out of a Dom0->DomU network performance is about 500MBits, and
that''s
with Dom0 sending packets at close to 1GBit, with about 50% of packets
being lost. But it''s not consistent... things seem to stall at strange
times (some of that may be a driver or a windows problem - the time
between scheduling a Dpc and the Dpc being executed is up to 3 seconds
sometimes when this happens...)

How much overhead is introduced in the event channel -> HVM IRQ path, as
compared to the normal interdomain event channels? I think that the
delay there might be bringing me down, but maybe I''m looking in the
wrong place?

Thanks

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 18/11/08 12:02, "James Harper"
<james.harper@bendigoit.com.au> wrote:
> I''m finding some odd things during development of the GPLPV and am
> wondering if I''m just expecting too much of a HVM Windows DomU.
> 
> I''m using iperf for testing, and the most I can get on a 1.8GHz
AMD1210
> out of a Dom0->DomU network performance is about 500MBits, and
that''s
> with Dom0 sending packets at close to 1GBit, with about 50% of packets
> being lost. But it''s not consistent... things seem to stall at
strange
> times (some of that may be a driver or a windows problem - the time
> between scheduling a Dpc and the Dpc being executed is up to 3 seconds
> sometimes when this happens...)
> 
> How much overhead is introduced in the event channel -> HVM IRQ path, as
> compared to the normal interdomain event channels? I think that the
> delay there might be bringing me down, but maybe I''m looking in
the
> wrong place?
I don''t think evtchn->IRQ latency is particularly large. But also I
don''t
know what else might be causing your erratic behaviour.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> On 18/11/08 12:02, "James Harper"
<james.harper@bendigoit.com.au>
wrote:
> 
> > I''m finding some odd things during development of the GPLPV
and am
> > wondering if I''m just expecting too much of a HVM Windows
DomU.
> >
> > I''m using iperf for testing, and the most I can get on a
1.8GHz
AMD1210
> > out of a Dom0->DomU network performance is about 500MBits, and
that''s
> > with Dom0 sending packets at close to 1GBit, with about 50% of
packets
> > being lost. But it''s not consistent... things seem to stall
at
strange
> > times (some of that may be a driver or a windows problem - the time
> > between scheduling a Dpc and the Dpc being executed is up to 3
seconds
> > sometimes when this happens...)
> >
> > How much overhead is introduced in the event channel -> HVM IRQ
path, as
> > compared to the normal interdomain event channels? I think that the
> > delay there might be bringing me down, but maybe I''m looking
in the
> > wrong place?
> 
> I don''t think evtchn->IRQ latency is particularly large. But
also I
don''t
> know what else might be causing your erratic behaviour.
> 
That''s probably all I needed to know for now. I think it might actually
be Windows that''s the problem...

Thanks

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

A couple of questions:
* You''re comparing dom0->domU performance; what about domU to
off-box?
 i.e., if it goes overr th enetwork card?
* Have you compared the same test with a PV domU?

  -George

On Tue, Nov 18, 2008 at 12:50 PM, James Harper
<james.harper@bendigoit.com.au> wrote:
>> On 18/11/08 12:02, "James Harper"
<james.harper@bendigoit.com.au>
> wrote:
>>
>> > I''m finding some odd things during development of the
GPLPV and am
>> > wondering if I''m just expecting too much of a HVM Windows
DomU.
>> >
>> > I''m using iperf for testing, and the most I can get on a
1.8GHz
> AMD1210
>> > out of a Dom0->DomU network performance is about 500MBits, and
> that''s
>> > with Dom0 sending packets at close to 1GBit, with about 50% of
> packets
>> > being lost. But it''s not consistent... things seem to
stall at
> strange
>> > times (some of that may be a driver or a windows problem - the
time
>> > between scheduling a Dpc and the Dpc being executed is up to 3
> seconds
>> > sometimes when this happens...)
>> >
>> > How much overhead is introduced in the event channel -> HVM IRQ
> path, as
>> > compared to the normal interdomain event channels? I think that
the
>> > delay there might be bringing me down, but maybe I''m
looking in the
>> > wrong place?
>>
>> I don''t think evtchn->IRQ latency is particularly large.
But also I
> don''t
>> know what else might be causing your erratic behaviour.
>>
>
> That''s probably all I needed to know for now. I think it might
actually
> be Windows that''s the problem...
>
> Thanks
>
> James
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> >
> > I don''t think evtchn->IRQ latency is particularly large.
But also I
> > don''t know what else might be causing your erratic behaviour.
> >
> 
> That''s probably all I needed to know for now. I think it might
actually
> be Windows that''s the problem...
> 
*sigh*

I was queuing the next dpc (under windows, the dpc is the bit that get
scheduled by the isr and runs at a lower priority than the isr but
higher than normal user code) and then measuring the time until the next
dpc occurred. I also re-queue another dpc when too much work has been
done by the current instance to try and even out the load a bit - dpcs
are supposed to complete in microseconds. When there was a large delay
between the queuing and the executing of the dpc I assumed it was
windows delaying it for some reason, but it''s my code after the
re-queuing and the end of the dpc that is delaying things. There is
still a problem with Windows taking too long to do things, but it''s not
where I thought it was.

I guess I''m just too used to everything being Windows''s fault
:)

Thanks

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> I don''t think evtchn->IRQ latency is particularly large. But
also I
don''t
> know what else might be causing your erratic behaviour.
> 
I think I figured out what the problem was. I had dom0 with 1 vcpu, and
my testing DomU with 2 vcpu''s, all on a machine which only had 2
core''s.
I changed my DomU to 1 vcpu''s and now it behaves itself. Is this the
expected behaviour?

Thanks

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel