Xen users - Apr 2009 - Does I/O from Dom0 skip the PV backend drivers ?

I am trying to determine the amount of overhead (if any) that a PV guest
incurs as compared to running on bare metal.  I was thinking of running some
performance tests on both a bare metal system and then in a PV guest – but I
only have one machine.  I am trying to figure out if  booting from a xen to
non-xen kernel is needed in order to get real numbers.

So the question boils down to – When Dom0 performs I/O does it go directly
to the physical device driver or does it also route through the PV backend
drivers ?

Also, my assumption is that the xen hypervisor has no role in any I/O
operations so it being present or not should have almost no impact on I/O
performance.

Thanks for any info.

Joe

-- 
View this message in context:
http://www.nabble.com/Does-I-O-from-Dom0-skip-the-PV-backend-drivers---tp23302372p23302372.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,
>From what I''ve read at [1]:
When network packets are received by the NIC, the NIC will
raise an interrupt. While the NIC’s device driver in the driver do-
main may directly access the NIC, all interrupts in the system must
first go through the hypervisor. The hypervisor receives the inter-
rupt, determines that the NIC is owned by the driver domain, and
then generates a virtual interrupt for the NIC’s device driver in the
driver domain. The device driver receives the virtual interrupt, pro-
cesses the packets received by the NIC and sends them to the Eth-
ernet bridge. The Ethernet bridge demultiplexes the packet based
on its Ethernet address and delivers it to the appropriate netback
interface. The netback driver then sends the packet to the netfront
driver in the guest domain over an I/O channel. The netfront driver
then delivers the packet to the guest operating system as if it had
come directly from the NIC. The process is basically reversed to
send packets from the guest domains.

So if when they say "processes" at "The device driver receives
the virtual
interrupt, processes the packets received by the NIC" they mean
"receive if
the destination is Dom0" then yes, it does perform I/O directly - at least
on the receive path.

Actually, running the benchmark at Dom0 was how they "simulated" a
direct
I/O on [2] :-)

Please correct me if I''m mistaken. I hope it helps,
~Paolo

[1] http://www.hpl.hp.com/techreports/2009/HPL-2009-25.pdf
[2]
http://www.usenix.org/events/usenix08/tech/full_papers/santos/santos_html/

On Wed, Apr 29, 2009 at 5:45 PM, Joe Armstrong
<Joe.Armstrong@webex.com>wrote:
>
> I am trying to determine the amount of overhead (if any) that a PV guest
> incurs as compared to running on bare metal.  I was thinking of running
> some
> performance tests on both a bare metal system and then in a PV guest – but
> I
> only have one machine.  I am trying to figure out if  booting from a xen to
> non-xen kernel is needed in order to get real numbers.
>
> So the question boils down to – When Dom0 performs I/O does it go directly
> to the physical device driver or does it also route through the PV backend
> drivers ?
>
> Also, my assumption is that the xen hypervisor has no role in any I/O
> operations so it being present or not should have almost no impact on I/O
> performance.
>
> Thanks for any info.
>
> Joe
>
> --
> View this message in context:
>
http://www.nabble.com/Does-I-O-from-Dom0-skip-the-PV-backend-drivers---tp23302372p23302372.html
> Sent from the Xen - User mailing list archive at Nabble.com.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Apr 30, 2009 at 3:45 AM, Joe Armstrong <Joe.Armstrong@webex.com>
wrote:
> So the question boils down to – When Dom0 performs I/O does it go directly
> to the physical device driver
Should be. There is an exception though.
> or does it also route through the PV backend
> drivers ?
>
The exception is that if you deliberately use netfront/blkfront on
dom0. Examples :
- the old (3.0, also in RHEL) Xen bridged networking setup where
dom0''s eth0 is renamed to peth0, and veth0 is renamed to eth0. In this
case network traffic goes through netfront/netback drivers.
- mapping a storage to dom0 via "xm block-attach". For example,
running
xm block-attach 0 tap:aio:/var/lib/xen/root.img xvda
would create a block device called xvda on dom0, which goes through
blkfront/blkback drivers. Useful if you need to mount domU''s storage
on dom0.
> Also, my assumption is that the xen hypervisor has no role in any I/O
> operations so it being present or not should have almost no impact on I/O
> performance.
In current implementation, correct. This post from Mark WIlliamson
might be of interest to you
http://lists.xensource.com/archives/html/xen-users/2009-03/msg00018.html
Note that the hypervisor still have some impact on CPU performance, so
if your benchmark is cpu-bound you might get different result from
native Linux compared to xenified Linux.

Regards,

Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks for the info - since I am running CentOs (with xen 3.0.3) it sounds
like my best bet for real network numbers is to run on bare metal, but disk
I/O numbers from dom0 should be consistent with a bare metal machine.

Also, thanks for the hint on mounting domU''s disk...

Joe
-- 
View this message in context:
http://www.nabble.com/Does-I-O-from-Dom0-skip-the-PV-backend-drivers---tp23302372p23317269.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Apr 30, 2009 at 9:29 PM, Joe Armstrong <Joe.Armstrong@webex.com>
wrote:
> Thanks for the info - since I am running CentOs (with xen 3.0.3) it sounds
> like my best bet for real network numbers is to run on bare metal,
Actually it depends on what you''re trying to do.
Network PV drivers are good enough so (for example) if you have a
recent enough CPU, you can easily saturate 1 GB link with only a minor
difference in CPU load.

If your benchmark is something like sysbench for MySQL, the limiting
factor is usually disk I/O. In this case testing on a native Linux and
testing on dom0 would yield very similar results.
> but disk
> I/O numbers from dom0 should be consistent with a bare metal machine.
Correct.

Another thing to note is memory allocation. By default Xen
automatically baloons down dom0 memory as needed when domU is started,
but it does NOT automatically baloon-up later when domU is stopped. So
make sure your dom0 is using all memory (try "xm list" or "xm
info")
before starting your benchmark.

Regards,

Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users