Xen devel - Nov 2007 - Third release of GPL''d PV drivers for Windows

This is the third release of my GPL''d PV drivers for Windows. Only
block
device drivers so far. Also only really useful to anyone who knows
something about windows driver development.

My test environment is Xen 3.1.1 (64 bit Hypervisor) and Windows 2003
sp2 (32 bit PAE). YMMV.

You can download the source zip file at
http://www.meadowcourt.org/WindowsXenPV-0.0.3.zip
No binary release yet!

Please do not in any way consider it ready for production use.
Definitely do not use them on a production DomU. While I have never seen
them cause a problem with the hypervisor or Dom0, I would also be very
wary of using them on a production physical machine too. It now seems
fairly sane in use, I haven''t had a crash for a while, but I
haven''t
done nearly enough testing yet.

To build, you''ll need the Windows WDF 6000 (not the later one which is
Longhorn only I think) which you can get from Microsoft. It''s not
small.
You''ll also need the xen/include/public folder from the xen source
tree,
which needs to be in common/include/public in the Windows PV drivers
source tree. I''ve been doing checked build''s and using Dbgview
from
SysInternals for testing. Run ''build'' from both the xenpci and
xenvbd
folders, then copy everything from target up to your test server.

To install, browse to the drivers on your test server and select as a
driver for the ''Unknown PCI'' device, and then for the
''vbd'' device which
gets created by the Xen PCI driver.

I''ve probably omitted some vitally important step from the above, so
please let me know if you need any assistance compiling.

To get the drivers to automatically probe your block devices on boot,
you''ll need to add /GPLPV to your boot.ini file. I haven''t
figured out a
nice way of stopping windows loading the qemu Intel IDE drivers yet
though (the best I got was to change the PCI ID). So you might have
problems if you do it that way.

The next best option is to do a block-attach at runtime, eg:

xm block-attach virtdemo phy:/dev/vg00/virt-virtdemo-1 /dev/hdc w

I''m pretty sure that you need to specify the ''phy:''
otherwise the
backend doesn''t seem to publish the block and sector sizes correctly.

If you do the block-attach, then your block device should show up under
windows, and you can partition it etc.

Changes:
. I think i''ve fixed all the race conditions etc that were causing
hangs
. Fixed lots of crashes
. Can now add multiple block devices
. Detects the block device size correctly

The source code still needs a tidy up. The Windows DDK stuff has its own
coding style which I''ve imitated in places, and then xen has
it''s own
too, so this code is mostly one, the other, both, or neither. I''ll get
to that as soon as I figure out what coding style to use and where.

Major limitations
. Need a way to disable the qemu provided Intel IDE device when /GPLPV
is specified...
. Only tested under 32-bit PAE Windows 2003. Almost certainly won''t
work
under 64 bit
. Doesn''t unload properly
. No power management
. Almost certainly won''t support migration or suspend/resume.
. Probably really poor performance - no optimisation done at all yet.
. I''m not sure that I''ve implemented the block device in a way
that
Microsoft would approve of (eg maybe not guaranteed to work under newer
windows operating systems)... maybe it should be a scsi miniport driver,
except those have other requirements that don''t mesh well with xen.
. Probably lots of other things too.

As soon as I can get it booting nicely i''ll release some binaries so it
can get some more general testing.

Future plans:
. Improve performance. Currently I have to do a memcpy between the
buffers Windows provides and Xen''s buffers, because it was the easiest
way. But Windows appears to mostly give buffers aligned to a multiple of
512 bytes (sector size) which may allow me to just grant Dom0 access to
the page... although what happens to the other bytes in the page?
. Virtual IRQ''s... not sure if I can do this under Windows...
everything
is just a port event handler currently.
. Better resource allocation to drivers (eg the PCI enumerator should be
able to hand out IRQ''s and address space to the lower drivers, which
would make the scsi miniport driver do-able).
. proper shutdown (need a user space driver to do this)
. network adapters (vif) (Andy Grover is looking at this)
. virtual scsi (eg a front end for the scsi passthrough stuff)
. balloon drivers (this should actually be pretty easy)

Enjoy!

James


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

James Harper, le Fri 09 Nov 2007 21:37:23 +1100, a écrit
:
> Windows appears to mostly give buffers aligned to a multiple of
> 512 bytes (sector size) which may allow me to just grant Dom0 access to
> the page... although what happens to the other bytes in the page?
They are untouched, so you can make the optimization, unless you are
paranoid enough about information leakage and don''t trust dom0.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> James Harper, le Fri 09 Nov 2007 21:37:23 +1100, a écrit :
> > Windows appears to mostly give buffers aligned to a multiple of
> > 512 bytes (sector size) which may allow me to just grant Dom0 access
to
> > the page... although what happens to the other bytes in the page?
> 
> They are untouched, so you can make the optimization, unless you are
> paranoid enough about information leakage and don''t trust dom0.
> 
I was more thinking along the lines of what happens when I need to grant the
same page more than once... eg windows request 1 (read) comes along and wants to
read a sector into offset 1024 in PFN 12345, and while that request is in
progress, windows request 2 (write) comes along and wants to write a sector from
offset 1536 in PFN 12345 (eg same PFN). Is anything going to break if I grant
the same page twice (two separate grant table references but the same PFN)? What
about if I grant it once as read only and then as read/write (or in the other
order)?

I''m hopeful that all of those problems have been considered and solved
previously and that it will ''just work''...

Thanks

James



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> James Harper, le Fri 09 Nov 2007 21:37:23 +1100, a écrit :
> > Windows appears to mostly give buffers aligned to a multiple of
> > 512 bytes (sector size) which may allow me to just grant Dom0 access
to
> > the page... although what happens to the other bytes in the page?
> 
> They are untouched, so you can make the optimization, unless you are
> paranoid enough about information leakage and don''t trust dom0.
Just tried it out. I''m not taking any metrics on how often the fast
path is followed (eg how often Windows provides a buffer aligned to a 512 byte
boundry), but some clock watching shows that an operation that takes 16-19
seconds using the Qemu drivers takes 6-9 seconds using my PV drivers. The only
issue so far is that while the PV drivers are doing their thing, Windows appears
to get a bit starved for resources (appears to hang on occasion etc)... could be
another bug causing that of course...

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Nov 09, 2007 at 09:37:23PM +1100, James Harper
wrote:
> This is the third release of my GPL''d PV drivers for Windows. Only
block
> device drivers so far. Also only really useful to anyone who knows
> something about windows driver development.
> 
> My test environment is Xen 3.1.1 (64 bit Hypervisor) and Windows 2003
> sp2 (32 bit PAE). YMMV.
> 
> You can download the source zip file at
> http://www.meadowcourt.org/WindowsXenPV-0.0.3.zip
> No binary release yet!
> 
Hi!

Really nice to see opensource PV drivers for windows!

Did you investigate microsoft/windows (ddk) licensing? Any problems releasing
these drivers
as GPL ?

-- Pasi

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

James Harper wrote:
> Future plans:
> . network adapters (vif) (Andy Grover is looking at this)
I have a netxen driver successfully loading against vif and responding
to NDIS OIDs. Then it tries to get an IP but send and recv aren''t
implemented yet. Progress has been faster than I anticipated, having
both xenvbd.c and netfront.c to use as examples.

Once I get send and recv working I''ll be ready to share.

Regards -- Andy


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

hum... the first binary release is 0.1.0 version of source code release? or
it`s 0.3.0?

I`m confusing!

Regards,
Luiz Vitor Martinez Cardoso.

On Nov 9, 2007 10:20 AM, James Harper <james.harper@bendigoit.com.au>
wrote:
> > James Harper, le Fri 09 Nov 2007 21:37:23 +1100, a écrit :
> > > Windows appears to mostly give buffers aligned to a multiple of
> > > 512 bytes (sector size) which may allow me to just grant Dom0
access
> to
> > > the page... although what happens to the other bytes in the page?
> >
> > They are untouched, so you can make the optimization, unless you are
> > paranoid enough about information leakage and don''t trust
dom0.
>
> Just tried it out. I''m not taking any metrics on how often the
fast path
> is followed (eg how often Windows provides a buffer aligned to a 512 byte
> boundry), but some clock watching shows that an operation that takes 16-19
> seconds using the Qemu drivers takes 6-9 seconds using my PV drivers. The
> only issue so far is that while the PV drivers are doing their thing,
> Windows appears to get a bit starved for resources (appears to hang on
> occasion etc)... could be another bug causing that of course...
>
> James
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>


-- 
Atenciosamente,
Luiz Vitor.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Have a look at the ''sent'' date on my email. For some reason
the Xen mailing lists re-post previous messages from time to time, just to keep
us on our toes.

James
> -----Original Message-----
> From: Grabber [mailto:grabber@gmail.com]
> Sent: Tuesday, 20 November 2007 11:46
> To: James Harper
> Cc: Samuel Thibault; xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] Third release of GPL''d PV drivers for
Windows
> 
> hum... the first binary release is 0.1.0 version of source code release?
> or it`s 0.3.0?
> 
> I`m confusing!
> 
> Regards,
> Luiz Vitor Martinez Cardoso.
> 
> 
> On Nov 9, 2007 10:20 AM, James Harper <
james.harper@bendigoit.com.au>
> wrote:
> 
> 
> 	> James Harper, le Fri 09 Nov 2007 21:37:23 +1100, a écrit :
> 	> > Windows appears to mostly give buffers aligned to a multiple of
> 	> > 512 bytes (sector size) which may allow me to just grant Dom0
> access to
> 	> > the page... although what happens to the other bytes in the
> page?
> 	>
> 	> They are untouched, so you can make the optimization, unless you
> are
> 	> paranoid enough about information leakage and don''t trust
dom0.
> 
> 
> 	Just tried it out. I''m not taking any metrics on how often the
fast
> path is followed (eg how often Windows provides a buffer aligned to a 512
> byte boundry), but some clock watching shows that an operation that takes
> 16-19 seconds using the Qemu drivers takes 6-9 seconds using my PV
> drivers. The only issue so far is that while the PV drivers are doing
> their thing, Windows appears to get a bit starved for resources (appears
> to hang on occasion etc)... could be another bug causing that of course...
> 
> 	James
> 
> 
> 	_______________________________________________
> 	Xen-devel mailing list
> 	Xen-devel@lists.xensource.com
> 	http://lists.xensource.com/xen-devel
> 
> 
> 
> 
> 
> --
> Atenciosamente,
> Luiz Vitor.
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel