Xen devel - May 2008 - Release 0.9.2 of GPL PV Drivers for Windows

Another update. This one should fix the problems people were having with
chkdsk on boot failing. Chkdsk never aligns its buffers to a 512 byte
boundary, so I''ve put in some code which will handle this case when the
transfer size is > PAGE_SIZE. This might fix some other random crashes
too.

A word of warning though, on my system a windows bug check (eg BSoD)
will hang my Dom0 hard for upwards of 5 minutes, and when it finally
comes back it reports:

"
BUG: soft lockup detected on CPU#0!

Call Trace:
 <IRQ> [<ffffffff8029fb2c>] softlockup_tick+0xdb/0xed
 [<ffffffff80267dba>] timer_interrupt+0x38d/0x3db
 [<ffffffff80211154>] handle_IRQ_event+0x2d/0x60
 [<ffffffff8029fe6b>] __do_IRQ+0xa4/0x105
 [<ffffffff8028364e>] _local_bh_enable+0x59/0xb3
 [<ffffffff802665a8>] do_IRQ+0x65/0x73
 [<ffffffff80360feb>] evtchn_do_upcall+0x86/0xe0
 [<ffffffff8025c616>] do_hypervisor_callback+0x1e/0x2c
 <EOI> [<ffffffff8020622a>] hypercall_page+0x22a/0x1000
 [<ffffffff8020622a>] hypercall_page+0x22a/0x1000
 [<ffffffff803607c4>] force_evtchn_callback+0xa/0xb
 [<ffffffff8036a7a3>] make_response+0xeb/0x131
 [<ffffffff8036b020>] blkif_schedule+0x24f/0x328
 [<ffffffff8028fa70>] autoremove_wake_function+0x0/0x2e
 [<ffffffff8028f8ad>] keventd_create_kthread+0x0/0x61
 [<ffffffff8036add1>] blkif_schedule+0x0/0x328
 [<ffffffff8028f8ad>] keventd_create_kthread+0x0/0x61
 [<ffffffff8023352b>] kthread+0xd4/0x107
 [<ffffffff8025c86c>] child_rip+0xa/0x12
 [<ffffffff8028f8ad>] keventd_create_kthread+0x0/0x61
 [<ffffffff80233457>] kthread+0x0/0x107
 [<ffffffff8025c862>] child_rip+0x0/0x12
"

Dom0 just plods along fine again after that though. Not sure what to
make of it at this point. This probably also happened on 0.9.1 too, and
maybe 0.9.0. It''s to do with a crash in xenvbd when Windows calls it in
''dump mode''.

You can trigger this behaviour manually on 0.9.2 with a ''xm sysrq
<domain> B'' if you want to see it for yourself, and maybe give me
a clue
as to why it''s happening :)

Have a look at the wiki page for more details:
http://wiki.xensource.com/xenwiki/XenWindowsGplPv

Download from:
http://www.meadowcourt.org/downloads/
(there is a link to the download site from the wiki entry).

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Sunday May 25 2008 10:09:28 pm James Harper wrote:
> Another update. This one should fix the problems people were having with
> chkdsk on boot failing. Chkdsk never aligns its buffers to a 512 byte
> boundary, so I''ve put in some code which will handle this case
when the
> transfer size is > PAGE_SIZE. This might fix some other random crashes
> too.
Kind of a similar upgrade path as 0.9.1: Reboot w/o /gplpv, run install.bat, 
get a BSOD when installing xennet:

DRIVER_IRQL_NOT_LESS_OR_EQUAL
[...]
*** STOP: 0x0000000A (0x00000016,0x0000001C,0x00000000,0x80502392)

(no driver mentioned)

and the rest of the install is uneventful after rebooting. Next time around, 
I''m going to alter the order of install in install.bat to see if I can
get
xennet to install last. (I guess just hiding xennet.inf the first time around 
will work.)

I can verify that chkdsk-on-boot failed w/ 0.9.1, and works w/ 0.9.2. However, 
once chkdsk finishes & reboots (or any time you reboot after booting 
w/ /gplpv) you get the annoying BSOD:
DRIVER_POWER_STATE_FAILURE
[...]
*** STOP: 0x0000009F (0x00000500,0x00000002,0x82588EB0,0x8258C218)

(no driver name mentioned)

(Annoying because you have to remember to kill qemu-dm before you ''xm 
destroy'', or else you have to reboot dom0.)

Keyboard still isn''t functional, and no keyboard device in Device
Manager.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,

booting /gplpv the emulated realtek nic''s and the ide driver are marked
with a yellow sign, but I assume this can be silently ignored.

If I try to activate/deactivate a nic, a bluescreen occurs. This happens
for both - the realtek and the xennet.

*** STOP: 0x0000007E (0xC0000005,0x808925CF,0xF78DA4B8,0xF78DA1B4)
(for xennet)

System is a plain W2K3 Server Standard Edition / German.
Xen: 3.2 x86_64.

cheers,

Stephan




James Harper schrieb:
> Another update. This one should fix the problems people were having with
> chkdsk on boot failing. Chkdsk never aligns its buffers to a 512 byte
> boundary, so I''ve put in some code which will handle this case
when the
> transfer size is > PAGE_SIZE. This might fix some other random crashes
> too.
> 
> A word of warning though, on my system a windows bug check (eg BSoD)
> will hang my Dom0 hard for upwards of 5 minutes, and when it finally
> comes back it reports:
> 
> "
> BUG: soft lockup detected on CPU#0!
> 
> Call Trace:
>  <IRQ> [<ffffffff8029fb2c>] softlockup_tick+0xdb/0xed
>  [<ffffffff80267dba>] timer_interrupt+0x38d/0x3db
>  [<ffffffff80211154>] handle_IRQ_event+0x2d/0x60
>  [<ffffffff8029fe6b>] __do_IRQ+0xa4/0x105
>  [<ffffffff8028364e>] _local_bh_enable+0x59/0xb3
>  [<ffffffff802665a8>] do_IRQ+0x65/0x73
>  [<ffffffff80360feb>] evtchn_do_upcall+0x86/0xe0
>  [<ffffffff8025c616>] do_hypervisor_callback+0x1e/0x2c
>  <EOI> [<ffffffff8020622a>] hypercall_page+0x22a/0x1000
>  [<ffffffff8020622a>] hypercall_page+0x22a/0x1000
>  [<ffffffff803607c4>] force_evtchn_callback+0xa/0xb
>  [<ffffffff8036a7a3>] make_response+0xeb/0x131
>  [<ffffffff8036b020>] blkif_schedule+0x24f/0x328
>  [<ffffffff8028fa70>] autoremove_wake_function+0x0/0x2e
>  [<ffffffff8028f8ad>] keventd_create_kthread+0x0/0x61
>  [<ffffffff8036add1>] blkif_schedule+0x0/0x328
>  [<ffffffff8028f8ad>] keventd_create_kthread+0x0/0x61
>  [<ffffffff8023352b>] kthread+0xd4/0x107
>  [<ffffffff8025c86c>] child_rip+0xa/0x12
>  [<ffffffff8028f8ad>] keventd_create_kthread+0x0/0x61
>  [<ffffffff80233457>] kthread+0x0/0x107
>  [<ffffffff8025c862>] child_rip+0x0/0x12
> "
> 
> Dom0 just plods along fine again after that though. Not sure what to
> make of it at this point. This probably also happened on 0.9.1 too, and
> maybe 0.9.0. It''s to do with a crash in xenvbd when Windows calls
it in
> ''dump mode''.
> 
> You can trigger this behaviour manually on 0.9.2 with a ''xm sysrq
> <domain> B'' if you want to see it for yourself, and maybe
give me a clue
> as to why it''s happening :)
> 
> Have a look at the wiki page for more details:
> http://wiki.xensource.com/xenwiki/XenWindowsGplPv
> 
> Download from:
> http://www.meadowcourt.org/downloads/
> (there is a link to the download site from the wiki entry).
> 
> James
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users

-- 
Stephan Seitz
Senior System Administrator

*netz-haut* e.K.
multimediale kommunikation

zweierweg 22
97074 würzburg

fon: +49 931 2876247
fax: +49 931 2876248

web: www.netz-haut.de <http://www.netz-haut.de/>

registriergericht: amtsgericht würzburg, hra 5054



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> booting /gplpv the emulated realtek nic''s and the ide driver are
marked
> with a yellow sign, but I assume this can be silently ignored.
> 
> If I try to activate/deactivate a nic, a bluescreen occurs. This
happens
> for both - the realtek and the xennet.
> 
> *** STOP: 0x0000007E (0xC0000005,0x808925CF,0xF78DA4B8,0xF78DA1B4)
> (for xennet)
> 
> System is a plain W2K3 Server Standard Edition / German.
> Xen: 3.2 x86_64.
> 
Since 0.9.2 I have fixed one memory alloc/free error, but I''m not
confident that it''s the error you are seeing. The strange thing is,
that
error should have meant that xennet would crash every single time it
loaded, but obviously it wasn''t. It might explain why some people got
BSoD''s on loading xennet sometimes though...

I can enable and disable my xennet network adapter without problems at
the current hg version (eg post 0.9.2), but I don''t remember trying
previously.

I''ve just gotten xenscsi working properly (allows scsi passthrough to
windows, needs the vscsi patches posted to the list ages ago) and I''m
testing a Symantec Backup Exec ''baremetal'' restore in a
windows DomU...
so far so good. I''ll upload 0.9.3 in the next few days assuming nothing
breaks.

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James, if you can find some free time, could you please explain us how this
vscsi thing works and how to enable it? Would it also gain us some
performance, or is for serving other purpose?

Emre

On Tue, May 27, 2008 at 3:19 PM, James Harper
<james.harper@bendigoit.com.au>
wrote:
> > booting /gplpv the emulated realtek nic''s and the ide driver
are
> marked
> > with a yellow sign, but I assume this can be silently ignored.
> >
> > If I try to activate/deactivate a nic, a bluescreen occurs. This
> happens
> > for both - the realtek and the xennet.
> >
> > *** STOP: 0x0000007E (0xC0000005,0x808925CF,0xF78DA4B8,0xF78DA1B4)
> > (for xennet)
> >
> > System is a plain W2K3 Server Standard Edition / German.
> > Xen: 3.2 x86_64.
> >
>
> Since 0.9.2 I have fixed one memory alloc/free error, but I''m not
> confident that it''s the error you are seeing. The strange thing
is, that
> error should have meant that xennet would crash every single time it
> loaded, but obviously it wasn''t. It might explain why some people
got
> BSoD''s on loading xennet sometimes though...
>
> I can enable and disable my xennet network adapter without problems at
> the current hg version (eg post 0.9.2), but I don''t remember
trying
> previously.
>
> I''ve just gotten xenscsi working properly (allows scsi passthrough
to
> windows, needs the vscsi patches posted to the list ages ago) and
I''m
> testing a Symantec Backup Exec ''baremetal'' restore in a
windows DomU...
> so far so good. I''ll upload 0.9.3 in the next few days assuming
nothing
> breaks.
>
> James
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


-- 
Emre


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> James, if you can find some free time, could you please explain us how
> this vscsi thing works and how to enable it? Would it also gain us
some
> performance, or is for serving other purpose?
> 
pvSCSI was a series of patches posted by Jun Kamada on or around 18th
Feb 2008 (look for subject "[Patch x/7] pvSCSI driver]. Jun was given a
heap of feedback in response to those patches and went away to dwell on
it. A recent post indicated that he would be releasing an updated
version soon.

xenscsi is a windows frontend driver to the pvSCSI backend driver, and
allows (via the sg interface) raw scsi devices to be exported from Linux
to Windows. I am now using it to do test restores of Symantec''s Backup
Exec backups using Symantec''s IDR CD. Because the PV drivers are all
WDM
instead of WDF, it''s easy enough to make a windows text mode setup
floppy so you can activate the drivers via F6 when installing Windows.

As for performance, I''m getting between 1-2GB/minute from the tape
drive, this performance is probably limited by other factors, eg I''m
still using the qemu harddisk drivers at that stage.

If you had a scsi device (or sas/sata possibly) that you were willing to
dedicate solely to one domain, xenscsi would allow you to do it. I''ve
only tested it on tape drives at the moment. YMMV :)

James

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi James-san and all,

I''m planning to post the latest pvSCSI driver tomorrow evening.
Following are brief explanation of current design and implementation 
status.

# I will describe same contents on tomorrow''s mail. :-)

-----
As you know, after the post of pvSCSI driver to Xen community on Feb.
18th, we have gotten a lot of requests for improvements. We recognize
that the primary improvements, which affects the basic design of the
pvSCSI driver archtecture, are as follows.

1. Create a "virtual (SCSI) host" on Dom0, and attach it to 
   appropriate guest domain.
2. Hot-plug a LUN to the virtual host on the Dom0 created by above 1.
   The LUN appears and becomes usable immediately on the guest domain. 
   (By repeating this procedure, you can attach multiple LUNs to the
   guest domain.)
3. Support arbitrary SCSI ID mapping between physical host(s) on the 
   Dom0 and the virtual host. (Physical ID "host:channel:target:lun" 
   on the Dom0 can be mapped to arbitrary virtual ID 
   "host:channel:target:lun".)
4. The above 1. to 3. causes needs for "munging" of request and reply
   packet on Dom0. (e.g. The LUN list in reply packet for REPORT_LUN
   command should be appropriately altered in order to report correct
   LUNs actually attached to the virtual host.)
5. Support the Netchannel2, new communication mechanism between Dom0 
   and guest domain, in order to improve performance.


We have implemented mainly above 1. to 3.

As for 4., we are trying to develop a framework, which can support
various "munging" functions according to user requirements, device
specific functions and so on. (Ideally, the munging function can be
dynamically plugged in, and such like a bitmap table, which controls
invocation of the munging functions, can be specifiable by user.)
How ever, this is a very heavy work, so design and implementation are
still in progress.

And also above 5., we recognize that the source code is not yet 
available, therefore current pvSCSI driver does not support it. If it
became available, we would like to replace current communication 
mechanism with it.


Jun Kamada


On Wed, 28 May 2008 19:31:58 +1000
"James Harper" <james.harper@bendigoit.com.au> wrote:
> > 
> > James, if you can find some free time, could you please explain us how
> > this vscsi thing works and how to enable it? Would it also gain us
> some
> > performance, or is for serving other purpose?
> > 
> 
> pvSCSI was a series of patches posted by Jun Kamada on or around 18th
> Feb 2008 (look for subject "[Patch x/7] pvSCSI driver]. Jun was given
a
> heap of feedback in response to those patches and went away to dwell on
> it. A recent post indicated that he would be releasing an updated
> version soon.
> 
> xenscsi is a windows frontend driver to the pvSCSI backend driver, and
> allows (via the sg interface) raw scsi devices to be exported from Linux
> to Windows. I am now using it to do test restores of Symantec''s
Backup
> Exec backups using Symantec''s IDR CD. Because the PV drivers are
all WDM
> instead of WDF, it''s easy enough to make a windows text mode setup
> floppy so you can activate the drivers via F6 when installing Windows.
> 
> As for performance, I''m getting between 1-2GB/minute from the tape
> drive, this performance is probably limited by other factors, eg
I''m
> still using the qemu harddisk drivers at that stage.
> 
> If you had a scsi device (or sas/sata possibly) that you were willing to
> dedicate solely to one domain, xenscsi would allow you to do it.
I''ve
> only tested it on tape drives at the moment. YMMV :)
> 
> James
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi Jun,

Could you please tell us if it will be possible to "export" a LVM
Volume as
a SCSI drive? Or maybe I shall ask, would it make sense? If not, is your
pvSCSI implementation only efficient when there''s an actual physical
scsi
device on the dom?

Emre


On Thu, May 29, 2008 at 4:55 AM, Jun Kamada <kama@jp.fujitsu.com> wrote:
> Hi James-san and all,
>
> I''m planning to post the latest pvSCSI driver tomorrow evening.
> Following are brief explanation of current design and implementation
> status.
>
> # I will describe same contents on tomorrow''s mail. :-)
>
> -----
> As you know, after the post of pvSCSI driver to Xen community on Feb.
> 18th, we have gotten a lot of requests for improvements. We recognize
> that the primary improvements, which affects the basic design of the
> pvSCSI driver archtecture, are as follows.
>
> 1. Create a "virtual (SCSI) host" on Dom0, and attach it to
>   appropriate guest domain.
> 2. Hot-plug a LUN to the virtual host on the Dom0 created by above 1.
>   The LUN appears and becomes usable immediately on the guest domain.
>   (By repeating this procedure, you can attach multiple LUNs to the
>   guest domain.)
> 3. Support arbitrary SCSI ID mapping between physical host(s) on the
>   Dom0 and the virtual host. (Physical ID
"host:channel:target:lun"
>   on the Dom0 can be mapped to arbitrary virtual ID
>   "host:channel:target:lun".)
> 4. The above 1. to 3. causes needs for "munging" of request and
reply
>   packet on Dom0. (e.g. The LUN list in reply packet for REPORT_LUN
>   command should be appropriately altered in order to report correct
>   LUNs actually attached to the virtual host.)
> 5. Support the Netchannel2, new communication mechanism between Dom0
>   and guest domain, in order to improve performance.
>
>
> We have implemented mainly above 1. to 3.
>
> As for 4., we are trying to develop a framework, which can support
> various "munging" functions according to user requirements,
device
> specific functions and so on. (Ideally, the munging function can be
> dynamically plugged in, and such like a bitmap table, which controls
> invocation of the munging functions, can be specifiable by user.)
> How ever, this is a very heavy work, so design and implementation are
> still in progress.
>
> And also above 5., we recognize that the source code is not yet
> available, therefore current pvSCSI driver does not support it. If it
> became available, we would like to replace current communication
> mechanism with it.
>
>
> Jun Kamada
>
>
> On Wed, 28 May 2008 19:31:58 +1000
> "James Harper" <james.harper@bendigoit.com.au> wrote:
>
> > >
> > > James, if you can find some free time, could you please explain
us how
> > > this vscsi thing works and how to enable it? Would it also gain
us
> > some
> > > performance, or is for serving other purpose?
> > >
> >
> > pvSCSI was a series of patches posted by Jun Kamada on or around 18th
> > Feb 2008 (look for subject "[Patch x/7] pvSCSI driver]. Jun was
given a
> > heap of feedback in response to those patches and went away to dwell
on
> > it. A recent post indicated that he would be releasing an updated
> > version soon.
> >
> > xenscsi is a windows frontend driver to the pvSCSI backend driver, and
> > allows (via the sg interface) raw scsi devices to be exported from
Linux
> > to Windows. I am now using it to do test restores of
Symantec''s Backup
> > Exec backups using Symantec''s IDR CD. Because the PV drivers
are all WDM
> > instead of WDF, it''s easy enough to make a windows text mode
setup
> > floppy so you can activate the drivers via F6 when installing Windows.
> >
> > As for performance, I''m getting between 1-2GB/minute from the
tape
> > drive, this performance is probably limited by other factors, eg
I''m
> > still using the qemu harddisk drivers at that stage.
> >
> > If you had a scsi device (or sas/sata possibly) that you were willing
to
> > dedicate solely to one domain, xenscsi would allow you to do it.
I''ve
> > only tested it on tape drives at the moment. YMMV :)
> >
> > James
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
>
>
>

-- 
Emre


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> Hi Jun,
> 
> Could you please tell us if it will be possible to "export" a LVM
Volume
> as a SCSI drive? Or maybe I shall ask, would it make sense? If not, is
> your pvSCSI implementation only efficient when there''s an actual
physical
> scsi device on the dom?
> 
It wouldn''t make sense. I can''t think of any advantage to
turning a
block device into a scsi device for that purpose. The windows xenvbd
driver does this on the windows side, but only because it appears to be
the best way to present a disk device to Windows.

The pvSCSI is a passthrough mechanism for ''sg'' devices (eg
/dev/sgX) and
allows the passing through of raw scsi commands. Great for CD burners,
tape drives, scanners (do SCSI scanners still exist???) and other
non-block-device devices, but not necessary for lvm backed or file
backed block devices.

Did you have a particular reason in mind? Maybe you thought of something
I didn''t :)

James


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Well, not :)

I think this will be most useful when we have a drive to assign to a guest,
and also especially good for CD Burners. In the current implementation, I
think we can''t export such devices to domains. :)

Emre

On Thu, May 29, 2008 at 1:02 PM, James Harper
<james.harper@bendigoit.com.au>
wrote:
> > Hi Jun,
> >
> > Could you please tell us if it will be possible to "export"
a LVM
> Volume
> > as a SCSI drive? Or maybe I shall ask, would it make sense? If not, is
> > your pvSCSI implementation only efficient when there''s an
actual
> physical
> > scsi device on the dom?
> >
>
> It wouldn''t make sense. I can''t think of any advantage to
turning a
> block device into a scsi device for that purpose. The windows xenvbd
> driver does this on the windows side, but only because it appears to be
> the best way to present a disk device to Windows.
>
> The pvSCSI is a passthrough mechanism for ''sg'' devices
(eg /dev/sgX) and
> allows the passing through of raw scsi commands. Great for CD burners,
> tape drives, scanners (do SCSI scanners still exist???) and other
> non-block-device devices, but not necessary for lvm backed or file
> backed block devices.
>
> Did you have a particular reason in mind? Maybe you thought of something
> I didn''t :)
>
> James
>
>

-- 
Emre


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi, Emre-san,

As James-san already answered, target to be controlled by the pvSCSI 
is SCSI hardware device. The pvSCSI backend cannot and doesn''t 
recognize "volume" layer, for example fdisk or LVM, and "file
system"
layer, for example a file on ext3.

Thanks,


On Thu, 29 May 2008 21:02:25 +1000
"James Harper" <james.harper@bendigoit.com.au> wrote:
> > Hi Jun,
> > 
> > Could you please tell us if it will be possible to "export"
a LVM
> Volume
> > as a SCSI drive? Or maybe I shall ask, would it make sense? If not, is
> > your pvSCSI implementation only efficient when there''s an
actual
> physical
> > scsi device on the dom?
> > 
> 
> It wouldn''t make sense. I can''t think of any advantage to
turning a
> block device into a scsi device for that purpose. The windows xenvbd
> driver does this on the windows side, but only because it appears to be
> the best way to present a disk device to Windows.
> 
> The pvSCSI is a passthrough mechanism for ''sg'' devices
(eg /dev/sgX) and
> allows the passing through of raw scsi commands. Great for CD burners,
> tape drives, scanners (do SCSI scanners still exist???) and other
> non-block-device devices, but not necessary for lvm backed or file
> backed block devices.
> 
> Did you have a particular reason in mind? Maybe you thought of something
> I didn''t :)
> 
> James
> 

-----
Jun Kamada



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel