Xen users - Nov 2008 - GPLPV (9.11pre20) in Win2003 x64 on XenServer Enterprise 5.0 (CD drive missing)

Hello,

Introduction
------------
I''m introducing myself since it''s my first post here.
My name is Roel Broersma and my company is using Xenserver 5.0, we''re
using
Xenserver since it was from Xensource. Nowadays it''s Citrix and we have
several expensive licenses, paid support and are Citrix certified (Silver
Solution Advisor).  However, we''re experiencing a lot of problems last
months and Support (India) is getting worse...  ("Hello sir, please upload
a
server-bug-report, the developers will investigate", but mostly
it''s  "we
couldn''t find anything" ).
That''s why we''re moving more and more to complete Open Source.

Problem
-------
We were having major problems with one of our Windows 2003 Server R2 x64
(standard) system with Ipswitch Imail server.   There were a lot of
''connection resets''  and connections broken suddenly with the
SMTP protocol.
These problems came after upgrading the Xen (pv-tools) from 4.0.1 to 4.1
(even the 4.1 beta already had thse problems).  In Xenserver 5.0, we still
got the problems.  I think it had to do with TCP Checksum / offload, etc. 
We tried everything but couldn''t fix it (setting the windows registry, 
using ethtool -K ... on the Xenserver, etc.) I also saw the errors on from a
Windows Command prompt on this machine, typing: "netstat -e".

That''s why we finally moved to the GPLPV drivers (9.11pre20).
I can say:  THESE DRIVERS ROCK!   All our problems are gone!
When i type "netstat -e"  i don''t see any errors anymore and
we''ve no more
complaining customers about broken POP3 sessions (from Outlook) or messages
which couldn''t be send (using SMTP).
These drivers are really fast too!

However, we are missing the CD drive in Windows.  When i boot windows
without the GPLPV drivers, i see the CD drive again...  (i already checked
Drive Manager and did a rescan for new drives but it isn''t there...  
In
Xenserver 5.0  we have 2 VBD''s,  a HDA and a HDD  (the HDD is the cd
drive)).

It''s also not possible to hot-plug a network interface or Harddisk
drive,
(even after doing a new-hardware-scan or "rescan disks").  Hotplugging
is
not a real issue for us.. but the missing CD drive is.


Real question ;)
--------------
Is there any way to get the CD drive working in a Windows2003 guest with
GPLPV drivers on Xenserver 5.0 ?

-- 
View this message in context:
http://www.nabble.com/GPLPV-%289.11pre20%29-in-Win2003-x64--on-XenServer-Enterprise-5.0-%28CD-drive-missing%29-tp20499705p20499705.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> That''s why we finally moved to the GPLPV drivers (9.11pre20).
> I can say:  THESE DRIVERS ROCK!   All our problems are gone!
> When i type "netstat -e"  i don''t see any errors anymore
and we''ve no
more
> complaining customers about broken POP3 sessions (from Outlook) or
> messages
> which couldn''t be send (using SMTP).
> These drivers are really fast too!
Thanks for the feedback. I''m only using the Open Sourced version of
xen,
so that''s all I''ve tested with. I''m a little
surprised (but pleased)
that it works at all under XenSource - I hope you tested on a test
machine first :)
> 
> However, we are missing the CD drive in Windows.  When i boot windows
> without the GPLPV drivers, i see the CD drive again...  (i already
checked
> Drive Manager and did a rescan for new drives but it isn''t
there...
In
> Xenserver 5.0  we have 2 VBD''s,  a HDA and a HDD  (the HDD is the
cd
> drive)).
> 
> It''s also not possible to hot-plug a network interface or Harddisk
drive,
> (even after doing a new-hardware-scan or "rescan disks"). 
Hotplugging
is
> not a real issue for us.. but the missing CD drive is.
Hot adding of network and disk devices has worked at various points
along the way, at least under the Open Sourced version of Xen. The
interaction between the various Xen device states and the Windows
devices states is complicated :)

I can''t test it right now as my test box keeps BSoD-ing. I think I did
something stupid with ntfs-3g while trying to resurrect it :(
> Real question ;)
> --------------
> Is there any way to get the CD drive working in a Windows2003 guest
with
> GPLPV drivers on Xenserver 5.0 ?
> 
I did fix a bug in (I think) 0.9.12-pre2 which was affecting DVD-sized
cd/dvd images. That would cause the CD drive to appear but fail to read
though, which doesn''t sound like the problem you are having.

I''m not sure if all of these commands are available under the XenSource
version, but:

. send me the output of ''xenstore-ls
/local/domain/<id>/device''
(substitute <id> for the domain id of the domain in question)
. In device manager, you should see one ''Xen Block Device
Driver''
adapter per device (disk or cdrom). For each one, can you tell me the
value of ''Device Instance Id'' in the Properties -> Details
tab?
. send me a copy of your DomU config
. if you know how to use the windows debugger, connect that to the DomU
and send me the output. If you don''t know, then just the above stuff
might be sufficient to get started - it may be that the XenSource
version does things a little differently for CDROM''s or something which
I might be able to tell immediately.

Thanks

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi James, well.. about that test machine...      ;)  But at least i have lots
of feedback now:



James Harper wrote:
> 
> . send me the output of ''xenstore-ls
/local/domain/<id>/device''
> (substitute <id> for the domain id of the domain in question)
> . In device manager, you should see one ''Xen Block Device
Driver''
> adapter per device (disk or cdrom). For each one, can you tell me the
> value of ''Device Instance Id'' in the Properties ->
Details tab?
> . send me a copy of your DomU config
> . if you know how to use the windows debugger, connect that to the DomU
> and send me the output. If you don''t know, then just the above
stuff
> might be sufficient to get started - it may be that the XenSource
> version does things a little differently for CDROM''s or something
which
> I might be able to tell immediately.
> 
I did a "xe vm-list params=dom-id,name-label"  to see a list of
VM''s and
there IDs.
Then i did "xenstore-ls /local/domain/28/device"  which have me:

[root@xensvr2 ~]# xenstore-ls /local/domain/28/device
vbd = ""
 832 = ""
  backend = "/local/domain/0/backend/vbd/28/832"
  state = "4"
  backend-id = "0"
  device-type = "disk"
  virtual-device = "832"
  event-channel = "6"
  ring-ref = "16383"
 768 = ""
  backend = "/local/domain/0/backend/vbd/28/768"
  state = "4"
  backend-id = "0"
  device-type = "disk"
  virtual-device = "768"
  event-channel = "7"
  ring-ref = "16238"
 5632 = ""
  backend = "/local/domain/0/backend/vbd/28/5632"
  state = "4"
  backend-id = "0"
  device-type = "disk"
  virtual-device = "5632"
  event-channel = "8"
  ring-ref = "16093"
 5696 = ""
  backend = "/local/domain/0/backend/vbd/28/5696"
  state = "4"
  backend-id = "0"
  device-type = "cdrom"
  virtual-device = "5696"
  event-channel = "9"
  ring-ref = "15948"
vif = ""
 0 = ""
  backend = "/local/domain/0/backend/vif/28/0"
  backend-id = "0"
  state = "4"
  handle = "0"
  mac = "1a:87:80:a6:b9:a2"
  tx-ring-ref = "15947"
  rx-ring-ref = "15946"
  event-channel = "10"
  feature-no-csum-offload = "0"
  feature-sg = "1"
  feature-gso-tcpv4 = "1"
  request-rx-copy = "1"
  feature-rx-notify = "1"
[root@xensvr2 ~]#

I think that is the Xen equivalent of XenServer-api: "xe vbd-list
params=all
vm-name-label=mailsvr1" which gives me this:  (see attached file 
file1.txt)
http://www.nabble.com/file/p20515016/file1.txt file1.txt 

Driver instance IDs:
- XEN\VBD\4&32FE5319&1&5632
- XEN\VBD\4&32FE5319&1&5696
- XEN\VBD\4&32FE5319&1&768
- XEN\VBD\4&32FE5319&1&832

(btw: i have now 3 drives connected and should have 1 cd-drive connected,..
which i couldn''t see)

Behavior
--------
When i hot-plug a device from the Xenserver, i can not see it in the Windows
2003 VM.  (even not after a rescan disk  or  hardware detect)   When i
reboot the VM, it will detect a new device when starting Windows. I click..
next..next.. and it adds another "Xen Block Device Driver".


Other BAD behavior
-------------------
Most of our VM''s are on the SAN and connected with iSCSI to the
Xenserver.
When the shit-hits-the-fan and the SAN is going down (broken switch.. cable
broken.. or just something else)  all our Windows VM''s give
BSOD''s.  Which
is a quite normal behavior.  99% of the time we can reboot thse VM''s
later
without any problems,.. very sometimes we need to run a chkdsk.  (luckily
NTFS is a journalling filesystem).
BUT:  With the GPLPV drivers.. we do NOT get a BSOD''s,  i''ve
waited 10
minutes for it.  First i see some popups: "Can''t write to
<filename>  or
<disk>"  and it will raise many..many popups.  Finally i did a
force-shutdown from the Xenserver.  Then when rebooting this VM, the Master
Filesystem Table (MFT) was corrupt and couldn''t be repaired with
chkdsk. The
were lots of errors on the drive and i had to recover some files with
"GetDataBack for NTFS".  ... a long night...   :(
I never had this with the XenServer PV-tools.   I think the GPLPV drivers
have a too large disk-cache (write cache?) or something ?  The best is too: 
freeze the OS (i''ve seen that on Linux) or to give a BSOD within a
short
time  (Windows)... otherwise you''re really screwing up things...
Just test it:  Put 5 VM''s on.  4 with the Xenserver PV-tools and 1 with
the
GPLPV drivers,  then pull-off the storage.  The 4 VM''s are the first
give a
BSOD.. and the GPLPV is probably... never.. ?
(the thing i don''t understand is that when storage is completely
broken, it
wouldn''t matter if the VM is on for 10secs. or 10mins..  it
can''t write
through the storage so it can''t corrupt things...    This thought let
me
think about a too-big storage buffer maybe?  So a too-big piece is
missing... or journalling is not in sync.. ?)

I''m still using 9.11pre20.  I will try to find ou the Windows debugger
stuf..

Thanks for your response, i appreciate it!
-- 
View this message in context:
http://www.nabble.com/GPLPV-%289.11pre20%29-in-Win2003-x64--on-XenServer-Enterprise-5.0-%28CD-drive-missing%29-tp20499705p20515016.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> James Harper wrote:
> >
> > . send me the output of ''xenstore-ls
/local/domain/<id>/device''
> > (substitute <id> for the domain id of the domain in question)
> > . In device manager, you should see one ''Xen Block Device
Driver''
> > adapter per device (disk or cdrom). For each one, can you tell me
the
> > value of ''Device Instance Id'' in the Properties
-> Details tab?
> > . send me a copy of your DomU config
> > . if you know how to use the windows debugger, connect that to the
DomU
> > and send me the output. If you don''t know, then just the
above stuff
> > might be sufficient to get started - it may be that the XenSource
> > version does things a little differently for CDROM''s or
something
which
> > I might be able to tell immediately.
> >
> 
> I did a "xe vm-list params=dom-id,name-label"  to see a list of
VM''s
and
> there IDs.
> Then i did "xenstore-ls /local/domain/28/device"  which have me:
> 
> [root@xensvr2 ~]# xenstore-ls /local/domain/28/device
> vbd = ""
>  832 = ""
>   backend = "/local/domain/0/backend/vbd/28/832"
>   state = "4"
>   backend-id = "0"
>   device-type = "disk"
>   virtual-device = "832"
>   event-channel = "6"
>   ring-ref = "16383"
>  768 = ""
>   backend = "/local/domain/0/backend/vbd/28/768"
>   state = "4"
>   backend-id = "0"
>   device-type = "disk"
>   virtual-device = "768"
>   event-channel = "7"
>   ring-ref = "16238"
>  5632 = ""
>   backend = "/local/domain/0/backend/vbd/28/5632"
>   state = "4"
>   backend-id = "0"
>   device-type = "disk"
>   virtual-device = "5632"
>   event-channel = "8"
>   ring-ref = "16093"
>  5696 = ""
>   backend = "/local/domain/0/backend/vbd/28/5696"
>   state = "4"
>   backend-id = "0"
>   device-type = "cdrom"
>   virtual-device = "5696"
>   event-channel = "9"
>   ring-ref = "15948"
> vif = ""
>  0 = ""
>   backend = "/local/domain/0/backend/vif/28/0"
>   backend-id = "0"
>   state = "4"
>   handle = "0"
>   mac = "1a:87:80:a6:b9:a2"
>   tx-ring-ref = "15947"
>   rx-ring-ref = "15946"
>   event-channel = "10"
>   feature-no-csum-offload = "0"
>   feature-sg = "1"
>   feature-gso-tcpv4 = "1"
>   request-rx-copy = "1"
>   feature-rx-notify = "1"
> [root@xensvr2 ~]#
> 
> I think that is the Xen equivalent of XenServer-api: "xe vbd-list
> params=all
> vm-name-label=mailsvr1" which gives me this:  (see attached file
> file1.txt)
> http://www.nabble.com/file/p20515016/file1.txt file1.txt
> 
> Driver instance IDs:
> - XEN\VBD\4&32FE5319&1&5632
> - XEN\VBD\4&32FE5319&1&5696
> - XEN\VBD\4&32FE5319&1&768
> - XEN\VBD\4&32FE5319&1&832
Well there are 4 devices that the gplpv frontend is seeing, but
obviously something is going wrong and the cdrom devices are never being
reported to windows properly.

See the 4
''backend="/local/domain/0/backend/vbd/<id>/<dev>"''
lines
above? Can you do a xenstore-ls against each of those too. The frontend
xenstore stuff looks okay, including ''state=4'' which means
that the
frontend and backends are connected, but maybe the backend is giving
some wrong information or something.
> (btw: i have now 3 drives connected and should have 1 cd-drive
> connected,..
> which i couldn''t see)
> 
> Behavior
> --------
> When i hot-plug a device from the Xenserver, i can not see it in the
> Windows
> 2003 VM.  (even not after a rescan disk  or  hardware detect)   When i
> reboot the VM, it will detect a new device when starting Windows. I
> click..
> next..next.. and it adds another "Xen Block Device Driver".
When I hot-add a network adapter it appears to work, but then all the
network adapters go into ''acquiring dhcp address'', but after
that is
done it all works again. Hot-removing a network adapter appears to work
too, although after I do it from Xen, I have to ''safely
remove'' the
device before it disappears from windows. Not sure exactly why that
would be the case but I suppose it can be fixed.

Block devices though aren''t going to work... I deliberately fail any
attempt by Windows to recognise block devices added after system boot,
just in case one of them is the same as the qemu devices (eg because
you''ve just installed the drivers), with all the problems that that
entails. I may be able to fix that too, but I''ll have to be careful.
> Other BAD behavior
> -------------------
> Most of our VM''s are on the SAN and connected with iSCSI to the
Xenserver.
> When the shit-hits-the-fan and the SAN is going down (broken switch..
> cable
> broken.. or just something else)  all our Windows VM''s give
BSOD''s.
Which
> is a quite normal behavior.  99% of the time we can reboot thse
VM''s
later
> without any problems,.. very sometimes we need to run a chkdsk.
(luckily
> NTFS is a journalling filesystem).
> BUT:  With the GPLPV drivers.. we do NOT get a BSOD''s, 
i''ve waited 10
> minutes for it.  First i see some popups: "Can''t write to
<filename>
or
> <disk>"  and it will raise many..many popups.  Finally i did a
> force-shutdown from the Xenserver.  Then when rebooting this VM, the
> Master
> Filesystem Table (MFT) was corrupt and couldn''t be repaired with
chkdsk.
> The
> were lots of errors on the drive and i had to recover some files with
> "GetDataBack for NTFS".  ... a long night...   :(
> I never had this with the XenServer PV-tools.   I think the GPLPV
drivers
> have a too large disk-cache (write cache?) or something ?  The best is
> too:
> freeze the OS (i''ve seen that on Linux) or to give a BSOD within a
short
> time  (Windows)... otherwise you''re really screwing up things...
> Just test it:  Put 5 VM''s on.  4 with the Xenserver PV-tools and 1
with
> the
> GPLPV drivers,  then pull-off the storage.  The 4 VM''s are the
first
give
> a
> BSOD.. and the GPLPV is probably... never.. ?
> (the thing i don''t understand is that when storage is completely
broken,
> it
> wouldn''t matter if the VM is on for 10secs. or 10mins..  it
can''t
write
> through the storage so it can''t corrupt things...    This thought
let
me
> think about a too-big storage buffer maybe?  So a too-big piece is
> missing... or journalling is not in sync.. ?)
Now that is interesting... yes, you are right in saying that once the
''plug'' is pulled to the storage it doesn''t really
matter (from a data
integrity point of view)  what happens thereafter... a BSoD may be the
correct thing to do. I wonder what the backend will tell me... will it
report a fail on the block request, or will it in turn wait for ages
relying on me to fail the request instead? I''m also not sure if my
drivers should be invoking the BSoD directly... I suspect that they
should fail the request in such a way that Windows knows that all hope
is lost and so Windows should instigate the BSoD.

Either way, it does sound like I''m doing something a bit strange that
is
causing problems. This may happen with requests that aren''t aligned to
a
512 byte boundary - requests larger than 4096 bytes may be written out
of order (wrt other write requests), but those are seldom (never?) seen
during normal use, just at boot time and during a few infrequent
operations like formats.

I have definitely seen filesystem corruption after a crash (hanging the
windows domu ''hard'' should have the same effect as you were
seeing -
data not getting committed to the disk - that I didn''t expect. I put it
down to the circumstances of the crash but maybe there is more to it.

I am definitely not doing any write caching though - I don''t tell
Windows that the write is completed until the backend has finished with
the write. The backend may, in turn, be doing write caching, but that
should be the same as with the xensource drivers too.

Can you tell me, during the time the DomU is ''hung'' because
the SAN has
disconnected, does the SAN come back online before the reboot? If I''m
not managing read or write failures correctly, and suddenly the SAN
comes back online again, then that could be causing problems. If you
think that''s the case I can look at the failure paths a bit closer.
> 
> I''m still using 9.11pre20.  I will try to find ou the Windows
debugger
> stuf..
Just give me the xenstore-ls of the backend for now. That may be enough
to figure out what is going on.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

James, thanks for your reply!

OK, first things first, so here are the xenstore-ls <4 devices>


[root@xensvr2 ~]# xenstore-ls /local/domain/0/backend/vbd/28/832
frontend = "/local/domain/28/device/vbd/832"
online = "1"
params
"/dev/VG_XenStorage-97538472-f24d-6a24-f880-7ba5e765a66f/LV-5d27e\..."
state = "4"
dev = "hdb"
physical-device = "fc:0"
removable = "1"
mode = "w"
sm-data = ""
 scsi = ""
  0x12 = ""
   0x83
"AIMAMQIBAC1YRU5TUkMgIDVkMjdlMzRjLTlkYzItNDdlYi05ZTM2LTc2NDNiMDM\..."
   0x80 = "AIAAJjVkMjdlMzRjLTlkYzItNDdlYi05ZTM2LTc2NDNiMDMxNGUzMCAg"
 vdi-uuid = "5d27e34c-9dc2-47eb-9e36-7643b0314e30"
frontend-id = "28"
type = "phy"
feature-barrier = "1"
sectors = "140509184"
info = "0"
sector-size = "512"
kthread-pid = "31456"
[root@xensvr2 ~]# 


[root@xensvr2 ~]# xenstore-ls /local/domain/0/backend/vbd/28/768
frontend = "/local/domain/28/device/vbd/768"
online = "1"
params
"/dev/VG_XenStorage-97538472-f24d-6a24-f880-7ba5e765a66f/LV-c07c7\..."
state = "4"
dev = "hda"
physical-device = "fc:1"
removable = "1"
mode = "w"
sm-data = ""
 scsi = ""
  0x12 = ""
   0x83
"AIMAMQIBAC1YRU5TUkMgIGMwN2M3NDY0LTUxNGQtNGUyNS05MWMwLTJhZDMwN2J\..."
   0x80 = "AIAAJmMwN2M3NDY0LTUxNGQtNGUyNS05MWMwLTJhZDMwN2JjMjIwYyAg"
 vdi-uuid = "c07c7464-514d-4e25-91c0-2ad307bc220c"
frontend-id = "28"
type = "phy"
feature-barrier = "1"
sectors = "62914560"
info = "0"
sector-size = "512"
kthread-pid = "31457"
[root@xensvr2 ~]# 


[root@xensvr2 ~]# xenstore-ls /local/domain/0/backend/vbd/28/5632
frontend = "/local/domain/28/device/vbd/5632"
online = "1"
params
"/dev/VG_XenStorage-130038a8-e836-3449-2805-c11e0e05660b/LV-74641\..."
state = "4"
dev = "hdc"
physical-device = "fc:7"
removable = "1"
mode = "w"
sm-data = ""
 scsi = ""
  0x12 = ""
   0x83
"AIMAMQIBAC1YRU5TUkMgIDc0NjQxZjg0LTVkYmYtNDg0MS04NjAzLTg5ZDEyMTN\..."
   0x80 = "AIAAJjc0NjQxZjg0LTVkYmYtNDg0MS04NjAzLTg5ZDEyMTNmMTc3MSAg"
 vdi-uuid = "74641f84-5dbf-4841-8603-89d1213f1771"
frontend-id = "28"
type = "phy"
feature-barrier = "1"
sectors = "140509184"
info = "0"
sector-size = "512"
kthread-pid = "31458"
[root@xensvr2 ~]# 


[root@xensvr2 ~]# xenstore-ls /local/domain/0/backend/vbd/28/5696
frontend = "/local/domain/28/device/vbd/5696"
online = "1"
params
"/var/run/sr-mount/7d66b2a3-3e7f-7718-db04-76ba3a57d0c5/en_win_sr\..."
state = "5"
dev = "hdd"
removable = "1"
mode = "r"
frontend-id = "28"
type = "file"
[root@xensvr2 ~]# 


(i think the last one it the cd-drive... it gives less data..)


And the answer on your other question:
"Can you tell me, during the time the DomU is ''hung''
because the SAN has
disconnected, does the SAN come back online before the reboot?"
NO, the SAN was very down...   and we first shut down all the Xenservers
(and VM''s) before starting up the SAN.
BTW, another thought:  The VM (with the GPLPV drivers and the BSOD i told
about) was a Mailserver, mailservers have a lot of small files (1 or2 Kb). 
I''ve head/saw somewhere that all files under 1,5Kb are not written to
the
disk but to the MFT directly because making a pointer in the MFT to the
address on the disk where the (small) file is, is too much overhead.  So,
maybe the fact that this server is a mailserver with small files,
''helped''
in getting the MFT down.

Roel



James Harper wrote:
> 
>> James Harper wrote:
>> >
>> > . send me the output of ''xenstore-ls
/local/domain/<id>/device''
>> > (substitute <id> for the domain id of the domain in
question)
>> > . In device manager, you should see one ''Xen Block Device
Driver''
>> > adapter per device (disk or cdrom). For each one, can you tell me
> the
>> > value of ''Device Instance Id'' in the Properties
-> Details tab?
>> > . send me a copy of your DomU config
>> > . if you know how to use the windows debugger, connect that to the
> DomU
>> > and send me the output. If you don''t know, then just the
above stuff
>> > might be sufficient to get started - it may be that the XenSource
>> > version does things a little differently for CDROM''s or
something
> which
>> > I might be able to tell immediately.
>> >
>> 
>> I did a "xe vm-list params=dom-id,name-label"  to see a list
of VM''s
> and
>> there IDs.
>> Then i did "xenstore-ls /local/domain/28/device"  which have
me:
>> 
>> [root@xensvr2 ~]# xenstore-ls /local/domain/28/device
>> vbd = ""
>>  832 = ""
>>   backend = "/local/domain/0/backend/vbd/28/832"
>>   state = "4"
>>   backend-id = "0"
>>   device-type = "disk"
>>   virtual-device = "832"
>>   event-channel = "6"
>>   ring-ref = "16383"
>>  768 = ""
>>   backend = "/local/domain/0/backend/vbd/28/768"
>>   state = "4"
>>   backend-id = "0"
>>   device-type = "disk"
>>   virtual-device = "768"
>>   event-channel = "7"
>>   ring-ref = "16238"
>>  5632 = ""
>>   backend = "/local/domain/0/backend/vbd/28/5632"
>>   state = "4"
>>   backend-id = "0"
>>   device-type = "disk"
>>   virtual-device = "5632"
>>   event-channel = "8"
>>   ring-ref = "16093"
>>  5696 = ""
>>   backend = "/local/domain/0/backend/vbd/28/5696"
>>   state = "4"
>>   backend-id = "0"
>>   device-type = "cdrom"
>>   virtual-device = "5696"
>>   event-channel = "9"
>>   ring-ref = "15948"
>> vif = ""
>>  0 = ""
>>   backend = "/local/domain/0/backend/vif/28/0"
>>   backend-id = "0"
>>   state = "4"
>>   handle = "0"
>>   mac = "1a:87:80:a6:b9:a2"
>>   tx-ring-ref = "15947"
>>   rx-ring-ref = "15946"
>>   event-channel = "10"
>>   feature-no-csum-offload = "0"
>>   feature-sg = "1"
>>   feature-gso-tcpv4 = "1"
>>   request-rx-copy = "1"
>>   feature-rx-notify = "1"
>> [root@xensvr2 ~]#
>> 
>> I think that is the Xen equivalent of XenServer-api: "xe vbd-list
>> params=all
>> vm-name-label=mailsvr1" which gives me this:  (see attached file
>> file1.txt)
>> http://www.nabble.com/file/p20515016/file1.txt file1.txt
>> 
>> Driver instance IDs:
>> - XEN\VBD\4&32FE5319&1&5632
>> - XEN\VBD\4&32FE5319&1&5696
>> - XEN\VBD\4&32FE5319&1&768
>> - XEN\VBD\4&32FE5319&1&832
> 
> Well there are 4 devices that the gplpv frontend is seeing, but
> obviously something is going wrong and the cdrom devices are never being
> reported to windows properly.
> 
> See the 4
''backend="/local/domain/0/backend/vbd/<id>/<dev>"''
lines
> above? Can you do a xenstore-ls against each of those too. The frontend
> xenstore stuff looks okay, including ''state=4'' which
means that the
> frontend and backends are connected, but maybe the backend is giving
> some wrong information or something.
> 
>> (btw: i have now 3 drives connected and should have 1 cd-drive
>> connected,..
>> which i couldn''t see)
>> 
>> Behavior
>> --------
>> When i hot-plug a device from the Xenserver, i can not see it in the
>> Windows
>> 2003 VM.  (even not after a rescan disk  or  hardware detect)   When i
>> reboot the VM, it will detect a new device when starting Windows. I
>> click..
>> next..next.. and it adds another "Xen Block Device Driver".
> 
> When I hot-add a network adapter it appears to work, but then all the
> network adapters go into ''acquiring dhcp address'', but
after that is
> done it all works again. Hot-removing a network adapter appears to work
> too, although after I do it from Xen, I have to ''safely
remove'' the
> device before it disappears from windows. Not sure exactly why that
> would be the case but I suppose it can be fixed.
> 
> Block devices though aren''t going to work... I deliberately fail
any
> attempt by Windows to recognise block devices added after system boot,
> just in case one of them is the same as the qemu devices (eg because
> you''ve just installed the drivers), with all the problems that
that
> entails. I may be able to fix that too, but I''ll have to be
careful.
> 
>> Other BAD behavior
>> -------------------
>> Most of our VM''s are on the SAN and connected with iSCSI to
the
> Xenserver.
>> When the shit-hits-the-fan and the SAN is going down (broken switch..
>> cable
>> broken.. or just something else)  all our Windows VM''s give
BSOD''s.
> Which
>> is a quite normal behavior.  99% of the time we can reboot thse
VM''s
> later
>> without any problems,.. very sometimes we need to run a chkdsk.
> (luckily
>> NTFS is a journalling filesystem).
>> BUT:  With the GPLPV drivers.. we do NOT get a BSOD''s, 
i''ve waited 10
>> minutes for it.  First i see some popups: "Can''t write to
<filename>
> or
>> <disk>"  and it will raise many..many popups.  Finally i did
a
>> force-shutdown from the Xenserver.  Then when rebooting this VM, the
>> Master
>> Filesystem Table (MFT) was corrupt and couldn''t be repaired
with
> chkdsk.
>> The
>> were lots of errors on the drive and i had to recover some files with
>> "GetDataBack for NTFS".  ... a long night...   :(
>> I never had this with the XenServer PV-tools.   I think the GPLPV
> drivers
>> have a too large disk-cache (write cache?) or something ?  The best is
>> too:
>> freeze the OS (i''ve seen that on Linux) or to give a BSOD
within a
> short
>> time  (Windows)... otherwise you''re really screwing up
things...
>> Just test it:  Put 5 VM''s on.  4 with the Xenserver PV-tools
and 1
> with
>> the
>> GPLPV drivers,  then pull-off the storage.  The 4 VM''s are the
first
> give
>> a
>> BSOD.. and the GPLPV is probably... never.. ?
>> (the thing i don''t understand is that when storage is
completely
> broken,
>> it
>> wouldn''t matter if the VM is on for 10secs. or 10mins..  it
can''t
> write
>> through the storage so it can''t corrupt things...    This
thought let
> me
>> think about a too-big storage buffer maybe?  So a too-big piece is
>> missing... or journalling is not in sync.. ?)
> 
> Now that is interesting... yes, you are right in saying that once the
> ''plug'' is pulled to the storage it doesn''t
really matter (from a data
> integrity point of view)  what happens thereafter... a BSoD may be the
> correct thing to do. I wonder what the backend will tell me... will it
> report a fail on the block request, or will it in turn wait for ages
> relying on me to fail the request instead? I''m also not sure if my
> drivers should be invoking the BSoD directly... I suspect that they
> should fail the request in such a way that Windows knows that all hope
> is lost and so Windows should instigate the BSoD.
> 
> Either way, it does sound like I''m doing something a bit strange
that is
> causing problems. This may happen with requests that aren''t
aligned to a
> 512 byte boundary - requests larger than 4096 bytes may be written out
> of order (wrt other write requests), but those are seldom (never?) seen
> during normal use, just at boot time and during a few infrequent
> operations like formats.
> 
> I have definitely seen filesystem corruption after a crash (hanging the
> windows domu ''hard'' should have the same effect as you
were seeing -
> data not getting committed to the disk - that I didn''t expect. I
put it
> down to the circumstances of the crash but maybe there is more to it.
> 
> I am definitely not doing any write caching though - I don''t tell
> Windows that the write is completed until the backend has finished with
> the write. The backend may, in turn, be doing write caching, but that
> should be the same as with the xensource drivers too.
> 
> Can you tell me, during the time the DomU is ''hung''
because the SAN has
> disconnected, does the SAN come back online before the reboot? If
I''m
> not managing read or write failures correctly, and suddenly the SAN
> comes back online again, then that could be causing problems. If you
> think that''s the case I can look at the failure paths a bit
closer.
> 
>> 
>> I''m still using 9.11pre20.  I will try to find ou the Windows
debugger
>> stuf..
> 
> Just give me the xenstore-ls of the backend for now. That may be enough
> to figure out what is going on.
> 
> James
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
-- 
View this message in context:
http://www.nabble.com/GPLPV-%289.11pre20%29-in-Win2003-x64--on-XenServer-Enterprise-5.0-%28CD-drive-missing%29-tp20499705p20515504.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> [root@xensvr2 ~]# xenstore-ls /local/domain/0/backend/vbd/28/5696
> frontend = "/local/domain/28/device/vbd/5696"
> online = "1"
> params >
"/var/run/sr-mount/7d66b2a3-3e7f-7718-db04-76ba3a57d0c5/en_win_sr\..."
> state = "5"
> dev = "hdd"
> removable = "1"
> mode = "r"
> frontend-id = "28"
> type = "file"
> [root@xensvr2 ~]#
> 
> (i think the last one it the cd-drive... it gives less data..)
Certainly looks that way. Notice the ''state = "5"''?
5 means it''s in a
''closing'' state, which is unusual as we should actually detect
that in
the frontend. When you retrieved the above info, you hadn''t tried any
hotplug events or anything had you? If not, it means that the drivers
are going through the motions of setting things up but then something is
going wrong and the backend is setting the state to 5, and waiting for
the frontend set it''s state to 1 to restart things again.

Download http://www.meadowcourt.org/downloads/windbg.tgz, build it, and
follow the instructions in there to get a log of the boot messages. Let
me know if you need assistance.
> And the answer on your other question:
> "Can you tell me, during the time the DomU is ''hung''
because the SAN
has
> disconnected, does the SAN come back online before the reboot?"
> NO, the SAN was very down...   and we first shut down all the
Xenservers
> (and VM''s) before starting up the SAN.
> BTW, another thought:  The VM (with the GPLPV drivers and the BSOD i
told
> about) was a Mailserver, mailservers have a lot of small files (1 or2
Kb).
> I''ve head/saw somewhere that all files under 1,5Kb are not written
to
the
> disk but to the MFT directly because making a pointer in the MFT to
the
> address on the disk where the (small) file is, is too much overhead.
So,
> maybe the fact that this server is a mailserver with small files,
''helped''
> in getting the MFT down.
> 
Interesting... I wonder if an ''xm destroy'' during some sort of
disk
activity would be sufficient for me to reproduce the same effect as the
SAN disconnecting. I''m not sure exactly what I''d be looking
for though.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hello James,

I just downloaded the PV 0.9.11 and installed to a Windows 2003 32-bit
guest. It seems like the CD ROM drive doesn''t turn up (shows) until a
CD
is inserted before the guest is booted. If the drive is empty before
boot, the drive is also gone from hardware.

If it''s loaded before boot, I can view the content of the CD. But if I
then eject the CD and load (insert) another one, the contents will not
change. It will still show the contents from the first CD.

/Peter



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

(corrected! I meant PV 0.9.12, not PV 0.9.11)

Hello James,

I just downloaded the PV 0.9.12 and installed to a Windows 2003 32-bit
guest. It seems like the CD ROM drive doesn''t turn up (shows) until a
CD
is inserted before the guest is booted. If the drive is empty before
boot, the drive is also gone from hardware.

If it''s loaded before boot, I can view the content of the CD. But if I
then eject the CD and load (insert) another one, the contents will not
change. It will still show the contents from the first CD.

/Peter



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> (corrected! I meant PV 0.9.12, not PV 0.9.11)
> 
> Hello James,
> 
> I just downloaded the PV 0.9.12 and installed to a Windows 2003 32-bit
> guest. It seems like the CD ROM drive doesn''t turn up (shows)
until a
CD
> is inserted before the guest is booted. If the drive is empty before
boot,
> the drive is also gone from hardware.
> 
> If it''s loaded before boot, I can view the content of the CD. But
if I
> then eject the CD and load (insert) another one, the contents will not
> change. It will still show the contents from the first CD.
> 
I don''t think that there is an easy way of detecting disk changes at
the
moment.

I rely on the backend presenting me with ''sectors'' and
''sector-size''
values, and obviously these can''t be computed if there is no actual CD
inserted so I fail the connection. XenServer appears to act a little
differently - I haven''t seen those scsi mode pages in the open source
version.

Roel doesn''t appear to be using a physical CD drive though, just an
image, so I don''t think this is his problem.

James

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I am now ''deploying'' a -new-  Windows 2003 R2 (x64, standard)
server with the
latest 9.12pre3 drivers.
I am using Xenserver5 and Xencenter5  and i mount the virtual CD drive to an
ISO.

I will test to load an ISO while booting (maybe it sees an CD-rom then. ?)
I will also test to mount the virtual CD drive to the real-physical
dvd-drive in the physical server (i will also try it with a cd loaded and
the startup the VM).

Please wait 1 or two hours till i get the results...

Roel Broersma


(another question:  Xencenter5 is also connecting to a deamon which it
installs inside the VM, to get memory, cpu, network-traffic, etc.
statistics.  Can we extract this deamon from the original-citrix-pc-tools
and install it separately ?  Maybe we can then also  Migrate (Xen-Motion) a
VM to another server.. (i don''t know if this also depends on this tools
or
not...)



Peter Privat wrote:
> 
> Hello James,
> 
> I just downloaded the PV 0.9.11 and installed to a Windows 2003 32-bit
> guest. It seems like the CD ROM drive doesn''t turn up (shows)
until a CD
> is inserted before the guest is booted. If the drive is empty before
> boot, the drive is also gone from hardware.
> 
> If it''s loaded before boot, I can view the content of the CD. But
if I
> then eject the CD and load (insert) another one, the contents will not
> change. It will still show the contents from the first CD.
> 
> /Peter
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
-- 
View this message in context:
http://www.nabble.com/GPLPV-%289.11pre20%29-in-Win2003-x64--on-XenServer-Enterprise-5.0-%28CD-drive-missing%29-tp20499705p20528078.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi James (and peter),

Here i am again, sorry for the delay but i had some problems  (and again...
lot''s of feedback ;) )
As promised i tried to install a NEW VM from scratch (Windows Server 2003
X64 R2 Standard) and then install the GPLPV drivers.. i needed to know if it
had to do with booting with-or-without a CD loaded...

So i installed a new VM from scratch.. and there things went wrong...
In a new VM with NO XenPVTools, i get a BSOD when installing the GPLPV
drivers.
I made a new topic for this problem:
http://www.nabble.com/PV-0.9.12pre3-gives-BSOD-when-installing-in-VM-in-Xenserver-5.-tt20607865.html

After we solved that and we are able to install the GPLPV drivers in a
brand-new-clean VM, i will continue testing for this topic.

Roel



James Harper wrote:
> 
>> (corrected! I meant PV 0.9.12, not PV 0.9.11)
>> 
>> Hello James,
>> 
>> I just downloaded the PV 0.9.12 and installed to a Windows 2003 32-bit
>> guest. It seems like the CD ROM drive doesn''t turn up (shows)
until a
> CD
>> is inserted before the guest is booted. If the drive is empty before
> boot,
>> the drive is also gone from hardware.
>> 
>> If it''s loaded before boot, I can view the content of the CD.
But if I
>> then eject the CD and load (insert) another one, the contents will not
>> change. It will still show the contents from the first CD.
>> 
> 
> I don''t think that there is an easy way of detecting disk changes
at the
> moment.
> 
> I rely on the backend presenting me with ''sectors'' and
''sector-size''
> values, and obviously these can''t be computed if there is no
actual CD
> inserted so I fail the connection. XenServer appears to act a little
> differently - I haven''t seen those scsi mode pages in the open
source
> version.
> 
> Roel doesn''t appear to be using a physical CD drive though, just
an
> image, so I don''t think this is his problem.
> 
> James
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
-- 
View this message in context:
http://www.nabble.com/GPLPV-%289.11pre20%29-in-Win2003-x64--on-XenServer-Enterprise-5.0-%28CD-drive-missing%29-tp20499705p20608007.html
Sent from the Xen - User mailing list archive at Nabble.com.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users