Xen users - Feb 2013 - Dom0 freeze on HVM DomU Windows reboot with VGA passthrough

Hi,


	I''m doing VGA passthrough for an ATI (HD 6770) graphics card to
a DomU Windows. The passthrough itself is working. However, everytime
when I reboot the DomU Windows, the whole machine freezes.


	I see no error messages from Dom0 kernel or the xen qemu log
(or perhaps I haven''t got the most verbose debugging turned on?).
Though from what I can tell, the the freeze happens when DomU
starts again, not the shutdown.


	Is there any pointers on how I could debug this?


	Xen 4.2.1, Dom0 is Linux 3.2.36, with xl passthrough. I have
tried both WinXP and Win7 and both shows the same result.


	Thanks!

-- 
Regards,
Howell.

> 	I''m doing VGA passthrough for an ATI (HD 6770) graphics card
> to a DomU Windows. The passthrough itself is working. However,
> everytime when I reboot the DomU Windows, the whole machine freezes.
	And FWIW, I tested with Xen latest tree and I still get the
freezing problem.

	I have tried to turn on more debugging with loglvl and
guest_loglvl but there''s nothing obvious.

	Any more suggestions anyone?

	Thanks.

-- 
Regards,
Howell.

> 
> > 	I''m doing VGA passthrough for an ATI (HD 6770) graphics card
> > to a DomU Windows. The passthrough itself is working. However,
> > everytime when I reboot the DomU Windows, the whole machine freezes.
> 
> 	And FWIW, I tested with Xen latest tree and I still get the
> freezing problem.
> 
> 	I have tried to turn on more debugging with loglvl and
> guest_loglvl but there''s nothing obvious.
> 
> 	Any more suggestions anyone?
> 
If you are using GPLPV, I have a very beta version that no longer tries to do
anything funky with interrupts and may make a difference here... unless you are
also freezing without using GPLPV in which case it won''t.

James

> If you are using GPLPV, I have a very beta version that no longer
> tries to do anything funky with interrupts and may make a difference
> here... unless you are also freezing without using GPLPV in which
> case it won''t.
	Thanks for the tip.

	I just uninstalled the GPLPV driver in the win7 guest, and now
that win7 won''t boot anymore, it just gives me a BSOD at startup.

	The BSOD stays if I disable the PCI/VGA passthrough so it''s not
related I guess.


-- 
Regards,
Howell.

> 
> > If you are using GPLPV, I have a very beta version that no longer
> > tries to do anything funky with interrupts and may make a difference
> > here... unless you are also freezing without using GPLPV in which
> > case it won''t.
> 
> 	Thanks for the tip.
> 
> 	I just uninstalled the GPLPV driver in the win7 guest, and now
> that win7 won''t boot anymore, it just gives me a BSOD at startup.
> 
> 	The BSOD stays if I disable the PCI/VGA passthrough so it''s not
> related I guess.
> 
Uninstalling GPLPV leaves a registry entry present that causes it to not boot.
Is the BSoD start with 0x0000007b? If so that''s the "cant find
boot device"

If you can boot of the win7 boot cd then into the recovery console, load the
registry hive (eg load c:\windows\system32\config\system) and delete xenpci from
anything called upperfilters it should boot again.

James

> Uninstalling GPLPV leaves a registry entry present that causes it to
> not boot. Is the BSoD start with 0x0000007b? If so that''s the
"cant
> find boot device"
> 
> If you can boot of the win7 boot cd then into the recovery console,
> load the registry hive (eg load c:\windows\system32\config\system)
> and delete xenpci from anything called upperfilters it should boot
> again.
	Thanks. The good news is, I did what you said and now the win7
boots again.

	The bad news is now, without the GPLPV drivers, shutting
down/restarting the win7 will still cause a complete machine freeze.



-- 
Regards,
Howell.

Hi,

have you found a solution for this behaviour? Cause I''m currently
seeing a similar result with my installation.

Hi Matthias and all,
> have you found a solution for this behaviour? Cause I''m currently
> seeing a similar result with my installation.

	Unfortunately not yet.

	I remember I do not have this problem at the very beginning,
with working VGA passthrough. I only have the common problem where
after restarting the DomU Win7, the VGA passthrough screen will become
blank, but does not cause any freeze.

	I''m not sure what has been changed and then the freezing
started.

-- 
Regards,
Howell.

Hi all again,

> > have you found a solution for this behaviour? Cause I''m
currently
> > seeing a similar result with my installation.

	I''ve been away for a few weeks and I''m finally back on trying
to figure
out this problem.


	I was using xl + xl pci-assignable-add when I get the full
machine freeze when I reboot the domU windows7.


	I just tried with the same domu config file, but with xm +
pci-stub, and I do not seem to get the freeze (yet, so far 3 reboots/shutdowns).


	The VGA passthrough (an HD7770) is still working even after the
reboot (i.e. no more blank screen).


	This is semi-good news. If the bug is in xl, then I guess we
will have to debug this since xm is deprecated.


-- 
Regards,
Howell.

Hi,

may I take an educated guess: You only see the dom0 freeze when you
are hiding 2 vgas from dom0?

Cause this is what my behaviour
(http://www.gossamer-threads.com/lists/xen/users/273630) and i''ve seen
this issue coming up on the mailing list by others..

Also, we got a patch for the blankscreen-on-domU-restart thing if you
are still experiancing this.. I posted it here:
http://lists.xen.org/archives/html/xen-users/2013-03/msg00127.html
(in this thread the guy has the dual vga passthrough as well, so it
might be worth reeding furthor)

Oh yeah, and one last thing: I don''t know what git branch you are
using but they switched default qemu in unstable and the new
qemu-upstream has a bug so vga passthrough doesn''t work with more then
4GB of Ram assigned to a domU (bug described here:
http://lists.xen.org/archives/html/xen-devel/2013-03/msg00211.html )..
so for the behaviour you are used to (using qemu-xen-traditional) you
have to now say that explicitly in your domU config)

2013/3/17 Howell Tam <pigeon@pigeond.net>:
>
> Hi all again,
>
>
>> > have you found a solution for this behaviour? Cause I''m
currently
>> > seeing a similar result with my installation.
>
>
>         I''ve been away for a few weeks and I''m finally
back on trying to figure
> out this problem.
>
>
>         I was using xl + xl pci-assignable-add when I get the full
> machine freeze when I reboot the domU windows7.
>
>
>         I just tried with the same domu config file, but with xm +
> pci-stub, and I do not seem to get the freeze (yet, so far 3
reboots/shutdowns).
>
>
>         The VGA passthrough (an HD7770) is still working even after the
> reboot (i.e. no more blank screen).
>
>
>         This is semi-good news. If the bug is in xl, then I guess we
> will have to debug this since xm is deprecated.
>
>
> --
> Regards,
> Howell.
>

Hi,

> may I take an educated guess: You only see the dom0 freeze when you
> are hiding 2 vgas from dom0?
	I do have two VGA cards, but I have tested passing through only
one card, using xl, without pciback.hide.

> Also, we got a patch for the blankscreen-on-domU-restart thing if you
> are still experiancing this..
	Thanks, that''s good to know.

> Oh yeah, and one last thing: I don''t know what git branch you are
> using but they switched default qemu in unstable and the new
> qemu-upstream has a bug so vga passthrough doesn''t work with more
then
> 4GB of Ram assigned to a domU (bug described here:
> http://lists.xen.org/archives/html/xen-devel/2013-03/msg00211.html )..
> so for the behaviour you are used to (using qemu-xen-traditional) you
> have to now say that explicitly in your domU config)
	I''ve been testing with xen 4.2.1 from git
(bb4d9534d139615190c752cb0b8a05ef39e2d84b). I guess I will some
more tests with xl and qemu-traditional tonight.


--
Regards,
Howell.

Hi,

My experience so far is also that xl gives more problems for VGA passthru
compared to xm. I have only tried xen <= 4.1, first 4.1.2 with xl on gentoo
and then 4.1.2 and 4.1.3 with xm on ubuntu. My problems with xl were
different from yours though, I experienced the performance degradation
issue (http://lists.xen.org/archives/html/xen-users/2012-02/msg00443.html),
which could be solved as indicated in the post. This did not happen with xm
on ubuntu (win7-64 vm in both cases, radeon 6850).

But the performance issue might also be unrelated to the blank screen
issue. If the issues are related though, it might have to do with how xm vs
xl handles resetting the card at VM reboot. It could lead to different
issues with different cards, which would explain our different problems.

Regards,
Andreas

2013/3/17 Howell Tam <pigeon@pigeond.net>
>
> Hi all again,
>
>
> > > have you found a solution for this behaviour? Cause I''m
currently
> > > seeing a similar result with my installation.
>
>
>         I''ve been away for a few weeks and I''m finally
back on trying to
> figure
> out this problem.
>
>
>         I was using xl + xl pci-assignable-add when I get the full
> machine freeze when I reboot the domU windows7.
>
>
>         I just tried with the same domu config file, but with xm +
> pci-stub, and I do not seem to get the freeze (yet, so far 3
> reboots/shutdowns).
>
>
>         The VGA passthrough (an HD7770) is still working even after the
> reboot (i.e. no more blank screen).
>
>
>         This is semi-good news. If the bug is in xl, then I guess we
> will have to debug this since xm is deprecated.
>
>
> --
> Regards,
> Howell.
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>

_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

> 	I''ve been testing with xen 4.2.1 from git
> (bb4d9534d139615190c752cb0b8a05ef39e2d84b). I guess I will some
> more tests with xl and qemu-traditional tonight.

	Just for the record, with xl + qemu-xen-traditional + qemu-dm,
the full freeze problem has gone away. And I get the
blank-screen-after-reboot problem, which is expected.


Regards,
Howell.