Hi,
okay, thank you for the additional information!
First of all some advices on your configs (these might not help you
with the vga passthrough but should eliminate other problems so you
have a good foundation):
1) You used mixed hdd types and not in any sequential order.. just use
either xvda or hda (xvda should be preferred) and increment step by
step a,b,c,d,e etc..
2) You try to pass the /dev/sr0 to both domUs at the same time.. don''t
know it that is supported.. (actually i never got the DVD-Rom running
perfectly, so you maybe just want to leave that out entirely if you
don''t need it)
3) You use ioemu for the network device. Since you have a hvm anyway
you can leave that out and might also see a performance gain on that..
4) You might consider switching to LVM instead of diskimages.. should
give you a I/O performance boost
Okay, now to the kernel:
Is this a vanilla one? Config looks good from what i can tell from the
tutorial you linked. I actually had problems getting the vanilla
kernel to run with vga passthrough at all and use an openSuse one ever
since cause they ship some xen patches with it.. but if vga
passthrough works for you (at least for one card) you should be fine,
i guess..
Xen-Version:
I think xen causes your problem here, and as i said before, dual vga
passthrough is currently broken for me, too. But first of all: You use
gfx_passthrough=1 in your config. Have you tried it with =0 so the
domUs only get the card as secondary? In my experiance this is much
more gracefully and easier to get running..
Now to what i think is your actual problem: Using passthrough with xl
as toolstack requires you to calculate some RAM-Ranges (Point 5 in the
tutorial you''ve referenced). My guess is that this formula / method
only works for one vga card. You basically (as far as I understand it,
I haven''t really looked into it, i explain why in a moment) tell xen /
the kernel that if you are doing vga passthrough use THESE specific
ranges for the PCI devices and since this is now hardcoded, when you
try the second card you use the ranges from the first one, which isn''t
good for the first domU because their graphic memory is overwritten
from something other then the domU, and it is not good for the second
domU because pci device and pci associated memory do not fit
together.. so in the end everything goes haywire..
What I would suggest to you is that you switch toolstacks from xl to
xm. On the one hand, xm is much more stable, especially with
experimental features like vga passthrough, and on the other hand it
does not require that you hardcode your VGA Bioses into xen or
calculated some adress ranges. The last thing I tested dual vga
passthrough successfully with is rev 23190 in xen-testing. I want to
start searching for the broken changeset in xen-testing next week, but
since then unfortunatly i can only advice to use this ~3 month old
change set. So if you want:
1) uninstall your current xen so there are no leftovers
2) cd /somewhere/you/want/to/compile/xen
3) hg clone -r 23190:5a00ccfc6391 http://xenbits.xen.org/hg/xen-4.1-testing.hg/
4) cd xen-4.1-testing.hg
5) make -j1 xen && make -j1 tools && make -j1 stubdom
6) make install-xen
7) make install-tools PYTHON_PREFIX_ARG8) make install-stubdom
9) update your grub entry and reboot so the new xen is loaded
10) if it is not started yet:
10a) /etc/init.d/xencommons start
10b) /etc/init.d/xend start
11) now you can start your domU with ''xm start domU''..
basically
everything should work the same but instead of using xl as the xen
command, use xm
P.S: with xm you can throw out the 00:01 device from pci too as well
as ignore the start script from the tutorial
Hope this works for you :)
2012/6/28 joe harper <jharper.research@gmail.com>:> Since this is being cc''d to xen''s mailing list I will
include a few more
> details.
>
> I am trying to assign 1 card per domU. Essentially I am trying to recreate
> Ubisoft''s experiment. http://www.youtube.com/watch?v=Gtmwnx-k2qg
>
> Here is my configuration:
>
> I followed David Techer''s tutorial for VGA passthrough with Nvidia
graphics
> cards.
>
http://www.davidgis.fr/blog/index.php?2011/12/07/860-xen-42unstable-patches-for-vga-pass-through
>
> One difference between my setup and his it that I am using Fedora 14 as my
> dom0.
>
> I am using a tyan s7025 motherboard with two Nvidia gtx 460 video cards.
>
> My dom0 kernel = 3.1.0-rc10
>
> Xen = xen-unstable.hg-rev-24407
>
> #xl info:
>
> host : vise
> release : 3.1.0-rc10
> version : #2 SMP Fri Jun 15 12:11:10 PDT 2012
> machine : x86_64
> nr_cpus : 4
> nr_nodes : 1
> cores_per_socket : 4
> threads_per_core : 1
> cpu_mhz : 1600
> hw_caps :
> bfebfbff:2c100800:00000000:00003f40:029ee3ff:00000000:00000001:00000000
> virt_caps : hvm hvm_directio
> total_memory : 12279
> free_memory : 11086
> free_cpus : 0
> xen_major : 4
> xen_minor : 2
> xen_extra : -unstable
> xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p hvm-3.0-x86_32 hvm-3.0-x86_32p
> hvm-3.0-x86_64
> xen_scheduler : credit
> xen_pagesize : 4096
> platform_params : virt_start=0xffff800000000000
> xen_changeset : Fri Dec 09 16:19:36 2011 +0000 24407:03138a08366b
> xen_commandline : dom0_mem=1024M loglvl=all guest_loglvl=all iommu=1
> cc_compiler : gcc version 4.5.1 20100924 (Red Hat 4.5.1-4) (GCC)
> cc_compile_by : root
> cc_compile_domain :
> cc_compile_date : Fri Jun 22 15:59:26 PDT 2012
> xend_config_format : 4
>
> Grub Conf:
>
> title Fedora Xen 4.2-unstable with Linux 3.1.0-rc10 pvops dom0
> root (hd0,0)
> kernel /xen.gz dom0_mem=1024M loglvl=all guest_loglvl=all iommu=1
> module /vmlinuz-3.1.0-rc10 ro root=/dev/mapper/vg_vise-lg_root nomodeset
> xen-pciback.hide=(06:00.0)(06:00.1)(08:00.0)(08:00.1)
>
>
> DomU (hvm_8):
>
>
#----------------------------------------------------------------------------
> builder=''hvm''
> memory = 1024
> vif = [ ''type=ioemu, bridge=virbr0'' ]
> monitor = 1
> name = "hvm_8"
> disk = [ ''file:
/var/lib/libvirt/images/centos5.3_8.img,hda,w'',
> ''phy:/dev/sr0,xvdc:cdrom,r'',]
> boot = ''c''
> vnc = 1
> vncdisplay = 1
> usb = 1
> usbdevice = ''tablet''
> localtime = 1
> vcpus = 1
> xen_platform_pci=1
> gfx_passthru=1
> pci = [
''08:00.0'',''08:00.1'',''00:1a.2''
]
> pci_msitranslate = 1
> pci_power_mgmt = 1
>
#----------------------------------------------------------------------------
>
>
> DomU(hvm_6):
> builder=''hvm''
> memory = 1024
> vif = [ ''type=ioemu, bridge=virbr0'' ]
> monitor = 1
> name = "hvm_6"
> disk = [ ''file:
/var/lib/libvirt/images/centos5.3.img,hda,w'',
> ''phy:/dev/sr0,xvdc:cdrom,r'',]
> boot = ''c''
> vnc = 1
> vncdisplay = 1
> usb = 1
> usbdevice = ''tablet''
> localtime = 1
> vcpus = 1
> xen_platform_pci=1
> gfx_passthru=1
> pci = [
''06:00.0'',''06:00.1'',''00:1d.0''
]
> pci_msitranslate = 1
> pci_power_mgmt = 1
>
#----------------------------------------------------------------------------
>
>
>
> Currently Video card 1''s BusId = "08:0.0" and video card
2''s BusId > "06:0.0"
>
> So far I have successfully configured VGA passthough for card 1.
>
> I have two domU guest named hvm_8 and hvm_6 accordingly. When dom0 starts
> it displays output on card 1. After dom0 comes up I start domU hvm_8. At
> this point Card 1 resets and displays the console for hvm_8. This all works
> as I would expect.
>
> Next I start hvm_6. What I expect to see is Card 2 display hvm_6''s
video
> output. Instead Card 1 goes blank.
>
> Both hmv_6 and hvm_8 show up in “#xl list” but neither display output on
> either card 1 or card 2. Both monitors connected to card 1 and card 2 are
> black and appear to be off.
>
> It seams like Xen is sending a reset to card 1 when I start domU associated
> with card 2.
>
> Joe,
>
>
> On Wed, Jun 27, 2012 at 11:03 AM, Matthias
> <matthias.kannenberg@googlemail.com> wrote:
>>
>> Hi
>>
>> A short question just to be save:
>>
>> Are you trying to assign one seperate vga card per domU or are you
trying
>> to assign the same vga card to different domUs and trying to run them
at the
>> same time?
>>
>> Also, is it okay for you if i keep adding the xen mailing list in CC?
One
>> benefit is that another user might have the same issue in the future
and can
>> find your post and the other is that others might see your post and
help
>> you, too..
>>
>> Am 27.06.2012 18:31 schrieb "joe harper"
<jharper.research@gmail.com>:
>>
>>> Thank you for your response.
>>>
>>> I have been trying to utilize primary
passthrough(gfx-passthrough-1) with
>>> nvidia GTX 460 cards. I am able to pass 1 card through.
>>>
>>> My goal is to have 1 card for each domU guest. The problem that I
see is
>>> when I start the second domU the card associated with the first
domain
>>> resets and goes blank. At this point neither card shows output and
then the
>>> domu that I Just started crashes.
>>>
>>> It seems to me that the problem lies within the power managment /
reset
>>> for the graphics cards.
>>>
>>> Thanks,
>>>
>>> On Tue, Jun 26, 2012 at 12:59 AM, Matthias
>>> <matthias.kannenberg@googlemail.com> wrote:
>>>>
>>>> Hi,
>>>>
>>>> sure I can give you some info, but unfortunately as explained
here
>>>>
(http://lists.xen.org/archives/html/xen-users/2012-06/msg00392.html),
>>>> dual vga is broken for me currently, too. But i guess this is
only a
>>>> matter of finding the currupted changeset.
>>>>
>>>> In general, I was never aware that this is even a problem. I
used to
>>>> simply hide both graphic cards from the dom0 at boot time via
the
>>>> pciback.hide and than assigned them as secondary (= without
>>>> gfx_passthrough=1) to differnet domUs, one to a Windows 7 for
Work /
>>>> Gaming and one to a Linux as Mediacenter. The dom0 boots just
fine
>>>> without graphic cards and the management (starting xen /
starting
>>>> domUs etc.) is done via SSH from either a Smartphone or a
laptop.
>>>>
>>>> My dom0 is a Debian Wheezy, but with an open Suse Kernel and
their
>>>> patches, which help a lot, and my xen is a testing from their
git
>>>> repository. I''m not compiling any vga bios into kernel
or doing some
>>>> other patching to xen (like the adress calculation) to get it
work.
>>>>
>>>> I think the main difference to most other is that I am still
using xm
>>>> over xl because it seems much more stable to me and vga
passthrough
>>>> works like a charm.
>>>>
>>>> I''m Using an AMD Phenom 2 X1090T on a Asus Crosshair
IV, 16GB
>>>> DDR3-1600 RAM, an AMD HD5700 and an AMD HD5450 (passive
cooling) card.
>>>>
>>>> Oh, and a an advice: If you want to seperate domUs with VGA you
should
>>>> keep in mind that you have to think of somethink concerning
your sound
>>>> because the emulated sound kind of sucks. I solved this for me
with
>>>> passing my onboard sound to the windows VM and using the
AudiOverHDMI
>>>> with my mediacenter because it is connected to my TV. If you
want to
>>>> connect both domUs to Monitors you might run into problems
here.. In
>>>> general it might be possible to just pass a dedicated audio
card to
>>>> the other domU, but at least for my, my PCI slots are full with
>>>> graphic cards and their coolers ;) . Same issue with sound goes
with
>>>> input devices. Either user two seperated keyboard / mouse
setups, or
>>>> you can use xen''s vnc server. With the usb=tablet
input option, this
>>>> is pretty decent.
>>>>
>>>> Well, I hope i could give you some pointers. These were the
genereal
>>>> information i could think of. If you want / need some more
detailed
>>>> information on a topic, just ask.
>>>>
>>>>
>>>>
>>>> 2012/6/26 joe harper <jharper.research@gmail.com>:
>>>> > Hello Matthias,
>>>> >
>>>> > I came across a recent post of yours that mentioned
success with Dual
>>>> > VGA
>>>> > passthrough in Xen. I am trying to do something similiar.
Would you
>>>> > please
>>>> > inform me of your set up? Which video cards are you using
and are you
>>>> > using
>>>> > primary or secondary passthrough?
>>>> >
>>>> >
>>>> > Thank you
>>>> >
>>>> > Joe.
>>>> >
>>>> >
>>>
>>>
>