Carsten Schiers
2010-Sep-08 05:47 UTC
AW: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny?
Hi, it definitely works. I am not sure though, whether we talk about distro xen and distro kernels here. What I confirm working is: Debian Etch and Lenny, Xen 4 self-compiled with XenLinux 2.6.18 Kernerls. It''s no miracle, thanks to Pasis XenWiki pages. Install tools, hg pull the source, make xen tools (at least), ./install.sh. Remember to setup the Xen network scripts correctly. Be very careful when using the options Pasi is describing. Be aware, whether you put in a Xen or a Kernel option. Read the logs carefully. Check with lspci in Dom0 and DomU. What doesn''t work is handling devices with MSI in Lenny distro Xen/Kernel. Use pci=nomsi as Kernel option and it should work with Lenny Xen kernel. It is the basis of a German distri (c''t server) and should work, although I still prefer XenLinux 2.6.18 for my DomUs that use PCI passthrough (for NICs and DVB-C cards). Be aware about the udev mechanism to create new entries in /etc/udev/rules.d/70-persistent-net-rules (or similar). You might have ended up with severeal entries and your NIC is not eth0, because the DomU was started with a Xen virtual device before, or some, as Xen sets a random MAC address. What is not working for me currently, is: pvops and PCI passthrough, but some of the DomUs are running a pvops 2.6.32 Kernel in parallel. I did not really try PCI passthrough with XenLinux 2.6.32, but sucessfully compiled it and run it w/o PCI passthrough. Hope this makes you a bit more confident in solving the problem instead of giving up ;o). If you already knew about all these things, please excuse. BR, Carsten. ----- Originalnachricht ----- Von: Mark Adams <mark@campbell-lange.net> Gesendet: Mit, 8.9.2010 00:39 An: Pasi Kärkkäinen <pasik@iki.fi> Cc: Meister Schieber <meisterschieber@ymail.com> ; xen-users@jbfavre.org ; xen-users@lists.xensource.com Betreff: Re: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny? As Pasi says it does work, I am using it to passthrough a nic. Out of your options only the xen-pciback.hide on the kernel command line was required for me, plus of course the config in the vm.cfg file. In your xm dmesg does it show that the device was successfully hidden? What do you get from xm list-pci-assignable-devices? (sorry that may not be 100% accurate command it''s something like that) I did have issues with 1 quad port Gb pci-e card. It would hide correctly but the domU would not load it as an interface (I think now maybe because the domU was Lenny not tested that theory though). Can give you some more detailed config tomorrow if required. Regards, Mark On 7 Sep 2010, at 18:19, Pasi Kärkkäinen <pasik@iki.fi> wrote:> On Tue, Sep 07, 2010 at 12:22:24PM +0000, Meister Schieber wrote: >> From: Mark Adams <mark@campbell-lange.net> >>> You should just use Squeeze (which is frozen now so on it''s way to >>> being the new stable). It works well simply by installing the >>> xen-linux-system and xen-qemu-dm packages. It includes pciback. >> >> Tried that for the last days, did not work either. The installation of >> squeeze and xen 4.0 was very easy indeed, only the pci-passthru does not >> work (tried 3 different computers with different hardware). >> >> Regarding to [1]http://wiki.xensource.com/xenwiki/XenPCIpassthrough I used >> the options: >> xen-pciback.hide=(09:00.0) >> and >> pciback.permissive pciback.hide=(09:00.0) pci=resource_alignment=09:00.0 >> and >> xen-pciback.permissive xen-pciback.hide=(09:00.0) >> pci=resource_alignment=(09:00.0) >> >> combined with "swiotlb=force" and "iommu=soft swiotlb=force" in the DomU >> (the "official" wiki has no special entries for the current release 4.0, >> so I tried every combination...) >> >> It just does not work, either the domU does not even start with some of >> the options, or the domU starts but the nic does not come up. >> >>> If you are so far into setup now that you want to stick with it, you >>> could try the linux-image-2.6.32-5-xen-amd64 kernel from the debian >>> repos. >> >> I am so far into several setups (plural) that I just does not want to give >> up and switch to vmware, so I desperately ask again: Did anyone succeeded >> in installing xen 4 on _any_ Debian version _and_ got a passed through NIC >> working in the DomU? >> > > Yeah just yesterday someone was doing it.. and writing to ##xen on irc. > > What domU kernel are you using? Does it have xen-pcifront? > What does "lspci" show in the domU? > > What pci related do you have in the domU dmesg? > > -- Pasi >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Carsten Schiers
2010-Sep-08 12:23 UTC
AW: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny?
Isn''t your Dom0 2.6.32-5 a XenLinux kernel <pciback.hide=()> or a pvops kernel <xen-pciback.hide=()> one? Also: have you put pci=nomsi in your DomU kernel line? Or check with cat /proc/interrupts, whether you use GSI or MSI interrupts... BR, Carsten. ----- Originalnachricht ----- Von: Meister Schieber <meisterschieber@ymail.com> Gesendet: Mit, 8.9.2010 13:34 An: Pasi Kärkkäinen <pasik@iki.fi> Cc: xen-users@lists.xensource.com Betreff: AW: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny? From: Pasi Kärkkäinen <pasik@iki.fi>>How much memory does your domU have? If more than 3 GB, try with 2GB or less.Dom0 has 2 GB total, domU has 128 MB reserved and "ballooning" is activated. > What''s the mode you''re using for xen pci backend driver in dom0 kernel? To be honest, I do not really understand your question. I just installed and used the kernel which comes with the xen-system (2.6.32-5). I was told it supports pcihide out of the box. I just added the line xen-pciback.hide=(09:00.0) to the grub-config. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Pasi Kärkkäinen
2010-Sep-08 12:37 UTC
Re: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny?
On Wed, Sep 08, 2010 at 02:23:45PM +0200, Carsten Schiers wrote:> Isn''t your Dom0 2.6.32-5 a XenLinux kernel <pciback.hide=()> or a pvops kernel <xen-pciback.hide=()> one? >Debian has pvops kernels in Squeeze. -- Pasi> Also: have you put pci=nomsi in your DomU kernel line? Or check with cat /proc/interrupts, whether you use > GSI or MSI interrupts... > > BR, > Carsten. > > ----- Originalnachricht ----- > Von: Meister Schieber <meisterschieber@ymail.com> > Gesendet: Mit, 8.9.2010 13:34 > An: Pasi Kärkkäinen <pasik@iki.fi> > Cc: xen-users@lists.xensource.com > Betreff: AW: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny? > > From: Pasi Kärkkäinen <pasik@iki.fi> > >How much memory does your domU have? If more than 3 GB, try with 2GB or less. > > Dom0 has 2 GB total, domU has 128 MB reserved and "ballooning" is activated. > > > What''s the mode you''re using for xen pci backend driver in dom0 kernel? > > To be honest, I do not really understand your question. I just installed and used the kernel which comes with the xen-system (2.6.32-5). I was told it supports pcihide out of the box. I just added the line > xen-pciback.hide=(09:00.0) to the grub-config. > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Bruce Edge
2010-Sep-08 18:20 UTC
Re: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny?
On Tue, Sep 7, 2010 at 10:47 PM, Carsten Schiers <carsten@schiers.de> wrote:> Hi, > > it definitely works. I am not sure though, whether we talk about distro xen > and distro > kernels here. > > What I confirm working is: Debian Etch and Lenny, Xen 4 self-compiled with > XenLinux 2.6.18 > Kernerls. It''s no miracle, thanks to Pasis XenWiki pages. Install tools, hg > pull the source, > make xen tools (at least), ./install.sh. Remember to setup the Xen network > scripts correctly. > > Be very careful when using the options Pasi is describing. Be aware, > whether you put in a Xen > or a Kernel option. Read the logs carefully. Check with lspci in Dom0 and > DomU. > > What doesn''t work is handling devices with MSI in Lenny distro Xen/Kernel. > Use pci=nomsi > as Kernel option and it should work with Lenny Xen kernel. It is the basis > of a German > distri (c''t server) and should work, although I still prefer XenLinux > 2.6.18 for my DomUs > that use PCI passthrough (for NICs and DVB-C cards). > > Be aware about the udev mechanism to create new entries in > /etc/udev/rules.d/70-persistent-net-rules (or similar). You might have ended > up with severeal entries and your NIC is not > eth0, because the DomU was started with a Xen virtual device before, or > some, as Xen sets > a random MAC address. >You can disable this above "feature" with the patch below: (At least this works for Ubuntu 10.04) It just comments out the line that writes the new rules entry. --- /lib/udev/rules.d/75-persistent-net-generator.rules 2009-05-14 02:28:14.000000000 -0700 +++ /lib/udev/rules.d/75-persistent-net-generator.rules.new 2010-05-26 14:34:25.000000000 -0700 @@ -54,7 +54,7 @@ ENV{COMMENT}=="", ENV{COMMENT}="net device ($attr{driver})" # write rule -DRIVERS=="?*", IMPORT{program}="write_net_rules" +# DRIVERS=="?*", IMPORT{program}="write_net_rules" # rename interface if needed ENV{INTERFACE_NEW}=="?*", NAME="$env{INTERFACE_NEW}" -Bruce> > What is not working for me currently, is: pvops and PCI passthrough, but > some of the DomUs > are running a pvops 2.6.32 Kernel in parallel. I did not really try PCI > passthrough with > XenLinux 2.6.32, but sucessfully compiled it and run it w/o PCI > passthrough. > > Hope this makes you a bit more confident in solving the problem instead of > giving up ;o). > > If you already knew about all these things, please excuse. > > BR, > Carsten. > > ----- Originalnachricht ----- > Von: Mark Adams <mark@campbell-lange.net> > Gesendet: Mit, 8.9.2010 00:39 > An: Pasi Kärkkäinen <pasik@iki.fi> > Cc: Meister Schieber <meisterschieber@ymail.com> ; xen-users@jbfavre.org ; > xen-users@lists.xensource.com > Betreff: Re: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on > Debian Lenny? > > As Pasi says it does work, I am using it to passthrough a nic. Out of your > options only the xen-pciback.hide on the kernel command line was required > for me, plus of course the config in the vm.cfg file. > > In your xm dmesg does it show that the device was successfully hidden? What > do you get from xm list-pci-assignable-devices? (sorry that may not be 100% > accurate command it''s something like that) > > I did have issues with 1 quad port Gb pci-e card. It would hide correctly > but the domU would not load it as an interface (I think now maybe because > the domU was Lenny not tested that theory though). > > Can give you some more detailed config tomorrow if required. > > Regards, > Mark > > On 7 Sep 2010, at 18:19, Pasi Kärkkäinen <pasik@iki.fi> wrote: > > > On Tue, Sep 07, 2010 at 12:22:24PM +0000, Meister Schieber wrote: > >> From: Mark Adams <mark@campbell-lange.net> > >>> You should just use Squeeze (which is frozen now so on it''s way to > >>> being the new stable). It works well simply by installing the > >>> xen-linux-system and xen-qemu-dm packages. It includes pciback. > >> > >> Tried that for the last days, did not work either. The installation of > >> squeeze and xen 4.0 was very easy indeed, only the pci-passthru does > not > >> work (tried 3 different computers with different hardware). > >> > >> Regarding to [1]http://wiki.xensource.com/xenwiki/XenPCIpassthrough I > used > >> the options: > >> xen-pciback.hide=(09:00.0) > >> and > >> pciback.permissive pciback.hide=(09:00.0) > pci=resource_alignment=09:00.0 > >> and > >> xen-pciback.permissive xen-pciback.hide=(09:00.0) > >> pci=resource_alignment=(09:00.0) > >> > >> combined with "swiotlb=force" and "iommu=soft swiotlb=force" in the > DomU > >> (the "official" wiki has no special entries for the current release > 4.0, > >> so I tried every combination...) > >> > >> It just does not work, either the domU does not even start with some > of > >> the options, or the domU starts but the nic does not come up. > >> > >>> If you are so far into setup now that you want to stick with it, you > >>> could try the linux-image-2.6.32-5-xen-amd64 kernel from the debian > >>> repos. > >> > >> I am so far into several setups (plural) that I just does not want to > give > >> up and switch to vmware, so I desperately ask again: Did anyone > succeeded > >> in installing xen 4 on _any_ Debian version _and_ got a passed through > NIC > >> working in the DomU? > >> > > > > Yeah just yesterday someone was doing it.. and writing to ##xen on irc. > > > > What domU kernel are you using? Does it have xen-pcifront? > > What does "lspci" show in the domU? > > > > What pci related do you have in the domU dmesg? > > > > -- Pasi > > > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Meister Schieber
2010-Sep-09 11:03 UTC
AW: AW: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny?
From: Carsten Schiers <carsten@schiers.de>> > Also: have you put pci=nomsi in your DomU kernel line?I tried, did not change anything: domU login: [ 71.986115] e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX/TX [ 72.801768] e1000: eth1: e1000_watchdog: NIC Link is Down [ 74.421975] e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX/TX [ 75.232677] e1000: eth1: e1000_watchdog: NIC Link is Down [ 76.945332] e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX/TX [ 77.749471] e1000: eth1: e1000_watchdog: NIC Link is Down [ 79.581691] e1000: eth1: e1000_watchdog: NIC Link is Up 100 Mbps Full Duplex, Flow Control: RX/TX> Or check with cat /proc/interrupts, whether you use > GSI or MSI interrupts...The output of cat /proc/interrupts does not have any entries about "msi" or "gsi": root@domU:~# cat /proc/interrupts | grep -ie gsi -ie msi root@domU:~# root@saturn:~# cat /proc/interrupts CPU0 1: 2 xen-pirq-ioapic-edge i8042 8: 0 xen-pirq-ioapic-edge rtc0 9: 0 xen-pirq-ioapic-edge acpi 12: 4 xen-pirq-ioapic-edge i8042 14: 90 xen-pirq-ioapic-edge pata_sis 15: 0 xen-pirq-ioapic-edge pata_sis 17: 5311 xen-pirq-ioapic-level sata_sis 18: 201 xen-pirq-ioapic-level 19: 2622 xen-pirq-ioapic-level peth0 20: 28 xen-pirq-ioapic-level ohci_hcd:usb2 21: 0 xen-pirq-ioapic-level ohci_hcd:usb3 22: 0 xen-pirq-ioapic-level ohci_hcd:usb4 23: 2 xen-pirq-ioapic-level ehci_hcd:usb1 499: 14 xen-dyn-event evtchn:xenstored 500: 5877 xen-dyn-event evtchn:xenstored 502: 0 xen-dyn-virq mce 505: 0 xen-dyn-virq pcpu 506: 10874 xen-dyn-event xenbus 507: 0 xen-dyn-ipi callfuncsingle0 508: 0 xen-dyn-virq debug0 509: 0 xen-dyn-ipi callfunc0 510: 0 xen-dyn-ipi resched0 511: 18433 xen-dyn-virq timer0 NMI: 0 Non-maskable interrupts LOC: 0 Local timer interrupts SPU: 0 Spurious interrupts PMI: 0 Performance monitoring interrupts PND: 0 Performance pending work RES: 0 Rescheduling interrupts CAL: 0 Function call interrupts TLB: 0 TLB shootdowns TRM: 0 Thermal event interrupts THR: 0 Threshold APIC interrupts MCE: 0 Machine check exceptions MCP: 3 Machine check polls ERR: 0 MIS: 0 ----- Originalnachricht ----- Von: Meister Schieber <meisterschieber@ymail.com> Gesendet: Mit, 8.9.2010 13:34 An: Pasi Kärkkäinen <pasik@iki.fi> Cc: xen-users@lists.xensource.com Betreff: AW: [Xen-users] Did anyone succeeded in installing xen 4.0.0 on Debian Lenny? From: Pasi Kärkkäinen <pasik@iki.fi>>How much memory does your domU have? If more than 3 GB, try with 2GB or less.Dom0 has 2 GB total, domU has 128 MB reserved and "ballooning" is activated.> What''s the mode you''re using for xen pci backend driver in dom0 kernel?To be honest, I do not really understand your question. I just installed and used the kernel which comes with the xen-system (2.6.32-5). I was told it supports pcihide out of the box. I just added the line xen-pciback.hide=(09:00.0) to the grub-config. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users