Hi all, I have a Toshiba Tecra M9 and have not been able to boot it dom0. This is running SXDE 01/08, snv79b. After booting under kmdb and setting moddebug=80000000 before booting the Solaris kernel (with help from Dan Mick), I was able to see mac_ether as the last thing loading, right after loading the e1000g driver. I cannot drop into kmdb via F1-A after it hangs. I''ve also tried booting 32-bit, but had the same behavior. Any recommendations? Is there any data I can gather? I searched both the solaris/driver and solaris/xvm bugs, with not hits. Thanks in advance, - Matt -- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
Matt, [CC: Garrett D''Amore <garrett@damore.org>...]> Hi all, > > I have a Toshiba Tecra M9 and have not been able to boot it dom0. This > is running SXDE 01/08, snv79b. > > After booting under kmdb and setting moddebug=80000000 before booting > the Solaris kernel (with help from Dan Mick), I was able to see > mac_ether as the last thing loading, right after loading the e1000g > driver. I cannot drop into kmdb via F1-A after it hangs. I''ve also > tried booting 32-bit, but had the same behavior. > > Any recommendations? Is there any data I can gather? I searched both > the solaris/driver and solaris/xvm bugs, with not hits. > > Thanks in advance, > > - MattIt seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; maybe Garrett could provide an additional data point if a current Nevada build boots xVM/dom0 on his Tecra M9 ? Does the Tecra M9 still have a serial port? Looking at the Tecra M9 specs on Toshiba''s website it seems so... Maybe you could try to setup a xen console on the serial port with something like this in grub''s menu.lst file: kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 ... Maybe there are additional xen hypervisor error messages logged on the xen serial console? And it might be possible to break to dom0''s kmdb (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial port console? [*] http://mail.opensolaris.org/pipermail/laptop-discuss/2007-October/008976.html
Hi Garrett/Juergen, Thanks for the pointers. More inline... Garrett D''Amore wrote:> Juergen Keil wrote: >> >> >> It seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; >> maybe Garrett could provide an additional data point if a current >> Nevada build boots xVM/dom0 on his Tecra M9 ? >> > > I haven''t tried xVM at all. This laptop has been in use running > recent OpenSolaris/Nevada bits for SDcard development. > > However, I can try to set up xVM if there is a need. (It would be > especially helpful if I had a pointer to how to do this... I''ve not > tried xVM at all yet.)If you set up a recent SXCE/SXDE, you''ll find an xVM boot option in the GRUB menu already there. Just select that and you should be off and running. At least, the parameters in the menu match what is on the OpenSolaris Xen website for installation and getting started.> >> >> Does the Tecra M9 still have a serial port? Looking at the Tecra M9 >> specs on Toshiba''s website it seems so... Maybe you could try to >> setup a xen console on the serial port with something like this in >> grub''s menu.lst file: >> > > Yep, its got a DB9. No idea whether its COM1 or something else though.That could be useful. I wouldn''t know what to do with a xen console though, so any pointers on this would be of use to me.> >> >> kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 >> ... >> >> Maybe there are additional xen hypervisor error messages logged on the >> xen serial console? And it might be possible to break to dom0''s kmdb >> (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial port >> console?There were certainly plenty that flew by, but there was no opportunity to capture them when they were on the screen. I''ll see if I can get something with the serial then. Thanks for the help, - Matt
Garrett wrote:> Juergen Keil wrote: > > > It seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; > > maybe Garrett could provide an additional data point if a > > current Nevada build boots xVM/dom0 on his Tecra M9 ? > > > > I haven''t tried xVM at all. This laptop has been in use running recent > OpenSolaris/Nevada bits for SDcard development. > > However, I can try to set up xVM if there is a need. (It would be > especially helpful if I had a pointer to how to do this... I''ve not > tried xVM at all yet.)Hmm, recent SXCE builds (build 75 or newer) should have automatically installed the following GRUB boot entry: title Solaris xVM X86 kernel$ /boot/$ISADIR/xen.gz module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/$ISADIR/unix module$ /platform/i86pc/$ISADIR/boot_archive http://blogs.sun.com/levon/entry/opensolaris_xvm_now_available_in http://www.opensolaris.org/os/community/on/flag-days/pages/2007091801/ So I thought testing it would be as simple as a reboot and selecting the "Solaris xVM X86" boot entry in the grub menu. I think that it''s possible to simply add the missing Xen packages to an old Nevada build which has already been bfu''ed to post snv75 ONNV bits (e.g. grab the missing Xen packages from a nevada installation dvd): http://blogs.sun.com/sdaven/entry/moving_to_xen The hypervisor binary /boot/$ISADIR/xen.gz is from the package SUNWxvmr, and the dom0 kernel /platform/i86xpv/kernel/$ISADIR/unix should be part of ONNV >= build 75.> > Does the Tecra M9 still have a serial port? Looking at the Tecra M9 > > specs on Toshiba''s website it seems so... Maybe you could try to > > setup a xen console on the serial port with something like this in > > grub''s menu.lst file: > > > > Yep, its got a DB9. No idea whether its COM1 or something else though. > > > > > kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 > > ... > > > > > > Maybe there are additional xen hypervisor error messages logged on the > > xen serial console? And it might be possible to break to dom0''s kmdb > > (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial port > > console? > > > > -- Garrett > > > > [*] > > http://mail.opensolaris.org/pipermail/laptop-discuss/2007-October/008976.html
Matt,> >> Does the Tecra M9 still have a serial port? Looking at the Tecra M9 > >> specs on Toshiba''s website it seems so... Maybe you could try to > >> setup a xen console on the serial port with something like this in > >> grub''s menu.lst file: > >> > > > > Yep, its got a DB9. No idea whether its COM1 or something else though. > > That could be useful. I wouldn''t know what to do with a xen console > though, so any pointers on this would be of use to me.See "How to Debug On xVM / xVM Console", at the end of this page: http://www.opensolaris.org/os/community/xen/docs/sysadmin/
Garrett D''Amore wrote:> Hmm... I just booted up snv82 on my Tecra M9, using the "Solaris xVM" > grub menu option. It just worked, and uname shows i86xpv, so I think > this is a properly functioning Dom0.I''ll do a live upgrade to 82, and hopefully that''ll "just work" for me too. Thanks for looking into it Garrett. If you happen to think of any modifications you''ve made to the BIOS or anything like that, please let me know. I assume there are none and it''s just bugs that have been squashed post build 79b. Thanks, - Matt> > I don''t have a b79 build laying around to test with. > > Additionally, I''ve now verified that my sdcard software works properly > in dom0. Yay. :-) > > -- Garrett > > Juergen Keil wrote: >> Garrett wrote: >> >> >>> Juergen Keil wrote: >>> >>> >>>> It seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; >>>> maybe Garrett could provide an additional data point if a current >>>> Nevada build boots xVM/dom0 on his Tecra M9 ? >>>> >>> I haven''t tried xVM at all. This laptop has been in use running >>> recent OpenSolaris/Nevada bits for SDcard development. >>> >>> However, I can try to set up xVM if there is a need. (It would be >>> especially helpful if I had a pointer to how to do this... I''ve not >>> tried xVM at all yet.) >>> >> >> >> Hmm, recent SXCE builds (build 75 or newer) should have automatically >> installed the following GRUB boot entry: >> >> title Solaris xVM X86 kernel$ /boot/$ISADIR/xen.gz >> module$ /platform/i86xpv/kernel/$ISADIR/unix >> /platform/i86xpv/kernel/$ISADIR/unix >> module$ /platform/i86pc/$ISADIR/boot_archive >> >> >> http://blogs.sun.com/levon/entry/opensolaris_xvm_now_available_in >> http://www.opensolaris.org/os/community/on/flag-days/pages/2007091801/ >> >> >> So I thought testing it would be as simple as a reboot and selecting the >> "Solaris xVM X86" boot entry in the grub menu. >> >> >> I think that it''s possible to simply add the missing Xen packages >> to an old Nevada build which has already been bfu''ed to post snv75 >> ONNV bits (e.g. grab the missing Xen packages from a nevada >> installation dvd): >> >> http://blogs.sun.com/sdaven/entry/moving_to_xen >> >> >> The hypervisor binary /boot/$ISADIR/xen.gz is from the package >> SUNWxvmr, and the dom0 kernel /platform/i86xpv/kernel/$ISADIR/unix >> should be part of ONNV >= build 75. >> >> >> >>>> Does the Tecra M9 still have a serial port? Looking at the Tecra M9 >>>> specs on Toshiba''s website it seems so... Maybe you could try to >>>> setup a xen console on the serial port with something like this in >>>> grub''s menu.lst file: >>>> >>> Yep, its got a DB9. No idea whether its COM1 or something else though. >>> >>> >>>> kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 >>>> ... >>>> >>>> Maybe there are additional xen hypervisor error messages logged on the >>>> xen serial console? And it might be possible to break to dom0''s kmdb >>>> (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial >>>> port >>>> console? >>>> >>> -- Garrett >>> >>>> [*] >>>> http://mail.opensolaris.org/pipermail/laptop-discuss/2007-October/008976.html >>>> >>>> >> >> >
Garret''s Tecra M9 seems to have onboard Intel chipset video. And it seems to be possible to order the Tecra M9 from Toshiba with optional nvidia video hardware. Just a wild guess: Matt, does your Tecra M9 use nvidia video hardware? IIRC, there is a special xVM nvidia driver update available. For example, see bug 6607517: http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6607517> Hmm... I just booted up snv82 on my Tecra M9, using the "Solaris xVM" > grub menu option. It just worked, and uname shows i86xpv, so I think > this is a properly functioning Dom0. > > I don''t have a b79 build laying around to test with. > > Additionally, I''ve now verified that my sdcard software works properly > in dom0. Yay. :-) > > -- Garrett > > Juergen Keil wrote: > > Garrett wrote: > > > > > >> Juergen Keil wrote: > >> > >> > >>> It seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; > >>> maybe Garrett could provide an additional data point if a > >>> current Nevada build boots xVM/dom0 on his Tecra M9 ? > >>> > >>> > >> I haven''t tried xVM at all. This laptop has been in use running recent > >> OpenSolaris/Nevada bits for SDcard development. > >> > >> However, I can try to set up xVM if there is a need. (It would be > >> especially helpful if I had a pointer to how to do this... I''ve not > >> tried xVM at all yet.) > >> > > > > > > Hmm, recent SXCE builds (build 75 or newer) should have automatically > > installed the following GRUB boot entry: > > > > title Solaris xVM X86 > > kernel$ /boot/$ISADIR/xen.gz > > module$ /platform/i86xpv/kernel/$ISADIR/unix/platform/i86xpv/kernel/$ISADIR/unix> > module$ /platform/i86pc/$ISADIR/boot_archive > > > > > > http://blogs.sun.com/levon/entry/opensolaris_xvm_now_available_in > > http://www.opensolaris.org/os/community/on/flag-days/pages/2007091801/ > > > > > > So I thought testing it would be as simple as a reboot and selecting the > > "Solaris xVM X86" boot entry in the grub menu. > > > > > > I think that it''s possible to simply add the missing Xen packages > > to an old Nevada build which has already been bfu''ed to post snv75 ONNV > > bits (e.g. grab the missing Xen packages from a nevada installation dvd): > > > > http://blogs.sun.com/sdaven/entry/moving_to_xen > > > > > > The hypervisor binary /boot/$ISADIR/xen.gz is from the package > > SUNWxvmr, and the dom0 kernel /platform/i86xpv/kernel/$ISADIR/unix > > should be part of ONNV >= build 75. > > > > > > > >>> Does the Tecra M9 still have a serial port? Looking at the Tecra M9 > >>> specs on Toshiba''s website it seems so... Maybe you could try to > >>> setup a xen console on the serial port with something like this in > >>> grub''s menu.lst file: > >>> > >>> > >> Yep, its got a DB9. No idea whether its COM1 or something else though. > >> > >> > >>> kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 > >>> ... > >>> > >>> > >>> Maybe there are additional xen hypervisor error messages logged on the > >>> xen serial console? And it might be possible to break to dom0''s kmdb > >>> (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial port > >>> console? > >>> > >>> > >> -- Garrett > >> > >>> [*] > >>>http://mail.opensolaris.org/pipermail/laptop-discuss/2007-October/008976.html> >>> > > > > >Juergen Keil jk@tools.de Tools GmbH +49 (228) 9858011 Vorgebirgsstraße 37-39 http://www.tools.de 53119 BONN Sitz- und Registergericht HRB Bonn 4026 Geschäftsführung Wolfgang Franke & Wolfgang Solfrank
Juergen Keil wrote:> Garrett wrote: > >> Juergen Keil wrote: >> >>> It seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; >>> maybe Garrett could provide an additional data point if a >>> current Nevada build boots xVM/dom0 on his Tecra M9 ? >>> >> I haven''t tried xVM at all. This laptop has been in use running recent >> OpenSolaris/Nevada bits for SDcard development. >> >> However, I can try to set up xVM if there is a need. (It would be >> especially helpful if I had a pointer to how to do this... I''ve not >> tried xVM at all yet.) > > > Hmm, recent SXCE builds (build 75 or newer) should have automatically > installed the following GRUB boot entry:In general, yes, but it''s not fully automatic in a couple of situations: 1. Live Upgrade needed some changes to add the menu.lst entry, and this wasn''t completed until build 80. (Reminder: Always upgrade the SUNWlu* packages before doing the live upgrade itself). In this case, you can use ''bootadm list-menu'' to find the boot menu, open it up, and manually add the entry as shown below (you''ll probably also need a root command). 2. As previously mentioned, bfu will not add the xVM packages. How to do so is covered below. After adding the packages, run ''bootadm -m upgrade'' to add the menu entry. Or, open /boot/grub/menu.lst and manually copy in the entry. -Ryan> > title Solaris xVM X86 > kernel$ /boot/$ISADIR/xen.gz > module$ /platform/i86xpv/kernel/$ISADIR/unix /platform/i86xpv/kernel/$ISADIR/unix > module$ /platform/i86pc/$ISADIR/boot_archive > > > http://blogs.sun.com/levon/entry/opensolaris_xvm_now_available_in > http://www.opensolaris.org/os/community/on/flag-days/pages/2007091801/ > > > So I thought testing it would be as simple as a reboot and selecting the > "Solaris xVM X86" boot entry in the grub menu. > > > I think that it''s possible to simply add the missing Xen packages > to an old Nevada build which has already been bfu''ed to post snv75 ONNV > bits (e.g. grab the missing Xen packages from a nevada installation dvd): > > http://blogs.sun.com/sdaven/entry/moving_to_xen > > > The hypervisor binary /boot/$ISADIR/xen.gz is from the package > SUNWxvmr, and the dom0 kernel /platform/i86xpv/kernel/$ISADIR/unix > should be part of ONNV >= build 75. > > >>> Does the Tecra M9 still have a serial port? Looking at the Tecra M9 >>> specs on Toshiba''s website it seems so... Maybe you could try to >>> setup a xen console on the serial port with something like this in >>> grub''s menu.lst file: >>> >> Yep, its got a DB9. No idea whether its COM1 or something else though. >> >>> kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 >>> ... >>> >>> >>> Maybe there are additional xen hypervisor error messages logged on the >>> xen serial console? And it might be possible to break to dom0''s kmdb >>> (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial port >>> console? >>> >> -- Garrett >>> [*] >>> http://mail.opensolaris.org/pipermail/laptop-discuss/2007-October/008976.html > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org
Juergen Keil wrote:> Garret''s Tecra M9 seems to have onboard Intel chipset video. > > And it seems to be possible to order the Tecra M9 from Toshiba with > optional nvidia video hardware. > > Just a wild guess: Matt, does your Tecra M9 use nvidia video hardware? >Indeed it does, but...> IIRC, there is a special xVM nvidia driver update available. > For example, see bug 6607517: > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6607517 >I looked into it and that driver seems to be in snv_73 or later so mine should be fine. Still, maybe I should just disable X to eliminate that as an issue. Thanks for the pointer and looking into it though. I''ll try the BIOS change first, and if that doesn''t do it, a Live Upgrade or reinstall to the latest snv bits. - Matt> > > >> Hmm... I just booted up snv82 on my Tecra M9, using the "Solaris xVM" >> grub menu option. It just worked, and uname shows i86xpv, so I think >> this is a properly functioning Dom0. >> >> I don''t have a b79 build laying around to test with. >> >> Additionally, I''ve now verified that my sdcard software works properly >> in dom0. Yay. :-) >> >> -- Garrett >> >> Juergen Keil wrote: >> >>> Garrett wrote: >>> >>> >>> >>>> Juergen Keil wrote: >>>> >>>> >>>> >>>>> It seems Garrett D''Amore (@sun) has the same Tecra M9 laptop[*]; >>>>> maybe Garrett could provide an additional data point if a >>>>> current Nevada build boots xVM/dom0 on his Tecra M9 ? >>>>> >>>>> >>>>> >>>> I haven''t tried xVM at all. This laptop has been in use running recent >>>> OpenSolaris/Nevada bits for SDcard development. >>>> >>>> However, I can try to set up xVM if there is a need. (It would be >>>> especially helpful if I had a pointer to how to do this... I''ve not >>>> tried xVM at all yet.) >>>> >>>> >>> Hmm, recent SXCE builds (build 75 or newer) should have automatically >>> installed the following GRUB boot entry: >>> >>> title Solaris xVM X86 >>> kernel$ /boot/$ISADIR/xen.gz >>> module$ /platform/i86xpv/kernel/$ISADIR/unix >>> > /platform/i86xpv/kernel/$ISADIR/unix > >>> module$ /platform/i86pc/$ISADIR/boot_archive >>> >>> >>> http://blogs.sun.com/levon/entry/opensolaris_xvm_now_available_in >>> http://www.opensolaris.org/os/community/on/flag-days/pages/2007091801/ >>> >>> >>> So I thought testing it would be as simple as a reboot and selecting the >>> "Solaris xVM X86" boot entry in the grub menu. >>> >>> >>> I think that it''s possible to simply add the missing Xen packages >>> to an old Nevada build which has already been bfu''ed to post snv75 ONNV >>> bits (e.g. grab the missing Xen packages from a nevada installation dvd): >>> >>> http://blogs.sun.com/sdaven/entry/moving_to_xen >>> >>> >>> The hypervisor binary /boot/$ISADIR/xen.gz is from the package >>> SUNWxvmr, and the dom0 kernel /platform/i86xpv/kernel/$ISADIR/unix >>> should be part of ONNV >= build 75. >>> >>> >>> >>> >>>>> Does the Tecra M9 still have a serial port? Looking at the Tecra M9 >>>>> specs on Toshiba''s website it seems so... Maybe you could try to >>>>> setup a xen console on the serial port with something like this in >>>>> grub''s menu.lst file: >>>>> >>>>> >>>>> >>>> Yep, its got a DB9. No idea whether its COM1 or something else though. >>>> >>>> >>>> >>>>> kernel$ /boot/$ISADIR/xen.gz com1=9600,8n1 console=com1 >>>>> ... >>>>> >>>>> >>>>> Maybe there are additional xen hypervisor error messages logged on the >>>>> xen serial console? And it might be possible to break to dom0''s kmdb >>>>> (by tree times ctrl-A, and sending a "q" [IIRC]) via the xen serial port >>>>> console? >>>>> >>>>> >>>>> >>>> -- Garrett >>>> >>>> >>>>> [*] >>>>> >>>>> > http://mail.opensolaris.org/pipermail/laptop-discuss/2007-October/008976.html > >>>>> >>>>> >>> >>> > > Juergen Keil jk@tools.de > Tools GmbH +49 (228) 9858011 > Vorgebirgsstraße 37-39 http://www.tools.de > 53119 BONN > > Sitz- und Registergericht HRB Bonn 4026 > Geschäftsführung Wolfgang Franke & Wolfgang Solfrank > >-- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
Garrett D''Amore wrote:> Matt Ingenthron wrote: >> >> I''ll do a live upgrade to 82, and hopefully that''ll "just work" for >> me too. Thanks for looking into it Garrett. >> >> If you happen to think of any modifications you''ve made to the BIOS >> or anything like that, please let me know. I assume there are none >> and it''s just bugs that have been squashed post build 79b. > > The one thing that I''ve done is change the BIOS PnP OS setting so that > BIOS configures all devices. That is necessary for the SDcard stuff. > I don''t think it makes a difference otherwise, but maybe its something > that Xen^WxVM is sensitive to.Is that the "Device Configuration", where the choices are "Setup by OS" or "All"? Why can''t they just call it PnP? I just changed that, and sure enough, I''ve booted under xVM. I also changed a Virtualization enabled/disabled setting which I did not see before. That would seem to be important for xVM. :) The NVidia driver is working just fine as well. Let me go through the matrix to see what we can figure out. There''s either a bug or a release note to be created here I think. - Matt -- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
Matt Ingenthron wrote:> Garrett D''Amore wrote: > > Matt Ingenthron wrote: > >> > >> I''ll do a live upgrade to 82, and hopefully that''ll "just work" for > >> me too. Thanks for looking into it Garrett. > >> > >> If you happen to think of any modifications you''ve made to the BIOS > >> or anything like that, please let me know. I assume there are none > >> and it''s just bugs that have been squashed post build 79b. > > > > The one thing that I''ve done is change the BIOS PnP OS setting so that > > BIOS configures all devices. That is necessary for the SDcard stuff. > > I don''t think it makes a difference otherwise, but maybe its something > > that Xen^WxVM is sensitive to. > > Is that the "Device Configuration", where the choices are "Setup by OS" > or "All"? Why can''t they just call it PnP? > > I just changed that, and sure enough, I''ve booted under xVM.Hmm, is there any other driver sharing the interrupt vector with the e1000g device? Run mdb -k and have a look at the ::interrupts output, when not booted under xVM and with the BIOS PnP setting reverted: echo ::interrupts | mdb -k echo ::interrupts -d | mdb -k Maybe e1000g is sharing the interrupt vector with another piece of hardware, and that other hardware has a pending interrupt during xVM dom0 boot. e1000g driver is the first driver that is loaded, installs it''s interrupt handler, the vector is unmasked, and we immediatelly get interrupts on that vector, but they are not from e1000g. There''s no driver to clear the interrupt condition => the system hangs. My Tecra S1 had such an issue with the ehci and ipw drivers 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 hangs on reboot" http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812> I also > changed a Virtualization enabled/disabled setting which I did not see > before. That would seem to be important for xVM. :)Yes, probable helps with HVM domUs...> The NVidia driver is working just fine as well. > > Let me go through the matrix to see what we can figure out. There''s > either a bug or a release note to be created here I think.
Well, after messing about with it a bit, the best I can figure is the thing is not deterministic. I was trying to see if between the PnP and Vzn settings in the BIOS, I could get it to hang. It''s booting every time now. I did select HW defaults at one point from the BIOS, so maybe that changed something else I didn''t spot. virt-install prompts for HVM if Vzn is turned on -OR- off in the BIOS. I don''t know what''s going on there. More below... Juergen Keil wrote:> Matt Ingenthron wrote: > >> Garrett D''Amore wrote: >> >>> Matt Ingenthron wrote: >>> >>>> I''ll do a live upgrade to 82, and hopefully that''ll "just work" for >>>> me too. Thanks for looking into it Garrett. >>>> >>>> If you happen to think of any modifications you''ve made to the BIOS >>>> or anything like that, please let me know. I assume there are none >>>> and it''s just bugs that have been squashed post build 79b. >>>> >>> The one thing that I''ve done is change the BIOS PnP OS setting so that >>> BIOS configures all devices. That is necessary for the SDcard stuff. >>> I don''t think it makes a difference otherwise, but maybe its something >>> that Xen^WxVM is sensitive to. >>> >> Is that the "Device Configuration", where the choices are "Setup by OS" >> or "All"? Why can''t they just call it PnP? >> >> I just changed that, and sure enough, I''ve booted under xVM. >> > > Hmm, is there any other driver sharing the interrupt vector with > the e1000g device? Run mdb -k and have a look at the ::interrupts > output, when not booted under xVM and with the BIOS PnP setting > reverted: > > echo ::interrupts | mdb -k > echo ::interrupts -d | mdb -k >I get "NOTICE: IRQ xxx shared" during boot under xVM, even though I''m not booting verbose so there are some buggy things going on there.> > Maybe e1000g is sharing the interrupt vector with another piece of hardware, > and that other hardware has a pending interrupt during xVM dom0 boot. > e1000g driver is the first driver that is loaded, installs it''s > interrupt handler, the vector is unmasked, and we immediatelly get > interrupts on that vector, but they are not from e1000g. There''s > no driver to clear the interrupt condition => the system hangs. >Hmm, this could be. For those I remember the name of above, the interrupts under xVM show hci1394 and uhci/ehci. Does xVM handle such things well? How many of these get passed through to Solaris. If there are any pointers to a better technical doc on how xVM works with dom0''s, I''d appreciate it. Most of the docs I see are pretty high level-- it''s hard to see how much xVM does and how much is left for the dom0 to do. At this point, I''m tempted to leave it alone since it''s working, but I''m happy to investigate interrupts if the data is valuable to someone.> My Tecra S1 had such an issue with the ehci and ipw drivers > 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 hangs on > reboot" > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812 > > >> I also >> changed a Virtualization enabled/disabled setting which I did not see >> before. That would seem to be important for xVM. :) >> > > Yes, probable helps with HVM domUs... > > >> The NVidia driver is working just fine as well. >> >> Let me go through the matrix to see what we can figure out. There''s >> either a bug or a release note to be created here I think. >> > > >-- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
Matt.Ingenthron@Sun.COM
2008-Feb-09 01:46 UTC
achieving deterministic config with xVM (was: Dom0 issues: snv_79b and Tecra M9)
Hi Jurgen/Garrett/all, This may be specific to my hardware, but I cannot get things to behave reliably on my system. For instance, I''ve had situations under xVM dom0 where snv81 can see the e1000g interface with dladm show-dev, but not plumb it. Then one boot later, I can''t see it with dladm, but can plumb and up it with DHCP. All of this of course wreaks havoc on NWAM. This doesn''t bother me so much, but I''d like to be able to use my network every time I boot. :) I get similar behavior when running straight on the hardware too. I''ve generated a whole bevy of kmdb interrupt output. I''ve attached it. It does seem that the HW has quite a bit of sharing going on. How does xVM handle that? My ideal scenario would be booting under xVM with my e1000g working well enough to use NWAM. This is, after all, a laptop. Thanks in advance for any advice, - Matt p.s.: the noUSB refers to my unplugging the mouse-- something I did to see if I could make the darn thing deterministic, but it didn''t help Juergen Keil wrote:> Matt Ingenthron wrote: > >> Garrett D''Amore wrote: >> >>> Matt Ingenthron wrote: >>> >>>> I''ll do a live upgrade to 82, and hopefully that''ll "just work" for >>>> me too. Thanks for looking into it Garrett. >>>> >>>> If you happen to think of any modifications you''ve made to the BIOS >>>> or anything like that, please let me know. I assume there are none >>>> and it''s just bugs that have been squashed post build 79b. >>>> >>> The one thing that I''ve done is change the BIOS PnP OS setting so that >>> BIOS configures all devices. That is necessary for the SDcard stuff. >>> I don''t think it makes a difference otherwise, but maybe its something >>> that Xen^WxVM is sensitive to. >>> >> Is that the "Device Configuration", where the choices are "Setup by OS" >> or "All"? Why can''t they just call it PnP? >> >> I just changed that, and sure enough, I''ve booted under xVM. >> > > Hmm, is there any other driver sharing the interrupt vector with > the e1000g device? Run mdb -k and have a look at the ::interrupts > output, when not booted under xVM and with the BIOS PnP setting > reverted: > > echo ::interrupts | mdb -k > echo ::interrupts -d | mdb -k > > > Maybe e1000g is sharing the interrupt vector with another piece of hardware, > and that other hardware has a pending interrupt during xVM dom0 boot. > e1000g driver is the first driver that is loaded, installs it''s > interrupt handler, the vector is unmasked, and we immediatelly get > interrupts on that vector, but they are not from e1000g. There''s > no driver to clear the interrupt condition => the system hangs. > > My Tecra S1 had such an issue with the ehci and ipw drivers > 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 hangs on > reboot" > http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812 > > >> I also >> changed a Virtualization enabled/disabled setting which I did not see >> before. That would seem to be important for xVM. :) >> > > Yes, probable helps with HVM domUs... > > >> The NVidia driver is working just fine as well. >> >> Let me go through the matrix to see what we can figure out. There''s >> either a bug or a release note to be created here I think. >> > > > >-- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
I would be surprised if your e1000g ever worked when booting a Solaris dom0 The ::interrupts output you provided when it worked on metal indicate is uses MSI interrupts. MSI is currently not supported by Xen and thus not by Solaris dom0. See CR #6451665 Matt.Ingenthron@Sun.COM wrote:> Hi Jurgen/Garrett/all, > > This may be specific to my hardware, but I cannot get things to behave > reliably on my system. For instance, I''ve had situations under xVM dom0 > where snv81 can see the e1000g interface with dladm show-dev, but not > plumb it. Then one boot later, I can''t see it with dladm, but can plumb > and up it with DHCP. > > All of this of course wreaks havoc on NWAM. This doesn''t bother me so > much, but I''d like to be able to use my network every time I boot. :) > > I get similar behavior when running straight on the hardware too. > > I''ve generated a whole bevy of kmdb interrupt output. I''ve attached > it. It does seem that the HW has quite a bit of sharing going on. How > does xVM handle that? > > My ideal scenario would be booting under xVM with my e1000g working well > enough to use NWAM. This is, after all, a laptop. > > Thanks in advance for any advice, > > - Matt > > p.s.: the noUSB refers to my unplugging the mouse-- something I did to > see if I could make the darn thing deterministic, but it didn''t help > > Juergen Keil wrote: >> Matt Ingenthron wrote: >> >>> Garrett D''Amore wrote: >>> >>>> Matt Ingenthron wrote: >>>> >>>>> I''ll do a live upgrade to 82, and hopefully that''ll "just work" for >>>>> me too. Thanks for looking into it Garrett. >>>>> >>>>> If you happen to think of any modifications you''ve made to the BIOS >>>>> or anything like that, please let me know. I assume there are none >>>>> and it''s just bugs that have been squashed post build 79b. >>>>> >>>> The one thing that I''ve done is change the BIOS PnP OS setting so that >>>> BIOS configures all devices. That is necessary for the SDcard stuff. >>>> I don''t think it makes a difference otherwise, but maybe its something >>>> that Xen^WxVM is sensitive to. >>>> >>> Is that the "Device Configuration", where the choices are "Setup by OS" >>> or "All"? Why can''t they just call it PnP? >>> >>> I just changed that, and sure enough, I''ve booted under xVM. >>> >> >> Hmm, is there any other driver sharing the interrupt vector with >> the e1000g device? Run mdb -k and have a look at the ::interrupts >> output, when not booted under xVM and with the BIOS PnP setting >> reverted: >> >> echo ::interrupts | mdb -k >> echo ::interrupts -d | mdb -k >> >> >> Maybe e1000g is sharing the interrupt vector with another piece of hardware, >> and that other hardware has a pending interrupt during xVM dom0 boot. >> e1000g driver is the first driver that is loaded, installs it''s >> interrupt handler, the vector is unmasked, and we immediatelly get >> interrupts on that vector, but they are not from e1000g. There''s >> no driver to clear the interrupt condition => the system hangs. >> >> My Tecra S1 had such an issue with the ehci and ipw drivers >> 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 hangs on >> reboot" >> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812 >> >> >>> I also >>> changed a Virtualization enabled/disabled setting which I did not see >>> before. That would seem to be important for xVM. :) >>> >> >> Yes, probable helps with HVM domUs... >> >> >>> The NVidia driver is working just fine as well. >>> >>> Let me go through the matrix to see what we can figure out. There''s >>> either a bug or a release note to be created here I think. >>> >> >> >> >> > > > -- > Matt Ingenthron - Web Infrastructure Solutions Architect > Sun Microsystems, Inc. - Global Systems Practice > http://blogs.sun.com/mingenthron/ > email: matt.ingenthron@sun.com Phone: 310-242-6439 > > > ------------------------------------------------------------------------ > > IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) > 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 > 4 0xb0 12 ISA Edg Fixed 1 1 0x0/0x4 asy#0 > 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 > 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata#0 > 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, nvidia#0 > 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 > 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 > 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 > 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic#0 > 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394#0 > 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd#0 > 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 > 24 0x61 6 PCI Edg MSI 0 1 - e1000g#0 > 160 0xa0 0 Edg IPI all 0 - poke_cpu > 192 0xc0 13 Edg IPI all 1 - xc_serv > 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr > 209 0xd1 14 Edg IPI all 1 - cbe_fire > 210 0xd3 14 Edg IPI all 1 - cbe_fire > 240 0xe0 15 Edg IPI all 1 - xc_serv > 241 0xe1 15 Edg IPI all 1 - apic_error_intr > > > ------------------------------------------------------------------------ > > IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# ISR(s) > 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr > 4 0xb0 12 ISA Edg Fixed 1 1 0x0/0x4 asyintr > 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr > 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata_intr > 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, uhci_intr, > nv_intr > 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr > 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr > 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr > 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic_intr > 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394_isr > 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd_intr > 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr > 24 0x61 6 PCI Edg MSI 0 1 - e1000g_intr_pciexpress > 160 0xa0 0 Edg IPI all 0 - poke_cpu > 192 0xc0 13 Edg IPI all 1 - xc_serv > 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr > 209 0xd1 14 Edg IPI all 1 - cbe_fire > 210 0xd3 14 Edg IPI all 1 - cbe_fire > 240 0xe0 15 Edg IPI all 1 - xc_serv > 241 0xe1 15 Edg IPI all 1 - apic_error_intr > > > ------------------------------------------------------------------------ > > IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) > 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 > 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 > 14 0x90 25 5 ISA Edg Fixed 0 1 0x0/0xe ata#0 > 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, > nvidia#0 > 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 > 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 > 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 > 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic#0 > 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394#0 > 22 0xd8 26 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd#0 > 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 > 256 - I 15 - Edg ipi all - - xc_serv > 257 - I 13 - Edg ipi all - - xc_serv > 258 - I 11 - Edg ipi all - - poke_cpu > 259 - 1 15 - Edg virq all - - xen_debug_handler > 260 - T 14 - Edg virq all - - cbe_fire > 261 - I 14 - Edg ipi all - - cbe_fire > 262 - D 1 xpvd Edg device 0 - - evtchn#0 > 263 - 22 1 - Edg evtchn 1 - - xenbus_intr > > > ------------------------------------------------------------------------ > > IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# ISR(s) > 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr > 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr > 14 0x90 25 5 ISA Edg Fixed 0 1 0x0/0xe ata_intr > 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, > uhci_intr, nv_intr > 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr > 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr > 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr > 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic_intr > 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394_isr > 22 0xd8 26 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd_intr > 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr > 256 - I 15 - Edg ipi all - - xc_serv > 257 - I 13 - Edg ipi all - - xc_serv > 258 - I 11 - Edg ipi all - - poke_cpu > 259 - 1 15 - Edg virq all - - xen_debug_handler > 260 - T 14 - Edg virq all - - cbe_fire > 261 - I 14 - Edg ipi all - - cbe_fire > 262 - D 1 xpvd Edg device 0 - - evtchn_device_upcall > 263 - 22 1 - Edg evtchn 1 - - xenbus_intr > > > ------------------------------------------------------------------------ > > IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) > 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 > 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 > 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata#0 > 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, nvidia#0 > 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 > 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 > 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 > 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic#0 > 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394#0 > 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd#0 > 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 > 160 0xa0 0 Edg IPI all 0 - poke_cpu > 192 0xc0 13 Edg IPI all 1 - xc_serv > 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr > 209 0xd1 14 Edg IPI all 1 - cbe_fire > 210 0xd3 14 Edg IPI all 1 - cbe_fire > 240 0xe0 15 Edg IPI all 1 - xc_serv > 241 0xe1 15 Edg IPI all 1 - apic_error_intr > > > ------------------------------------------------------------------------ > > IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# ISR(s) > 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr > 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr > 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata_intr > 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, uhci_intr, > nv_intr > 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr > 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr > 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr > 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic_intr > 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394_isr > 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd_intr > 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr > 160 0xa0 0 Edg IPI all 0 - poke_cpu > 192 0xc0 13 Edg IPI all 1 - xc_serv > 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr > 209 0xd1 14 Edg IPI all 1 - cbe_fire > 210 0xd3 14 Edg IPI all 1 - cbe_fire > 240 0xe0 15 Edg IPI all 1 - xc_serv > 241 0xe1 15 Edg IPI all 1 - apic_error_intr > > > ------------------------------------------------------------------------ > > IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) > 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 > 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 > 14 0x90 26 5 ISA Edg Fixed 0 1 0x0/0xe ata#0 > 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, > nvidia#0 > 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 > 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 > 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 > 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic#0 > 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394#0 > 22 0xd8 22 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd#0 > 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 > 256 - I 15 - Edg ipi all - - xc_serv > 257 - I 13 - Edg ipi all - - xc_serv > 258 - I 11 - Edg ipi all - - poke_cpu > 259 - 1 15 - Edg virq all - - xen_debug_handler > 260 - T 14 - Edg virq all - - cbe_fire > 261 - I 14 - Edg ipi all - - cbe_fire > 263 - 25 1 - Edg evtchn 1 - - xenbus_intr > 264 - D 1 xpvd Edg device 0 - - evtchn#0 > > > ------------------------------------------------------------------------ > > IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# ISR(s) > 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr > 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr > 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr > 14 0x90 26 5 ISA Edg Fixed 0 1 0x0/0xe ata_intr > 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, > uhci_intr, nv_intr > 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr > 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr > 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr > 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic_intr > 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394_isr > 22 0xd8 22 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd_intr > 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr > 256 - I 15 - Edg ipi all - - xc_serv > 257 - I 13 - Edg ipi all - - xc_serv > 258 - I 11 - Edg ipi all - - poke_cpu > 259 - 1 15 - Edg virq all - - xen_debug_handler > 260 - T 14 - Edg virq all - - cbe_fire > 261 - I 14 - Edg ipi all - - cbe_fire > 263 - 25 1 - Edg evtchn 1 - - xenbus_intr > 264 - D 1 xpvd Edg device 0 - - evtchn_device_upcall > > > ------------------------------------------------------------------------ > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org
Thanks Stuart, Stuart Maybee wrote:> > I would be surprised if your e1000g ever worked when booting a Solaris > dom0 The ::interrupts output you provided when it worked on metal > indicate is uses MSI interrupts. MSI is currently not supported by > Xen and thus not by Solaris dom0. See CR #6451665Perhaps it was too many iterations-- early on I wasn''t tracking them carefully. I was, at least at one point, able to plumb the interface under Solaris dom0. Perhaps it just saw the device, but couldn''t attach the driver though. Thanks again for the pointer. I''ll wait until that CR is addressed. - Matt> > Matt.Ingenthron@Sun.COM wrote: >> Hi Jurgen/Garrett/all, >> >> This may be specific to my hardware, but I cannot get things to >> behave reliably on my system. For instance, I''ve had situations >> under xVM dom0 where snv81 can see the e1000g interface with dladm >> show-dev, but not plumb it. Then one boot later, I can''t see it with >> dladm, but can plumb and up it with DHCP. >> >> All of this of course wreaks havoc on NWAM. This doesn''t bother me >> so much, but I''d like to be able to use my network every time I >> boot. :) >> >> I get similar behavior when running straight on the hardware too. >> I''ve generated a whole bevy of kmdb interrupt output. I''ve attached >> it. It does seem that the HW has quite a bit of sharing going on. >> How does xVM handle that? >> >> My ideal scenario would be booting under xVM with my e1000g working >> well enough to use NWAM. This is, after all, a laptop. >> >> Thanks in advance for any advice, >> >> - Matt >> >> p.s.: the noUSB refers to my unplugging the mouse-- something I did >> to see if I could make the darn thing deterministic, but it didn''t help >> >> Juergen Keil wrote: >>> Matt Ingenthron wrote: >>> >>>> Garrett D''Amore wrote: >>>> >>>>> Matt Ingenthron wrote: >>>>> >>>>>> I''ll do a live upgrade to 82, and hopefully that''ll "just work" >>>>>> for me too. Thanks for looking into it Garrett. >>>>>> >>>>>> If you happen to think of any modifications you''ve made to the >>>>>> BIOS or anything like that, please let me know. I assume there >>>>>> are none and it''s just bugs that have been squashed post build 79b. >>>>>> >>>>> The one thing that I''ve done is change the BIOS PnP OS setting so >>>>> that BIOS configures all devices. That is necessary for the >>>>> SDcard stuff. I don''t think it makes a difference otherwise, but >>>>> maybe its something that Xen^WxVM is sensitive to. >>>>> >>>> Is that the "Device Configuration", where the choices are "Setup by >>>> OS" or "All"? Why can''t they just call it PnP? >>>> >>>> I just changed that, and sure enough, I''ve booted under xVM. >>>> >>> >>> Hmm, is there any other driver sharing the interrupt vector with >>> the e1000g device? Run mdb -k and have a look at the ::interrupts >>> output, when not booted under xVM and with the BIOS PnP setting >>> reverted: >>> >>> echo ::interrupts | mdb -k >>> echo ::interrupts -d | mdb -k >>> >>> >>> Maybe e1000g is sharing the interrupt vector with another piece of >>> hardware, >>> and that other hardware has a pending interrupt during xVM dom0 >>> boot. e1000g driver is the first driver that is loaded, installs >>> it''s interrupt handler, the vector is unmasked, and we immediatelly get >>> interrupts on that vector, but they are not from e1000g. There''s >>> no driver to clear the interrupt condition => the system hangs. >>> >>> My Tecra S1 had such an issue with the ehci and ipw drivers >>> 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 >>> hangs on reboot" >>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812 >>> >>> >>>> I also changed a Virtualization enabled/disabled setting which I >>>> did not see before. That would seem to be important for xVM. :) >>>> >>> >>> Yes, probable helps with HVM domUs... >>> >>> >>>> The NVidia driver is working just fine as well. >>>> >>>> Let me go through the matrix to see what we can figure out. >>>> There''s either a bug or a release note to be created here I think. >>>> >>> >>> >>> >>> >> >> >> -- >> Matt Ingenthron - Web Infrastructure Solutions Architect >> Sun Microsystems, Inc. - Global Systems Practice >> http://blogs.sun.com/mingenthron/ >> email: matt.ingenthron@sun.com Phone: 310-242-6439 >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) >> 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 >> 4 0xb0 12 ISA Edg Fixed 1 1 0x0/0x4 asy#0 >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata#0 >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, >> uhci#0, nvidia#0 >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic#0 >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394#0 >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd#0 >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 >> 24 0x61 6 PCI Edg MSI 0 1 - e1000g#0 >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - >> kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# ISR(s) 1 0x41 >> 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 4 0xb0 12 ISA Edg Fixed 1 1 0x0/0x4 asyintr >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata_intr >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, >> uhci_intr, nv_intr >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic_intr >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394_isr >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd_intr >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr >> 24 0x61 6 PCI Edg MSI 0 1 - >> e1000g_intr_pciexpress >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - >> kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# Driver >> Name(s) 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 >> i8042#0 >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 >> acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x90 25 5 ISA Edg Fixed 0 1 0x0/0xe ata#0 >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 >> pcie_pci#2, uhci#0, nvidia#0 >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, >> ehci#0 >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, >> ahci#0 >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic#0 >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394#0 >> 22 0xd8 26 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd#0 >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, >> ehci#1 >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - >> xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 262 - D 1 xpvd Edg device 0 - - evtchn#0 >> 263 - 22 1 - Edg evtchn 1 - - xenbus_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# ISR(s) >> 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 >> acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x90 25 5 ISA Edg Fixed 0 1 0x0/0xe ata_intr >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 >> pepb_intx_intr, uhci_intr, nv_intr >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, >> ehci_intr >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, >> ahci_intr >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic_intr >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394_isr >> 22 0xd8 26 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd_intr >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, >> ehci_intr >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - >> xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 262 - D 1 xpvd Edg device 0 - - >> evtchn_device_upcall >> 263 - 22 1 - Edg evtchn 1 - - xenbus_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) >> 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata#0 >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, >> uhci#0, nvidia#0 >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic#0 >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394#0 >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd#0 >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - >> kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# ISR(s) 1 0x41 >> 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata_intr >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, >> uhci_intr, nv_intr >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic_intr >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394_isr >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd_intr >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - >> kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# Driver >> Name(s) 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 >> i8042#0 >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 >> acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x90 26 5 ISA Edg Fixed 0 1 0x0/0xe ata#0 >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 >> pcie_pci#2, uhci#0, nvidia#0 >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, >> ehci#0 >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, >> ahci#0 >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic#0 >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394#0 >> 22 0xd8 22 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd#0 >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, >> ehci#1 >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - >> xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 263 - 25 1 - Edg evtchn 1 - - xenbus_intr >> 264 - D 1 xpvd Edg device 0 - - evtchn#0 >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# ISR(s) >> 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 >> acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x90 26 5 ISA Edg Fixed 0 1 0x0/0xe ata_intr >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 >> pepb_intx_intr, uhci_intr, nv_intr >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, >> ehci_intr >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, >> ahci_intr >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic_intr >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394_isr >> 22 0xd8 22 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd_intr >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, >> ehci_intr >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - >> xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 263 - 25 1 - Edg evtchn 1 - - xenbus_intr >> 264 - D 1 xpvd Edg device 0 - - >> evtchn_device_upcall >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> xen-discuss mailing list >> xen-discuss@opensolaris.org >-- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
Garrett D''Amore wrote:> Matt.Ingenthron@Sun.COM wrote: >> Well, after messing about with it a bit, the best I can figure is the >> thing is not deterministic. I was trying to see if between the PnP >> and Vzn settings in the BIOS, I could get it to hang. It''s booting >> every time now. I did select HW defaults at one point from the BIOS, >> so maybe that changed something else I didn''t spot. > > The switching back and forth in PnP is *meaningless*. Because, there > is some bug in the Toshiba BIOS were changing the value doesn''t really > take effect unless a full power down is done. This includes removing > the battery physically from the system. The Toshiba M9 BIOS is kinda > funky here, and I''ve had a heard time qualifying this properly, but > lets just say that after switching to All, then back to PnP OS, the > problem I had with my particular device''s BIOS settings went away.Well, this may explain what I''m seeing then. I initially had hangs during xVM dom0 boot, but ever since messing with the BIOS it boots. One other Sun colleague, George Asaad, still has one in a hanging state. His is hanging like mine was, loading that ether_mac module. Now that I know xVM doesn''t support MSI (thanks to Stuart''s email), it''s probably trying to initialize the device and never hearing back. Why it should go further after messing with the BIOS, I don''t know. Maybe the device is further initialized to some point? We''re at the edges of my understanding of how such things work...> Until I allowed the battery to drain fully. Then mysteriously it > reappeared, causing me about 3 days of painful debugging trying to > find a problem I thought was in my driver (SDcard), but which > ultimately boiled down to this weird BIOS thing. Setting the value > back to "All" cleared my problems up.Had you raised this issue with Toshiba? I think I know some folks who might be able to do so if you''ve not yet, but I don''t want to cause any unnecessary noise (unless you think that''s warranted as well). - Matt p.s.: I couldn''t get my SDCard to work in either case, but I didn''t mess with it that much. I probably missed something simple.> > -- Garrett >> >> virt-install prompts for HVM if Vzn is turned on -OR- off in the >> BIOS. I don''t know what''s going on there. >> >> More below... >> >> Juergen Keil wrote: >>> Matt Ingenthron wrote: >>> >>>> Garrett D''Amore wrote: >>>> >>>>> Matt Ingenthron wrote: >>>>> >>>>>> I''ll do a live upgrade to 82, and hopefully that''ll "just work" >>>>>> for me too. Thanks for looking into it Garrett. >>>>>> >>>>>> If you happen to think of any modifications you''ve made to the >>>>>> BIOS or anything like that, please let me know. I assume there >>>>>> are none and it''s just bugs that have been squashed post build 79b. >>>>>> >>>>> The one thing that I''ve done is change the BIOS PnP OS setting so >>>>> that BIOS configures all devices. That is necessary for the >>>>> SDcard stuff. I don''t think it makes a difference otherwise, but >>>>> maybe its something that Xen^WxVM is sensitive to. >>>>> >>>> Is that the "Device Configuration", where the choices are "Setup by >>>> OS" or "All"? Why can''t they just call it PnP? >>>> >>>> I just changed that, and sure enough, I''ve booted under xVM. >>>> >>> >>> Hmm, is there any other driver sharing the interrupt vector with >>> the e1000g device? Run mdb -k and have a look at the ::interrupts >>> output, when not booted under xVM and with the BIOS PnP setting >>> reverted: >>> >>> echo ::interrupts | mdb -k >>> echo ::interrupts -d | mdb -k >>> >> >> I get "NOTICE: IRQ xxx shared" during boot under xVM, even though I''m >> not booting verbose so there are some buggy things going on there. >>> >>> Maybe e1000g is sharing the interrupt vector with another piece of >>> hardware, >>> and that other hardware has a pending interrupt during xVM dom0 >>> boot. e1000g driver is the first driver that is loaded, installs >>> it''s interrupt handler, the vector is unmasked, and we immediatelly get >>> interrupts on that vector, but they are not from e1000g. There''s >>> no driver to clear the interrupt condition => the system hangs. >>> >> >> Hmm, this could be. >> >> For those I remember the name of above, the interrupts under xVM show >> hci1394 and uhci/ehci. >> >> Does xVM handle such things well? How many of these get passed >> through to Solaris. If there are any pointers to a better technical >> doc on how xVM works with dom0''s, I''d appreciate it. Most of the >> docs I see are pretty high level-- it''s hard to see how much xVM does >> and how much is left for the dom0 to do. >> >> At this point, I''m tempted to leave it alone since it''s working, but >> I''m happy to investigate interrupts if the data is valuable to someone. >> >>> My Tecra S1 had such an issue with the ehci and ipw drivers >>> 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 >>> hangs on reboot" >>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812 >>> >>> >>>> I also changed a Virtualization enabled/disabled setting which I >>>> did not see before. That would seem to be important for xVM. :) >>>> >>> >>> Yes, probable helps with HVM domUs... >>> >>> >>>> The NVidia driver is working just fine as well. >>>> >>>> Let me go through the matrix to see what we can figure out. >>>> There''s either a bug or a release note to be created here I think. >>>> >>> >>> >>> >> >> >> >-- Matt Ingenthron - Web Infrastructure Solutions Architect Sun Microsystems, Inc. - Global Systems Practice http://blogs.sun.com/mingenthron/ email: matt.ingenthron@sun.com Phone: 310-242-6439
Actually a quick look at the driver shows that it does attempt to fall back to non-msi interrupts if MSI is not available, so this may not actually be the problem. Stuart Maybee wrote:> I would be surprised if your e1000g ever worked when booting a Solaris > dom0 The ::interrupts output you provided when it worked on metal > indicate is uses MSI interrupts. MSI is currently not supported by Xen > and thus not by Solaris dom0. See CR #6451665 > > Matt.Ingenthron@Sun.COM wrote: >> Hi Jurgen/Garrett/all, >> >> This may be specific to my hardware, but I cannot get things to behave >> reliably on my system. For instance, I''ve had situations under xVM dom0 >> where snv81 can see the e1000g interface with dladm show-dev, but not >> plumb it. Then one boot later, I can''t see it with dladm, but can plumb >> and up it with DHCP. >> >> All of this of course wreaks havoc on NWAM. This doesn''t bother me so >> much, but I''d like to be able to use my network every time I boot. :) >> >> I get similar behavior when running straight on the hardware too. >> >> I''ve generated a whole bevy of kmdb interrupt output. I''ve attached >> it. It does seem that the HW has quite a bit of sharing going on. How >> does xVM handle that? >> >> My ideal scenario would be booting under xVM with my e1000g working well >> enough to use NWAM. This is, after all, a laptop. >> >> Thanks in advance for any advice, >> >> - Matt >> >> p.s.: the noUSB refers to my unplugging the mouse-- something I did to >> see if I could make the darn thing deterministic, but it didn''t help >> >> Juergen Keil wrote: >>> Matt Ingenthron wrote: >>> >>>> Garrett D''Amore wrote: >>>> >>>>> Matt Ingenthron wrote: >>>>> >>>>>> I''ll do a live upgrade to 82, and hopefully that''ll "just work" for >>>>>> me too. Thanks for looking into it Garrett. >>>>>> >>>>>> If you happen to think of any modifications you''ve made to the BIOS >>>>>> or anything like that, please let me know. I assume there are none >>>>>> and it''s just bugs that have been squashed post build 79b. >>>>>> >>>>> The one thing that I''ve done is change the BIOS PnP OS setting so that >>>>> BIOS configures all devices. That is necessary for the SDcard stuff. >>>>> I don''t think it makes a difference otherwise, but maybe its something >>>>> that Xen^WxVM is sensitive to. >>>>> >>>> Is that the "Device Configuration", where the choices are "Setup by OS" >>>> or "All"? Why can''t they just call it PnP? >>>> >>>> I just changed that, and sure enough, I''ve booted under xVM. >>>> >>> Hmm, is there any other driver sharing the interrupt vector with >>> the e1000g device? Run mdb -k and have a look at the ::interrupts >>> output, when not booted under xVM and with the BIOS PnP setting >>> reverted: >>> >>> echo ::interrupts | mdb -k >>> echo ::interrupts -d | mdb -k >>> >>> >>> Maybe e1000g is sharing the interrupt vector with another piece of hardware, >>> and that other hardware has a pending interrupt during xVM dom0 boot. >>> e1000g driver is the first driver that is loaded, installs it''s >>> interrupt handler, the vector is unmasked, and we immediatelly get >>> interrupts on that vector, but they are not from e1000g. There''s >>> no driver to clear the interrupt condition => the system hangs. >>> >>> My Tecra S1 had such an issue with the ehci and ipw drivers >>> 6353812 "ipw driver must disable interrupts at shutdown; tecra s1 hangs on >>> reboot" >>> http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6353812 >>> >>> >>>> I also >>>> changed a Virtualization enabled/disabled setting which I did not see >>>> before. That would seem to be important for xVM. :) >>>> >>> Yes, probable helps with HVM domUs... >>> >>> >>>> The NVidia driver is working just fine as well. >>>> >>>> Let me go through the matrix to see what we can figure out. There''s >>>> either a bug or a release note to be created here I think. >>>> >>> >>> >>> >> >> -- >> Matt Ingenthron - Web Infrastructure Solutions Architect >> Sun Microsystems, Inc. - Global Systems Practice >> http://blogs.sun.com/mingenthron/ >> email: matt.ingenthron@sun.com Phone: 310-242-6439 >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) >> 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 >> 4 0xb0 12 ISA Edg Fixed 1 1 0x0/0x4 asy#0 >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata#0 >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, nvidia#0 >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic#0 >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394#0 >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd#0 >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 >> 24 0x61 6 PCI Edg MSI 0 1 - e1000g#0 >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# ISR(s) >> 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 4 0xb0 12 ISA Edg Fixed 1 1 0x0/0x4 asyintr >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata_intr >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, uhci_intr, >> nv_intr >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic_intr >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394_isr >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd_intr >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr >> 24 0x61 6 PCI Edg MSI 0 1 - e1000g_intr_pciexpress >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) >> 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x90 25 5 ISA Edg Fixed 0 1 0x0/0xe ata#0 >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, >> nvidia#0 >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic#0 >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394#0 >> 22 0xd8 26 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd#0 >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 262 - D 1 xpvd Edg device 0 - - evtchn#0 >> 263 - 22 1 - Edg evtchn 1 - - xenbus_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# ISR(s) >> 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x90 25 5 ISA Edg Fixed 0 1 0x0/0xe ata_intr >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, >> uhci_intr, nv_intr >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic_intr >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394_isr >> 22 0xd8 26 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd_intr >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 262 - D 1 xpvd Edg device 0 - - evtchn_device_upcall >> 263 - 22 1 - Edg evtchn 1 - - xenbus_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) >> 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata#0 >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, nvidia#0 >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic#0 >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394#0 >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd#0 >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect IPL Bus Trg Type CPU Share APIC/INT# ISR(s) >> 1 0x41 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 9 0x81 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x42 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x43 5 ISA Edg Fixed 1 1 0x0/0xe ata_intr >> 16 0x82 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, uhci_intr, >> nv_intr >> 17 0x22 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0x20 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr >> 19 0x40 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr >> 20 0x60 6 PCI Lvl Fixed 0 1 0x0/0x14 pcic_intr >> 21 0x83 9 PCI Lvl Fixed 1 1 0x0/0x15 hci1394_isr >> 22 0x84 9 PCIe Lvl Fixed 0 1 0x0/0x16 audiohd_intr >> 23 0x21 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr >> 160 0xa0 0 Edg IPI all 0 - poke_cpu >> 192 0xc0 13 Edg IPI all 1 - xc_serv >> 208 0xd0 14 Edg IPI all 1 - kcpc_hw_overflow_intr >> 209 0xd1 14 Edg IPI all 1 - cbe_fire >> 210 0xd3 14 Edg IPI all 1 - cbe_fire >> 240 0xe0 15 Edg IPI all 1 - xc_serv >> 241 0xe1 15 Edg IPI all 1 - apic_error_intr >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# Driver Name(s) >> 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042#0 >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042#0 >> 14 0x90 26 5 ISA Edg Fixed 0 1 0x0/0xe ata#0 >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pcie_pci#2, uhci#0, >> nvidia#0 >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci#1 >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci#4, ehci#0 >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci#3, ahci#0 >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic#0 >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394#0 >> 22 0xd8 22 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd#0 >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci#2, ehci#1 >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 263 - 25 1 - Edg evtchn 1 - - xenbus_intr >> 264 - D 1 xpvd Edg device 0 - - evtchn#0 >> >> >> ------------------------------------------------------------------------ >> >> IRQ Vect Evtchn IPL Bus Trg Type CPU Share APIC/INT# ISR(s) >> 1 0x28 10 5 ISA Edg Fixed 0 1 0x0/0x1 i8042_intr >> 9 0x60 5 9 PCI Lvl Fixed 1 1 0x0/0x9 acpi_wrapper_isr >> 12 0x78 11 5 ISA Edg Fixed 0 1 0x0/0xc i8042_intr >> 14 0x90 26 5 ISA Edg Fixed 0 1 0x0/0xe ata_intr >> 16 0xa8 9 9 PCIe Lvl Fixed 1 3 0x0/0x10 pepb_intx_intr, >> uhci_intr, nv_intr >> 17 0xc0 14 1 PCI Lvl Fixed 0 1 0x0/0x11 uhci_intr >> 18 0xb0 12 1 PCI Lvl Fixed 1 2 0x0/0x12 uhci_intr, ehci_intr >> 19 0xa0 8 5 PCI Lvl Fixed 0 2 0x0/0x13 uhci_intr, ahci_intr >> 20 0xd0 21 6 PCI Lvl Fixed 1 1 0x0/0x14 pcic_intr >> 21 0xc8 20 9 PCI Lvl Fixed 0 1 0x0/0x15 hci1394_isr >> 22 0xd8 22 9 PCIe Lvl Fixed 1 1 0x0/0x16 audiohd_intr >> 23 0xb8 13 1 PCI Lvl Fixed 1 2 0x0/0x17 uhci_intr, ehci_intr >> 256 - I 15 - Edg ipi all - - xc_serv >> 257 - I 13 - Edg ipi all - - xc_serv >> 258 - I 11 - Edg ipi all - - poke_cpu >> 259 - 1 15 - Edg virq all - - xen_debug_handler >> 260 - T 14 - Edg virq all - - cbe_fire >> 261 - I 14 - Edg ipi all - - cbe_fire >> 263 - 25 1 - Edg evtchn 1 - - xenbus_intr >> 264 - D 1 xpvd Edg device 0 - - evtchn_device_upcall >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> xen-discuss mailing list >> xen-discuss@opensolaris.org > > _______________________________________________ > xen-discuss mailing list > xen-discuss@opensolaris.org