xen discuss - Mar 2008 - Build 85 reboots when choosing xVM in grub menu (blade x6220)

Hi, Am trying to test xVM on a blade 6220. I installed the build 85 
fresh from the dvd, no patches, nothing else.When I boot Solaris there 
is no problem but if I try the xVM option it goes all the way to the 
login screen and then it reboots.
Am I missing something ? .

Thanks
Pedro

Try booting xVM with kmdb. At the xVM grub line you''ll need to edit the
module$
line that contains unix to add " -k" to the end.

Pedro Gomez wrote:
> Hi, Am trying to test xVM on a blade 6220. I installed the build 85 
> fresh from the dvd, no patches, nothing else.When I boot Solaris there 
> is no problem but if I try the xVM option it goes all the way to the 
> login screen and then it reboots.
> Am I missing something ? .
> 
> Thanks
> Pedro
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
-- 

-----------------------------------------------------
Russ Blaine | Solaris Kernel | russell.blaine@sun.com

Interesting, I had (still have) similar problems:
when I upgraded from nv_77 to 85, I noticed that glxgears showed some 3,xyz FPS
(yes, three point something!) when booted into xVM...
Okay, I "upgraded" the nvidia driver (same version on b85 dvd and on
nvidia''s site, but who knows...), then my machine began to behave
exactly as described above.
Then I upgraded the nvidia driver to the current beta, same issues.
Then I reverted back to the opensolaris-dvd nvidia drivers, and now it works,
but under xVM the same poor performance.
I forgot to add, that without xVM I get about 3350-3400 FPS with glxgears.
So based on the above findings, I think that there might be something with X, or
the nvidia drivers, but I''ll give a go to the kmdb-option, and post it
here, just to give more feedback.
My config is based on: C2D E6600, 4*512 DDR, Geforce 7600 GS, Asrock i865-based
motherboard.
Interesting is though that I have an other pc, an Asus barebone with an
i965-based motherboard, Asus 7600GS vga, 4*1GB RAM, and a C2Q 6600 cpcu, and no
issues there. Albeit the configs are somewhat similar...
 
 
This message posted from opensolaris.org

Attila Nagy wrote:
> Interesting, I had (still have) similar problems:
> when I upgraded from nv_77 to 85, I noticed that
 > glxgears showed some 3,xyz FPS (yes, three point
 > something!) when booted into xVM...

That sounds like OpenGL in S/W performance to me.
Can you look through /var/log/Xorg.log and see if
you see any error messages?

I don''t see your problem.

: alpha[1]#; /usr/X11/demo/glxgears -display :0
27273 frames in 5.0 seconds = 5454.424 FPS
27269 frames in 5.0 seconds = 5453.799 FPS
27263 frames in 5.0 seconds = 5452.488 FPS
27290 frames in 5.0 seconds = 5457.977 FPS
^C
: alpha[1]#; uname -a
SunOS alpha 5.11 snv_85 i86pc i386 i86xpv
: alpha[1]#;

> Okay, I "upgraded" the nvidia driver (same version
 > on b85 dvd and on nvidia''s site, but who knows...),
 > then my machine began to behave exactly as described
above.
> Then I upgraded the nvidia driver to the current beta, same issues.
They are not the same driver :-) The driver on the
nVidia site needs to work on s10, and all versions
of Nevada.

The driver included with OpenSolaris is modified
to know about Xen and will not work on older
versions of Solaris.

You should not see the same performance running on
metal vs running on Xen (running on Xen should be
slower).  There will always be performance lost
going through the hypervisor and we don''t use the
PAT in the i86xpv kernel (on top of Xen) today.


> Then I reverted back to the opensolaris-dvd nvidia drivers, 
 > and now it works, but under xVM the same poor
performance.
> I forgot to add, that without xVM I get about 3350-3400 FPS
 > with glxgears.
> So based on the above findings, I think that there might 
 > be something with X, or the nvidia drivers, but I''ll give
 > a go to the kmdb-option, and post it here, just to give
 > more feedback.
> My config is based on: C2D E6600, 4*512 DDR, Geforce
 > 7600 GS, Asrock i865-based motherboard.
> Interesting is though that I have an other pc, an Asus
 > barebone with an i965-based motherboard, Asus 7600GS vga,
 > 4*1GB RAM, and a C2Q 6600 cpcu, and no issues there.
 > Albeit the configs are somewhat similar...
> 
I have a GIGABYTE GA-G33M-DS2R, with a Core 2 Duo
E6400 (Conroe), 4G of memory, and an EVGA
256-P2-N549-TR GeForce 7600GS.




MRJ

Hi,

thanks for the quick response!
> 
> That sounds like OpenGL in S/W performance to me.
> Can you look through /var/log/Xorg.log and see if
> you see any error messages?
No errors, see at the end! (&other info too!)
> 
> I don't see your problem.
> 
> : alpha[1]#; /usr/X11/demo/glxgears -display :0
> 27273 frames in 5.0 seconds = 5454.424 FPS
> 27269 frames in 5.0 seconds = 5453.799 FPS
> 27263 frames in 5.0 seconds = 5452.488 FPS
> 27290 frames in 5.0 seconds = 5457.977 FPS
> ^C
> : alpha[1]#; uname -a
> SunOS alpha 5.11 snv_85 i86pc i386 i86xpv
> : alpha[1]#;
> 
> 
[...]
> 
> They are not the same driver :-)
I thought it! :) I should have been more specific:
the Version string in both drivers are the same, according to the "NVidia
Xserver Settings Utility".
> The driver on the
> nVidia site needs to work on s10, and all versions
> of Nevada.
> 
> The driver included with OpenSolaris is modified
> to know about Xen and will not work on older
> versions of Solaris.
> 
> You should not see the same performance running on
> metal vs running on Xen (running on Xen should be
> slower).  There will always be performance lost
> going through the hypervisor and we don't use the
> PAT in the i86xpv kernel (on top of Xen) today.
> 
Aha, I thought something similar.
> 
> 
[...]
>  
> I have a GIGABYTE GA-G33M-DS2R, with a Core 2 Duo
> E6400 (Conroe), 4G of memory, and an EVGA
> 256-P2-N549-TR GeForce 7600GS.
 :)
> 
> 
> MRJ
> 
> 
I got reboots only, if I did _not_ use the drivers from the Nevada dvd, but the
drivers from Nvidia's site; that might be because what you wrote about the
driver's "Xen-awareness".


Now a few outputs: (see the degraded performance under i86pv!)

# uname -a
SunOS home 5.11 snv_85 i86pc i386 i86pc
# /usr/X11/demo/glxgears 
33325 frames in 5.0 seconds = 6664.886 FPS
33945 frames in 5.0 seconds = 6788.911 FPS
33944 frames in 5.0 seconds = 6788.621 FPS
32085 frames in 5.0 seconds = 6388.705 FPS
X connection to :0.0 broken (explicit kill or server shutdown).

(a few minutes, and a reboot later; from now on all outputs were made under
i86pv!)

# uname -a                         
SunOS home 5.11 snv_85 i86pc i386 i86xpv
# /usr/X11/demo/glxgears                           
19 frames in 5.0 seconds =  3.766 FPS
18 frames in 5.0 seconds =  3.582 FPS
18 frames in 5.0 seconds =  3.589 FPS
18 frames in 5.0 seconds =  3.582 FPS

# pwd
/var/log
# l | grep Xorg.0.log                              
-rw-r--r--    1 root     root        25818 ápr.  1 14:02 Xorg.0.log        
(<-- i86pv)
-rw-r--r--    1 root     root        25874 ápr.  1 13:58 Xorg.0.log.old  
(<-- i86pc)
# grep EE Xorg.0.log
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(II) Loading extension MIT-SCREEN-SAVER
# grep WW Xorg.0.log                               
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(WW) The directory "/usr/X11R6/lib/X11/fonts/Type1/sun/" does not
exist.
(WW) Ignoring unrecognized extension "TwinView"
# grep EE Xorg.0.log.old                           
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(II) Loading extension MIT-SCREEN-SAVER
# grep WW Xorg.0.log.old                           
	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(WW) The directory "/usr/X11R6/lib/X11/fonts/Type1/sun/" does not
exist.
(WW) Ignoring unrecognized extension "TwinView"
# /usr/X11/bin/glxinfo 
name of display: :0.0
display: :0  screen: 0
direct rendering: Yes
server glx vendor string: NVIDIA Corporation
server glx version string: 1.4
server glx extensions:
[...]
client glx vendor string: NVIDIA Corporation
client glx version string: 1.4
client glx extensions:
[...]
GLX version: 1.3
GLX extensions:
[...]
OpenGL vendor string: NVIDIA Corporation
OpenGL renderer string: GeForce 7600 GS/AGP/SSE2
OpenGL version string: 2.1.2 NVIDIA 169.12
 
 
This message posted from opensolaris.org
_______________________________________________
xen-discuss mailing list
xen-discuss@opensolaris.org

Attila Nagy wrote:
> Hi,
> 
> I got reboots only, if I did _not_ use the drivers 
 > from the Nevada dvd, but the drivers from Nvidia's
 > site; that might be because what you wrote about the
 > driver's "Xen-awareness".

Yes, the drivers from Nvidia's site will be doing all
sorts of bad things on top of Xen. Physical addresses
are virtual :-) in i86xpv.  They need to be translated
to machine addresses.  There are other things too
which the nvidia driver and associated code has to
do differently on i86xpv.



> 
> Now a few outputs: (see the degraded performance under i86pv!)
> 
> # uname -a
> SunOS home 5.11 snv_85 i86pc i386 i86pc
> # /usr/X11/demo/glxgears 
> 33325 frames in 5.0 seconds = 6664.886 FPS
> 33945 frames in 5.0 seconds = 6788.911 FPS
> 33944 frames in 5.0 seconds = 6788.621 FPS
> 32085 frames in 5.0 seconds = 6388.705 FPS
> X connection to :0.0 broken (explicit kill or server shutdown).
> 
> (a few minutes, and a reboot later; from now on all outputs were made under
i86pv!)
> 
> # uname -a                         
> SunOS home 5.11 snv_85 i86pc i386 i86xpv
> # /usr/X11/demo/glxgears                           
> 19 frames in 5.0 seconds =  3.766 FPS
> 18 frames in 5.0 seconds =  3.582 FPS
> 18 frames in 5.0 seconds =  3.589 FPS
> 18 frames in 5.0 seconds =  3.582 FPS
There performance shouldn't be that bad :-) I would
expect you would see ~ 3 1/2 FPS with the nv driver
which is S/W only.

John M, any ideas on how to debug this further?




MRJ



> # pwd
> /var/log
> # l | grep Xorg.0.log                              
> -rw-r--r--    1 root     root        25818 ápr.  1 14:02 Xorg.0.log        
(<-- i86pv)
> -rw-r--r--    1 root     root        25874 ápr.  1 13:58 Xorg.0.log.old  
(<-- i86pc)
> # grep EE Xorg.0.log
> 	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
> (II) Loading extension MIT-SCREEN-SAVER
> # grep WW Xorg.0.log                               
> 	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
> (WW) The directory "/usr/X11R6/lib/X11/fonts/Type1/sun/" does not
exist.
> (WW) Ignoring unrecognized extension "TwinView"
> # grep EE Xorg.0.log.old                           
> 	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
> (II) Loading extension MIT-SCREEN-SAVER
> # grep WW Xorg.0.log.old                           
> 	(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
> (WW) The directory "/usr/X11R6/lib/X11/fonts/Type1/sun/" does not
exist.
> (WW) Ignoring unrecognized extension "TwinView"
> # /usr/X11/bin/glxinfo 
> name of display: :0.0
> display: :0  screen: 0
> direct rendering: Yes
> server glx vendor string: NVIDIA Corporation
> server glx version string: 1.4
> server glx extensions:
> [...]
> client glx vendor string: NVIDIA Corporation
> client glx version string: 1.4
> client glx extensions:
> [...]
> GLX version: 1.3
> GLX extensions:
> [...]
> OpenGL vendor string: NVIDIA Corporation
> OpenGL renderer string: GeForce 7600 GS/AGP/SSE2
> OpenGL version string: 2.1.2 NVIDIA 169.12
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org
_______________________________________________
xen-discuss mailing list
xen-discuss@opensolaris.org

> Yes, the drivers from Nvidia''s site will be doing all
> sorts of bad things on top of Xen. Physical addresses
> are virtual :-) in i86xpv.  They need to be
> translated
> to machine addresses.  There are other things too
> which the nvidia driver and associated code has to
> do differently on i86xpv.
The only reason I downloaded the newer drivers from nvidia''s site was
compiz. But as compiz now works with the currently shipped driver, there is no
need to do that anymore.
Just wanted to give it a try.
My success is only virtual... :) - I can run compiz in i86pv, but it
doesn''t give much joy with 3.xx fps :)
Anyways, it''s been so only with nv_85; until 77 there wasn''t
such  problem. I skipped nv78-84 though.
> 
> There performance shouldn''t be that bad :-) I would
> expect you would see ~ 3 1/2 FPS with the nv driver
> which is S/W only.
:) Yep, but this with hw renderer... 
Ah, yes, and add to that, that I work with an opengl application...! ;) (Well,
not always, but often)
> 
> John M, any ideas on how to debug this further?
> 
> 
> 
> 
> MRJ
> 
Thanks for the help!
A
 
 
This message posted from opensolaris.org

Mark Johnson wrote:
>
>> Then I upgraded the nvidia driver to the current beta, same issues.
>
> They are not the same driver :-) The driver on the
> nVidia site needs to work on s10, and all versions
> of Nevada.
>
> The driver included with OpenSolaris is modified
> to know about Xen and will not work on older
> versions of Solaris.
Not true.  The driver I deliver to Nevada/OpenSolaris is supposed to 
work all
the way back to S10 GA.  AFAIK, I''ve only broken S10 support
once (a Makefile goof that won''t happen again).  This driver also
has xVM support enabled conditionally, so it should "just work"
if/when xVM graphics support is back ported to S10.

As of this writing, none of the graphics drivers at the NVIDIA download
site have xVM support.  xVM support is only available in the drivers
delivered by Sun.  If you load a driver directly from NVIDIA and
boot dom0, assume bad things will happen.
>
> You should not see the same performance running on
> metal vs running on Xen (running on Xen should be
> slower).  There will always be performance lost
> going through the hypervisor and we don''t use the
> PAT in the i86xpv kernel (on top of Xen) today.
>
Unless you hit this problem:

  6661292 nv_devmap errors running Boing screen saver

the worst case hit I''ve seen running xVM dom0 is 50%, not a couple
of orders of magnitude.  This bug is fixed in the 169.11 driver
delivered to b83.

If Attila can privately send me the output of 
/usr/bin/nvidia-SunOS-bug-report.sh
after glxgears shows single digit frame rates, I will investigate.

> Unless you hit this problem:
> 
>   6661292 nv_devmap errors running Boing screen saver
You are hitting this problem or at least a variant of it:

  NOTICE: NVRM: nv_devmap: invalid mmap(): 0x00100000 @ 0x80000000e0000000

You have the fix for 6661292 on your system, so suspect it has
something to do with the high bit set in the 64 bit physical address.
I need to recheck all the code paths for a potential failure in this
case.  I suspect you don''t get physical addresses like this on you
i965 motherboard.  I''m tracking this under:

  6683369 nv_devmap errors under xVM dom0 on i865 motherboard

In the mean time, please try a workaround adding this option to
the nvidia section in /etc/X11/xorg.conf:

  Option "ConstantFrameRateHint" "True"

> > Unless you hit this problem:
> > 
> >   6661292 nv_devmap errors running Boing screen
> saver
> 
> You are hitting this problem or at least a variant of
> it:
> 
> NOTICE: NVRM: nv_devmap: invalid mmap(): 0x00100000
>  @ 0x80000000e0000000
> You have the fix for 6661292 on your system, so
> suspect it has
> something to do with the high bit set in the 64 bit
> physical address.
> I need to recheck all the code paths for a potential
> failure in this
> case.  I suspect you don''t get physical addresses
> like this on you
> i965 motherboard. 
Indeed, no such problems with the other machine! (I just re-checked it today,
just to be sure)
>  I''m tracking this under:
> 
> 6683369 nv_devmap errors under xVM dom0 on i865
>  motherboard
> In the mean time, please try a workaround adding this
> option to
> the nvidia section in /etc/X11/xorg.conf:
> 
>   Option "ConstantFrameRateHint" "True"
I added it 
(see in the log:
(**) NVIDIA(0): Option "ConstantFrameRateHint" "True" )
, but still 3-4 fps.

# /usr/X11/demo/glxgears           
21 frames in 5.2 seconds =  4.036 FPS
17 frames in 5.1 seconds =  3.349 FPS
17 frames in 5.1 seconds =  3.337 FPS
17 frames in 5.1 seconds =  3.329 FPS
17 frames in 5.2 seconds =  3.252 FPS
35 frames in 5.1 seconds =  6.920 FPS
56 frames in 5.2 seconds = 10.837 FPS
46 frames in 5.1 seconds =  9.081 FPS
159 frames in 5.0 seconds = 31.755 FPS
326 frames in 5.0 seconds = 65.101 FPS
82 frames in 5.1 seconds = 15.936 FPS
X connection to :0.0 broken (explicit kill or server shutdown).

Again, higher framerates, when a window is on top of the opengl window. 9-10
fps, when the glxgears window was only partially "overlaid" by an
other window (the terminal in this case)

A
 
 
This message posted from opensolaris.org