Just wondering what it will take to get the nvidia binary driver working on a pv_ops kernel. It makes it difficult to debug without the source to the nvidia driver, but I think it should be possible to get it to work without changing the binary driver. If the dom0 kernel had access to all the resources that a bare metal kernel did, then it should work right? I''m using Jeremy''s current master branch, and hypervisor 3.4.1-rc6 on Ubuntu 9.04 64 bit. Attached is my kern.log and Xorg.0.log when I attempt to start Xorg. The screen goes blank and my monitor enters powersave mode, but I am able to run an xterm and it appears to connect to the X server. Also when the X server exits, it returns the console to a sane state. I''ve also tried to get the nvidia driver to work with rebased suse kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel it blacks the screen. I can remote ssh in to my pc, but I can''t reboot the pc. It needs a hard reset. -- Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-12 06:03 UTC
Re: [Xen-users] pv_ops kernel and nvidia binary driver
A while ago Jeremy responded the same question.
Per Jeremy:- unmodified binary Nvidia driver would crash under Xen.
Modified driver should in the pvops kernel.
It''s also my major concern as well.
Boris
--- On Sun, 7/12/09, Michael Ralston <michael@ralston.id.au> wrote:
From: Michael Ralston <michael@ralston.id.au>
Subject: [Xen-users] pv_ops kernel and nvidia binary driver
To: xen-users@lists.xensource.com
Date: Sunday, July 12, 2009, 1:48 AM
Just wondering what it will take to get the nvidia binary driver
working on a pv_ops kernel.
It makes it difficult to debug without the source to the nvidia
driver, but I think it should be possible to get it to work without
changing the binary driver. If the dom0 kernel had access to all the
resources that a bare metal kernel did, then it should work right?
I''m using Jeremy''s current master branch, and hypervisor
3.4.1-rc6 on
Ubuntu 9.04 64 bit.
Attached is my kern.log and Xorg.0.log when I attempt to start Xorg.
The screen goes blank and my monitor enters powersave mode, but I am
able to run an xterm and it appears to connect to the X server. Also
when the X server exits, it returns the console to a sane state.
I''ve also tried to get the nvidia driver to work with rebased suse
kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel
it blacks the screen. I can remote ssh in to my pc, but I can''t reboot
the pc. It needs a hard reset.
-- Michael
-----Inline Attachment Follows-----
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-12 06:22 UTC
Re: [Xen-users] pv_ops kernel and nvidia binary driver
>I''ve also tried to get the nvidia driver to work with rebased suse >kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel >it blacks the screen. I can remote ssh in to my pc, but I can''t reboot >the pc. It needs a hard reset.Native Ubuntu 9.04 X-Server works for me with any xenified 2.6.29.X,2.6.30.X kernel (aka Suse) . GeForce 8500 GT installed on both boxes. Xen host boots into Gnome desktop environment right away. I just replaced GeForce 7X00 cards about year and half ago due to problems with tuning Solaris Nevada X-Server Boris. --- On Sun, 7/12/09, Michael Ralston <michael@ralston.id.au> wrote: From: Michael Ralston <michael@ralston.id.au> Subject: [Xen-users] pv_ops kernel and nvidia binary driver To: xen-users@lists.xensource.com Date: Sunday, July 12, 2009, 1:48 AM Just wondering what it will take to get the nvidia binary driver working on a pv_ops kernel. It makes it difficult to debug without the source to the nvidia driver, but I think it should be possible to get it to work without changing the binary driver. If the dom0 kernel had access to all the resources that a bare metal kernel did, then it should work right? I''m using Jeremy''s current master branch, and hypervisor 3.4.1-rc6 on Ubuntu 9.04 64 bit. Attached is my kern.log and Xorg.0.log when I attempt to start Xorg. The screen goes blank and my monitor enters powersave mode, but I am able to run an xterm and it appears to connect to the X server. Also when the X server exits, it returns the console to a sane state. I''ve also tried to get the nvidia driver to work with rebased suse kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel it blacks the screen. I can remote ssh in to my pc, but I can''t reboot the pc. It needs a hard reset. -- Michael -----Inline Attachment Follows----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Michael Ralston
2009-Jul-12 08:27 UTC
Re: [Xen-users] pv_ops kernel and nvidia binary driver
On Sun, Jul 12, 2009 at 4:22 PM, Boris Derzhavets<bderzhavets@yahoo.com> wrote:>>I''ve also tried to get the nvidia driver to work with rebased suse >>kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel >>it blacks the screen. I can remote ssh in to my pc, but I can''t reboot >>the pc. It needs a hard reset. > > Native Ubuntu 9.04 X-Server works for me with any xenified 2.6.29.X,2.6.30.X > kernel (aka Suse) . GeForce 8500 GT installed on both boxes. Xen host boots > into Gnome desktop environment right away. > > I just replaced GeForce 7X00 cards about year and half ago due to problems > with tuning Solaris Nevada X-Server >What version nvidia driver are you using with that? I''ve tried 185.18.14, 180.44, 173.14.16 and 96.43.10. All with identical results. -- Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-12 09:11 UTC
Re: [Xen-users] pv_ops kernel and nvidia binary driver
Michael, I have no any idea. It gets built in 2.6.29.4 and 2.6.30.1 after xen patching via rebased patch set from Andy. 1. I build Xen 2. I build kernel 3. Create Grub entry for Xen Host 4. Reboot I am at Ubuntu 9.04 Gnome login prompt Login into Gnome Desktop. Driver gets patched to run under Xen Hypervisor when i apply Andy''s patch set either 2.6.29.4 or 2.6.30. I believe Suse src rpm has to be analyzed carefully to find location of Nvidia patch. It never bothered me. Boris. --- On Sun, 7/12/09, Michael Ralston <michael@ralston.id.au> wrote: From: Michael Ralston <michael@ralston.id.au> Subject: Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: xen-users@lists.xensource.com Date: Sunday, July 12, 2009, 4:27 AM On Sun, Jul 12, 2009 at 4:22 PM, Boris Derzhavets<bderzhavets@yahoo.com> wrote:>>I''ve also tried to get the nvidia driver to work with rebased suse >>kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel >>it blacks the screen. I can remote ssh in to my pc, but I can''t reboot >>the pc. It needs a hard reset. > > Native Ubuntu 9.04 X-Server works for me with any xenified 2.6.29.X,2.6.30.X > kernel (aka Suse) . GeForce 8500 GT installed on both boxes. Xen host boots > into Gnome desktop environment right away. > > I just replaced GeForce 7X00 cards about year and half ago due to problems > with tuning Solaris Nevada X-Server >What version nvidia driver are you using with that? I''ve tried 185.18.14, 180.44, 173.14.16 and 96.43.10. All with identical results. -- Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-12 13:57 UTC
Re: [Xen-users] pv_ops kernel and nvidia binary driver
Staff like this has to be done to xen 3.4.1 http://fluxcoil.net/doku.php/xen/docs Quote:- currently running: a patched NVIDIA-Linux-x86-100.14.11 on debian unstable 32bit with xen-3.1.1 compiled from mercurial testing the patch for nvidia-driver 100.14.11 is here: http://fluxcoil.net/files/patch-xen3.0-nvidia100.14.11.txt (got it from http://www.nvnews.net ) To install make xen-kernelsources availabe, execute driver-binary with -x switch for extraction-only, cd to subdirectory usr/src, appy patch with ''patch -p0 </path/patch.file'', execute nvidia-installer from the patch-directory to to install. the patch for nvidia-driver 1.0-9746 is here: http://fluxcoil.net/files/NVIDIA-Linux-x86-1.0-9746-pkg1_xendom0.patch (to be patched directly from the NVIDIA-Linux-x86-1.0-9746 directory) Boris --- On Sun, 7/12/09, Michael Ralston <michael@ralston.id.au> wrote: From: Michael Ralston <michael@ralston.id.au> Subject: Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: xen-users@lists.xensource.com Date: Sunday, July 12, 2009, 4:27 AM On Sun, Jul 12, 2009 at 4:22 PM, Boris Derzhavets<bderzhavets@yahoo.com> wrote:>>I''ve also tried to get the nvidia driver to work with rebased suse >>kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel >>it blacks the screen. I can remote ssh in to my pc, but I can''t reboot >>the pc. It needs a hard reset. > > Native Ubuntu 9.04 X-Server works for me with any xenified 2.6.29.X,2.6.30.X > kernel (aka Suse) . GeForce 8500 GT installed on both boxes. Xen host boots > into Gnome desktop environment right away. > > I just replaced GeForce 7X00 cards about year and half ago due to problems > with tuning Solaris Nevada X-Server >What version nvidia driver are you using with that? I''ve tried 185.18.14, 180.44, 173.14.16 and 96.43.10. All with identical results. -- Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-12 14:10 UTC
Re: [Xen-users] pv_ops kernel and nvidia binary driver
Googling with "xen patch for nvidia-driver" also provide some understanding of the issue, which i believe is already resolved in 2.6.27 and higher xenified aka Suse kernels for GeForce 8000,9000 Series drivers. Boris. --- On Sun, 7/12/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote: From: Boris Derzhavets <bderzhavets@yahoo.com> Subject: Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Michael Ralston" <michael@ralston.id.au> Cc: xen-users@lists.xensource.com Date: Sunday, July 12, 2009, 9:57 AM Staff like this has to be done to xen 3.4.1 http://fluxcoil.net/doku.php/xen/docs Quote:- currently running: a patched NVIDIA-Linux-x86-100.14.11 on debian unstable 32bit with xen-3.1.1 compiled from mercurial testing the patch for nvidia-driver 100.14.11 is here: http://fluxcoil.net/files/patch-xen3.0-nvidia100.14.11.txt (got it from http://www.nvnews.net ) To install make xen-kernelsources availabe, execute driver-binary with -x switch for extraction-only, cd to subdirectory usr/src, appy patch with ''patch -p0 </path/patch.file'', execute nvidia-installer from the patch-directory to to install. the patch for nvidia-driver 1.0-9746 is here: http://fluxcoil.net/files/NVIDIA-Linux-x86-1.0-9746-pkg1_xendom0.patch (to be patched directly from the NVIDIA-Linux-x86-1.0-9746 directory) Boris --- On Sun, 7/12/09, Michael Ralston <michael@ralston.id.au> wrote: From: Michael Ralston <michael@ralston.id.au> Subject: Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: xen-users@lists.xensource.com Date: Sunday, July 12, 2009, 4:27 AM On Sun, Jul 12, 2009 at 4:22 PM, Boris Derzhavets<bderzhavets@yahoo.com> wrote:>>I''ve also tried to get the nvidia driver to work with rebased suse >>kernel 2.6.29.6 and hypervisor 3.3 and 3.4. With that version kernel >>it blacks the screen. I can remote ssh in to my pc, but I can''t reboot >>the pc. It needs a hard reset. > >Native Ubuntu 9.04 X-Server works for me with any xenified 2.6.29.X,2.6.30.X> kernel (aka Suse) . GeForce 8500 GT installed on both boxes. Xen host boots > into Gnome desktop environment right away. > > I just replaced GeForce 7X00 cards about year and half ago due to problems > with tuning Solaris Nevada X-Server >What version nvidia driver are you using with that? I''ve tried 185.18.14, 180.44, 173.14.16 and 96.43.10. All with identical results. -- Michael _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jeremy Fitzhardinge
2009-Jul-14 19:52 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
On 07/11/09 23:03, Boris Derzhavets wrote:> A while ago Jeremy responded the same question. > Per Jeremy:- unmodified binary Nvidia driver would crash under Xen. > Modified driver should in the pvops kernel. > > It''s also my major concern as well. > > Boris > > --- On *Sun, 7/12/09, Michael Ralston /<michael@ralston.id.au>/* wrote: > > > From: Michael Ralston <michael@ralston.id.au> > Subject: [Xen-users] pv_ops kernel and nvidia binary driver > To: xen-users@lists.xensource.com > Date: Sunday, July 12, 2009, 1:48 AM > > Just wondering what it will take to get the nvidia binary driver > working on a pv_ops kernel. > > It makes it difficult to debug without the source to the nvidia > driver, but I think it should be possible to get it to work without > changing the binary driver. If the dom0 kernel had access to all the > resources that a bare metal kernel did, then it should work right? >The problem comes down to whether the nvidia driver assumes the kernel''s (pseudo-)physical addresses are really machine physical or not. If it doesn''t do the appropriate conversions between physical and machine addresses using the standard Linux DMA API (or similar), then it will end up misprogramming the hardware and reading/writing random memory. There''s not a lot we can do about that if that happens within the binary part of the nvidia driver. If the binary code calls out to the source-available parts of the driver to do those conversions, then it would be possible to fix there. There are other potential pitfalls the nvidia driver could be falling into, but that''s the most likely. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Boris Derzhavets
2009-Jul-14 20:57 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
By running "sh NVIDIA-Linux-x86_64-185.18.14-pkg2.run -x" i was able to get some extraction in folder bellow. It''s hard for me to decide could it be patched similar schema in http://fluxcoil.net/doku.php/xen/docs for xenified kernel or no. Kernels 2.6.29.4 , 2.6.30.1 xenified via rebased patch set from Andy do contain patched driver for GeForce 8000 series . It just works fine. Nvidia driver extraction :- root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2# ls -CR .: LICENSE nvidia-installer pkg-history.txt usr ./usr: bin include lib lib32 share src X11R6 ./usr/bin: makeself.sh nvidia-bug-report.sh nvidia-smi tls_test mkprecompiled nvidia-settings nvidia-xconfig tls_test_dso.so ./usr/include: cuda GL vdpau ./usr/include/cuda: cudaGL.h cuda.h ./usr/include/GL: glext.h gl.h glxext.h glx.h ./usr/include/vdpau: vdpau.h vdpau_x11.h ./usr/lib: libcuda.so.185.18.14 libGL.so.185.18.14 libvdpau_nvidia.so.185.18.14 tls libGLcore.so.185.18.14 libnvidia-cfg.so.185.18.14 libvdpau.so.185.18.14 libGL.la libnvidia-tls.so.185.18.14 libvdpau_trace.so.185.18.14 ./usr/lib/tls: libnvidia-tls.so.185.18.14 ./usr/lib32: libcuda.so.185.18.14 libGL.la libnvidia-tls.so.185.18.14 libvdpau.so.185.18.14 tls libGLcore.so.185.18.14 libGL.so.185.18.14 libvdpau_nvidia.so.185.18.14 libvdpau_trace.so.185.18.14 ./usr/lib32/tls: libnvidia-tls.so.185.18.14 ./usr/share: applications doc man pixmaps ./usr/share/applications: nvidia-settings.desktop ./usr/src: nv ./usr/src/nv: conftest.sh Makefile.kbuild nv.h nv-memdbg.h nvtypes.h os-agp.h README cpuopsys.h Makefile.nvidia nv-i2c.c nv-misc.h nv-vm.c os-interface.c rmretval.h gcc-version-check.c nvacpi.c nv-kernel.o nvreadme.h nv-vm.h os-interface.h makefile nv.c nv-linux.h nv-reg.h os-agp.c os-registry.c ./usr/X11R6: lib ./usr/X11R6/lib: libXvMCNVIDIA.a libXvMCNVIDIA.so.185.18.14 modules ./usr/X11R6/lib/modules: drivers extensions libnvidia-wfb.so.185.18.14 ./usr/X11R6/lib/modules/drivers: nvidia_drv.so ./usr/X11R6/lib/modules/extensions: libglx.so.185.18.14 Boris. --- On Tue, 7/14/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote: From: Jeremy Fitzhardinge <jeremy@goop.org> Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: xen-users@lists.xensource.com, "Michael Ralston" <michael@ralston.id.au>, xen-devel@lists.xensource.com Date: Tuesday, July 14, 2009, 3:52 PM On 07/11/09 23:03, Boris Derzhavets wrote:> A while ago Jeremy responded the same question. > Per Jeremy:- unmodified binary Nvidia driver would crash under Xen. > Modified driver should in the pvops kernel. > > It''s also my major concern as well. > > Boris > > --- On *Sun, 7/12/09, Michael Ralston /<michael@ralston.id.au>/* wrote: > > > From: Michael Ralston <michael@ralston.id.au> > Subject: [Xen-users] pv_ops kernel and nvidia binary driver > To: xen-users@lists.xensource.com > Date: Sunday, July 12, 2009, 1:48 AM > > Just wondering what it will take to get the nvidia binary driver > working on a pv_ops kernel. > > It makes it difficult to debug without the source to the nvidia > driver, but I think it should be possible to get it to work without > changing the binary driver. If the dom0 kernel had access to all the > resources that a bare metal kernel did, then it should work right? >The problem comes down to whether the nvidia driver assumes the kernel''s (pseudo-)physical addresses are really machine physical or not. If it doesn''t do the appropriate conversions between physical and machine addresses using the standard Linux DMA API (or similar), then it will end up misprogramming the hardware and reading/writing random memory. There''s not a lot we can do about that if that happens within the binary part of the nvidia driver. If the binary code calls out to the source-available parts of the driver to do those conversions, then it would be possible to fix there. There are other potential pitfalls the nvidia driver could be falling into, but that''s the most likely. J _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-14 21:17 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
Following README i attemped :- root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# make SYSSRC=/usr/src/linux-2.6-xen module . . . . . . . . ld -r -m elf_x86_64 --build-id -o /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.o /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.mod.o NVIDIA: left KBUILD. root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# make SYSSRC=/usr/src/linux-2.6-xen module root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# ls -l total 40416 -rw-r--r-- 1 root root 1148 2009-07-15 01:05 conftest.h -rw-r--r-- 1 root root 49341 2009-05-27 13:02 conftest.sh -rw-r--r-- 1 root root 8517 2009-05-27 13:02 cpuopsys.h -rw-r--r-- 1 root root 9077 2009-05-27 13:02 gcc-version-check.c -rw-r--r-- 1 root root 1834 2009-05-27 13:02 makefile lrwxrwxrwx 1 root root 15 2009-07-15 01:05 Makefile -> Makefile.kbuild -rw-r--r-- 1 root root 9174 2009-05-27 13:02 Makefile.kbuild -rw-r--r-- 1 root root 3726 2009-05-27 13:02 Makefile.nvidia -rw-r--r-- 1 root root 45 2009-07-15 01:05 Module.markers -rw-r--r-- 1 root root 72 2009-07-15 01:05 modules.order -rw-r--r-- 1 root root 104 2009-07-15 01:05 Module.symvers -rw-r--r-- 1 root root 33436 2009-05-27 13:02 nvacpi.c -rw-r--r-- 1 root root 178616 2009-07-15 01:05 nvacpi.o -rw-r--r-- 1 root root 131773 2009-05-27 13:02 nv.c -rw-r--r-- 1 root root 65 2009-07-15 01:05 nv_compiler.h -rw-r--r-- 1 root root 23476 2009-05-27 13:02 nv.h -rw-r--r-- 1 root root 9980 2009-05-27 13:02 nv-i2c.c -rw-r--r-- 1 root root 161448 2009-07-15 01:05 nv-i2c.o ++++++++++++++++++++++++++++++++++++++ -rw-r--r-- 1 root root 13577047 2009-07-15 01:05 nvidia.ko ++++++++++++++++++++++++++++++++++++++ -rw-r--r-- 1 root root 5268 2009-07-15 01:05 nvidia.mod.c -rw-r--r-- 1 root root 53264 2009-07-15 01:05 nvidia.mod.o -rw-r--r-- 1 root root 13525278 2009-07-15 01:05 nvidia.o -rw-r--r-- 1 root root 12118040 2009-05-27 13:02 nv-kernel.o -rw-r--r-- 1 root root 50152 2009-05-27 13:02 nv-linux.h -rw-r--r-- 1 root root 2967 2009-05-27 13:02 nv-memdbg.h -rw-r--r-- 1 root root 714 2009-05-27 13:02 nv-misc.h -rw-r--r-- 1 root root 403968 2009-07-15 01:05 nv.o -rw-r--r-- 1 root root 5072 2009-05-27 13:02 nvreadme.h -rw-r--r-- 1 root root 19284 2009-05-27 13:02 nv-reg.h -rw-r--r-- 1 root root 12883 2009-05-27 13:02 nvtypes.h -rw-r--r-- 1 root root 23062 2009-05-27 13:02 nv-vm.c -rw-r--r-- 1 root root 768 2009-05-27 13:02 nv-vm.h -rw-r--r-- 1 root root 158584 2009-07-15 01:05 nv-vm.o -rw-r--r-- 1 root root 9249 2009-05-27 13:02 os-agp.c -rw-r--r-- 1 root root 804 2009-05-27 13:02 os-agp.h -rw-r--r-- 1 root root 156520 2009-07-15 01:05 os-agp.o -rw-r--r-- 1 root root 31069 2009-05-27 13:02 os-interface.c -rw-r--r-- 1 root root 8785 2009-05-27 13:02 os-interface.h -rw-r--r-- 1 root root 212096 2009-07-15 01:05 os-interface.o -rw-r--r-- 1 root root 2395 2009-05-27 13:02 os-registry.c -rw-r--r-- 1 root root 160400 2009-07-15 01:05 os-registry.o -rw-r--r-- 1 root root 157 2009-07-15 01:05 patches.h Boris. --- On Tue, 7/14/09, Boris Derzhavets <bderzhavets@yahoo.com> wrote: From: Boris Derzhavets <bderzhavets@yahoo.com> Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Jeremy Fitzhardinge" <jeremy@goop.org> Cc: "Michael Ralston" <michael@ralston.id.au>, xen-devel@lists.xensource.com, xen-users@lists.xensource.com Date: Tuesday, July 14, 2009, 4:57 PM By running "sh NVIDIA-Linux-x86_64-185.18.14-pkg2.run -x" i was able to get some extraction in folder bellow. It''s hard for me to decide could it be patched similar schema in http://fluxcoil.net/doku.php/xen/docs for xenified kernel or no. Kernels 2.6.29.4 , 2.6.30.1 xenified via rebased patch set from Andy do contain patched driver for GeForce 8000 series . It just works fine. Nvidia driver extraction :- root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2# ls -CR .: LICENSE nvidia-installer pkg-history.txt usr ./usr: bin include lib lib32 share src X11R6 ./usr/bin: makeself.sh nvidia-bug-report.sh nvidia-smi tls_test mkprecompiled nvidia-settings nvidia-xconfig tls_test_dso.so ./usr/include: cuda GL vdpau ./usr/include/cuda: cudaGL.h cuda.h ./usr/include/GL: glext.h gl.h glxext.h glx.h ./usr/include/vdpau: vdpau.h vdpau_x11.h ./usr/lib: libcuda.so.185.18.14 libGL.so.185.18.14 libvdpau_nvidia.so.185.18.14 tls libGLcore.so.185.18.14 libnvidia-cfg.so.185.18.14 libvdpau.so.185.18.14 libGL.la libnvidia-tls.so.185.18.14 libvdpau_trace.so.185.18.14 ./usr/lib/tls: libnvidia-tls.so.185.18.14 ./usr/lib32: libcuda.so.185.18.14 libGL.la libnvidia-tls.so.185.18.14 libvdpau.so.185.18.14 tls libGLcore.so.185.18.14 libGL.so.185.18.14 libvdpau_nvidia.so.185.18.14 libvdpau_trace.so.185.18.14 ./usr/lib32/tls: libnvidia-tls.so.185.18.14 ./usr/share: applications doc man pixmaps ./usr/share/applications: nvidia-settings.desktop ./usr/src: nv ./usr/src/nv: conftest.sh Makefile.kbuild nv.h nv-memdbg.h nvtypes.h os-agp.h README cpuopsys.h Makefile.nvidia nv-i2c.c nv-misc.h nv-vm.c os-interface.c rmretval.h gcc-version-check.c nvacpi.c nv-kernel.o nvreadme.h nv-vm.h os-interface.h makefile nv.c nv-linux.h nv-reg.h os-agp.c os-registry.c ./usr/X11R6: lib ./usr/X11R6/lib: libXvMCNVIDIA.a libXvMCNVIDIA.so.185.18.14 modules ./usr/X11R6/lib/modules: drivers extensions libnvidia-wfb.so.185.18.14 ./usr/X11R6/lib/modules/drivers: nvidia_drv.so ./usr/X11R6/lib/modules/extensions: libglx.so.185.18.14 Boris. --- On Tue, 7/14/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote: From: Jeremy Fitzhardinge <jeremy@goop.org> Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: xen-users@lists.xensource.com, "Michael Ralston" <michael@ralston.id.au>, xen-devel@lists.xensource.com Date: Tuesday, July 14, 2009, 3:52 PM On 07/11/09 23:03, Boris Derzhavets wrote:> A while ago Jeremy responded the same question. > Per Jeremy:- unmodified binary Nvidia driver would crash under Xen. > Modified driver should in the pvops kernel. > > It''s also my major concern as well. > > Boris > > --- On *Sun, 7/12/09, Michael Ralston /<michael@ralston.id.au>/* wrote: > > > From: Michael Ralston <michael@ralston.id.au> > Subject: [Xen-users] pv_ops kernel and nvidia binary driver > To: xen-users@lists.xensource.com > Date: Sunday, July 12, 2009, 1:48 AM > > Just wondering what it will take to get the nvidia binary driver > working on a pv_ops kernel. > > It makes it difficult to debug without the source to the nvidia > driver, but I think it should be possible to get it to work without > changing the binary driver. If the dom0 kernel had access to all the > resources that a bare metal kernel did, then it should work right? >The problem comes down to whether the nvidia driver assumes the kernel''s (pseudo-)physical addresses are really machine physical or not. If it doesn''t do the appropriate conversions between physical and machine addresses using the standard Linux DMA API (or similar), then it will end up misprogramming the hardware and reading/writing random memory. There''s not a lot we can do about that if that happens within the binary part of the nvidia driver. If the binary code calls out to the source-available parts of the driver to do those conversions, then it would be possible to fix there. There are other potential pitfalls the nvidia driver could be falling into, but that''s the most likely. J -----Inline Attachment Follows----- _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Jeremy Fitzhardinge
2009-Jul-14 21:21 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
On 07/14/09 14:17, Boris Derzhavets wrote:> Following README i attemped :- > > root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# > make SYSSRC=/usr/src/linux-2.6-xen module > . . . . . . . . > ld -r -m elf_x86_64 --build-id -o > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.o > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.mod.o > NVIDIA: left KBUILD. > root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# > make SYSSRC=/usr/src/linux-2.6-xen module > root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# > ls -l > total 40416 > -rw-r--r-- 1 root root 1148 2009-07-15 01:05 conftest.h > -rw-r--r-- 1 root root 49341 2009-05-27 13:02 conftest.sh > -rw-r--r-- 1 root root 8517 2009-05-27 13:02 cpuopsys.h > -rw-r--r-- 1 root root 9077 2009-05-27 13:02 gcc-version-check.c > -rw-r--r-- 1 root root 1834 2009-05-27 13:02 makefile > lrwxrwxrwx 1 root root 15 2009-07-15 01:05 Makefile -> > Makefile.kbuild > -rw-r--r-- 1 root root 9174 2009-05-27 13:02 Makefile.kbuild > -rw-r--r-- 1 root root 3726 2009-05-27 13:02 Makefile.nvidia > -rw-r--r-- 1 root root 45 2009-07-15 01:05 Module.markers > -rw-r--r-- 1 root root 72 2009-07-15 01:05 modules.order > -rw-r--r-- 1 root root 104 2009-07-15 01:05 Module.symvers > -rw-r--r-- 1 root root 33436 2009-05-27 13:02 nvacpi.c > -rw-r--r-- 1 root root 178616 2009-07-15 01:05 nvacpi.o > -rw-r--r-- 1 root root 131773 2009-05-27 13:02 nv.c > -rw-r--r-- 1 root root 65 2009-07-15 01:05 nv_compiler.h > -rw-r--r-- 1 root root 23476 2009-05-27 13:02 nv.h > -rw-r--r-- 1 root root 9980 2009-05-27 13:02 nv-i2c.c > -rw-r--r-- 1 root root 161448 2009-07-15 01:05 nv-i2c.o > ++++++++++++++++++++++++++++++++++++++ > -rw-r--r-- 1 root root 13577047 2009-07-15 01:05 nvidia.ko > ++++++++++++++++++++++++++++++++++++++ >What are you saying here? It built? It works? Either way, its up to nvidia to incorporate the support into their driver. (Or for someone to maintain it separately.) J _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Boris Derzhavets
2009-Jul-14 21:43 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
Seems to build and fails to install :- root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# make SYSSRC=/usr/src/linux-2.6-xen install NVIDIA: calling KBUILD... make CC=cc KBUILD_VERBOSE=1 -C /usr/src/linux-2.6-xen SUBDIRS=/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv modules test -e include/linux/autoconf.h -a -e include/config/auto.conf || ( \ echo; \ echo " ERROR: Kernel configuration is invalid."; \ echo " include/linux/autoconf.h or include/config/auto.conf are missing."; \ echo " Run ''make oldconfig && make prepare'' on kernel src to fix it."; \ echo; \ /bin/false) mkdir -p /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/.tmp_versions ; rm -f /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/.tmp_versions/* make -f scripts/Makefile.build obj=/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv (cat /dev/null; echo kernel//usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko;) > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/modules.order make -f /usr/src/linux-2.6-xen/scripts/Makefile.modpost scripts/mod/modpost -m -a -i /usr/src/linux-2.6-xen/Module.symvers -I /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/Module.symvers -o /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/Module.symvers -S -K /usr/src/linux-2.6-xen/Module.markers -M /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/Module.markers -w -s WARNING: could not find /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/.nv-kernel.o.cmd for /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nv-kernel.o NVIDIA: left KBUILD. PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/bin:/sbin" modprobe nvidia && \ echo "nvidia.ko installed successfully."; FATAL: Module nvidia not found. make[1]: *** [package-install] Error 1 Boris --- On Tue, 7/14/09, Jeremy Fitzhardinge <jeremy@goop.org> wrote: From: Jeremy Fitzhardinge <jeremy@goop.org> Subject: Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver To: "Boris Derzhavets" <bderzhavets@yahoo.com> Cc: "Michael Ralston" <michael@ralston.id.au>, xen-devel@lists.xensource.com, xen-users@lists.xensource.com Date: Tuesday, July 14, 2009, 5:21 PM On 07/14/09 14:17, Boris Derzhavets wrote:> Following README i attemped :- > > root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# > make SYSSRC=/usr/src/linux-2.6-xen module > . . . . . . . . > ld -r -m elf_x86_64 --build-id -o > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.ko > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.o > /usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv/nvidia.mod.o > NVIDIA: left KBUILD. > root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# > make SYSSRC=/usr/src/linux-2.6-xen module > root@SeverUbuntuJaunty:/usr/src/NVIDIA-Linux-x86_64-185.18.14-pkg2/usr/src/nv# > ls -l > total 40416 > -rw-r--r-- 1 root root 1148 2009-07-15 01:05 conftest.h > -rw-r--r-- 1 root root 49341 2009-05-27 13:02 conftest.sh > -rw-r--r-- 1 root root 8517 2009-05-27 13:02 cpuopsys.h > -rw-r--r-- 1 root root 9077 2009-05-27 13:02 gcc-version-check.c > -rw-r--r-- 1 root root 1834 2009-05-27 13:02 makefile > lrwxrwxrwx 1 root root 15 2009-07-15 01:05 Makefile -> > Makefile.kbuild > -rw-r--r-- 1 root root 9174 2009-05-27 13:02 Makefile.kbuild > -rw-r--r-- 1 root root 3726 2009-05-27 13:02 Makefile.nvidia > -rw-r--r-- 1 root root 45 2009-07-15 01:05 Module.markers > -rw-r--r-- 1 root root 72 2009-07-15 01:05 modules.order > -rw-r--r-- 1 root root 104 2009-07-15 01:05 Module.symvers > -rw-r--r-- 1 root root 33436 2009-05-27 13:02 nvacpi.c > -rw-r--r-- 1 root root 178616 2009-07-15 01:05 nvacpi.o > -rw-r--r-- 1 root root 131773 2009-05-27 13:02 nv.c > -rw-r--r-- 1 root root 65 2009-07-15 01:05 nv_compiler.h > -rw-r--r-- 1 root root 23476 2009-05-27 13:02 nv.h > -rw-r--r-- 1 root root 9980 2009-05-27 13:02 nv-i2c.c > -rw-r--r-- 1 root root 161448 2009-07-15 01:05 nv-i2c.o > ++++++++++++++++++++++++++++++++++++++ > -rw-r--r-- 1 root root 13577047 2009-07-15 01:05 nvidia.ko > ++++++++++++++++++++++++++++++++++++++ >What are you saying here? It built? It works? Either way, its up to nvidia to incorporate the support into their driver. (Or for someone to maintain it separately.) J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2009-Jul-15 08:14 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
On Tue, 2009-07-14 at 20:52 +0100, Jeremy Fitzhardinge wrote:> The problem comes down to whether the nvidia driver assumes the kernel''s > (pseudo-)physical addresses are really machine physical or not. If it > doesn''t do the appropriate conversions between physical and machine > addresses using the standard Linux DMA API (or similar), then it will > end up misprogramming the hardware and reading/writing random memory. > There''s not a lot we can do about that if that happens within the binary > part of the nvidia driver. If the binary code calls out to the > source-available parts of the driver to do those conversions, then it > would be possible to fix there.I''ve been running the Nvidia driver on an old style Xen kernel for quite a while now so I guess it will be possible to make it work for pvops too. The glue layer contains: /* * Traditionally, CONFIG_XEN indicated that the target kernel was * built exclusively for use under a Xen hypervisor, requiring * modifications to or disabling of a variety of NVIDIA graphics * driver code paths. As of the introduction of CONFIG_PARAVIRT * and support for Xen hypervisors within the CONFIG_PARAVIRT_GUEST * architecture, CONFIG_XEN merely indicates that the target * kernel can run under a Xen hypervisor, but not that it will. * * If CONFIG_XEN and CONFIG_PARAVIRT are defined, the old Xen * specific code paths are disabled. If the target kernel executes * stand-alone, the NVIDIA graphics driver will work fine. If the * kernels executes under a Xen (or other) hypervisor, however, the * NVIDIA graphics driver has no way of knowing and is unlikely * to work correctly. */ #if defined(CONFIG_XEN) && !defined(CONFIG_PARAVIRT) #include <asm/maddr.h> #include <xen/interface/memory.h> #define NV_XEN_SUPPORT_OLD_STYLE_KERNEL #endif which suggests it doesn''t currently work but I would guess that the issue could be fixed in the glue layer and the usages of NV_XEN_SUPPORT_OLD_STYLE_KERNEL will point towards the areas which need consideration, there aren''t too many of them... Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2009-Jul-15 16:57 UTC
Re: [Xen-devel] Re: [Xen-users] pv_ops kernel and nvidia binary driver
On 07/15/09 01:14, Ian Campbell wrote:> On Tue, 2009-07-14 at 20:52 +0100, Jeremy Fitzhardinge wrote: > >> The problem comes down to whether the nvidia driver assumes the kernel''s >> (pseudo-)physical addresses are really machine physical or not. If it >> doesn''t do the appropriate conversions between physical and machine >> addresses using the standard Linux DMA API (or similar), then it will >> end up misprogramming the hardware and reading/writing random memory. >> There''s not a lot we can do about that if that happens within the binary >> part of the nvidia driver. If the binary code calls out to the >> source-available parts of the driver to do those conversions, then it >> would be possible to fix there. >> > > I''ve been running the Nvidia driver on an old style Xen kernel for quite > a while now so I guess it will be possible to make it work for pvops > too. The glue layer contains: > /* > * Traditionally, CONFIG_XEN indicated that the target kernel was > * built exclusively for use under a Xen hypervisor, requiring > * modifications to or disabling of a variety of NVIDIA graphics > * driver code paths. As of the introduction of CONFIG_PARAVIRT > * and support for Xen hypervisors within the CONFIG_PARAVIRT_GUEST > * architecture, CONFIG_XEN merely indicates that the target > * kernel can run under a Xen hypervisor, but not that it will. > * > * If CONFIG_XEN and CONFIG_PARAVIRT are defined, the old Xen > * specific code paths are disabled. If the target kernel executes > * stand-alone, the NVIDIA graphics driver will work fine. If the > * kernels executes under a Xen (or other) hypervisor, however, the > * NVIDIA graphics driver has no way of knowing and is unlikely > * to work correctly. >The whole point of pvops and the other changes is that they don''t need to care whether its running under Xen. They must be going around the standard APIs, and doing something special for the Xen case. Of course, they can always use the xen_domain() predicates to make that code runtime conditional if they want to do it that way... J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Apparently Analagous Threads
- pv_ops kernel and nvidia binary driver
- 2.6.30.1 dom0 Xen patches
- Opensolaris domU unable to get dhcp lease
- Advice on PCIe graphics card for Xen+X86_64 for X
- Serial logs for 2.6.31 (commit 12e8537b6b29807cb9e13d728b2e5bab40424df7) under Xen Unstable on top of Ubuntu 9.04 Server