llvm dev - Mar 2018 - Success : Ubuntu 17.10 x86_64 on LLVM/clang-built kernel with LLVM/clang-built Nvidia 390.25 driver

Hello All,

This is about my successful experience in bringing up Ubuntu 17.10 x86_64
on the clang-built Linux 4.15.7 kernel(latest stable as of early March
2018) and clang-built proprietary Nvidia 390.25(latest as of early March
2018) driver for my Nvidia GTX GeForce 860M graphics card. I wanted to
share the same here for any similarly interested people out here. By the
way, appreciate the efforts of people involved in LLVM/clang community for
all the so far work.

Cheers,
Raghavan Santhanam

======
*Stage 1 : *Linux kernel 4.15.7 was built using clang as the host and
target compiler and installed along with the modules based on default
kernel config. I had to disable exofs module as it had non-standard "VLAs
embedded within structs" that clang doesn't support. Then I also
compiled
Nvidia 390.25 driver source after extracting it from its propitiatory
installer - I had to ignore the compiler version mismatch psuedo-warning as
kernel headers included the thread model and also the platform information
while Nvidia driver script check resulted in just the version. After
installing thus built Nvidia driver modules, I rebooted my system, the
Ubuntu 17.10 booting didn't complete - it got stuck with "Loading ...
4.15.7". The system journalctl/logs didn't help much.

*Stage 2 : *To further isolate the compatibility issues(if any) of kernel
with gcc/clang vs Nvidia driver with gcc/clang, Compiled Linux kernel
4.15.7 with gcc and Nvidia 390.25 driver with clang, and vice versa. For
the first, the booting process passed the "Loading ... 4.15.7"phase
and
moved onto some systemd service, but got stuck there. For the second,
again, it got stuck in the very "Loading ... 4.15.7" phase itself. The
system journalctl/logs didn't help much.

*Stage 3 : *Having worked with Qemu/Kvm earlier, I wanted to get hold of
the actual failing point in bringing the clang-built kernel up. So, I tried
to bring up the clang-built Linux kernel 4.15.7image using the x86_64
default config to begin with and also clang-built busybox x86_64 for the
root file system(initrd), on Qemu/kvm x86_64. This worked. I got the shell
and busyboxutilities were working and so on.

*On host Ubuntu 17.10 x86_64 :*

Code:
*exp at exp:~$ *
*exp at exp:~$ strings -a defconfig-linux-4.15.7/linux-4.15.7/vmlinux | grep
"clang version" | head -1*
*Linux version 4.15.7 (exp at exp) (clang version 5.0.0-3
(tags/RELEASE_500/final)) #1 SMP Fri Mar 2 21:15:24 PST 2018*
*exp at exp:~$ *

*exp at exp:~$ lsb_release -a*
*No LSB modules are available.*
*Distributor ID: Ubuntu*
*Description:    Ubuntu 17.10*
*Release:    17.10*
*Codename:   artful*

*exp at exp:~$ clang -v*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*Target: x86_64-pc-linux-gnu*
*Thread model: posix*
*InstalledDir: /usr/bin*
*Within Qemu/kvm guest, Linux kernel 4.15.7(x86_64 defconfig)/Busybox
64-bit :*

Code:
*/ # cat /proc/version *
*Linux version 4.15.7 (exp at exp) (clang version 5.0.0-3
(tags/RELEASE_500/final)) #1 SMP Fri Mar 2 21:15:24 PST 2018*
*/ # *

*/ # uname -arv*
*Linux (none) 4.15.7 #1 SMP Fri Mar 2 21:15:24 PST 2018 x86_64 GNU/Linux*
*/ # *
*/ # *
*/ # strings -a bin/busybox | grep "clang version"*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*/ # *
*/ # *
*/ # dmesg | grep QEMU*
*[    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org
<http://rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org> 04/01/2014*
*[    0.024000] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (family:
0x6, model: 0x6, stepping: 0x3)*
*[    0.337415] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100*
*[    0.339072] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM
 2.5+ PQ: 0 ANSI: 5*
*/ # *
*/ #*

*Stage 4 : *With the above initial success with defconfig kernel, I took
the host Ubuntu 17.10 kernel config and rebuilt the Linux kernel 4.15.7
image. With this I kept rest of the busybox the same and tried to bring
them up on Qemu/kvm as earlier - this didn't work. Resulted in general
protection faults and kernel oops - kernel panic. I looked into the kernel
stack dump. I noticed the first issue was within kernel irq work tick
logic. Then the next issue was with kernel live instruction update logic. I
worked on both of them. Rebuilt the kernel. Tried again with Qemu/kvm - got
the shell, no panic this time. Next step was to install kernel modules and
Nvidia driver modules as well. I did that but this time, with all the
Ubuntu kernel modules and Nvidia driver module, Qemu/kvm showed some
colored pixels and got stuck - mostly due the difference between host's
hardware and guest's virtualized hardware. So, I wanted to try only with
Nvidia driver as it was of prime concern as it's the one that I use on my
host for the graphics. So, I included(didn't install) Nvidia driver modules
in the busybox rootfs and tried to bring up Qemu/kvm as usual - this time,
there was some kernel panic due to rootfs-not-found though there was a
rootfs. So, I tried to compress all of the Nvidia driver modules onto a zip
and include it in the rootfs. But, when I tried to extract the zip with the
Qemu/kvm guest, there was "no space left on device". So, I tried to
construct a 10G virtual qemu disk and install the rootfs within that disk
to address the space concern - but, after a while I called this off as I
was more interested in getting the clang-built Nvidia driver on real system
hardware than on the emulated Qemu/kvm environment(I know it doesn't have
the real Nvidia graphics card available via GPU-passthrough so that it can
test the clang-built Nvidia 390.25 driver on it before I could try that on
host's real system hardware/graphics card, but gave it a shot anyway just
to see if clang-built Nvidia driver exhibits any issues within Qemu/kvm
when loaded).

*Within Qemu/kvm guest, Linux kernel 4.15.7(host Ubuntu 17.10 x86_64
config)/Busybox64-bit :*

Code:
*/ # *
*/ # *
*/ # cat /proc/version *
*Linux version 4.15.7 (exp at exp) (clang version 5.0.0-3
(tags/RELEASE_500/final)) #20 SMP Sat Mar 3 20:46:15 PST 2018*
*/ # *
*/ # *
*/ # uname -arv*
*Linux (none) 4.15.7 #20 SMP Sat Mar 3 20:46:15 PST 2018 x86_64 GNU/Linux*
*/ # *
*/ # *
*/ # *
*/ # dmesg | grep QEMU*
*[    0.000000] DMI: QEMU Standard PC (i440FX + PIIX, 1996), BIOS
rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org
<http://rel-1.9.3-0-ge2fc41e-prebuilt.qemu-project.org> 04/01/2014*
*[    0.024000] smpboot: CPU0: Intel QEMU Virtual CPU version 2.5+ (family:
0x6, model: 0x6, stepping: 0x3)*
*[    0.340776] ata2.00: ATAPI: QEMU DVD-ROM, 2.5+, max UDMA/100*
*[    0.343135] scsi 1:0:0:0: CD-ROM            QEMU     QEMU DVD-ROM
 2.5+ PQ: 0 ANSI: 5*
*/ # *
*/ # *
*/ # *
*/ # strings -a bin/busybox | grep "clang version"*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*/ # *
*/ # *
*/ #*

*Stage 5 and Success : *As my main and actual target was my host(Ubuntu
17.10), I installed thus clang-built kernel(with above issues worked upon)
and Nvidia 390.25 drivers on my host system directly. Rebooted my system.
Bam! I got my Ubuntu 17.10 x86_64 up and running with clang-built 4.15.7
kernel and clang-built Nvidia 390.25 driver without any issues.
On host Ubuntu 17.10 x86_64 with clang-built Linux kernel 4.15.7 and
clang-built Nvidia 390.25 GeForce GTX 860M proprietary
driver - my successful research and experience :

Code:
*exp at exp:~$ *
*exp at exp:~$ dmesg | grep clang*
*[    0.000000] Linux version 4.15.7 (exp at exp) (clang version 5.0.0-3
(tags/RELEASE_500/final)) #20 SMP Sat Mar 3 20:46:15 PST 2018*
*exp at exp:~$ *
*exp at exp:~$ *
*exp at exp:~$ cat /proc/version *
*Linux version 4.15.7 (exp at exp) (clang version 5.0.0-3
(tags/RELEASE_500/final)) #20 SMP Sat Mar 3 20:46:15 PST 2018*
*exp at exp:~$ *
*exp at exp:~$*
*exp at exp:~$ *
*exp at exp:~$ uname -arv*
*Linux exp 4.15.7 #20 SMP Sat Mar 3 20:46:15 PST 2018 x86_64 x86_64 x86_64
GNU/Linux*
*exp at exp:~$ *

*exp at exp:~$ nvidia-smi*
*Sun Mar  4 18:15:39 2018       *
*+-----------------------------------------------------------------------------+*
*| NVIDIA-SMI 390.25                 Driver Version: 390.25
    |*
*|-------------------------------+----------------------+----------------------+*
*| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr.
ECC |*
*| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute
M. |*
*|===============================+======================+======================|*
*|   0  GeForce GTX 860M    Off  | 00000000:01:00.0 Off |
N/A |*
*| N/A   54C    P0    N/A /  N/A |    370MiB /  4046MiB |      0%
Default |*
*+-------------------------------+----------------------+----------------------+*

*exp at exp:~$ dmesg | grep NVIDIA | grep 390.25*
*[   17.643496] NVRM: loading NVIDIA UNIX x86_64 Kernel Module  390.25  Wed
Jan 24 20:02:43 PST 2018 (using threaded interrupts)*
*[   17.689835] nvidia-modeset: Loading NVIDIA Kernel Mode Setting Driver
for UNIX platforms  390.25  Wed Jan 24 19:29:37 PST 2018*
*exp at exp:~$ *

*exp at exp:~$ lsmod | grep nvidia*
*nvidia_uvm            815104  0 *
*nvidia_drm             24576  2 *
*nvidia_modeset       1097728  5 nvidia_drm*
*nvidia              14352384  334 nvidia_uvm,nvidia_modeset*
*ipmi_msghandler        61440  2 nvidia,ipmi_devintf*
*drm                   454656  6 nvidia_drm,i915,drm_kms_helper*
*exp at exp:~$ *


*#*
*# clang-built Linux kernel 4.15.7 image and Nvidia 390.25 driver modules
having clang compiler specific*
*# metadata within them.*
*#*
*exp at exp:~$ *
*exp at exp:~$ eclang /lib/modules/4.15.7/build/vmlinux | grep
"Linux"*
*++ strings -a /lib/modules/4.15.7/build/vmlinux*
*++ grep clang*
*Linux version 4.15.7 (exp at exp) (clang version 5.0.0-3
(tags/RELEASE_500/final)) #20 SMP Sat Mar 3 20:46:15 PST 2018*
*exp at exp:~$ eclang /lib/modules/4.15.7/kernel/drivers/video/nvidia.ko |
head -1*
*++ strings -a /lib/modules/4.15.7/kernel/drivers/video/nvidia.ko*
*++ grep clang*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*exp at exp:~$ eclang /lib/modules/4.15.7/kernel/drivers/video/nvidia-drm.ko |
head -1*
*++ strings -a /lib/modules/4.15.7/kernel/drivers/video/nvidia-drm.ko*
*++ grep clang*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*exp at exp:~$ eclang /lib/modules/4.15.7/kernel/drivers/video/nvidia-uvm.ko |
head -1*
*++ strings -a /lib/modules/4.15.7/kernel/drivers/video/nvidia-uvm.ko*
*++ grep clang*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*exp at exp:~$ eclang
/lib/modules/4.15.7/kernel/drivers/video/nvidia-modeset.ko | head -1*
*++ strings -a /lib/modules/4.15.7/kernel/drivers/video/nvidia-modeset.ko*
*++ grep clang*
*clang version 5.0.0-3 (tags/RELEASE_500/final)*
*exp at exp:~$ *

*#*
*# DMI specific hardware information - verifies and confirms the physical
hardware used*
*# instead of QEMU or any other virtual platform for running clang-built
kernel and*
*# Nvidia 390.25 driver.*
*#
     *
*exp at exp:~$ *
*exp at exp:~$ *
*exp at exp:~$ sudo dmidecode -t bios | grep Vendor*
*    Vendor: LENOVO*
*exp at exp:~$ *
*exp at exp:~$ *
*exp at exp:~$ sudo dmidecode -t system | grep
"Manufacturer:\|Version"*
*    Manufacturer: LENOVO*
*    Version: Lenovo Y50-70 Touch*
*exp at exp:~$ *
*exp at exp:~$ *
*exp at exp:~$ sudo dmidecode -t processor | grep Version*
*    Version: Intel(R) Core(TM) i7-4710HQ CPU @ 2.50GHz*
*exp at exp:~$*

*NOTE 1 : *For all the detailed logs(from each stage), you can read my
answer for LLVM-Clang/Linux kernel question here : Building Linux kernel
with LLVM/clang.

*NOTE 2 : *clang's warning are more pedantic than gcc as I saw. But, for
time being, to get the kernel/nvidia driver up and running, I have ignored
them to see if they are not serious(so far they are not).

*NOTE 3 : *I have posted this above experience in the relevant communities
as under. Interested people can follow them as well.


   1. *(**Ubuntu Forums* <http://ubuntuforums.org/>*) : **Success : Ubuntu
   17.10 x86_64 on LLVM/clang-built kernel with Nvidia 390.25 driver
   <https://ubuntuforums.org/showthread.php?t=2386423> :
https://ubuntuforums.org/showthread.php?t=2386423
   <https://ubuntuforums.org/showthread.php?t=2386423>*
   2. *(**Ask Ubuntu* <http://askubuntu.com/>*) **Success : Ubuntu 17.10
   x86_64 on LLVM/clang-built kernel with LLVM/clang-built Nvidia 390.25
   x86_64 driver
  
<https://askubuntu.com/questions/1012186/success-ubuntu-17-10-x86-64-on-llvm-clang-built-kernel-with-llvm-clang-built-n>
:
https://askubuntu.com/questions/1012186/success-ubuntu-17-10-x86-64-on-llvm-clang-built-kernel-with-llvm-clang-built-n
  
<https://askubuntu.com/questions/1012186/success-ubuntu-17-10-x86-64-on-llvm-clang-built-kernel-with-llvm-clang-built-n>*
   3. (*Nvidia GeForce Forum* <https://forums.geforce.com/>) *Success :
   Clang-built Nvidia 390.25 GTX GeForce 860M driver on clang-built Linux
   kernel 4.15.7
  
<https://forums.geforce.com/default/topic/1039280/geforce-drivers/success-clang-built-nvidia-390-25-gtx-geforce-860m-driver-on-clang-built-linux-kernel-4-15-7/>
:
https://forums.geforce.com/default/topic/1039280/geforce-drivers/success-clang-built-nvidia-390-25-gtx-geforce-860m-driver-on-clang-built-linux-kernel-4-15-7/
  
<https://forums.geforce.com/default/topic/1039280/geforce-drivers/success-clang-built-nvidia-390-25-gtx-geforce-860m-driver-on-clang-built-linux-kernel-4-15-7/>*
   4. *(**Bug 4068* <https://bugs.llvm.org/show_bug.cgi?id=4068>*
update*) *[Meta]
   Compiling the Linux kernel with clang
   <https://bugs.llvm.org/show_bug.cgi?id=4068#c27> :
https://bugs.llvm.org/show_bug.cgi?id=4068#c27
   <https://bugs.llvm.org/show_bug.cgi?id=4068#c27> *
   5. *What is the state of the art of compiling the Linux Kernel using
   LLVM/Clang?
  
<https://www.quora.com/What-is-the-state-of-the-art-of-compiling-the-Linux-Kernel-using-LLVM-Clang>
:*

  
https://www.quora.com/What-is-the-state-of-the-art-of-compiling-the-Linux-Kernel-using-LLVM-Clang/answer/Raghavan-Santhanam
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20180315/41434698/attachment-0001.html>