Xen devel - Apr 2010 - HOWTO make PV drivers for 2.6.32.11 domU HVM

I have had some success compiling xen 4.0 PV drivers for 2.6.32.11, and
seeing as this appears to be non-trivial, I thought it worth documenting.
All the information is out there, but I thought I''d publish a HOWTO for
other people; perhaps I am very stupid, but it took me a while to work
out, partly as the unmodified_drivers documentation is misleading.
My target was Ubuntu, but as I start off with kernel.org sources, this
is largely irrelevant.

Question: this may be a very naive question, but would it not make
life simpler for people if those distributing the xenify parch set
(the ones I grabbed from google) also distributed an
"unmodified_drivers"
directory with the path mangling done by mkbuildtree already in place.
Then it would be simply a matter of going into unmodified_drivers
and calling a script that sets XEN= and XL= and calls make. This would
avoid the need to bring down the entire xen source, and would mean
only combining two potions, rather than three. As far as I can tell,
it doesn''t actually /need/ the xen source, though I may be wrong. And
pv drivers built this way work with older versions (I''ve tried 3.3)
as well as Xen 4.0.

HOWTO make PV drivers for 2.6.32.11 domU HVM

Step 1
=====
Build and install an HVM kernel built from xenified kernel sources, but
*without* any of the Xen options turned on (this got me for a while). You
want a normal guest configuration which knows as little about Xen as
possible.

$ mkdir ~/test
$ cd ~/test
$ wget http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.32.11.tar.bz2
$ tar xvjf linux-2.6.32.11.tar.bz2

Now xenify it (patch script courtesy of Boris Derzhavets)

$ cat >patcher
#!/bin/bash
for P in `ls ../xen-patches-2.6.32-1/6*.patch1 | sort `
do
patch -p1 -s -i $P
if [ $? = 0 ]; then
echo $P applied
else
echo "Error processing "$P
exit 1
fi
done
^D
$ chmod ogua+x patcher

Get the patches from Gentoo:
$ wget 
http://gentoo-xen-kernel.googlecode.com/files/xen-patches-2.6.32-1.tar.bz2
$ mkdir xen-patches-2.6.32-1
$ cd xen-patches-2.6.32-1
$ tar -xvjf ../xen-patches-2.6.32-1.tar.bz2
$ cd ..

Apply the patches
$ cd linux-2.6.32.11
$ ../patcher

Make an appropriate .config - NOTE: DO NOT TURN ON ANY XEN OPTIONS
USE A STANDARD x86_64 config.
$ cp /boot/config-`uname -r` .config
$ make menuconfig
[make any changes and exit]
To speed up build:
$ make localmodconfig

Build kernel, either traditionally (make), or via
$  fakeroot make-kpkg --initrd --append-to-version=-xen-hvm-domu \
    kernel-image kernel-headers


NB: for me this refused to build an initrd, which I had to do manually (i.e.
make modules ; make modules_install
 mkinitramfs -o initrd.img-2.6.32.11-xen-hvm-domu 2.6.32.11-xen-hvm-domu
) - this step should be unnecessary.

Step 2
=====
Make pv drivers

This assumes you are inside the linux directory (i.e. at the end of step
1), which has a built kernel in it (it will need a generated .config, plus
the kernel symbols, possibly plus other stuff).

First make a file with this patch in
$ cat > pvdrivers.patch
diff -r 88d9619e21c3 unmodified_drivers/linux-2.6/blkfront/Kbuild
--- a/unmodified_drivers/linux-2.6/blkfront/Kbuild	Wed Apr 21 08:36:58 2010 
+0100
+++ b/unmodified_drivers/linux-2.6/blkfront/Kbuild	Sat Apr 24 22:03:59 2010 
+0100
@@ -2,4 +2,4 @@

 obj-m += xen-vbd.o

-xen-vbd-objs := blkfront.o vbd.o
+xen-vbd-objs := blkfront.o vbd.o vcd.o
^D

First get the xen source
$ mkdir pvdrivers
$ cd pvdrivers
$ hg clone http://xenbits.xen.org/xen-4.0-testing.hg
$ cd xen-4.0-testing.hg

Now fix a broken makefile (otherwise insmod on xen-vbd.ko will give you
unresolved symbols for vcd_register, vcd_unregister)
$ perl -pi -e ''s/ vbd.o$/ vbd.o vcd.o/'' 
unmodified_drivers/linux-2.6/blkfront/Kbuild

Now set paths up for build, and build it, totally ignoring the instructions
which I believe are wrong. Note that both XL and XEN point into kernel
source you have built, not into the xen source tree.

$ export XL=`cd ../..;pwd`
$ export XEN=$XL/include/xen
$ cd unmodified_drivers/linux-2.6/
$ ./mkbuildtree
$ make -C $XL modules M=$PWD
$ find . -name ''*.ko''

That gives you the locations of your pvdrivers as kernel modules.

What I did was install the kernel (without the pvdrivers first)
and then insmod the pvdrivers in this order:
$ sudo insmod xen-platform-pci.ko
$ sudo insmod xen-balloon.ko
$ sudo insmod xen-vnif.ko
$ sudo insmod xen-vbd.ko
$ sudo insmod xen-scsi.ko


-- 
Alex Bligh

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

--On 24 April 2010 22:47:24 +0100 Alex Bligh <alex@alex.org.uk> wrote:
> First make a file with this patch in
> $ cat > pvdrivers.patch
> diff -r 88d9619e21c3 unmodified_drivers/linux-2.6/blkfront/Kbuild
> --- a/unmodified_drivers/linux-2.6/blkfront/Kbuild	Wed Apr 21 08:36:58
> 2010 +0100
> +++ b/unmodified_drivers/linux-2.6/blkfront/Kbuild	Sat Apr 24 22:03:59
> 2010 +0100
> @@ -2,4 +2,4 @@
>
>  obj-m += xen-vbd.o
>
> -xen-vbd-objs := blkfront.o vbd.o
> +xen-vbd-objs := blkfront.o vbd.o vcd.o
> ^D
You can omit this step as I do it with perl later, but I''m interested
in
why it''s needed.

-- 
Alex Bligh

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

--On 24 April 2010 22:47:24 +0100 Alex Bligh <alex@alex.org.uk> wrote:
> Question: this may be a very naive question, but would it not make
> life simpler for people if those distributing the xenify parch set
> (the ones I grabbed from google) also distributed an
"unmodified_drivers"
> directory with the path mangling done by mkbuildtree already in place.
> Then it would be simply a matter of going into unmodified_drivers
> and calling a script that sets XEN= and XL= and calls make. This would
> avoid the need to bring down the entire xen source, and would mean
> only combining two potions, rather than three. As far as I can tell,
> it doesn''t actually /need/ the xen source, though I may be wrong.
And
> pv drivers built this way work with older versions (I''ve tried
3.3)
> as well as Xen 4.0.
By which I mean, if this tgz
  http://www.alex.org.uk/pvdrivers.tgz
were just expanded into (say) Andrew Lyon''s gentoo patch, all you would
need to do instead of my step 2 would be
  cd pvdrivers
  ./build.sh
and you''d be done. No hg clone etc.

-- 
Alex Bligh

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Sat, Apr 24, 2010 at 11:44:51PM +0100, Alex Bligh
wrote:
>
>
> --On 24 April 2010 22:47:24 +0100 Alex Bligh <alex@alex.org.uk>
wrote:
>
>> Question: this may be a very naive question, but would it not make
>> life simpler for people if those distributing the xenify parch set
>> (the ones I grabbed from google) also distributed an
"unmodified_drivers"
>> directory with the path mangling done by mkbuildtree already in place.
>> Then it would be simply a matter of going into unmodified_drivers
>> and calling a script that sets XEN= and XL= and calls make. This would
>> avoid the need to bring down the entire xen source, and would mean
>> only combining two potions, rather than three. As far as I can tell,
>> it doesn''t actually /need/ the xen source, though I may be
wrong. And
>> pv drivers built this way work with older versions (I''ve tried
3.3)
>> as well as Xen 4.0.
>
> By which I mean, if this tgz
>  http://www.alex.org.uk/pvdrivers.tgz
> were just expanded into (say) Andrew Lyon''s gentoo patch, all you
would
> need to do instead of my step 2 would be
>  cd pvdrivers
>  ./build.sh
> and you''d be done. No hg clone etc.
>
Nice work with the HOWTO! 

And yeah.. providing .tgz like that sounds like a good thing, for the time
being,
until pv-on-hvm drivers are in the mainline Linux.

You want to do it / host it? :)

-- Pasi


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel