Xen devel - Dec 2009 - HVM support to be removed from Debian Squeeze: call for volunteers

[message cross posted to the pkg-xen and xen-devel list]

Dear everyone,

Bastian Blank - which is the person (among others, but mainly him) that
is packaging Xen in Debian -, has decided last summer that he doesn't
want to deal with the qemu-dm of Xen, thus removing Xen Qemu and support
for HVM in Debian. Here is what he wrote:

http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2009-July/002366.html
http://lists.alioth.debian.org/pipermail/pkg-xen-devel/2009-July/002382.html

Many people have expressed their concerns about this regression.

Before anyone flames Bastian Blank, I'd like to highlight the fact that
maintaining a PV dom0 in a distribution needs quite some work, he does
it for free, and that he should be thanks for his work within Debian,
rather than being blame for his maintainers choice. If he is not
interested or doesn't have time to maintain the Qemu version of Xen, he
has right to do that, and in fact I think it's not a bad idea to have a
binary package separated from the hypervisor itself.

Now, if the qemu-dm binary have been removed from the hypervisor
packages, nothing prevents anyone to make a new Debian package that
would have only the HVM bits of Xen, depending on the hypervisor
package, and then having it uploaded to SID (so that it would go in
testing later). Later, we could ask the maintainers of the
xen-linux-system package to add yet another dependency to the new
"xen-hvm-support" package that we would create (are you ok with this
new
package name?).

The issue here is that Squeeze (the current Debian testing) is said to
be going to be functionality frozen next march. That means that if we
want to have a xen-hvm-support package in the next Debian, and recover
from Bastian's choice, then we need to act FAST, otherwise after the
deadline, there's a big chance that the release managers will not accept
the package into testing.

I am quite comfortable with Debian packaging myself, so I believe I can
write Debian packages with a quality good enough to enter the
distribution without too much hassle. I manage already a dozen of
packages in Debian, and I intend to add even more. But I do not feel
comfortable enough with the Xen sources to do it alone. So I'm calling
here for volunteers to work together with me on this package. I also
don't want to be the only person responsible for the package, as I
wouldn't be able to address any security bug fixing issue (as I don't
know the internals of the Xen and Qemu code, and don't have the time to
dive into it). Someone that is willing to do this job of security
maintenance HAS to raise hand here, otherwise I will NOT shoot myself in
the foot by asking for an upload of a package that would have a bad
security maintenance team (this is the kind of behavior that people hate
in Debian, which I think is right: no package with poor maintenance
should enter the distribution).

Mainly, the work would be to have a script (or some Makefile entries
with DESTDIR= for example) that would install all needed files in
debian/xen-hvm-support (if we call that package xen-hvm-support). Also,
the source package should contain ONLY what is strictly concerning
Qemu-dm (not a big issue, but it would be cleaner this way and make the
source package smaller). As of now, I don't know who is maintaining Qemu
in Xen at all. Any volunteer here that would work with me does NOT need
to know anything about Debian or Debian packaging, but only about Qemu
and Xen itself.

Also, I'm not a registered Debian developer (yet), so I can't upload
directly in the distribution without going by a sponsor for my package,
which can seriously slow down everything. If a registered DD could raise
hand, that would help as well.

I think that the best way here, is to build a (small, 3 or 4 persons)
team to work on this issue so it can be addressed fast. I'd like to
highlight also, that my company (GPLHost) doesn't really need the HVM
part of Xen (we sell PV only VMs), so I'd be doing it only for the
community, and not for myself. So please volunteer, as this is the only
way it can work. Best could be to create a project in Alioth (which I
never did, maybe someone else who had this experience should do it for
the team), or anywhere else. GPLHost can host the git repository and/or
web space for that.

If you are interested in helping, please reply to this message.

Thomas

P.S. to Bastian: Are you willing to help the team with advices, even if
you don't take any responsibility in the package?

Thomas Goirand writes ("[Xen-devel] HVM support to be removed from Debian
Squeeze: call for volunteers"):
>  Someone that is willing to do this job of security
> maintenance HAS to raise hand here,
Hello.  Yes, I volunteer (with my day job hat on).  I've done security
patching before in other contexts.
> Mainly, the work would be to have a script (or some Makefile entries
> with DESTDIR= for example) that would install all needed files in
You mean in the upstream package ?  This isn't quite trivial but it
should be doable.  There is a slight difficulty in that the filesystem
layout of the Debian Xen packages is quite different to upstream's.

But partitioning the output of `make dist-tools' etc. to provide
something that can be used for qemu-dm build should be easy enough.
> debian/xen-hvm-support (if we call that package xen-hvm-support). Also,
> the source package should contain ONLY what is strictly concerning
> Qemu-dm (not a big issue, but it would be cleaner this way and make the
> source package smaller). As of now, I don't know who is maintaining
Qemu
> in Xen at all. Any volunteer here that would work with me does NOT need
> to know anything about Debian or Debian packaging, but only about Qemu
> and Xen itself.
I'm very happy to help out with this
> Also, I'm not a registered Debian developer (yet), so I can't
upload
> directly in the distribution without going by a sponsor for my package,
> which can seriously slow down everything. If a registered DD could raise
> hand, that would help as well.
[swaps hats]

Also, I am a Debian Developer although I haven't been very active in
Debian recently.  I would be happy to sponsor uploads, provided they
seemed sane to me.
> I think that the best way here, is to build a (small, 3 or 4 persons)
> team to work on this issue so it can be addressed fast.
Yes.

Regards,
Ian.

On Tue, Dec 15, 2009 at 12:59:13PM +0800, Thomas Goirand
wrote:
> Now, if the qemu-dm binary have been removed from the hypervisor
> packages, nothing prevents anyone to make a new Debian package that
> would have only the HVM bits of Xen, depending on the hypervisor
> package, and then having it uploaded to SID (so that it would go in
> testing later).
I fact I already worked on such a package. The qemu-fork just need some
small patches to link against the provided libraries. However it is
again a fork and the patches are not clean enough to merge that into
qemu upstream yet and build it from their.
>                 Later, we could ask the maintainers of the
> xen-linux-system package to add yet another dependency to the new
> "xen-hvm-support" package that we would create (are you ok with
this new
> package name?).
No, this is xen-qemu-dm. HVM works properly without, using a stubdomain.
>                Someone that is willing to do this job of security
> maintenance HAS to raise hand here, otherwise I will NOT shoot myself in
> the foot by asking for an upload of a package that would have a bad
> security maintenance team (this is the kind of behavior that people hate
> in Debian, which I think is right: no package with poor maintenance
> should enter the distribution).
The Xen in stable and oldstable is currently in this state. I fixed
several bugs in the core hypervisor but ignored the qemu part.
>                  Best could be to create a project in Alioth (which I
> never did, maybe someone else who had this experience should do it for
> the team), or anywhere else. GPLHost can host the git repository and/or
> web space for that.
This is best maintained as part of the existing Xen team, which needs
some cleanups.
> P.S. to Bastian: Are you willing to help the team with advices, even if
> you don't take any responsibility in the package?
Sure. I'm always available for advices.

I would like to have some further packages anyway: pv-grub,
qemu-stubdom. But the build environment for that needs a complete
rewrite to be able to produce acceptable packages. It needs to reuse
existing packages.

Bastian

-- 
A Vulcan can no sooner be disloyal than he can exist without breathing.
		-- Kirk, "The Menagerie", stardate 3012.4

On Tue, Dec 15, 2009 at 07:02:28PM +0100, Bastian Blank
wrote:
> I fact I already worked on such a package. The qemu-fork just need some
> small patches to link against the provided libraries. However it is
> again a fork and the patches are not clean enough to merge that into
> qemu upstream yet and build it from their.
http://hermes.jura.uni-tuebingen.de/~blank/debian/xen-test/
d7ac980cf0ba90ed2aa8dd0122643fc591132f605f73e98bcdc984dd10e0b1df 
qemu-xen_3.4.2-1.debian.tar.gz
2fc5b931ae8f706f0a9a1da9dd1c9f5158d3d50a5cfc4b214dc88c9fb122a815 
qemu-xen_3.4.2.orig.tar.gz

A little bit raw, but should work.

Bastian

-- 
There is a multi-legged creature crawling on your shoulder.
		-- Spock, "A Taste of Armageddon", stardate 3193.9
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: Digital signature
URL:
<http://lists.alioth.debian.org/pipermail/pkg-xen-devel/attachments/20091216/d183b8a1/attachment.pgp>

Thomas Goirand writes ("Qemu-dm compiling error with
SDL"):
> Doing the following:
> 
> cp -f /usr/share/misc/config.sub .
> cp -f /usr/share/misc/config.guess
> ./configure --audio-drv-list="oss alsa sdl pa esd" \
> --audio-card-list="ac97 es1370 sb16 cs4231a adlib gus"
--enable-mixemu
> $(MAKE) install DESTDIR=debian/$(PKG_NAME)
The qemu-xen-*.git tree needs to be built with the "xen-setup"
script, rather than just running ./configure.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Thomas Goirand writes ("Qemu-dm compiling error with SDL"):
>> Doing the following:
>>
>> cp -f /usr/share/misc/config.sub .
>> cp -f /usr/share/misc/config.guess
>> ./configure --audio-drv-list="oss alsa sdl pa esd" \
>> --audio-card-list="ac97 es1370 sb16 cs4231a adlib gus"
--enable-mixemu
>> $(MAKE) install DESTDIR=debian/$(PKG_NAME)
> 
> The qemu-xen-*.git tree needs to be built with the "xen-setup"
> script, rather than just running ./configure.
> 
> Ian.
Thanks for this, I have now a package (which is litian clean). Resulting
work is here:

http://ftparchive.gplhost.com/debian/pool/lenny/main/x/xen-qemu-dm-3.4/

I have few questions though.

1- What is qemu-nbd-xen for, and do I need to package it (together with
its manpage)? Should it go in /usr/bin?

2- Is there anything else than qemu-dm, or qemu-img-xen, that needs to
be present in this package?

3- By default, the xen-setup script does:

./configure --disable-gfx-check --disable-curses \
   --disable-slirp "$@" --prefix=/usr

but I saw in the help of ./configure that there is:
--audio-drv-list

that can have the value oss alsa sdl esd pa fmod. I really believe that
OSS support only is not a good idea, and that ALSA, Pulse Audio, SDL and
ESD support would be good. Are they activated by default?

4- Is this normal, and could it be avoided:

dpkg-shlibdeps: warning: dependency on libm.so.6 could be avoided if
"debian/xen-qemu-dm-3.4/usr/lib/xen-3.4/boot/qemu-dm" were not
uselessly
linked against it (they use none of its symbols).
dpkg-shlibdeps: warning: dependency on libXext.so.6 could be avoided if
"debian/xen-qemu-dm-3.4/usr/lib/xen-3.4/boot/qemu-dm" were not
uselessly
linked against it (they use none of its symbols).

Another topic,

The list of persons that have volunteered for working on the package are
the persons in Cc: in this messages. Now, I need to make a common email
address for it. I can make a xenqemudm@gplhost.com, redirecting to us 4,
but I''m open to any other solution. Especially considering that I will
still need sponsoring and that Ian J. would be the one uploading, maybe
an email address @debian.org, signed with Ian J.''s Debian key would
make
more sense. Ian J., what do you think, and can you take care of that?

Thomas

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel