Fedora xen - Aug 2009 - [Xen-devel] [ANNOUNCE] Xen 3.3.2 and 3.4.1 released

----- Forwarded message from Keir Fraser <keir.fraser@eu.citrix.com> -----

From: Keir Fraser <keir.fraser@eu.citrix.com>
To: "xen-devel@lists.xensource.com"
<xen-devel@lists.xensource.com>,
	xen-users@lists.xensource.com
Cc: 
Date: Thu, 06 Aug 2009 23:35:08 +0100
Subject: [Xen-devel] [ANNOUNCE] Xen 3.3.2 and 3.4.1 released

Folks,

New releases in the 3.3 and 3.4 stable branches are now tagged and released.

You can pick up the source repositories here:
http://xenbits.xensource.com/xen-3.3-testing.hg (tagged RELEASE-3.3.2)
http://xenbits.xensource.com/xen-3.4-testing.hg (tagged RELEASE-3.4.1)

Or you can pick up tarballs from the following webpages:
3.3.2: http://www.xen.org/download/index_3.3.2.html
3.4.1: http://www.xen.org/download/

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

----- End forwarded message -----

Hi,
> 3.4.1: http://www.xen.org/download/
Great timing.  They managed to do the release just one day after the 
alpha freeze.  Packages are built, but they will not hit the repos until 
the alpha is done.  You can grab them from koji though:

http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/1.fc12/

enjoy,
   Gerd

Hi Gerd,

On Tue, 2009-08-11 at 10:24 +0200, Gerd Hoffmann wrote:
> Hi,
> 
> > 3.4.1: http://www.xen.org/download/
> 
> Great timing.  They managed to do the release just one day after the 
> alpha freeze.  Packages are built, but they will not hit the repos until 
> the alpha is done.  You can grab them from koji though:
> 
> http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/1.fc12/
See:

  https://fedoraproject.org/wiki/ReleaseEngineering/DevelFreezePolicy

You can file a tag request to have it tagged into f12-alpha. I''ve done
a
number of these:

  https://fedorahosted.org/rel-eng/ticket/2051
  https://fedorahosted.org/rel-eng/ticket/2052
  https://fedorahosted.org/rel-eng/ticket/2053
  https://fedorahosted.org/rel-eng/ticket/2060
  https://fedorahosted.org/rel-eng/ticket/2088

The alpha has been delayed a week now and since problems in the xen
package are only going to affect a small number of bleeding edge
testers, I don''t see why rel-eng would refuse the request.

Cheers,
Mark.

On Tue, Aug 11, 2009 at 10:24:29AM +0200, Gerd Hoffmann
wrote:
>   Hi,
> 
> >3.4.1: http://www.xen.org/download/
> 
> Great timing.  They managed to do the release just one day after the 
> alpha freeze.  Packages are built, but they will not hit the repos until 
> the alpha is done.  You can grab them from koji though:
> 
> http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/1.fc12/
> 
Gerd: Would you like to apply some bugfix patches for 3.4.1-2 ?

List of important fixes:

- fix pygrub timeout=0 handling, enable Fedora 11 PV domUs to boot:
  http://xenbits.xen.org/staging/xen-3.4-testing.hg?rev/7a57911ff09e

- fix qemu-dm memory leak on block/disk IO
  http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00335.html


Nice to have:

- fix qemu problem when stdvga=1 and videoram=16 are specified, starting hvm
domain fails
  http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00281.html

- fix stubdoms bridge info parsing, use correct bridge
  http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00393.html


-- Pasi

On 08/20/09 16:45, Pasi Kärkkäinen wrote:
> Gerd: Would you like to apply some bugfix patches for 3.4.1-2 ?
I''ll have a look next week.

Having bzip2+lzma sorted until then would be great,
so I can put that in as well.

cheers,
   Gerd

On Fri, Aug 21, 2009 at 12:35:17PM +0200, Gerd Hoffmann
wrote:
> On 08/20/09 16:45, Pasi Kärkkäinen wrote:
> >Gerd: Would you like to apply some bugfix patches for 3.4.1-2 ?
> 
> I''ll have a look next week.
> 
Thanks.
> Having bzip2+lzma sorted until then would be great,
> so I can put that in as well.
> 
Working on it now..

-- Pasi

On Fri, Aug 21, 2009 at 05:16:04PM +0300, Pasi Kärkkäinen
wrote:
> On Fri, Aug 21, 2009 at 12:35:17PM +0200, Gerd Hoffmann wrote:
> > On 08/20/09 16:45, Pasi Kärkkäinen wrote:
> > >Gerd: Would you like to apply some bugfix patches for 3.4.1-2 ?
> > 
> > I''ll have a look next week.
> > 
> 
Did you have time to take a look at those?
> 
> > Having bzip2+lzma sorted until then would be great,
> > so I can put that in as well.
> > 
> 
> Working on it now..
> 
I haven''t had time to get done with the stubdom makefile voodoo yet..

-- Pasi

On 08/27/09 16:38, Pasi Kärkkäinen wrote:
> On Fri, Aug 21, 2009 at 05:16:04PM +0300, Pasi Kärkkäinen wrote:
>> On Fri, Aug 21, 2009 at 12:35:17PM +0200, Gerd Hoffmann wrote:
>>> On 08/20/09 16:45, Pasi Kärkkäinen wrote:
>>>> Gerd: Would you like to apply some bugfix patches for 3.4.1-2 ?
>>>
>>> I''ll have a look next week.
>>>
>>
>
> Did you have time to take a look at those?
Didn''t look, was waiting for the stubdom stuff ...
>>> Having bzip2+lzma sorted until then would be great,
>>> so I can put that in as well.
>>>
>>
>> Working on it now..
>>
>
> I haven''t had time to get done with the stubdom makefile voodoo
yet..
Ok, I''ll try to update the package today or tomorrow then and make a
one
more update once the stubdom bits are finally sorted.

cheers,
   Gerd

On Thu, Aug 27, 2009 at 04:44:45PM +0200, Gerd Hoffmann
wrote:
> On 08/27/09 16:38, Pasi Kärkkäinen wrote:
> >On Fri, Aug 21, 2009 at 05:16:04PM +0300, Pasi Kärkkäinen wrote:
> >>On Fri, Aug 21, 2009 at 12:35:17PM +0200, Gerd Hoffmann wrote:
> >>>On 08/20/09 16:45, Pasi Kärkkäinen wrote:
> >>>>Gerd: Would you like to apply some bugfix patches for
3.4.1-2 ?
> >>>
> >>>I''ll have a look next week.
> >>>
> >>
> >
> >Did you have time to take a look at those?
> 
> Didn''t look, was waiting for the stubdom stuff ...
> 
> >>>Having bzip2+lzma sorted until then would be great,
> >>>so I can put that in as well.
> >>>
> >>
> >>Working on it now..
> >>
> >
> >I haven''t had time to get done with the stubdom makefile
voodoo yet..
> 
> Ok, I''ll try to update the package today or tomorrow then and make
a one
> more update once the stubdom bits are finally sorted.
> 
OK. 

You can include the userspace pv bootloader bits already.

http://xenbits.xen.org/staging/xen-unstable.hg?rev/a687c4d35fdd
http://xenbits.xen.org/staging/xen-unstable.hg?rev/7dad2e23bf89
http://xenbits.xen.org/staging/xen-unstable.hg?rev/c5125c0ea051

I believe those are the needed changesets from xen-unstable. 

xen.spec needs to have xz-devel and bzip2-devel in the builddeps.

-- Pasi

> - fix pygrub timeout=0 handling, enable Fedora 11 PV domUs to boot:
>    http://xenbits.xen.org/staging/xen-3.4-testing.hg?rev/7a57911ff09e
>
> - fix qemu-dm memory leak on block/disk IO
>    http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00335.html
Committed, build running.
> - fix qemu problem when stdvga=1 and videoram=16 are specified, starting
hvm domain fails
>    http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00281.html
skipped -- ia64 only.
> - fix stubdoms bridge info parsing, use correct bridge
>    http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00393.html
Patch doesn''t apply cleanly.

cheers,
   Gerd

Hi,
> http://xenbits.xen.org/staging/xen-unstable.hg?rev/a687c4d35fdd
> http://xenbits.xen.org/staging/xen-unstable.hg?rev/7dad2e23bf89
Committed.
> http://xenbits.xen.org/staging/xen-unstable.hg?rev/c5125c0ea051
Doesn''t apply, probably depends on other changesets ...

cheers,
   Gerd

On Tue, Sep 01, 2009 at 03:04:16PM +0200, Gerd Hoffmann
wrote:
>   Hi,
> 
> >http://xenbits.xen.org/staging/xen-unstable.hg?rev/a687c4d35fdd
> >http://xenbits.xen.org/staging/xen-unstable.hg?rev/7dad2e23bf89
> 
> Committed.
> 
> >http://xenbits.xen.org/staging/xen-unstable.hg?rev/c5125c0ea051
> 
> Doesn''t apply, probably depends on other changesets ...
> 
I''ll check these and get back to you soon..

-- Pasi

On Tue, Sep 01, 2009 at 03:03:23PM +0200, Gerd Hoffmann
wrote:
> >- fix pygrub timeout=0 handling, enable Fedora 11 PV domUs to boot:
> >   http://xenbits.xen.org/staging/xen-3.4-testing.hg?rev/7a57911ff09e
> >
> >- fix qemu-dm memory leak on block/disk IO
> >  
http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00335.html
> 
> Committed, build running.
> 
> >- fix qemu problem when stdvga=1 and videoram=16 are specified,
starting
> >hvm domain fails
> >  
http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00281.html
> 
> skipped -- ia64 only.
> 
Actually no, it shouldn''t be ia64 only.

See:
http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00276.html
> >- fix stubdoms bridge info parsing, use correct bridge
> >  
http://lists.xensource.com/archives/html/xen-devel/2009-08/msg00393.html
> 
> Patch doesn''t apply cleanly.
> 
Oh yeah, that''d need backporting to 3.4 first. I guess we can skip this
at this point.

-- Pasi

On Tue, Sep 01, 2009 at 04:40:51PM +0300, Pasi Kärkkäinen
wrote:
> On Tue, Sep 01, 2009 at 03:04:16PM +0200, Gerd Hoffmann wrote:
> >   Hi,
> > 
> > >http://xenbits.xen.org/staging/xen-unstable.hg?rev/a687c4d35fdd
> > >http://xenbits.xen.org/staging/xen-unstable.hg?rev/7dad2e23bf89
> > 
> > Committed.
> > 
> > >http://xenbits.xen.org/staging/xen-unstable.hg?rev/c5125c0ea051
> > 
> > Doesn''t apply, probably depends on other changesets ...
> > 
> 
> I''ll check these and get back to you soon..
> 
Attached is a patch that backports the bzip2/lzma-stuff from current
xen-unstable
to xen-3.4.1. Please use this patch instead of the above changesets from
xen-unstable (this patch replaces all of them).

It applies cleanly to xen-3.4.1-1 src.rpm (with -p1) and the resulting
compiled binaries seem to work. I tried with booting lzma compressed
rawhide/F12 kernel.

Still no stubdom/pvgrub bzip2/lzma support though. I''ll look more into
it tonight.

-- Pasi


--9jxsPFA5p3P2qPhR
Content-Type: text/x-diff; charset=us-ascii
Content-Disposition: attachment;
	filename="xen-341-add-bzip2-lzma-pv-bootloader-support-v2-no-stubdom-pvgrub.patch"

--- xen-3.4-testing.hg/tools/libxc/Makefile	2009-09-01 17:19:44.000000000 +0300
+++ xen-unstable.hg/tools/libxc/Makefile	2009-09-01 17:12:45.000000000 +0300
@@ -149,6 +151,25 @@
 libxenguest.so.$(MAJOR): libxenguest.so.$(MAJOR).$(MINOR)
 	ln -sf $< $@
 
+ifeq ($(CONFIG_MiniOS),y)
+zlib-options +else
+zlib-options = $(shell					\
+	(. ../check/funcs.sh;				\
+	 if has_header bzlib.h; then			\
+           echo "-DHAVE_BZLIB";				\
+	   echo "-lbz2";				\
+	 fi;						\
+	 if has_header lzma.h; then			\
+	   echo "-DHAVE_LZMA";				\
+	   echo "-llzma";				\
+	 fi) | grep $(1))
+endif
+
+xc_dom_bzimageloader.o: CFLAGS += $(call zlib-options,D)
+xc_dom_bzimageloader.opic: CFLAGS += $(call zlib-options,D)
+
+libxenguest.so.$(MAJOR).$(MINOR): LDFLAGS += $(call zlib-options,l)
 libxenguest.so.$(MAJOR).$(MINOR): $(GUEST_PIC_OBJS) libxenctrl.so
 	$(CC) $(CFLAGS) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenguest.so.$(MAJOR)
$(SHLIB_CFLAGS) -o $@ $(GUEST_PIC_OBJS) -lz -lxenctrl $(PTHREAD_LIBS)
 
--- xen-3.4-testing.hg/tools/libxc/xc_dom_bzimageloader.c	2009-09-01
17:19:44.000000000 +0300
+++ xen-unstable.hg/tools/libxc/xc_dom_bzimageloader.c	2009-09-01
17:12:45.000000000 +0300
@@ -11,6 +11,7 @@
  * written 2006 by Gerd Hoffmann <kraxel@suse.de>.
  * written 2007 by Jeremy Fitzhardinge <jeremy@xensource.com>
  * written 2008 by Ian Campbell <ijc@hellion.org.uk>
+ * written 2009 by Chris Lalancette <clalance@redhat.com>
  *
  */
 #include <stdio.h>
@@ -20,43 +21,289 @@
 #include "xg_private.h"
 #include "xc_dom.h"
 
+#if defined(HAVE_BZLIB)
+
+#include <bzlib.h>
+
+static int xc_try_bzip2_decode(
+    struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    bz_stream stream;
+    int ret;
+    char *out_buf;
+    int retval = -1;
+    int outsize;
+    uint64_t total;
+
+    stream.bzalloc = NULL;
+    stream.bzfree = NULL;
+    stream.opaque = NULL;
+
+    ret = BZ2_bzDecompressInit(&stream, 0, 0);
+    if ( ret != BZ_OK )
+    {
+        xc_dom_printf("Error initting bz2 stream\n");
+        return -1;
+    }
+
+    /* sigh.  We don''t know up-front how much memory we are going to
need
+     * for the output buffer.  Allocate the output buffer to be equal
+     * the input buffer to start, and we''ll realloc as needed.
+     */
+    outsize = dom->kernel_size;
+    out_buf = malloc(outsize);
+    if ( out_buf == NULL )
+    {
+        xc_dom_printf("Failed to alloc memory\n");
+        goto bzip2_cleanup;
+    }
+
+    stream.next_in = dom->kernel_blob;
+    stream.avail_in = dom->kernel_size;
+
+    stream.next_out = out_buf;
+    stream.avail_out = dom->kernel_size;
+
+    for ( ; ; )
+    {
+        ret = BZ2_bzDecompress(&stream);
+        if ( (stream.avail_out == 0) || (ret != BZ_OK) )
+        {
+            out_buf = realloc(out_buf, outsize * 2);
+            if ( out_buf == NULL )
+            {
+                xc_dom_printf("Failed to realloc memory\n");
+                break;
+            }
+
+            stream.next_out = out_buf + outsize;
+            stream.avail_out = (outsize * 2) - outsize;
+            outsize *= 2;
+        }
+
+        if ( ret != BZ_OK )
+        {
+            if ( ret == BZ_STREAM_END )
+            {
+                xc_dom_printf("Saw data stream end\n");
+                retval = 0;
+                break;
+            }
+            xc_dom_printf("BZIP error\n");
+        }
+    }
+
+    total = (stream.total_out_hi32 << 31) | stream.total_out_lo32;
+
+    xc_dom_printf("%s: BZIP2 decompress OK, 0x%zx -> 0x%lx\n",
+                  __FUNCTION__, *size, (long unsigned int) total);
+
+    *blob = out_buf;
+    *size = total;
+
+ bzip2_cleanup:
+    BZ2_bzDecompressEnd(&stream);
+
+    return retval;
+}
+
+#else /* !defined(HAVE_BZLIB) */
+
+static int xc_try_bzip2_decode(
+    struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    xc_dom_printf("%s: BZIP2 decompress support unavailable\n",
+                  __FUNCTION__);
+    return -1;
+}
+
+#endif
+
+#if defined(HAVE_LZMA)
+
+#include <lzma.h>
+
+static uint64_t physmem(void)
+{
+    uint64_t ret = 0;
+    const long pagesize = sysconf(_SC_PAGESIZE);
+    const long pages = sysconf(_SC_PHYS_PAGES);
+
+    if ( (pagesize != -1) || (pages != -1) )
+    {
+        /*
+         * According to docs, pagesize * pages can overflow.
+         * Simple case is 32-bit box with 4 GiB or more RAM,
+         * which may report exactly 4 GiB of RAM, and "long"
+         * being 32-bit will overflow. Casting to uint64_t
+         * hopefully avoids overflows in the near future.
+         */
+        ret = (uint64_t)(pagesize) * (uint64_t)(pages);
+    }
+
+    return ret;
+}
+
+static int xc_try_lzma_decode(
+    struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    lzma_stream stream = LZMA_STREAM_INIT;
+    lzma_ret ret;
+    lzma_action action = LZMA_RUN;
+    unsigned char *out_buf;
+    int retval = -1;
+    int outsize;
+    const char *msg;
+
+    ret = lzma_alone_decoder(&stream, physmem() / 3);
+    if ( ret != LZMA_OK )
+    {
+        xc_dom_printf("Failed to init lzma stream decoder\n");
+        return -1;
+    }
+
+    /* sigh.  We don''t know up-front how much memory we are going to
need
+     * for the output buffer.  Allocate the output buffer to be equal
+     * the input buffer to start, and we''ll realloc as needed.
+     */
+    outsize = dom->kernel_size;
+    out_buf = malloc(outsize);
+    if ( out_buf == NULL )
+    {
+        xc_dom_printf("Failed to alloc memory\n");
+        goto lzma_cleanup;
+    }
+
+    stream.next_in = dom->kernel_blob;
+    stream.avail_in = dom->kernel_size;
+
+    stream.next_out = out_buf;
+    stream.avail_out = dom->kernel_size;
+
+    for ( ; ; )
+    {
+        ret = lzma_code(&stream, action);
+        if ( (stream.avail_out == 0) || (ret != LZMA_OK) )
+        {
+            out_buf = realloc(out_buf, outsize * 2);
+            if ( out_buf == NULL )
+            {
+                xc_dom_printf("Failed to realloc memory\n");
+                break;
+            }
+
+            stream.next_out = out_buf + outsize;
+            stream.avail_out = (outsize * 2) - outsize;
+            outsize *= 2;
+        }
+
+        if ( ret != LZMA_OK )
+        {
+            if ( ret == LZMA_STREAM_END )
+            {
+                xc_dom_printf("Saw data stream end\n");
+                retval = 0;
+                break;
+            }
+
+            switch ( ret )
+            {
+            case LZMA_MEM_ERROR:
+                msg = strerror(ENOMEM);
+                break;
+
+            case LZMA_MEMLIMIT_ERROR:
+                msg = "Memory usage limit reached";
+                break;
+
+            case LZMA_FORMAT_ERROR:
+                msg = "File format not recognized";
+                break;
+
+            case LZMA_OPTIONS_ERROR:
+                // FIXME: Better message?
+                msg = "Unsupported compression options";
+                break;
+
+            case LZMA_DATA_ERROR:
+                msg = "File is corrupt";
+                break;
+
+            case LZMA_BUF_ERROR:
+                msg = "Unexpected end of input";
+                break;
+
+            default:
+                msg = "Internal program error (bug)";
+                break;
+            }
+            xc_dom_printf("%s: LZMA decompression error %s\n",
+                          __FUNCTION__, msg);
+            break;
+        }
+    }
+
+    xc_dom_printf("%s: LZMA decompress OK, 0x%zx -> 0x%zx\n",
+                  __FUNCTION__, *size, (size_t)stream.total_out);
+
+    *blob = out_buf;
+    *size = stream.total_out;
+
+ lzma_cleanup:
+    lzma_end(&stream);
+
+    return retval;
+}
+
+#else /* !defined(HAVE_LZMA) */
+
+static int xc_try_lzma_decode(
+    struct xc_dom_image *dom, void **blob, size_t *size)
+{
+    xc_dom_printf("%s: LZMA decompress support unavailable\n",
+                  __FUNCTION__);
+    return -1;
+}
+
+#endif
+
 struct setup_header {
-	uint8_t		_pad0[0x1f1];		/* skip uninteresting stuff */
-	uint8_t		setup_sects;
-	uint16_t	root_flags;
-	uint32_t	syssize;
-	uint16_t	ram_size;
-	uint16_t	vid_mode;
-	uint16_t	root_dev;
-	uint16_t	boot_flag;
-	uint16_t	jump;
-	uint32_t	header;
-#define HDR_MAGIC		"HdrS"
-#define HDR_MAGIC_SZ	4
-	uint16_t	version;
-#define VERSION(h,l)	(((h)<<8) | (l))
-	uint32_t	realmode_swtch;
-	uint16_t	start_sys;
-	uint16_t	kernel_version;
-	uint8_t		type_of_loader;
-	uint8_t		loadflags;
-	uint16_t	setup_move_size;
-	uint32_t	code32_start;
-	uint32_t	ramdisk_image;
-	uint32_t	ramdisk_size;
-	uint32_t	bootsect_kludge;
-	uint16_t	heap_end_ptr;
-	uint16_t	_pad1;
-	uint32_t	cmd_line_ptr;
-	uint32_t	initrd_addr_max;
-	uint32_t	kernel_alignment;
-	uint8_t		relocatable_kernel;
-	uint8_t		_pad2[3];
-	uint32_t	cmdline_size;
-	uint32_t	hardware_subarch;
-	uint64_t	hardware_subarch_data;
-	uint32_t	payload_offset;
-	uint32_t	payload_length;
+    uint8_t  _pad0[0x1f1];  /* skip uninteresting stuff */
+    uint8_t  setup_sects;
+    uint16_t root_flags;
+    uint32_t syssize;
+    uint16_t ram_size;
+    uint16_t vid_mode;
+    uint16_t root_dev;
+    uint16_t boot_flag;
+    uint16_t jump;
+    uint32_t header;
+#define HDR_MAGIC  "HdrS"
+#define HDR_MAGIC_SZ 4
+    uint16_t version;
+#define VERSION(h,l) (((h)<<8) | (l))
+    uint32_t realmode_swtch;
+    uint16_t start_sys;
+    uint16_t kernel_version;
+    uint8_t  type_of_loader;
+    uint8_t  loadflags;
+    uint16_t setup_move_size;
+    uint32_t code32_start;
+    uint32_t ramdisk_image;
+    uint32_t ramdisk_size;
+    uint32_t bootsect_kludge;
+    uint16_t heap_end_ptr;
+    uint16_t _pad1;
+    uint32_t cmd_line_ptr;
+    uint32_t initrd_addr_max;
+    uint32_t kernel_alignment;
+    uint8_t  relocatable_kernel;
+    uint8_t  _pad2[3];
+    uint32_t cmdline_size;
+    uint32_t hardware_subarch;
+    uint64_t hardware_subarch_data;
+    uint32_t payload_offset;
+    uint32_t payload_length;
 } __attribute__((packed));
 
 extern struct xc_dom_loader elf_loader;
@@ -70,22 +317,22 @@
     return off;
 }
 
-static int check_bzimage_kernel(struct xc_dom_image *dom, int verbose)
+static int xc_dom_probe_bzimage_kernel(struct xc_dom_image *dom)
 {
     struct setup_header *hdr;
+    int ret;
 
     if ( dom->kernel_blob == NULL )
     {
-        if ( verbose )
-            xc_dom_panic(XC_INTERNAL_ERROR, "%s: no kernel image
loaded\n",
-                         __FUNCTION__);
+        xc_dom_panic(XC_INTERNAL_ERROR, "%s: no kernel image
loaded\n",
+                     __FUNCTION__);
         return -EINVAL;
     }
+
     if ( dom->kernel_size < sizeof(struct setup_header) )
     {
-        if ( verbose )
-            xc_dom_panic(XC_INTERNAL_ERROR, "%s: kernel image too
small\n",
-                         __FUNCTION__);
+        xc_dom_panic(XC_INTERNAL_ERROR, "%s: kernel image too
small\n",
+                     __FUNCTION__);
         return -EINVAL;
     }
 
@@ -93,39 +340,64 @@
 
     if ( memcmp(&hdr->header, HDR_MAGIC, HDR_MAGIC_SZ) != 0 )
     {
-        if ( verbose )
-            xc_dom_panic(XC_INVALID_KERNEL, "%s: kernel is not a
bzImage\n",
-                         __FUNCTION__);
+        xc_dom_panic(XC_INVALID_KERNEL, "%s: kernel is not a
bzImage\n",
+                     __FUNCTION__);
         return -EINVAL;
     }
 
     if ( hdr->version < VERSION(2,8) )
     {
-        if ( verbose )
-            xc_dom_panic(XC_INVALID_KERNEL, "%s: boot protocol too old
(%04x)\n",
-                         __FUNCTION__, hdr->version);
+        xc_dom_panic(XC_INVALID_KERNEL, "%s: boot protocol too old
(%04x)\n",
+                     __FUNCTION__, hdr->version);
         return -EINVAL;
     }
 
     dom->kernel_blob = dom->kernel_blob + payload_offset(hdr);
     dom->kernel_size = hdr->payload_length;
 
-    if ( xc_dom_try_gunzip(dom, &dom->kernel_blob,
&dom->kernel_size) == -1 )
+    if ( memcmp(dom->kernel_blob, "\037\213", 2) == 0 )
+    {
+        ret = xc_dom_try_gunzip(dom, &dom->kernel_blob,
&dom->kernel_size);
+        if ( ret == -1 )
+        {
+            xc_dom_panic(XC_INVALID_KERNEL,
+                         "%s: unable to gzip decompress kernel\n",
+                         __FUNCTION__);
+            return -EINVAL;
+        }
+    }
+    else if ( memcmp(dom->kernel_blob, "\102\132\150", 3) == 0 )
     {
-        if ( verbose )
-            xc_dom_panic(XC_INVALID_KERNEL, "%s: unable to decompress
kernel\n",
+        ret = xc_try_bzip2_decode(dom, &dom->kernel_blob,
&dom->kernel_size);
+        if ( ret < 0 )
+        {
+            xc_dom_panic(XC_INVALID_KERNEL,
+                         "%s unable to BZIP2 decompress kernel",
                          __FUNCTION__);
+            return -EINVAL;
+        }
+    }
+    else if ( memcmp(dom->kernel_blob, "\135\000", 2) == 0 )
+    {
+        ret = xc_try_lzma_decode(dom, &dom->kernel_blob,
&dom->kernel_size);
+        if ( ret < 0 )
+        {
+            xc_dom_panic(XC_INVALID_KERNEL,
+                         "%s unable to LZMA decompress kernel\n",
+                         __FUNCTION__);
+            return -EINVAL;
+        }
+    }
+    else
+    {
+        xc_dom_panic(XC_INVALID_KERNEL, "%s: unknown compression
format\n",
+                     __FUNCTION__);
         return -EINVAL;
     }
 
     return elf_loader.probe(dom);
 }
 
-static int xc_dom_probe_bzimage_kernel(struct xc_dom_image *dom)
-{
-    return check_bzimage_kernel(dom, 0);
-}
-
 static int xc_dom_parse_bzimage_kernel(struct xc_dom_image *dom)
 {
     return elf_loader.parser(dom);

--9jxsPFA5p3P2qPhR--

On Tue, Sep 01, 2009 at 05:50:20PM +0300, Pasi Kärkkäinen
wrote:
> On Tue, Sep 01, 2009 at 04:40:51PM +0300, Pasi Kärkkäinen wrote:
> > On Tue, Sep 01, 2009 at 03:04:16PM +0200, Gerd Hoffmann wrote:
> > >   Hi,
> > > 
> > >
>http://xenbits.xen.org/staging/xen-unstable.hg?rev/a687c4d35fdd
> > >
>http://xenbits.xen.org/staging/xen-unstable.hg?rev/7dad2e23bf89
> > > 
> > > Committed.
> > > 
> > >
>http://xenbits.xen.org/staging/xen-unstable.hg?rev/c5125c0ea051
> > > 
> > > Doesn''t apply, probably depends on other changesets ...
> > > 
> > 
> > I''ll check these and get back to you soon..
> > 
> 
> Attached is a patch that backports the bzip2/lzma-stuff from current
xen-unstable
> to xen-3.4.1. Please use this patch instead of the above changesets from
> xen-unstable (this patch replaces all of them).
> 
> It applies cleanly to xen-3.4.1-1 src.rpm (with -p1) and the resulting
> compiled binaries seem to work. I tried with booting lzma compressed
> rawhide/F12 kernel.
> 
> Still no stubdom/pvgrub bzip2/lzma support though. I''ll look more
into
> it tonight.
> 
.. but don''t wait for the final patch, since I don''t know yet
how long
it takes to get the stubdom/pvgrub stuff sorted out.

-- Pasi

Hi,
>> Still no stubdom/pvgrub bzip2/lzma support though. I''ll look
more into
>> it tonight.
>
> .. but don''t wait for the final patch, since I don''t know
yet how long
> it takes to get the stubdom/pvgrub stuff sorted out.
Bits committed, koji builds done.

cheers,
   Gerd

On Wed, Sep 02, 2009 at 02:25:59PM +0200, Gerd Hoffmann
wrote:
>   Hi,
> 
> >>Still no stubdom/pvgrub bzip2/lzma support though. I''ll
look more into
> >>it tonight.
> >
> >.. but don''t wait for the final patch, since I don''t
know yet how long
> >it takes to get the stubdom/pvgrub stuff sorted out.
> 
> Bits committed, koji builds done.
> 
Great, thanks!

Does 3.4.1-3 end up in F12 automatically? 

-- Pasi

Hi,
> Does 3.4.1-3 end up in F12 automatically?
I think so, alpha freeze is over and beta freeze not yet active ...

cheers,
   Gerd

On Wed, Sep 02, 2009 at 03:41:09PM +0300, Pasi Kärkkäinen
wrote:
> On Wed, Sep 02, 2009 at 02:25:59PM +0200, Gerd Hoffmann wrote:
> >   Hi,
> > 
> > >>Still no stubdom/pvgrub bzip2/lzma support though.
I''ll look more into
> > >>it tonight.
> > >
> > >.. but don''t wait for the final patch, since I
don''t know yet how long
> > >it takes to get the stubdom/pvgrub stuff sorted out.
> > 
> > Bits committed, koji builds done.
> > 
> 
> Great, thanks!
> 
> Does 3.4.1-3 end up in F12 automatically? 
> 
btw the url is this:
http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/3.fc12/

Changelog:

* Wed Sep 02 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-3
- swap bzip2+xz linux kernel compression support patches.
- backport one more bugfix (videoram option).

* Tue Sep 01 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-2
- backport bzip2+xz linux kernel compression support.
- backport a few bugfixes.


-- Pasi

On Wed, Sep 02, 2009 at 03:59:31PM +0300, Pasi Kärkkäinen
wrote:
> On Wed, Sep 02, 2009 at 03:41:09PM +0300, Pasi Kärkkäinen wrote:
> > On Wed, Sep 02, 2009 at 02:25:59PM +0200, Gerd Hoffmann wrote:
> > >   Hi,
> > > 
> > > >>Still no stubdom/pvgrub bzip2/lzma support though.
I''ll look more into
> > > >>it tonight.
> > > >
> > > >.. but don''t wait for the final patch, since I
don''t know yet how long
> > > >it takes to get the stubdom/pvgrub stuff sorted out.
> > > 
> > > Bits committed, koji builds done.
> > > 
> > 
> > Great, thanks!
> > 
> > Does 3.4.1-3 end up in F12 automatically? 
> > 
> 
> btw the url is this:
> http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/3.fc12/
> 
> Changelog:
> 
> * Wed Sep 02 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-3
> - swap bzip2+xz linux kernel compression support patches.
> - backport one more bugfix (videoram option).
> 
> * Tue Sep 01 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-2
> - backport bzip2+xz linux kernel compression support.
> - backport a few bugfixes.
> 
btw 3.4.1-2 and 3.4.1-3 do contain the fix for pygrub timeout=0
handling, so F11 PV domUs/guests will work out-of-the-box.

-- Pasi

On Wed, Sep 02, 2009 at 04:08:32PM +0300, Pasi Kärkkäinen
wrote:
> On Wed, Sep 02, 2009 at 03:59:31PM +0300, Pasi Kärkkäinen wrote:
> > On Wed, Sep 02, 2009 at 03:41:09PM +0300, Pasi Kärkkäinen wrote:
> > > On Wed, Sep 02, 2009 at 02:25:59PM +0200, Gerd Hoffmann wrote:
> > > >   Hi,
> > > > 
> > > > >>Still no stubdom/pvgrub bzip2/lzma support though.
I''ll look more into
> > > > >>it tonight.
> > > > >
> > > > >.. but don''t wait for the final patch, since I
don''t know yet how long
> > > > >it takes to get the stubdom/pvgrub stuff sorted out.
> > > > 
> > > > Bits committed, koji builds done.
> > > > 
> > > 
> > > Great, thanks!
> > > 
> > > Does 3.4.1-3 end up in F12 automatically? 
> > > 
> > 
> > btw the url is this:
> > http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/3.fc12/
> > 
> > Changelog:
> > 
> > * Wed Sep 02 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-3
> > - swap bzip2+xz linux kernel compression support patches.
> > - backport one more bugfix (videoram option).
> > 
> > * Tue Sep 01 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-2
> > - backport bzip2+xz linux kernel compression support.
> > - backport a few bugfixes.
> > 
> 
> btw 3.4.1-2 and 3.4.1-3 do contain the fix for pygrub timeout=0
> handling, so F11 PV domUs/guests will work out-of-the-box.
> 
Hello,

I just noticed something weird.. I upgraded my host to F12/rawhide, so
now I''m running everything straight from rawhide, except my custom
pv_ops dom0 kernel.

Anyway, it seems the F12/rawhide xen-3.4.1-3 rpms contain pygrub that
doesn''t support ext4.

# /usr/bin/pygrub --output=/tmp/debug /dev/vg_dom0test/test3vm
Traceback (most recent call last):
  File "/usr/bin/pygrub", line 680, in <module>
    chosencfg = run_grub(file, entry, fs, incfg["args"])
  File "/usr/bin/pygrub", line 515, in run_grub
    g = Grub(file, fs)
  File "/usr/bin/pygrub", line 202, in __init__
    self.read_config(file, fs)
  File "/usr/bin/pygrub", line 397, in read_config
    raise RuntimeError, "couldn''t find bootloader config file in
the image provided."
RuntimeError: couldn''t find bootloader config file in the image
provided.


However, If I do ''rpmbuild --bb xen.spec'' for
xen-3.4.1-3.fc12.src.rpm and install
the rebuilt rpms, pygrub works just fine!

So is there some lib missing from the build requirements or? 

Any ideas?

-- Pasi

>Anyway, it seems the F12/rawhide xen-3.4.1-3 rpms contain pygrub that
>doesn''t support ext4.
Please, view :-

http://www.fsdaily.com/HighEnd/Setup_Xen_3_4_1_Libvirt_on_top_F12_Alpha_rawhide

It''s on 64-bit.

Boris.

--- On Sun, 9/13/09, Pasi Kärkkäinen <pasik@iki.fi> wrote:

From: Pasi Kärkkäinen <pasik@iki.fi>
Subject: Re: [Fedora-xen] [Xen-devel] [ANNOUNCE] Xen 3.3.2 and 3.4.1 released /
3.4.1-3 rpm available from koji / F12 problem with pygrub
To: "Gerd Hoffmann" <kraxel@redhat.com>
Cc: fedora-xen@redhat.com
Date: Sunday, September 13, 2009, 12:16 PM

On Wed, Sep 02, 2009 at 04:08:32PM +0300, Pasi Kärkkäinen
wrote:
> On Wed, Sep 02, 2009 at 03:59:31PM +0300, Pasi Kärkkäinen wrote:
> > On Wed, Sep 02, 2009 at 03:41:09PM +0300, Pasi Kärkkäinen wrote:
> > > On Wed, Sep 02, 2009 at 02:25:59PM +0200, Gerd Hoffmann wrote:
> > > >   Hi,
> > > > 
> > > > >>Still no stubdom/pvgrub bzip2/lzma support though.
I''ll look more into
> > > > >>it tonight.
> > > > >
> > > > >.. but don''t wait for the final patch, since I
don''t know yet how long
> > > > >it takes to get the stubdom/pvgrub stuff sorted out.
> > > > 
> > > > Bits committed, koji builds done.
> > > > 
> > > 
> > > Great, thanks!
> > > 
> > > Does 3.4.1-3 end up in F12 automatically? 
> > > 
> > 
> > btw the url is this:
> > http://kojipkgs.fedoraproject.org/packages/xen/3.4.1/3.fc12/
> > 
> > Changelog:
> > 
> > * Wed Sep 02 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-3
> > - swap bzip2+xz linux kernel compression support patches.
> > - backport one more bugfix (videoram option).
> > 
> > * Tue Sep 01 2009 Gerd Hoffmann <kraxel@redhat.com> - 3.4.1-2
> > - backport bzip2+xz linux kernel compression support.
> > - backport a few bugfixes.
> > 
> 
> btw 3.4.1-2 and 3.4.1-3 do contain the fix for pygrub timeout=0
> handling, so F11 PV domUs/guests will work out-of-the-box.
> 
Hello,

I just noticed something weird.. I upgraded my host to F12/rawhide, so
now I''m running everything straight from rawhide, except my custom
pv_ops dom0 kernel.

Anyway, it seems the F12/rawhide xen-3.4.1-3 rpms contain pygrub that
doesn''t support ext4.

# /usr/bin/pygrub --output=/tmp/debug /dev/vg_dom0test/test3vm
Traceback (most recent call last):
  File "/usr/bin/pygrub", line 680, in <module>
    chosencfg = run_grub(file, entry, fs, incfg["args"])
  File "/usr/bin/pygrub", line 515, in run_grub
    g = Grub(file, fs)
  File "/usr/bin/pygrub", line 202, in __init__
    self.read_config(file, fs)
  File "/usr/bin/pygrub", line 397, in read_config
    raise RuntimeError, "couldn''t find bootloader config file in
the image provided."
RuntimeError: couldn''t find bootloader config file in the image
provided.


However, If I do ''rpmbuild --bb xen.spec'' for
xen-3.4.1-3.fc12.src.rpm and install
the rebuilt rpms, pygrub works just fine!

So is there some lib missing from the build requirements or? 

Any ideas?

-- Pasi

--
Fedora-xen mailing list
Fedora-xen@redhat.com
https://www.redhat.com/mailman/listinfo/fedora-xen

Hi,
> However, If I do ''rpmbuild --bb xen.spec'' for
xen-3.4.1-3.fc12.src.rpm and install
> the rebuilt rpms, pygrub works just fine!
>
> So is there some lib missing from the build requirements or?
e2fsprogs-devel most likely.

cheers,
   Gerd

On Mon, Sep 14, 2009 at 08:55:23AM +0200, Gerd Hoffmann
wrote:
>   Hi,
> 
> >However, If I do ''rpmbuild --bb xen.spec'' for
xen-3.4.1-3.fc12.src.rpm and
> >install
> >the rebuilt rpms, pygrub works just fine!
> >
> >So is there some lib missing from the build requirements or?
> 
> e2fsprogs-devel most likely.
> 
Yeah, it seems I have e2fsprogs-devel installed on my testbox. So that
explains the difference.

The attached xen.spec patch fixes this issue. Please add it for xen-3.4.1-4.

-- Pasi

On Mon, Sep 14, 2009 at 07:27:45PM +0300, Pasi Kärkkäinen
wrote:
> On Mon, Sep 14, 2009 at 08:55:23AM +0200, Gerd Hoffmann wrote:
> >   Hi,
> > 
> > >However, If I do ''rpmbuild --bb xen.spec'' for
xen-3.4.1-3.fc12.src.rpm and
> > >install
> > >the rebuilt rpms, pygrub works just fine!
> > >
> > >So is there some lib missing from the build requirements or?
> > 
> > e2fsprogs-devel most likely.
> > 
> 
> Yeah, it seems I have e2fsprogs-devel installed on my testbox. So that
> explains the difference.
> 
> The attached xen.spec patch fixes this issue. Please add it for
xen-3.4.1-4.
> 
.. and it''s needed to be able to boot F12 Xen PV guests. 
They use ext4 /boot as a default.

-- Pasi
> --- xen.spec.orig	2009-09-14 19:14:57.000000000 +0300
> +++ xen.spec	2009-09-14 19:16:06.000000000 +0300
> @@ -62,6 +62,8 @@
>  BuildRequires: gettext
>  BuildRequires: gnutls-devel
>  BuildRequires: openssl-devel
> +# For pygrub EXT4 support
> +BuildRequires: e2fsprogs-devel
>  # For ioemu PCI passthrough
>  BuildRequires: pciutils-devel
>  # modern compressed kernels
> --
> Fedora-xen mailing list
> Fedora-xen@redhat.com
> https://www.redhat.com/mailman/listinfo/fedora-xen

Xen 3.4.1  Dom0 has been built via xen-3.1.4-4.src.rpm on top of F11 ( yum
updated)
 This failure didn''t happen with rc8 (or rc9),  not sure.
At the same time via my experience xend starts fine on  the same 3.4.1 Dom0
(same 2.6.31 kernel) been built on top of F12 rawhide (yum updated).
/etc/fstab contains ''xenfs'' entry (3.4.1) in both cases and in
both cases message
/proc/xen is busy comes to console, however on F12 xend starts on F11
doesn''t.
I believe it makes sense to backport via xen-3.1.4-4.src.rpm CS  20185 plus some
other
CSs to make patching consistent (i cannot find out which ones)  and get an
option to remove ''xenfs'' entry from /etc/fstab, mounting
''xenfs'' at xend start up , what seems
to be a default behaviour of the most recent 2.6.31 pvops enabled kernel. 

Boris.

On Thu, Sep 17, 2009 at 11:07:08AM -0700, Boris Derzhavets
wrote:
> Xen 3.4.1  Dom0 has been built via xen-3.1.4-4.src.rpm on top of F11 ( yum
updated)
>  This failure didn''t happen with rc8 (or rc9),  not sure.
> At the same time via my experience xend starts fine on  the same 3.4.1 Dom0
(same 2.6.31 kernel) been built on top of F12 rawhide (yum updated).
> /etc/fstab contains ''xenfs'' entry (3.4.1) in both cases
and in both cases message
> /proc/xen is busy comes to console, however on F12 xend starts on F11
doesn''t.
I haven''t had problems starting xend on F11, with 2.6.31 pv_ops dom0
kernel..
> I believe it makes sense to backport via xen-3.1.4-4.src.rpm CS  20185 plus
some other
> CSs to make patching consistent (i cannot find out which ones)  and get an
option to remove ''xenfs'' entry from /etc/fstab, mounting
''xenfs'' at xend start up , what seems
> to be a default behaviour of the most recent 2.6.31 pvops enabled kernel. 
> 
Mounting xenfs from xend has nothing to do with the kernel. It''s a
feature of xen-unstable tools, not kernel

-- Pasi

Hi,
> /etc/fstab contains ''xenfs'' entry (3.4.1) in both cases
Hmm, where does that one come from?
It should not be there, it is not needed.
The xen packages never ever touch /etc/fstab.
>  mounting ''xenfs'' at xend start up , what seems
> to be a default behaviour of the most recent 2.6.31 pvops enabled kernel.
Both F11 (3.3.x) and F12 (3.4.x) xen packages mount the filesystem 
automatically.  Very early xen packages with pv_ops kernel support 
didn''t do that though, so maybe the /etc/fstab entry was added manually
back then and was never deleted?

cheers,
   Gerd

I removed ''xenfs'' entry from /etc/fstab and it brought back to
life Xen 3.4.1 (via xen-3.4.1-4.src.rpm) & Libvirt on the first F11 box, and
didn''t help on the second.
I would guess it''s xen&pvops unrelated issue on second box.

Thanks.
Boris.

--- On Thu, 9/17/09, Gerd Hoffmann <kraxel@redhat.com> wrote:

From: Gerd Hoffmann <kraxel@redhat.com>
Subject: Re: Xend doesn''t start at Xen 3.4.1 Dom(2.6.31 commit
ebf47a1fe1aec22756c32fdba3dd94b05d5b8a84) on top of F11
To: "Boris Derzhavets" <bderzhavets@yahoo.com>
Cc: "Pasi Kärkkäinen" <pasik@iki.fi>, fedora-xen@redhat.com
Date: Thursday, September 17, 2009, 4:53 PM

  Hi,
> /etc/fstab contains ''xenfs'' entry (3.4.1) in both cases
Hmm, where does that one come from?
It should not be there, it is not needed.
The xen packages never ever touch /etc/fstab.
>  mounting ''xenfs'' at xend start up , what seems
> to be a default behaviour of the most recent 2.6.31 pvops enabled kernel.
Both F11 (3.3.x) and F12 (3.4.x) xen packages mount the filesystem
automatically.  Very early xen packages with pv_ops kernel support
didn''t do that though, so maybe the /etc/fstab entry was added manually
back then and was never deleted?

cheers,
  Gerd

On Mon, Sep 14, 2009 at 08:06:34PM +0300, Pasi Kärkkäinen
wrote:
> On Mon, Sep 14, 2009 at 07:27:45PM +0300, Pasi Kärkkäinen wrote:
> > On Mon, Sep 14, 2009 at 08:55:23AM +0200, Gerd Hoffmann wrote:
> > >   Hi,
> > > 
> > > >However, If I do ''rpmbuild --bb xen.spec''
for xen-3.4.1-3.fc12.src.rpm and
> > > >install
> > > >the rebuilt rpms, pygrub works just fine!
> > > >
> > > >So is there some lib missing from the build requirements or?
> > > 
> > > e2fsprogs-devel most likely.
> > > 
> > 
> > Yeah, it seems I have e2fsprogs-devel installed on my testbox. So that
> > explains the difference.
> > 
> > The attached xen.spec patch fixes this issue. Please add it for
xen-3.4.1-4.
> > 
> 
> .. and it''s needed to be able to boot F12 Xen PV guests. 
> They use ext4 /boot as a default.
> 
xen-3.4.1-4 rpms from f12/rawhide seem to work OK, booting f12 PV domU
with /boot on ext4 works OK.

-- Pasi