Xen devel - Mar 2006 - Problems with pygrub: ValueError: unable to open file

I am currently struggling with getting pygrub to work. I am using unstable
9327:cf89e8f0831b with 2.6.16-rc6-git3 on CentOS 4 (although the kernel
version should not matter here).

I dedicated the partition /dev/sdb1 to a virtual machine, used fdisk to
sub-partition this into partitions again. Finally I used dmsetup (device
mapper) to temporarily make the partitions available to format and mount
them, and install the guest OS (linux) from a backup archive.

Using a regular xm config, I can happily boot the guest and it works
perfectly (mounts all partitions and just works). I use disk = [
''phy:sdb1,sda,w'' ].

When I use bootloader = "/root/pygrub" instead, I get the following
trace
when trying to do xm create:

Traceback (most recent call last):
  File "/root/pygrub", line 258, in ?
    cf = get_config(file, isconfig)
  File "/root/pygrub", line 121, in get_config
    fs = fstype.open_fs(fn, offset)
  File "/usr/lib/python2.3/site-packages/grub/fsys/ext2/__init__.py",
line
35, in open_fs
    return Ext2Fs(fn, offset = offset)
ValueError: unable to open file
Error: Boot loader didn''t return any data!

I checked the values of fn and offset:
fn = /dev/sdb1
offset = 512

This seems perfectly ok, considering the output of fdisk:

Disk /dev/sdb1: 15.0 GB, 15011103744 bytes
255 heads, 63 sectors/track, 1824 cylinders, total 29318562 sectors
Units = sectors of 1 * 512 = 512 bytes

     Device Boot      Start         End      Blocks   Id  System
/dev/sdb1p1   *           1      208844      104422   83  Linux
/dev/sdb1p2          208845     2313359     1052257+  82  Linux swap
/dev/sdb1p3         2313360    29302559    13494600   83  Linux

One sector is indeed 512 bytes, so the offset of one sector into /dev/sdb1
seems perfectly ok.

Additionally I tried to copy the partition to an image file with dd. Result: 
exactly
the same error as above. Now, using lomount gives this error: "vm1.img:
File
too
large".
Copying only the first gigabyte into the image (the boot partition is only
100 mb) makes lomount work (mounts the boot partition correctly), but pygrub
will still say "ValueError: unable to open file" in Ext2Fs, even for
the
truncated image file.

Please, help me find and fix this problem.

Best Regards,
Michael Paesold

-- 
Michael Paesold | Emerion WebHosting Gmbh
mpaesold gmx at | http://www.emerion.com




_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I wrote:
> I am currently struggling with getting pygrub to work. I am using unstable
> 9327:cf89e8f0831b with 2.6.16-rc6-git3 on CentOS 4 (although the kernel
> version should not matter here).
...
> Traceback (most recent call last):
>  File "/root/pygrub", line 258, in ?
>    cf = get_config(file, isconfig)
>  File "/root/pygrub", line 121, in get_config
>    fs = fstype.open_fs(fn, offset)
>  File
"/usr/lib/python2.3/site-packages/grub/fsys/ext2/__init__.py", line
> 35, in open_fs
>    return Ext2Fs(fn, offset = offset)
> ValueError: unable to open file
> Error: Boot loader didn''t return any data!
> 
> I checked the values of fn and offset:
> fn = /dev/sdb1
> offset = 512
> 
> This seems perfectly ok, considering the output of fdisk:
...

Well, after further investigation, the problem is that the CentOS/RHEL 4
version of e2fsprogs does not have a ext2fs_open2 that allows to use the
offset. The "bug" is that this is not reported at all.

     if (offset != 0) {
         snprintf(offsetopt, 29, "offset=%d", offset);
     }

#ifdef HAVE_EXT2FS_OPEN2
     err = ext2fs_open2(name, offsetopt, flags, superblock, block_size,
                        unix_io_manager, &efs);
#else
     err = ext2fs_open(name, flags, superblock, block_size,
                       unix_io_manager, &efs);
#endif


Here, HAVE_EXT2FS_OPEN2 is not defined. An offset cannot be used if 
ext2fs_open2 is not available. Does anyone see a way to work around this 
issue other than either update e2fsprogs or otherwise teach pygrub to use 
devicemapper or similar to be able to open an ext2 filesystem in a 
partition that is not directly accessible to the host (e.g. in a 
partitioned image)?

Best Regards,
Michael Paesold

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Improve pygrub error reporting when opening ext2 fs is not possible

As reported in the thread:
http://lists.xensource.com/archives/html/xen-users/2006-03/msg00721.html,
pygrub does not open ext2 file systems in partitioned images or sub
partitions on e.g. CentOS/RHEL 4, because e2fsprogs ext2fs_open does not
support an offset into the file to be opened.

With this patch, the error is correctly reported instead of a generic
"unable to open file" (and leaving the user searching in the dark).

Signed-off-by: Michael Paesold <mpaesold@gmx.at>

diff -r f0e14b4e535c tools/pygrub/src/fsys/ext2/ext2module.c
--- a/tools/pygrub/src/fsys/ext2/ext2module.c   Thu Mar 30 14:37:22 2006 +0100
+++ b/tools/pygrub/src/fsys/ext2/ext2module.c   Thu Mar 30 20:52:41 2006 +0200
@@ -225,19 +225,24 @@ ext2_fs_open (Ext2Fs *fs, PyObject *args
          return NULL;
      }

+#ifdef HAVE_EXT2FS_OPEN2
      if (offset != 0) {
          snprintf(offsetopt, 29, "offset=%d", offset);
      }

-#ifdef HAVE_EXT2FS_OPEN2
      err = ext2fs_open2(name, offsetopt, flags, superblock, block_size,
                         unix_io_manager, &efs);
  #else
+    if (offset != 0) {
+        PyErr_SetString(PyExc_ValueError, "offset argument not
supported");
+        return NULL;
+    }
+
      err = ext2fs_open(name, flags, superblock, block_size,
                        unix_io_manager, &efs);
  #endif
      if (err) {
-        PyErr_SetString(PyExc_ValueError, "unable to open file");
+        PyErr_SetString(PyExc_ValueError, "unable to open
filesystem");
          return NULL;
      }


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I am sorry, this patches produces a warning if HAVE_EXT2FS_OPEN2 is not 
defined (unused variable) and will break the build with -Werror. I will send 
a new patch.

Best Regards,
Michael Paesold

I wrote:
> Improve pygrub error reporting when opening ext2 fs is not possible
>
> As reported in the thread:
> http://lists.xensource.com/archives/html/xen-users/2006-03/msg00721.html,
> pygrub does not open ext2 file systems in partitioned images or sub
> partitions on e.g. CentOS/RHEL 4, because e2fsprogs ext2fs_open does not
> support an offset into the file to be opened.
>
> With this patch, the error is correctly reported instead of a generic
> "unable to open file" (and leaving the user searching in the
dark).
>
> Signed-off-by: Michael Paesold <mpaesold@gmx.at>
>
> diff -r f0e14b4e535c tools/pygrub/src/fsys/ext2/ext2module.c
> --- a/tools/pygrub/src/fsys/ext2/ext2module.c   Thu Mar 30 14:37:22 2006 
> +0100
> +++ b/tools/pygrub/src/fsys/ext2/ext2module.c   Thu Mar 30 20:52:41 2006 
> +0200
> @@ -225,19 +225,24 @@ ext2_fs_open (Ext2Fs *fs, PyObject *args
>          return NULL;
>      }
>
> +#ifdef HAVE_EXT2FS_OPEN2
>      if (offset != 0) {
>          snprintf(offsetopt, 29, "offset=%d", offset);
>      }
>
> -#ifdef HAVE_EXT2FS_OPEN2
>      err = ext2fs_open2(name, offsetopt, flags, superblock, block_size,
>                         unix_io_manager, &efs);
>  #else
> +    if (offset != 0) {
> +        PyErr_SetString(PyExc_ValueError, "offset argument not 
> supported");
> +        return NULL;
> +    }
> +
>      err = ext2fs_open(name, flags, superblock, block_size,
>                        unix_io_manager, &efs);
>  #endif
>      if (err) {
> -        PyErr_SetString(PyExc_ValueError, "unable to open
file");
> +        PyErr_SetString(PyExc_ValueError, "unable to open
filesystem");
>          return NULL;
>      }
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Sorry for the last empty mail (hit the wrong button). Attached is a patch 
against xen-unstable to improve the error messages in pygrub when the 
e2fsprogs do not support the newer ext2fs_open2 function. This should help 
beginners to identify the real problem when pygrub does not work (e.g. on 
RHEL4).

Please apply to xen-unstable.

Best Regards,
Michael Paesold

# HG changeset patch
# User mpaesold@gmx.at
# Node ID 6d90c95b919b324b7d8e4c7a568040334c8862e9
# Parent  f0e14b4e535c7d99c56c286384b0d512c5220884
Improve pygrub error reporting when opening ext2 fs is not possible

As reported in the thread:
http://lists.xensource.com/archives/html/xen-users/2006-03/msg00721.html,
pygrub does not open ext2 file systems in partitioned images or sub
partitions on e.g. CentOS/RHEL 4, because e2fsprogs ext2fs_open does not
support an offset into the file to be opened.

With this patch, the error is correctly reported instead of a generic
"unable to open file" (and leaving the user searching in the dark).

Signed-off-by: Michael Paesold <mpaesold@gmx.at> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Mar 31, 2006 at 05:28:54PM +0200, Michael Paesold wrote:
> Sorry for the last empty mail (hit the wrong button). Attached is a patch 
> against xen-unstable to improve the error messages in pygrub when the 
> e2fsprogs do not support the newer ext2fs_open2 function. This should help 
> beginners to identify the real problem when pygrub does not work (e.g. on 
> RHEL4).
> 
> Please apply to xen-unstable.
> 
> Best Regards,
> Michael Paesold
> 
> # HG changeset patch
> # User mpaesold@gmx.at
> # Node ID 6d90c95b919b324b7d8e4c7a568040334c8862e9
> # Parent  f0e14b4e535c7d99c56c286384b0d512c5220884
> Improve pygrub error reporting when opening ext2 fs is not possible
> 
> As reported in the thread:
> http://lists.xensource.com/archives/html/xen-users/2006-03/msg00721.html,
> pygrub does not open ext2 file systems in partitioned images or sub
> partitions on e.g. CentOS/RHEL 4, because e2fsprogs ext2fs_open does not
> support an offset into the file to be opened.
> 
> With this patch, the error is correctly reported instead of a generic
> "unable to open file" (and leaving the user searching in the
dark).
> 
> Signed-off-by: Michael Paesold <mpaesold@gmx.at> 
> # HG changeset patch
> # User mip@xencore04.1virtual.net
> # Node ID 6d90c95b919b324b7d8e4c7a568040334c8862e9
> # Parent  f0e14b4e535c7d99c56c286384b0d512c5220884
> Improve pygrub error reporting when opening ext2 fs is not possible
> 
> As reported in the thread:
> http://lists.xensource.com/archives/html/xen-users/2006-03/msg00721.html,
> pygrub does not open ext2 file systems in partitioned images or sub
> partitions on e.g. CentOS/RHEL 4, because e2fsprogs ext2fs_open does not
> support an offset into the file to be opened.
> 
> With this patch, the error is correctly reported instead of a generic
> "unable to open file" (and leaving the user searching in the
dark).
> 
> Signed-off-by: Michael Paesold <mpaesold@gmx.at>
Applied, thanks.

Ewan.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Michael Paesold wrote:
> Sorry for the last empty mail (hit the wrong button). Attached is a 
> patch against xen-unstable to improve the error messages in pygrub when 
> the e2fsprogs do not support the newer ext2fs_open2 function. This 
> should help beginners to identify the real problem when pygrub does not 
> work (e.g. on RHEL4).
I also have that problem on fedora core 5. As far as I can see, your fix 
is only about the error message. But how can the actual problem be solved?
I''d really like to use a simple xen config as given in xmexample1, and 
use the images from xen-get.org.

My post to fedora-xen can be found at: 
https://www.redhat.com/archives/fedora-xen/2006-April/msg00085.html

Henning

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Henning Sprang wrote:
> Michael Paesold wrote:
>> Sorry for the last empty mail (hit the wrong button). Attached is a 
>> patch against xen-unstable to improve the error messages in pygrub 
>> when the e2fsprogs do not support the newer ext2fs_open2 function. 
>> This should help beginners to identify the real problem when pygrub 
>> does not work (e.g. on RHEL4).
>
> I also have that problem on fedora core 5. As far as I can see, your 
> fix is only about the error message. But how can the actual problem be 
> solved?
> I''d really like to use a simple xen config as given in xmexample1,
and
> use the images from xen-get.org.
>
> My post to fedora-xen can be found at: 
> https://www.redhat.com/archives/fedora-xen/2006-April/msg00085.html
Henning,

I also haven''t managed to boot a domU different from FC5 on my FC5
dom0.
But I have different problems. Maybe there are some points of interest 
for you in this anyway...

You report that when trying to boot xend complains about not finding the 
kernel image file.
I disabled selinux in domU and dom0 and then that went away...

I also have no trouble at all working the normal way of booting with a 
kernel in dom0 and not using pygrub (which you obviously need if you 
want to domU to boot a kernel which is placed in domU itself).

My domUs (debian 3.1, fedora 4 - all happily managing to run on 
different dom0) boot up to different points. Usually they get stuck 
around mounting the ext3 filesystem... So I''m still working on that...

So I''d be happy if we could mutually keep track of our efforts and 
hopefully successes!

Bye,
Andrej

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Henning Sprang wrote:
> Michael Paesold wrote:
>> Sorry for the last empty mail (hit the wrong button). Attached is a 
>> patch against xen-unstable to improve the error messages in pygrub 
>> when the e2fsprogs do not support the newer ext2fs_open2 function. 
>> This should help beginners to identify the real problem when pygrub 
>> does not work (e.g. on RHEL4).
> 
> I also have that problem on fedora core 5. As far as I can see, your fix 
> is only about the error message. But how can the actual problem be solved?
> I''d really like to use a simple xen config as given in xmexample1,
and
> use the images from xen-get.org.
Yes, my patch only improved the error reporting, not the underlying problem 
of a too old version of e2fsprogs. Nevertheless FC5 has an appropriate 
version of e2fsprogs (as well as FC4 btw.).

 > My post to fedora-xen can be found at:
 > https://www.redhat.com/archives/fedora-xen/2006-April/msg00085.html

So your problem is a different problem. As Andrej noted in the other mail, 
disabling SELinux might help. I remember reading about this already on one 
of these lists here.


Just for the archives or anyone using pygrub on RHEL4/CentOS4: I have 
successfully rebuilt and used e2fsprogs from FC4 on RHEL4/CentOS4 and have 
had no issues so far. YMMV. (Don''t forget to rebuild pygrub after 
installing the new e2fsprogs!)

Best Regards,
Michael Paesold

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Henning Sprang wrote:
> Michael Paesold wrote:
>> Sorry for the last empty mail (hit the wrong button). Attached is a
patch
>> against xen-unstable to improve the error messages in pygrub when the 
>> e2fsprogs do not support the newer ext2fs_open2 function. This should 
>> help beginners to identify the real problem when pygrub does not work 
>> (e.g. on RHEL4).
>
> I also have that problem on fedora core 5. As far as I can see, your fix 
> is only about the error message. But how can the actual problem be solved?
> I''d really like to use a simple xen config as given in xmexample1,
and use
> the images from xen-get.org.
>
> My post to fedora-xen can be found at: 
> https://www.redhat.com/archives/fedora-xen/2006-April/msg00085.html
At the time of my first reply, I didn''t see that you replied to your
own
post. So here is a more appropriate answer.

Your first issue is that you are not able to load a domU kernel that is 
located in dom0, right? In that case make sure that the file is really there 
(ls -l /boot/vmlinuz-2.6.16-1.2080_FC5xenU) and is accessible by xend. 
SELinux might block access to /boot from Xen. You should try to disable 
SELinux to check if that is true. If you do not want to disable SELinux, you 
might also cp vmlinuz and the corresponding initrd (.img) to a place that is 
accessible for xend (e.g. on FC5 this should be /var/lib/xen for instance).

The second issue is that pygrub does not work. Since FC5 has recent 
e2fsprogs, your issue should be different than the one my patch addressed. 
The message "ValueError: unable to open file" could have several
reasons:

- kernel and initrd were not properly installed in the *guest* filesystem
- no valid grub configuration /boot/grub/grub.conf exists in the guest 
filesystem
- the filesystem image file was no created properly and pygrub fails to open 
the filesystem (with my patch applied, this would read "unable to open 
filesystem")

I hope that helps.

Best Regards,
Michael Paesold 



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users