Xen devel - Jan 2013 - [PATCH] fix wrong path while calling pygrub and libxl-save-helper

in current xen x86_64, the default libexec directory is /usr/lib/xen/bin,
while the private binder is /usr/lib64/xen/bin. but some commands(pygrub,
libxl-save-helper) located in private binder directory is called from
libexec directory which lead to the following error:
1, for pygrub bootloader:
libxl: debug: libxl_bootloader.c:429:bootloader_disk_attached_cb:
/usr/lib/xen/bin/pygrub doesn''t exist, falling back to config path
2, for libxl-save-helper:
libxl: cannot execute /usr/lib/xen/bin/libxl-save-helper: No such file or
directory
libxl: error: libxl_utils.c:363:libxl_read_exactly: file/stream truncated
reading ipc msg header from domain 3 save/restore helper stdout pipe
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: domain 3
save/restore helper [10222] exited with error status 255

there are two ways to fix above error. the first one is make such command
store in the /usr/lib/xen/bin and /usr/lib64/xen/bin(symbol link to
previous), e.g. qemu-dm. The second way is using private binder dir
instead of libexec dir. e.g. xenconsole.
For these cases, the latter one is suitable.

diff -r c4114a042410 -r 68e54ac2d3ed tools/libxl/libxl_bootloader.c
--- a/tools/libxl/libxl_bootloader.c	Fri Dec 21 17:05:38 2012 +0000
+++ b/tools/libxl/libxl_bootloader.c	Sat Jan 05 16:02:53 2013 +0800
@@ -419,7 +419,7 @@ static void bootloader_disk_attached_cb(
         const char *bltmp;
         struct stat st;
 
-        bltmp = libxl__abs_path(gc, bootloader, libxl__libexec_path());
+        bltmp = libxl__abs_path(gc, bootloader, libxl__private_bindir_path());
         /* Check to see if the file exists in this location; if not,
          * fall back to checking the path */
         LOG(DEBUG, "Checking for bootloader in libexec path: %s",
bltmp);
diff -r c4114a042410 -r 68e54ac2d3ed tools/libxl/libxl_save_callout.c
--- a/tools/libxl/libxl_save_callout.c	Fri Dec 21 17:05:38 2012 +0000
+++ b/tools/libxl/libxl_save_callout.c	Sat Jan 05 16:02:53 2013 +0800
@@ -172,7 +172,7 @@ static void run_helper(libxl__egc *egc, 
     shs->stdout_what = GCSPRINTF("domain %"PRIu32"
save/restore helper"
                                  " stdout pipe", domid);
 
-    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: LIBEXEC "/"
"libxl-save-helper";
+    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: PRIVATE_BINDIR
"/" "libxl-save-helper";
     *arg++ = mode_arg;
     const char **stream_fd_arg = arg++;
     for (i=0; i<num_argnums; i++)

On Sat, Jan 05, 2013 at 04:10:52PM +0800, Bamvor Jian Zhang
wrote:
> in current xen x86_64, the default libexec directory is /usr/lib/xen/bin,
> while the private binder is /usr/lib64/xen/bin. but some commands(pygrub,
> libxl-save-helper) located in private binder directory is called from
> libexec directory which lead to the following error:
> 1, for pygrub bootloader:
> libxl: debug: libxl_bootloader.c:429:bootloader_disk_attached_cb:
> /usr/lib/xen/bin/pygrub doesn''t exist, falling back to config path
> 2, for libxl-save-helper:
> libxl: cannot execute /usr/lib/xen/bin/libxl-save-helper: No such file or
> directory
> libxl: error: libxl_utils.c:363:libxl_read_exactly: file/stream truncated
> reading ipc msg header from domain 3 save/restore helper stdout pipe
> libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: domain 3
> save/restore helper [10222] exited with error status 255
> 
> there are two ways to fix above error. the first one is make such command
> store in the /usr/lib/xen/bin and /usr/lib64/xen/bin(symbol link to
> previous), e.g. qemu-dm. The second way is using private binder dir
> instead of libexec dir. e.g. xenconsole.
> For these cases, the latter one is suitable.
Hi Bamvor,

Please include a "Signed-off-by:" line in your patches. See
"Signing
off a patch" in http://wiki.xen.org/wiki/Submitting_Xen_Patches

Do you see this problem when you run "./configure
--libdir=/usr/lib64"?
See the 4.2 release notes for more information:
  http://wiki.xen.org/wiki/Xen_4.2_Release_Notes#Build_Requirements

Matt
> diff -r c4114a042410 -r 68e54ac2d3ed tools/libxl/libxl_bootloader.c
> --- a/tools/libxl/libxl_bootloader.c	Fri Dec 21 17:05:38 2012 +0000
> +++ b/tools/libxl/libxl_bootloader.c	Sat Jan 05 16:02:53 2013 +0800
> @@ -419,7 +419,7 @@ static void bootloader_disk_attached_cb(
>          const char *bltmp;
>          struct stat st;
>  
> -        bltmp = libxl__abs_path(gc, bootloader, libxl__libexec_path());
> +        bltmp = libxl__abs_path(gc, bootloader,
libxl__private_bindir_path());
>          /* Check to see if the file exists in this location; if not,
>           * fall back to checking the path */
>          LOG(DEBUG, "Checking for bootloader in libexec path:
%s", bltmp);
> diff -r c4114a042410 -r 68e54ac2d3ed tools/libxl/libxl_save_callout.c
> --- a/tools/libxl/libxl_save_callout.c	Fri Dec 21 17:05:38 2012 +0000
> +++ b/tools/libxl/libxl_save_callout.c	Sat Jan 05 16:02:53 2013 +0800
> @@ -172,7 +172,7 @@ static void run_helper(libxl__egc *egc, 
>      shs->stdout_what = GCSPRINTF("domain %"PRIu32"
save/restore helper"
>                                   " stdout pipe", domid);
>  
> -    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: LIBEXEC
"/" "libxl-save-helper";
> +    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: PRIVATE_BINDIR
"/" "libxl-save-helper";
>      *arg++ = mode_arg;
>      const char **stream_fd_arg = arg++;
>      for (i=0; i<num_argnums; i++)

>>>Matt Wilson <msw@amazon.com> wrote: 
> On Sat, Jan 05, 2013 at 04:10:52PM +0800, Bamvor Jian Zhang wrote: 
> > in current xen x86_64, the default libexec directory is
/usr/lib/xen/bin,
> > while the private binder is /usr/lib64/xen/bin. but some
commands(pygrub,
> > libxl-save-helper) located in private binder directory is called from 
> > libexec directory which lead to the following error: 
> > 1, for pygrub bootloader: 
> > libxl: debug: libxl_bootloader.c:429:bootloader_disk_attached_cb: 
> > /usr/lib/xen/bin/pygrub doesn''t exist, falling back to config
path
> > 2, for libxl-save-helper: 
> > libxl: cannot execute /usr/lib/xen/bin/libxl-save-helper: No such file
or
> > directory 
> > libxl: error: libxl_utils.c:363:libxl_read_exactly: file/stream
truncated
> > reading ipc msg header from domain 3 save/restore helper stdout pipe 
> > libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: domain 3
> > save/restore helper [10222] exited with error status 255 
> >  
> > there are two ways to fix above error. the first one is make such
command
> > store in the /usr/lib/xen/bin and /usr/lib64/xen/bin(symbol link to 
> > previous), e.g. qemu-dm. The second way is using private binder dir 
> > instead of libexec dir. e.g. xenconsole. 
> > For these cases, the latter one is suitable. 
>  
> Hi Bamvor, 
>  
> Please include a "Signed-off-by:" line in your patches. See
"Signing
> off a patch" in http://wiki.xen.org/wiki/Submitting_Xen_Patches
sorry missing it. i will add it next time.
> Do you see this problem when you run "./configure
--libdir=/usr/lib64"?
> See the 4.2 release notes for more information:
>   http://wiki.xen.org/wiki/Xen_4.2_Release_Notes#Build_Requirements
I got this issue while using the above libdir, here is my command of configure 
./configure --enable-xenapi --prefix=/usr --exec_prefix=/usr --bindir=/usr/bin
--sbindir=/usr/sbin --libdir=/usr/lib64 --datadir=/usr/share
but the LIBEXEC is force set to /usr/lib/xen/bin in config/StdGNU.mk. So, the
libdir is /usr/lib64 while libexec is /usr/bin/xen/bin in tools/libxl/_paths.h

meanwhile, the libxl-save-helper is install to "PRIVATE_BINDER" in
tools/libxl/Makefile not LIBEXEC:
$(INSTALL_PROG) libxl-save-helper $(DESTDIR)$(PRIVATE_BINDIR)
using the "PRIVATE_BINDER" in run_helper make it coherence with this
Makefile.

BTW, these files are not changed after xen4.2, but i only found this issue in
xen-unstable.
> Matt 
>  
> > diff -r c4114a042410 -r 68e54ac2d3ed tools/libxl/libxl_bootloader.c 
> > --- a/tools/libxl/libxl_bootloader.c	Fri Dec 21 17:05:38 2012 +0000 
> > +++ b/tools/libxl/libxl_bootloader.c	Sat Jan 05 16:02:53 2013 +0800 
> > @@ -419,7 +419,7 @@ static void bootloader_disk_attached_cb( 
> >          const char *bltmp; 
> >          struct stat st; 
> >   
> > -        bltmp = libxl__abs_path(gc, bootloader,
libxl__libexec_path());
> > +        bltmp = libxl__abs_path(gc, bootloader,  
> libxl__private_bindir_path()); 
> >          /* Check to see if the file exists in this location; if not, 
> >           * fall back to checking the path */ 
> >          LOG(DEBUG, "Checking for bootloader in libexec path:
%s", bltmp);
> > diff -r c4114a042410 -r 68e54ac2d3ed tools/libxl/libxl_save_callout.c 
> > --- a/tools/libxl/libxl_save_callout.c	Fri Dec 21 17:05:38 2012 +0000 
> > +++ b/tools/libxl/libxl_save_callout.c	Sat Jan 05 16:02:53 2013 +0800 
> > @@ -172,7 +172,7 @@ static void run_helper(libxl__egc *egc,  
> >      shs->stdout_what = GCSPRINTF("domain %"PRIu32"
save/restore helper"
> >                                   " stdout pipe", domid); 
> >   
> > -    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: LIBEXEC
"/" "libxl-save-helper";
> > +    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: PRIVATE_BINDIR
"/"
> "libxl-save-helper"; 
> >      *arg++ = mode_arg; 
> >      const char **stream_fd_arg = arg++; 
> >      for (i=0; i<num_argnums; i++) 
>  
>

On Sun, 2013-01-06 at 08:35 +0000, Bamvor Jian Zhang
wrote:
> 
>  >>>Matt Wilson <msw@amazon.com> wrote: 
> > On Sat, Jan 05, 2013 at 04:10:52PM +0800, Bamvor Jian Zhang wrote: 
> > > in current xen x86_64, the default libexec directory is
/usr/lib/xen/bin,
> > > while the private binder is /usr/lib64/xen/bin. but some
commands(pygrub,
> > > libxl-save-helper) located in private binder directory is called
from
> > > libexec directory which lead to the following error: 
> > > 1, for pygrub bootloader: 
> > > libxl: debug: libxl_bootloader.c:429:bootloader_disk_attached_cb:
> > > /usr/lib/xen/bin/pygrub doesn''t exist, falling back to
config path
> > > 2, for libxl-save-helper: 
> > > libxl: cannot execute /usr/lib/xen/bin/libxl-save-helper: No such
file or
> > > directory 
> > > libxl: error: libxl_utils.c:363:libxl_read_exactly: file/stream
truncated
> > > reading ipc msg header from domain 3 save/restore helper stdout
pipe
> > > libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus:
domain 3
> > > save/restore helper [10222] exited with error status 255 
> > >  
> > > there are two ways to fix above error. the first one is make such
command
> > > store in the /usr/lib/xen/bin and /usr/lib64/xen/bin(symbol link
to
> > > previous), e.g. qemu-dm. The second way is using private binder
dir
> > > instead of libexec dir. e.g. xenconsole. 
> > > For these cases, the latter one is suitable. 
This change seems correct to me.
> > Hi Bamvor, 
> >  
> > Please include a "Signed-off-by:" line in your patches. See
"Signing
> > off a patch" in http://wiki.xen.org/wiki/Submitting_Xen_Patches
> sorry missing it. i will add it next time.
Please do.
> 
> > Do you see this problem when you run "./configure
--libdir=/usr/lib64"?
> > See the 4.2 release notes for more information:
> >   http://wiki.xen.org/wiki/Xen_4.2_Release_Notes#Build_Requirements
> I got this issue while using the above libdir, here is my command of
configure
> ./configure --enable-xenapi --prefix=/usr --exec_prefix=/usr
--bindir=/usr/bin --sbindir=/usr/sbin --libdir=/usr/lib64 --datadir=/usr/share
> but the LIBEXEC is force set to /usr/lib/xen/bin in config/StdGNU.mk. So,
the libdir is /usr/lib64 while libexec is /usr/bin/xen/bin in
tools/libxl/_paths.h
You aren''t saying that _paths.h is inconsistent with the actual install
locations for either LIBEXEC or PRIVATE_BINDIR, are you? Just that the
wrong one is being used in the code.

I''m not entirely sure what the intended distinction between libexec and
private-bindir is. We should perhaps consider nuking one or the other.
> meanwhile, the libxl-save-helper is install to "PRIVATE_BINDER"
in tools/libxl/Makefile not LIBEXEC:
> $(INSTALL_PROG) libxl-save-helper $(DESTDIR)$(PRIVATE_BINDIR)
> using the "PRIVATE_BINDER" in run_helper make it coherence with
this Makefile.
> 
> BTW, these files are not changed after xen4.2, but i only found this issue
in xen-unstable.
That''s odd, both branches contain 25594:ad08cd8e7097 which is when the
libexec dir changed. I can''t see anything else which looks like it
would
cause the two to differ.

Ian.

in current xen x86_64, the default libexec directory is /usr/lib/xen/bin,
while the private binder is /usr/lib64/xen/bin. but some commands(pygrub,
libxl-save-helper) located in private binder directory is called from
libexec directory which lead to the following error:
1, for pygrub bootloader:
libxl: debug: libxl_bootloader.c:429:bootloader_disk_attached_cb:
/usr/lib/xen/bin/pygrub doesn''t exist, falling back to config path
2, for libxl-save-helper:
libxl: cannot execute /usr/lib/xen/bin/libxl-save-helper: No such file or
directory
libxl: error: libxl_utils.c:363:libxl_read_exactly: file/stream truncated
reading ipc msg header from domain 3 save/restore helper stdout pipe
libxl: error: libxl_exec.c:118:libxl_report_child_exitstatus: domain 3
save/restore helper [10222] exited with error status 255

there are two ways to fix above error. the first one is make such command
store in the /usr/lib/xen/bin and /usr/lib64/xen/bin(symbol link to
previous), e.g. qemu-dm. The second way is using private binder dir
instead of libexec dir. e.g. xenconsole.
For these cases, the latter one is suitable.

Signed-off-by: Bamvor Jian Zhang <bjzhang@suse.com>

Changes since v1:
 * add missing sign off string

diff -r e93843a89e79 -r a4f85de195cf tools/libxl/libxl_bootloader.c
--- a/tools/libxl/libxl_bootloader.c	Wed Jan 09 17:49:13 2013 +0800
+++ b/tools/libxl/libxl_bootloader.c	Wed Jan 09 17:53:01 2013 +0800
@@ -419,7 +419,7 @@ static void bootloader_disk_attached_cb(
         const char *bltmp;
         struct stat st;
 
-        bltmp = libxl__abs_path(gc, bootloader, libxl__libexec_path());
+        bltmp = libxl__abs_path(gc, bootloader, libxl__private_bindir_path());
         /* Check to see if the file exists in this location; if not,
          * fall back to checking the path */
         LOG(DEBUG, "Checking for bootloader in libexec path: %s",
bltmp);
diff -r e93843a89e79 -r a4f85de195cf tools/libxl/libxl_save_callout.c
--- a/tools/libxl/libxl_save_callout.c	Wed Jan 09 17:49:13 2013 +0800
+++ b/tools/libxl/libxl_save_callout.c	Wed Jan 09 17:53:01 2013 +0800
@@ -172,7 +172,7 @@ static void run_helper(libxl__egc *egc, 
     shs->stdout_what = GCSPRINTF("domain %"PRIu32"
save/restore helper"
                                  " stdout pipe", domid);
 
-    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: LIBEXEC "/"
"libxl-save-helper";
+    *arg++ = getenv("LIBXL_SAVE_HELPER") ?: PRIVATE_BINDIR
"/" "libxl-save-helper";
     *arg++ = mode_arg;
     const char **stream_fd_arg = arg++;
     for (i=0; i<num_argnums; i++)

>>>Ian Campbell <Ian.Campbell@citrix.com> wrote: 
> On Sun, 2013-01-06 at 08:35 +0000, Bamvor Jian Zhang wrote: 
> >  
> >  >>>Matt Wilson <msw@amazon.com> wrote:  
> > > On Sat, Jan 05, 2013 at 04:10:52PM +0800, Bamvor Jian Zhang
wrote:
> > > > in current xen x86_64, the default libexec directory is
/usr/lib/xen/bin,
>  
> > > > while the private binder is /usr/lib64/xen/bin. but some
commands(pygrub,
>  
> > > > libxl-save-helper) located in private binder directory is
called from
> > > > libexec directory which lead to the following error:  
> > > > 1, for pygrub bootloader:  
> > > > libxl: debug:
libxl_bootloader.c:429:bootloader_disk_attached_cb:
> > > > /usr/lib/xen/bin/pygrub doesn''t exist, falling back
to config path
> > > > 2, for libxl-save-helper:  
> > > > libxl: cannot execute /usr/lib/xen/bin/libxl-save-helper: No
such file or
>  
> > > > directory  
> > > > libxl: error: libxl_utils.c:363:libxl_read_exactly:
file/stream truncated
>  
> > > > reading ipc msg header from domain 3 save/restore helper
stdout pipe
> > > > libxl: error:
libxl_exec.c:118:libxl_report_child_exitstatus: domain 3
> > > > save/restore helper [10222] exited with error status 255  
> > > >   
> > > > there are two ways to fix above error. the first one is make
such command
>  
> > > > store in the /usr/lib/xen/bin and /usr/lib64/xen/bin(symbol
link to
> > > > previous), e.g. qemu-dm. The second way is using private
binder dir
> > > > instead of libexec dir. e.g. xenconsole.  
> > > > For these cases, the latter one is suitable.  
>  
> This change seems correct to me. 
>  
> > > Hi Bamvor,  
> > >   
> > > Please include a "Signed-off-by:" line in your patches.
See "Signing
> > > off a patch" in
http://wiki.xen.org/wiki/Submitting_Xen_Patches
> > sorry missing it. i will add it next time. 
>  
> Please do. 
>  
> >  
> > > Do you see this problem when you run "./configure
--libdir=/usr/lib64"?
> > > See the 4.2 release notes for more information: 
> > >  
http://wiki.xen.org/wiki/Xen_4.2_Release_Notes#Build_Requirements
> > I got this issue while using the above libdir, here is my command of  
> configure  
> > ./configure --enable-xenapi --prefix=/usr --exec_prefix=/usr  
> --bindir=/usr/bin --sbindir=/usr/sbin --libdir=/usr/lib64  
> --datadir=/usr/share 
> > but the LIBEXEC is force set to /usr/lib/xen/bin in config/StdGNU.mk.
So,
> the libdir is /usr/lib64 while libexec is /usr/bin/xen/bin in  
> tools/libxl/_paths.h 
sent it.
> You aren''t saying that _paths.h is inconsistent with the actual
install
> locations for either LIBEXEC or PRIVATE_BINDIR, are you? Just that the 
> wrong one is being used in the code. 
yes, i am not. i talking about how to use the path in the
code.
> I''m not entirely sure what the intended distinction between
libexec and
> private-bindir is. We should perhaps consider nuking one or the other. 
>  
> > meanwhile, the libxl-save-helper is install to
"PRIVATE_BINDER" in
> tools/libxl/Makefile not LIBEXEC: 
> > $(INSTALL_PROG) libxl-save-helper $(DESTDIR)$(PRIVATE_BINDIR) 
> > using the "PRIVATE_BINDER" in run_helper make it coherence
with this
> Makefile. 
> >  
> > BTW, these files are not changed after xen4.2, but i only found this
issue
> in xen-unstable. 
>  
> That''s odd, both branches contain 25594:ad08cd8e7097 which is when
the
> libexec dir changed. I can''t see anything else which looks like it
would
> cause the two to differ. 
i re-test in xen4.2, got the same error in xen-unstable. it is only ok in
xen4.1 in my test right now. Sorry for my wrong report in my last post.
this code(25539:0f1a974bfb38) has beed commit over 6 months. i should
not the only one who encounter this issue. but i do not found anything wrong
in my build and run-time environment.
> Ian. 
>  
>  
>

On Wed, 2013-01-09 at 10:33 +0000, Bamvor Jian Zhang
wrote:
> Signed-off-by: Bamvor Jian Zhang <bjzhang@suse.com>
Acked + applied, thanks,