Libguestfs - Nov 2018 - Re: guestfs_launch() fails when C application is started as a systemd service

Here are strace outputs per process.

strace_output.22076 is the plugin's pid. (A little before forking)
strace_output.22077  runs qemu-img
strace_output.22078 is a mystery to me
strace_output.22079 is rm -rf to the overlay image and its temporary dir.

Best Regards,
Peter


On Wed, Nov 7, 2018 at 4:38 PM Richard W.M. Jones <rjones@redhat.com>
wrote:
> On Wed, Nov 07, 2018 at 01:37:03PM +0200, Peter Dimitrov wrote:
> > > That makes no sense because we are supposed to have just forked
> > successfully
> >
> > I just realized libguestfs uses fork. Now we know why qemu-img worked
- I
> > launched it with popen.
> >
> > > So it must be something to do with collectd and how it runs
programs.
> > > Is it using LD_PRELOAD trickery, or replacing libc, or using
seccomp?
> >
> > If I understand the question correctly - it's about how collectd
loads
> its
> > plugins? If so it uses:
> >
> > static int plugin_load_file(const char *file, _Bool global) {
> >   void (*reg_handle)(void);
> >   int flags = RTLD_NOW;
> >     if (global)
> >       flags |= RTLD_GLOBAL;
> >   void *dlh = *dlopen*(file, flags);
> >   //...
> >   reg_handle = (void (*)(void))*dlsym*(dlh,
"module_register");
> >   //...
> >   *(*reg_handle)();*
> > }
> >
> >
> > Does this give any clues?
>
> Can you strace the collectd process around the point where qemu-img is
> run?  It may gives some clues.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> libguestfs lets you edit virtual machines.  Supports shell scripting,
> bindings from many languages.  http://libguestfs.org
>

On Thu, Nov 08, 2018 at 04:51:31PM +0200, Peter Dimitrov
wrote:
> Here are strace outputs per process.
> 
> strace_output.22076 is the plugin's pid. (A little before forking)
Ah ha.

Close reading of the waitpid(2) man page says:

       ECHILD (for  waitpid() or waitid()) The process specified by pid (wait‐
              pid()) or idtype and id (waitid()) does not exist or  is  not  a
              child  of  the  calling process.  (This can happen for one's
own
              child if the action for SIGCHLD is set to SIG_IGN.  See also the
              Linux Notes section about threads.)

I'm going to guess that collectd is leaking the signal handler setting
into the child process instead of resetting it.  This is a bug in
collectd.

It's surprisingly hard to correctly fork a process in Unix.  Here's
what libvirt does, which is the most comprehensive code that I know
of.  It involves resetting multiple things before running the child:

 
https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/util/vircommand.c;h=de937f6f9aa91abb518eac98bfac9dcf37e1f5df;hb=HEAD#l280

While you're getting the collectd bug fixed, the easiest workaround is
probably to add:

  signal (SIGCHLD, SIG_DFL);

in your code.

Rich.

-- 
Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones
Read my programming and virtualization blog: http://rwmj.wordpress.com
virt-top is 'top' for virtual machines.  Tiny program with many
powerful monitoring features, net stats, disk stats, logging, etc.
http://people.redhat.com/~rjones/virt-top

Confirming that:

signal (SIGCHLD, SIG_DFL);

works.

Thank you, Rich!
Best Regards,
Peter


On Thu, Nov 8, 2018 at 5:07 PM Richard W.M. Jones <rjones@redhat.com>
wrote:
> On Thu, Nov 08, 2018 at 04:51:31PM +0200, Peter Dimitrov wrote:
> > Here are strace outputs per process.
> >
> > strace_output.22076 is the plugin's pid. (A little before forking)
>
> Ah ha.
>
> Close reading of the waitpid(2) man page says:
>
>        ECHILD (for  waitpid() or waitid()) The process specified by pid
> (wait‐
>               pid()) or idtype and id (waitid()) does not exist or  is
> not  a
>               child  of  the  calling process.  (This can happen for
one's
> own
>               child if the action for SIGCHLD is set to SIG_IGN.  See also
> the
>               Linux Notes section about threads.)
>
> I'm going to guess that collectd is leaking the signal handler setting
> into the child process instead of resetting it.  This is a bug in
> collectd.
>
> It's surprisingly hard to correctly fork a process in Unix.  Here's
> what libvirt does, which is the most comprehensive code that I know
> of.  It involves resetting multiple things before running the child:
>
>
>
https://libvirt.org/git/?p=libvirt.git;a=blob;f=src/util/vircommand.c;h=de937f6f9aa91abb518eac98bfac9dcf37e1f5df;hb=HEAD#l280
>
> While you're getting the collectd bug fixed, the easiest workaround is
> probably to add:
>
>   signal (SIGCHLD, SIG_DFL);
>
> in your code.
>
> Rich.
>
> --
> Richard Jones, Virtualization Group, Red Hat
> http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-top is 'top' for virtual machines.  Tiny program with many
> powerful monitoring features, net stats, disk stats, logging, etc.
> http://people.redhat.com/~rjones/virt-top
>