Libguestfs - Mar 2016 - Re: [ovirt-devel] virt-v2v become zombie via python cpopen on error

> On 30 Mar 2016, at 11:49, Richard W.M. Jones <rjones@redhat.com>
wrote:
> 
> On Wed, Mar 30, 2016 at 12:19:35PM +0300, Shahar Havivi wrote:
>> Hi,
>> 
>> We encounter a problem in VDSM project that virt-v2v become zombie task
while
>> importing vm from vmware.
>> When virt-v2v is in 'copy disk' mode and we someone deletes the
vm at vmware
>> the process hang in read() method,
>> I am pretty sure that its not virt-v2v problem because when I run it
from the
>> shell virt-v2v exit with an error, still maybe someone have an idea....
>> 
>> I wrote a small python script that encounter the problem:
>> 
>>
----------------------------------------------------------------------------
>> from cpopen import CPopen
>> 
>> env = {'LIBGUESTFS_BACKEND': 'direct'}
>> cmd = ['/usr/bin/virt-v2v', '-ic',
>>       'vpx://....', '-o',
>>       'local', '-os', '/tmp', '-of',
'raw', '-oa', 'sparse',
>>       '--password-file', '/tmp/passwd',
'--machine-readable', 'bbb']
>> p = CPopen(cmd, env=env)
>> while p.returncode is None:
>>    c = p.stdout.read(1)
>>    print c
>>
----------------------------------------------------------------------------
> 
> An actual zombie task?  That would indicate that the parent process
> (your Python program) wasn't doing a wait system call.
> 
> I downloaded the cpopen-1.4 program, and it doesn't appear to call any
> of the wait*(2) system calls anywhere, so that could be the problem.
I suppose the cpopen parameters are not alright…I’m sure vdsm developers can
help with that.
> 
> Rich.
> 
> -- 
> Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
> Read my programming and virtualization blog: http://rwmj.wordpress.com
> virt-p2v converts physical machines to virtual machines.  Boot with a
> live CD or over the network (PXE) and turn machines into KVM guests.
> http://libguestfs.org/virt-v2v

On Wed, Mar 30, 2016 at 1:36 PM, Michal Skrivanek
<michal.skrivanek@redhat.com> wrote:
>
>> On 30 Mar 2016, at 11:49, Richard W.M. Jones <rjones@redhat.com>
wrote:
>>
>> On Wed, Mar 30, 2016 at 12:19:35PM +0300, Shahar Havivi wrote:
>>> Hi,
>>>
>>> We encounter a problem in VDSM project that virt-v2v become zombie
task while
>>> importing vm from vmware.
>>> When virt-v2v is in 'copy disk' mode and we someone deletes
the vm at vmware
>>> the process hang in read() method,
>>> I am pretty sure that its not virt-v2v problem because when I run
it from the
>>> shell virt-v2v exit with an error, still maybe someone have an
idea....
>>>
>>> I wrote a small python script that encounter the problem:
>>>
>>>
----------------------------------------------------------------------------
>>> from cpopen import CPopen
>>>
>>> env = {'LIBGUESTFS_BACKEND': 'direct'}
>>> cmd = ['/usr/bin/virt-v2v', '-ic',
>>>       'vpx://....', '-o',
>>>       'local', '-os', '/tmp',
'-of', 'raw', '-oa', 'sparse',
>>>       '--password-file', '/tmp/passwd',
'--machine-readable', 'bbb']
>>> p = CPopen(cmd, env=env)
>>> while p.returncode is None:
p.returncode just return the instance variable, there is no wait() involved.

The right way:

    while p.poll() is None:
        ...

p.returncode calling wait is non-standard feature in vdsm AsyncProc
wrapper. This is
the object used by v2v vdsm module, so there accessing p.returncode does call
p.poll().

These non-standard apis will be removed from vdsm, please do not use them.
>>>    c = p.stdout.read(1)
>>>    print c
>>>
----------------------------------------------------------------------------
>>
>> An actual zombie task?  That would indicate that the parent process
>> (your Python program) wasn't doing a wait system call.
>>
>> I downloaded the cpopen-1.4 program, and it doesn't appear to call
any
>> of the wait*(2) system calls anywhere, so that could be the problem.
>
> I suppose the cpopen parameters are not alright…I’m sure vdsm developers
can help with that.
>
>>
>> Rich.
>>
>> --
>> Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
>> Read my programming and virtualization blog: http://rwmj.wordpress.com
>> virt-p2v converts physical machines to virtual machines.  Boot with a
>> live CD or over the network (PXE) and turn machines into KVM guests.
>> http://libguestfs.org/virt-v2v
>
> _______________________________________________
> Devel mailing list
> Devel@ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel

On 30.03.16 14:28, Nir Soffer wrote:
> On Wed, Mar 30, 2016 at 1:36 PM, Michal Skrivanek
> <michal.skrivanek@redhat.com> wrote:
> >
> >> On 30 Mar 2016, at 11:49, Richard W.M. Jones
<rjones@redhat.com> wrote:
> >>
> >> On Wed, Mar 30, 2016 at 12:19:35PM +0300, Shahar Havivi wrote:
> >>> Hi,
> >>>
> >>> We encounter a problem in VDSM project that virt-v2v become
zombie task while
> >>> importing vm from vmware.
> >>> When virt-v2v is in 'copy disk' mode and we someone
deletes the vm at vmware
> >>> the process hang in read() method,
> >>> I am pretty sure that its not virt-v2v problem because when I
run it from the
> >>> shell virt-v2v exit with an error, still maybe someone have an
idea....
> >>>
> >>> I wrote a small python script that encounter the problem:
> >>>
> >>>
----------------------------------------------------------------------------
> >>> from cpopen import CPopen
> >>>
> >>> env = {'LIBGUESTFS_BACKEND': 'direct'}
> >>> cmd = ['/usr/bin/virt-v2v', '-ic',
> >>>       'vpx://....', '-o',
> >>>       'local', '-os', '/tmp',
'-of', 'raw', '-oa', 'sparse',
> >>>       '--password-file', '/tmp/passwd',
'--machine-readable', 'bbb']
> >>> p = CPopen(cmd, env=env)
> >>> while p.returncode is None:
> 
> p.returncode just return the instance variable, there is no wait()
involved.
> 
> The right way:
> 
>     while p.poll() is None:
the problem is proc.stdout.read(1) didn't raise and error when the stream
closed but return ''.
its a CPopen behaviour and works differently in subprocess.
>         ...
> 
> p.returncode calling wait is non-standard feature in vdsm AsyncProc
> wrapper. This is
> the object used by v2v vdsm module, so there accessing p.returncode does
call
> p.poll().
> 
> These non-standard apis will be removed from vdsm, please do not use them.
> 
> >>>    c = p.stdout.read(1)
> >>>    print c
> >>>
----------------------------------------------------------------------------
> >>
> >> An actual zombie task?  That would indicate that the parent
process
> >> (your Python program) wasn't doing a wait system call.
> >>
> >> I downloaded the cpopen-1.4 program, and it doesn't appear to
call any
> >> of the wait*(2) system calls anywhere, so that could be the
problem.
> >
> > I suppose the cpopen parameters are not alright…I’m sure vdsm
developers can help with that.
> >
> >>
> >> Rich.
> >>
> >> --
> >> Richard Jones, Virtualization Group, Red Hat
http://people.redhat.com/~rjones
> >> Read my programming and virtualization blog:
http://rwmj.wordpress.com
> >> virt-p2v converts physical machines to virtual machines.  Boot
with a
> >> live CD or over the network (PXE) and turn machines into KVM
guests.
> >> http://libguestfs.org/virt-v2v
> >
> > _______________________________________________
> > Devel mailing list
> > Devel@ovirt.org
> > http://lists.ovirt.org/mailman/listinfo/devel