libvirt users - Feb 2017 - Re: Determining domain job kind from job stats?

Jiri Denemark <jdenemar@redhat.com> writes:
> On Fri, Feb 17, 2017 at 12:38:24 +0100, Milan Zamazal wrote:
>> Jiri Denemark <jdenemar@redhat.com> writes:
>> 
>> > On Fri, Feb 10, 2017 at 21:50:19 +0100, Milan Zamazal wrote:
>> >> Hi, is there a reliable way to find out to what kind of job
does the
>> >> information returned from virDomainGetJobStats or provided in
>> >> VIR_DOMAIN_EVENT_ID_JOB_COMPLETED event callback belong to?
>> >
>> > No, libvirt expects that the caller knows what job it started. All
jobs
>> > currently reported using virDomainGetJobStats API or
>> > VIR_DOMAIN_EVENT_ID_JOB_COMPLETED event are internally implemented
as
>> > migration in QEMU driver (either to a file or to a network
socket),
>> > which may confuse any heuristics for detecting the job type from
the set
>> > of fields returned by libvirt.
>> 
>> I see, thank you for explanation.
>> 
>> > What is the problem you are trying to solve?
>> 
>> There are basically two problems:
>> 
>> - When the job completion callback is called, I need to distinguish
what
>>   kind of job was it to perform the appropriate actions.  It would be
>>   easier if I knew the job type directly in the callback (no need to
>>   coordinate anything), but "external" job tracking is also
possible.
>
> An immediate answer would be: "don't rely on the completion
callback and
> just check the return value of the API which started the job", but I
> guess you want it because checking the return value is not possible when
> the process which started the job is not running anymore as described
> below.
Well, avoiding using the completion callback is probably OK for me.
(In case of the process restart, I don't expect having everything
perfectly working, just some basic sanity.)
>> - If I lost track of my jobs (e.g. because of a crash and restart),
I'd
>>   like to find out whether a given VM is migrating.  Examining the job
>>   looked like a good candidate to get the information, but apparently
>>   it's not.  Again, I can probably arrange things to handle that,
but to
>>   get the information directly from libvirt (not necessarily via job
>>   info) would be easier and more reliable.
>
> Apparently you are talking about peer-to-peer migration, 
Yes.
> otherwise the migration would be automatically canceled when the
> process which started it disappears. I'm afraid this is not currently
> possible in general. You might be able to get something by checking
> the domain's status, but it won't work in all cases.
Too bad.  Could some future libvirt version provide that information?

Thank you for clarification,
Milan

[Starting to move to the development list.]

Milan Zamazal <mzamazal@redhat.com> writes:
> Jiri Denemark <jdenemar@redhat.com> writes:
>
>> On Fri, Feb 17, 2017 at 12:38:24 +0100, Milan Zamazal wrote:
>>> 
>>> There are basically two problems:
>>> 
>>> - When the job completion callback is called, I need to distinguish
what
>>>   kind of job was it to perform the appropriate actions.  It would
be
>>>   easier if I knew the job type directly in the callback (no need
to
>>>   coordinate anything), but "external" job tracking is
also possible.
>>
>> An immediate answer would be: "don't rely on the completion
callback and
>> just check the return value of the API which started the job", but
I
>> guess you want it because checking the return value is not possible
when
>> the process which started the job is not running anymore as described
>> below.
>
> Well, avoiding using the completion callback is probably OK for me.
Thinking about it more, it's not very nice: I have to use the callback
to get the completed job stats (I'm not guaranteed the domain still
exists on the source host when I ask it for the stats explicitly) *and*
to track the jobs outside the callback to know whether the callback is
related to the type of domain jobs I'm going to handle.

Although not absolutely necessary, it would be much nicer if the job
type was identified in the callback.
> (In case of the process restart, I don't expect having everything
> perfectly working, just some basic sanity.)
>
>>> - If I lost track of my jobs (e.g. because of a crash and restart),
I'd
>>>   like to find out whether a given VM is migrating.  Examining the
job
>>>   looked like a good candidate to get the information, but
apparently
>>>   it's not.  Again, I can probably arrange things to handle
that, but to
>>>   get the information directly from libvirt (not necessarily via
job
>>>   info) would be easier and more reliable.
>>
>> Apparently you are talking about peer-to-peer migration, 
>
> Yes.
>
>> otherwise the migration would be automatically canceled when the
>> process which started it disappears. I'm afraid this is not
currently
>> possible in general. You might be able to get something by checking
>> the domain's status, but it won't work in all cases.
>
> Too bad.  Could some future libvirt version provide that information?
If libvirt provided information about the job type, it would help with
several things, for instance: With the callback problem above, with
using libvirt as the ultimate single source of information about the
VMs, or with handling VMs not running under complete control of a
particular piece of software.

If some piece of information about a VM is missing from libvirt I need
to store it somewhere.  Domain metadata is a natural place for that,
since it's closely bound to the corresponding VM and keeps libvirt as
the ultimate single source of information.  But putting there
information about migration is weird at best.

Based on those thoughts I think libvirt should really provide a simple
way to find out whether the VM is migrating to another host and to
identify the domain job type in the job completed callback.  Is there
anything preventing to add that information?

Thanks,
Milan

On Tue, Feb 21, 2017 at 15:38:22 +0100, Milan Zamazal
wrote:
> [Starting to move to the development list.]
> 
> Milan Zamazal <mzamazal@redhat.com> writes:
> 
> > Jiri Denemark <jdenemar@redhat.com> writes:
> >
> >> On Fri, Feb 17, 2017 at 12:38:24 +0100, Milan Zamazal wrote:
> >>> 
> >>> There are basically two problems:
> >>> 
> >>> - When the job completion callback is called, I need to
distinguish what
> >>>   kind of job was it to perform the appropriate actions.  It
would be
> >>>   easier if I knew the job type directly in the callback (no
need to
> >>>   coordinate anything), but "external" job tracking
is also possible.
> >>
> >> An immediate answer would be: "don't rely on the
completion callback and
> >> just check the return value of the API which started the
job", but I
> >> guess you want it because checking the return value is not
possible when
> >> the process which started the job is not running anymore as
described
> >> below.
> >
> > Well, avoiding using the completion callback is probably OK for me.
> 
> Thinking about it more, it's not very nice: I have to use the callback
> to get the completed job stats (I'm not guaranteed the domain still
> exists on the source host when I ask it for the stats explicitly) *and*
> to track the jobs outside the callback to know whether the callback is
> related to the type of domain jobs I'm going to handle.
> 
> Although not absolutely necessary, it would be much nicer if the job
> type was identified in the callback.
The job completed event uses type parameters so adding a new parameter
describing the just completed job should not be a problem.

Jirka