libvirt users - Jan 2017 - Regarding Migration Statistics

Greetings,

I am writing a code using libvirt API to migrate VM between two physical
hosts *(QEMU/KVM) , *say some *n *number of times.

*1)* I am using right now* virDomainPtr virDomainMigrate (.......) *and to
calculate the total migration time I am using something like this:

*clock_gettime(CLOCK_MONOTONIC_RAW,&begin);  *
 *migrate*(domainToMigrate,nodeToMigrate);

*clock_gettime(CLOCK_MONOTONIC_RAW,&end);*



*Total Migration Time = end.tv_sec-begin.tv_sec*

Is this correct way to calculate total migration time. And is there some
way to calculate the downtime (not how to set it)?

*2) *I am interested in identifying in particular other statistics of
migration like :
*Number of iterations in Pre Copy*, *Memory transferred in each iteration*
etc.
I was going through the API and found* virDomainJobInfo
<http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainJobInfo> and
virDomainGetJobStats
<http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetJobStats>
*functions.But
how to use them is not very clear. Can anyone point me to right place to
achieve this objective?

Thanks in advance.
And sorry if that was too silly to ask.

Anubhav

Please, do not post to several libvirt lists at once.
> I am writing a code using libvirt API to migrate VM between two physical
> hosts *(QEMU/KVM) , *say some *n *number of times.
> 
> *1)* I am using right now* virDomainPtr virDomainMigrate (.......) *and to
> calculate the total migration time I am using something like this:
> 
> *clock_gettime(CLOCK_MONOTONIC_RAW,&begin);  *
>  *migrate*(domainToMigrate,nodeToMigrate);
> 
> *clock_gettime(CLOCK_MONOTONIC_RAW,&end);*
> 
> *Total Migration Time = end.tv_sec-begin.tv_sec*
> 
> Is this correct way to calculate total migration time. And is there some
> way to calculate the downtime (not how to set it)?
Well, just call virDomainGetJobStats and you will get both total time
and downtime and many more.
> virDomainGetJobStats
>
<http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetJobStats>
Yes, this is the right API to get all the statistics you want. It
returns a list of (key, type, value) entires in params. The keys are
VIR_DOMAIN_JOB_* (see
http://libvirt.org/html/libvirt-libvirt-domain.html#VIR_DOMAIN_JOB_AUTO_CONVERGE_THROTTLE
and a lot of other VIR_DOMAIN_JOB_* macros following this one for
details). You can look at
http://libvirt.org/git/?p=libvirt.git;a=blob;f=tools/virsh-domain.c;h=93587e8bc7c79cbb66b6ee107472dfec5f320dab;hb=HEAD

In short, just call

    virTypedParamsGetULLong(params, nparams, VIR_DOMAIN_JOB_DOWNTIME,
                            &value)

to fetch downtime from params into value if it was present there (check
return value of the API). The exact API to call is different for each
type, although all migration statistics are unsigned long long so this
one is all you need.

BTW, you can call virDomainGetJobStats anytime while migration is
running to monitor its progress. To check statistics of a completed
migration it's easier to just register a callback for
VIR_DOMAIN_EVENT_ID_JOB_COMPLETED event. See
http://libvirt.org/git/?p=libvirt.git;a=blob;f=examples/object-events/event-test.c;h=55c004f93f40de628ae76221bb8ef9a13b2acb3d;hb=HEAD#l940
for an example.

Jirka

On Tue, Jan 10, 2017 at 08:07:13PM +0530, Anubhav Guleria
wrote:
>Greetings,
>
>I am writing a code using libvirt API to migrate VM between two physical
>hosts *(QEMU/KVM) , *say some *n *number of times.
>
>*1)* I am using right now* virDomainPtr virDomainMigrate (.......) *and to
>calculate the total migration time I am using something like this:
>
>*clock_gettime(CLOCK_MONOTONIC_RAW,&begin);  *
> *migrate*(domainToMigrate,nodeToMigrate);
>
>*clock_gettime(CLOCK_MONOTONIC_RAW,&end);*
>
>
>
>*Total Migration Time = end.tv_sec-begin.tv_sec*
>
>Is this correct way to calculate total migration time. And is there some
>way to calculate the downtime (not how to set it)?
>
Libvirt provides this information, I believe in virDomainGetJobStats().
If you call this function with flags |= VIR_DOMAIN_JOB_STATS_COMPLETED,
it will return various information on a recently completed job
(e.g. migration).

You can also register a callback for VIR_DOMAIN_EVENT_ID_JOB_COMPLETED
event, which will tell you all that right when the migration is completed.

>*2) *I am interested in identifying in particular other statistics of
>migration like :
>*Number of iterations in Pre Copy*, *Memory transferred in each iteration*
>etc.
>I was going through the API and found* virDomainJobInfo
><http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainJobInfo>
and
>virDomainGetJobStats
><http://libvirt.org/html/libvirt-libvirt-domain.html#virDomainGetJobStats>
>*functions.But
>how to use them is not very clear. Can anyone point me to right place to
>achieve this objective?
>
int type = -1, nparams = 0;
virTypedParameterPtr params = NULL;

virDomainGetJobStats(domain, &type, &params, &nparams,
                     VIR_DOMAIN_JOB_STATS_COMPLETED)

This should be enough ^^.  Since migration should be a job on bothe
source and destination, it may return different data, I'm not too sure
about that.  You'll have to try that out or hope that someone else will
answer as well, sorry.
>Thanks in advance.
>And sorry if that was too silly to ask.
>
>Anubhav
>--
>libvir-list mailing list
>libvir-list@redhat.com
>https://www.redhat.com/mailman/listinfo/libvir-list

Actually my setup is like this:

I have one controller host that do this migration between other two nodes
say A and B. Now where should this virDomainGetJobStats method should be
invoked from ?

If at present VM is migrated from A to B then should B call this method?

It's not clear.



"CODENAME FREAK -47"

On Tue, Jan 10, 2017 at 8:48 PM, Jiri Denemark <jdenemar@redhat.com>
wrote:
> Please, do not post to several libvirt lists at once.
>
> > I am writing a code using libvirt API to migrate VM between two
physical
> > hosts *(QEMU/KVM) , *say some *n *number of times.
> >
> > *1)* I am using right now* virDomainPtr virDomainMigrate (.......)
*and
> to
> > calculate the total migration time I am using something like this:
> >
> > *clock_gettime(CLOCK_MONOTONIC_RAW,&begin);  *
> >  *migrate*(domainToMigrate,nodeToMigrate);
> >
> > *clock_gettime(CLOCK_MONOTONIC_RAW,&end);*
> >
> > *Total Migration Time = end.tv_sec-begin.tv_sec*
> >
> > Is this correct way to calculate total migration time. And is there
some
> > way to calculate the downtime (not how to set it)?
>
> Well, just call virDomainGetJobStats and you will get both total time
> and downtime and many more.
>
> > virDomainGetJobStats
> > <http://libvirt.org/html/libvirt-libvirt-domain.html#
> virDomainGetJobStats>
>
> Yes, this is the right API to get all the statistics you want. It
> returns a list of (key, type, value) entires in params. The keys are
> VIR_DOMAIN_JOB_* (see
> http://libvirt.org/html/libvirt-libvirt-domain.html#
> VIR_DOMAIN_JOB_AUTO_CONVERGE_THROTTLE
> and a lot of other VIR_DOMAIN_JOB_* macros following this one for
> details). You can look at
> http://libvirt.org/git/?p=libvirt.git;a=blob;f=tools/virsh-domain.c;h>
93587e8bc7c79cbb66b6ee107472dfec5f320dab;hb=HEAD
>
> In short, just call
>
>     virTypedParamsGetULLong(params, nparams, VIR_DOMAIN_JOB_DOWNTIME,
>                             &value)
>
> to fetch downtime from params into value if it was present there (check
> return value of the API). The exact API to call is different for each
> type, although all migration statistics are unsigned long long so this
> one is all you need.
>
> BTW, you can call virDomainGetJobStats anytime while migration is
> running to monitor its progress. To check statistics of a completed
> migration it's easier to just register a callback for
> VIR_DOMAIN_EVENT_ID_JOB_COMPLETED event. See
> http://libvirt.org/git/?p=libvirt.git;a=blob;f=examples/
> object-events/event-test.c;h=55c004f93f40de628ae76221bb8ef9
> a13b2acb3d;hb=HEAD#l940
> for an example.
>
> Jirka
>