Xen devel - Oct 2012 - How to get the checkpoint size in remus code?

Hi,

I''m doing my master research and I need to adapt remus code. Now... I
wanna
get the checkpoint size (memory + disk) on each period. Does someone know
what function does this? I think some *fd *object''s function in remus
code
could just get the memory size.

Does someone help me?

Thanks


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Remus takes advantage of the last iteration of live migration to get dirty
memory and transfer to the backup machine. So you can get the dirty memory size
from live migration''s dirty bitmap.

On Oct 1, 2012, at 10:48 AM, José Eduardo França wrote:
> Hi,
> 
> I''m doing my master research and I need to adapt remus code.
Now... I wanna get the checkpoint size (memory + disk) on each period. Does
someone know what function does this? I think some fd object''s function
in remus code could just get the memory size.
> 
> Does someone help me?
> 
> Thanks
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

I thought remus used xc_domain_save. Is this function used from live
migration?

Futhermore I have two doubts if really Remus takes the last iteration of
live migration

   1. What''s the function?
   2. And how to get de I/O disk size on each period?

Thanks

2012/10/1 lmingcsce <lmingcsce@gmail.com>
> Remus takes advantage of the last iteration of live migration to get dirty
> memory and transfer to the backup machine. So you can get the dirty memory
> size from live migration''s dirty bitmap.
>
> On Oct 1, 2012, at 10:48 AM, José Eduardo França wrote:
>
> Hi,
>
> I''m doing my master research and I need to adapt remus code.
Now... I
> wanna get the checkpoint size (memory + disk) on each period. Does someone
> know what function does this? I think some *fd *object''s function
in
> remus code could just get the memory size.
>
> Does someone help me?
>
> Thanks
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Wed, Oct 3, 2012 at 4:11 AM, José Eduardo França <jefranca@gmail.com>
wrote:
>
> I thought remus used xc_domain_save. Is this function used from live
> migration?
>
> Futhermore I have two doubts if really Remus takes the last iteration of
> live migration
>
> What''s the function?
There is no specific function. xc_domain_save is where everything
happens. The infinite loop
that basically keeps sending checkpoints @ a particular frequency

> And how to get de I/O disk size on each period?
>
This depends on the disk backend. With blktap2 (unfortunately not
available in 3.* kernels)
tap-remus driver can give you the number of disk blocks sent per checkpoint.

With DRBD, it needs a little bit of hacking into the kernel module to
return the number of disk blocks
being sent with each checkpoint.

>> I''m doing my master research and I need to adapt remus code.
Now... I
>> wanna get the checkpoint size (memory + disk) on each period. Does
someone
>> know what function does this? I think some fd object''s
function in remus
>> code could just get the memory size.
>>
You can get memory checkpoint stats for each iteration - like
number of pages dirtied, size of data actually transmitted after
compression (including headers, etc),
time to checkpoint, etc.

The attached patch (for xen-4.1.2) will give you the memory checkpoint
stats for each checkpoint and
can be easily parsed.

shriram


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

---------- Forwarded message ----------
From: José Eduardo França <jefranca@gmail.com>
Date: 2012/10/17
Subject: Re: [Xen-devel] How to get the checkpoint size in remus code?
To: rshriram@cs.ubc.ca


Hi Shriram,

Thank you for your reply. I''m sorry coz I saw your email yesterday and
my
English is bad.
Ok... I saw your patch but I need to explain more my problem.

In my master research, I intend to deploy a based-time dynamic checkpoint
that should work this way: if checkpoint size breaks *Lmax* (see attached
figure) I reduce checkpoint interval, and if checkpoint size doesn''t
break *
Lmin* I increase checkpoint interval. After that I will evaluate the
performace.

I had read remus code and I saw that remus control the elapsed time in

            endtime = time.time()
            elapsed = (endtime - closure.starttime) * 1000

            if elapsed < cfg.interval:
                time.sleep((cfg.interval - elapsed) / 1000.0)

Then I thought I could change the checkpoint interval close to code above,
but I suppose I need get checkpoint size. But here the remus code is python
and I saw (or thought) that checkpoint size is gotten on *xc_shadow_op_stats_t
*stats* or better on *stats->dirty_count*PAGE_SIZE*.

Would I get checkpoint size into remus code? I thought it''s easier this
way
for I intend to do.
Please, help me, coz my time is running out.

Thanks jefranca

PS: My English is terrible coz I''m not native


2012/10/5 Shriram Rajagopalan <rshriram@cs.ubc.ca>
> On Wed, Oct 3, 2012 at 4:11 AM, José Eduardo França
<jefranca@gmail.com>
> wrote:
> >
> > I thought remus used xc_domain_save. Is this function used from live
> > migration?
> >
> > Futhermore I have two doubts if really Remus takes the last iteration
of
> > live migration
> >
> > What''s the function?
>
> There is no specific function. xc_domain_save is where everything
> happens. The infinite loop
> that basically keeps sending checkpoints @ a particular frequency
>
>
> > And how to get de I/O disk size on each period?
> >
>
> This depends on the disk backend. With blktap2 (unfortunately not
> available in 3.* kernels)
> tap-remus driver can give you the number of disk blocks sent per
> checkpoint.
>
> With DRBD, it needs a little bit of hacking into the kernel module to
> return the number of disk blocks
> being sent with each checkpoint.
>
>
> >> I''m doing my master research and I need to adapt remus
code. Now... I
> >> wanna get the checkpoint size (memory + disk) on each period. Does
> someone
> >> know what function does this? I think some fd object''s
function in remus
> >> code could just get the memory size.
> >>
>
> You can get memory checkpoint stats for each iteration - like
> number of pages dirtied, size of data actually transmitted after
> compression (including headers, etc),
> time to checkpoint, etc.
>
> The attached patch (for xen-4.1.2) will give you the memory checkpoint
> stats for each checkpoint and
> can be easily parsed.
>
> shriram
>


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

Hi Guys,

I see too that I can get stats using struct xen_domctl.

Then could I put the code below anywhere and get the dirty_count? For
example:

    DECLARE_DOMCTL;
    uint32_t dirty_count = domctl.u.shadow_op.stats.dirty_count;

Can I do this?

Or are there a permanent data that return dirty_count or checkpoint_size?
If yes, what''s it?

thanks

2012/10/17 José Eduardo França <jefranca@gmail.com>
> Hi Shriram,
>
> Thank you for your reply. I''m sorry coz I saw your email yesterday
and my
> English is bad.
> Ok... I saw your patch but I need to explain more my problem.
>
> In my master research, I intend to deploy a based-time dynamic checkpoint
> that should work this way: if checkpoint size breaks *Lmax* (see attached
> figure) I reduce checkpoint interval, and if checkpoint size
doesn''t break
> *Lmin* I increase checkpoint interval. After that I will evaluate the
> performace.
>
> I had read remus code and I saw that remus control the elapsed time in
>
>             endtime = time.time()
>             elapsed = (endtime - closure.starttime) * 1000
>
>             if elapsed < cfg.interval:
>                 time.sleep((cfg.interval - elapsed) / 1000.0)
>
> Then I thought I could change the checkpoint interval close to code above,
> but I suppose I need get checkpoint size. But here the remus code is python
> and I saw (or thought) that checkpoint size is gotten on
*xc_shadow_op_stats_t
> *stats* or better on *stats->dirty_count*PAGE_SIZE*.
>
> Would I get checkpoint size into remus code? I thought it''s easier
this
> way for I intend to do.
> Please, help me, coz my time is running out.
>
> Thanks jefranca
>
> PS: My English is terrible coz I''m not native
>
>
> 2012/10/5 Shriram Rajagopalan <rshriram@cs.ubc.ca>
>
>> On Wed, Oct 3, 2012 at 4:11 AM, José Eduardo França
<jefranca@gmail.com>
>> wrote:
>> >
>> > I thought remus used xc_domain_save. Is this function used from
live
>> > migration?
>> >
>> > Futhermore I have two doubts if really Remus takes the last
iteration of
>> > live migration
>> >
>> > What''s the function?
>>
>> There is no specific function. xc_domain_save is where everything
>> happens. The infinite loop
>> that basically keeps sending checkpoints @ a particular frequency
>>
>>
>> > And how to get de I/O disk size on each period?
>> >
>>
>> This depends on the disk backend. With blktap2 (unfortunately not
>> available in 3.* kernels)
>> tap-remus driver can give you the number of disk blocks sent per
>> checkpoint.
>>
>> With DRBD, it needs a little bit of hacking into the kernel module to
>> return the number of disk blocks
>> being sent with each checkpoint.
>>
>>
>> >> I''m doing my master research and I need to adapt
remus code. Now... I
>> >> wanna get the checkpoint size (memory + disk) on each period.
Does
>> someone
>> >> know what function does this? I think some fd
object''s function in
>> remus
>> >> code could just get the memory size.
>> >>
>>
>> You can get memory checkpoint stats for each iteration - like
>> number of pages dirtied, size of data actually transmitted after
>> compression (including headers, etc),
>> time to checkpoint, etc.
>>
>> The attached patch (for xen-4.1.2) will give you the memory checkpoint
>> stats for each checkpoint and
>> can be easily parsed.
>>
>> shriram
>>
>
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel