Xen devel - Jun 2011 - Tracking guest pages modified by qemu-dm / backend

Consider an HVM domain with EPT. When qemu-dm, backend or any other
device model modifies DomU memory (like on a disk read or packet
received), how does Xen track the pages that have been modified? For
example, when an DomU is being migrated while IO is occurring.

Thanks,
AP

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi, 

At 15:43 -0700 on 21 Jun (1308671006), AP wrote:
> Consider an HVM domain with EPT. When qemu-dm, backend or any other
> device model modifies DomU memory (like on a disk read or packet
> received), how does Xen track the pages that have been modified? For
> example, when an DomU is being migrated while IO is occurring.
Backends use grant tables to map guest memory; the granted pages are
marked dirty when the backend disconnects.  Qemu-dm uses the
HVMOP_modified memory hypercall to tell Xen that it''s written to
the guest''s memory.

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, Jun 22, 2011 at 1:49 AM, Tim Deegan <Tim.Deegan@citrix.com>
wrote:
> Hi,
>
> At 15:43 -0700 on 21 Jun (1308671006), AP wrote:
>> Consider an HVM domain with EPT. When qemu-dm, backend or any other
>> device model modifies DomU memory (like on a disk read or packet
>> received), how does Xen track the pages that have been modified? For
>> example, when an DomU is being migrated while IO is occurring.
>
> Backends use grant tables to map guest memory; the granted pages are
> marked dirty when the backend disconnects.  Qemu-dm uses the
> HVMOP_modified memory hypercall to tell Xen that it''s written to
> the guest''s memory.
>From what I see for EPT/HAP, log dirty is turned on only during domain
save. Is that true or am I missing something. Plus how does dirty
tracking occur with EPT since those tables don''t have a dirty bit? Is
it using the log_dirty_bitmap?

Thanks,
AP

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 11:06 -0700 on 23 Jun (1308827188), AP wrote:
> On Wed, Jun 22, 2011 at 1:49 AM, Tim Deegan <Tim.Deegan@citrix.com>
wrote:
> > Hi,
> >
> > At 15:43 -0700 on 21 Jun (1308671006), AP wrote:
> >> Consider an HVM domain with EPT. When qemu-dm, backend or any
other
> >> device model modifies DomU memory (like on a disk read or packet
> >> received), how does Xen track the pages that have been modified?
For
> >> example, when an DomU is being migrated while IO is occurring.
> >
> > Backends use grant tables to map guest memory; the granted pages are
> > marked dirty when the backend disconnects.  Qemu-dm uses the
> > HVMOP_modified memory hypercall to tell Xen that it''s written
to
> > the guest''s memory.
> 
> From what I see for EPT/HAP, log dirty is turned on only during domain
> save.
It''s also used to track just the virtual framebuffer so qemu can avoid
having to compute which parts of the screen to send over VNC. 

But for general memory it''s only used during save, because
there''s no
need for it otherwise.  If you''re confused about that maybe you should
read the original paper on Xen live migration:
http://www.cl.cam.ac.uk/research/srg/netos/papers/2005-nsdi-migration.pdf
> Is that true or am I missing something. Plus how does dirty
> tracking occur with EPT since those tables don''t have a dirty bit?
Is
> it using the log_dirty_bitmap?
It sets all memory read-only in the EPT tables.  On an EPT fault it
marks the page dirty in the bitmap and resetd it to r/w. 
See hap_enable_log_dirty() and hvm_hap_nested_page_fault().

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Xen Platform Team
Citrix Systems UK Ltd.  (Company #02937203, SL9 0BG)

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Jun 24, 2011 at 1:40 AM, Tim Deegan <Tim.Deegan@citrix.com>
wrote:
> At 11:06 -0700 on 23 Jun (1308827188), AP wrote:
>> On Wed, Jun 22, 2011 at 1:49 AM, Tim Deegan
<Tim.Deegan@citrix.com> wrote:
>> > Hi,
>> >
>> > At 15:43 -0700 on 21 Jun (1308671006), AP wrote:
>> >> Consider an HVM domain with EPT. When qemu-dm, backend or any
other
>> >> device model modifies DomU memory (like on a disk read or
packet
>> >> received), how does Xen track the pages that have been
modified? For
>> >> example, when an DomU is being migrated while IO is occurring.
>> >
>> > Backends use grant tables to map guest memory; the granted pages
are
>> > marked dirty when the backend disconnects.  Qemu-dm uses the
>> > HVMOP_modified memory hypercall to tell Xen that it''s
written to
>> > the guest''s memory.
>>
>> From what I see for EPT/HAP, log dirty is turned on only during domain
>> save.
>
> It''s also used to track just the virtual framebuffer so qemu can
avoid
> having to compute which parts of the screen to send over VNC.
>
> But for general memory it''s only used during save, because
there''s no
> need for it otherwise.  If you''re confused about that maybe you
should
> read the original paper on Xen live migration:
> http://www.cl.cam.ac.uk/research/srg/netos/papers/2005-nsdi-migration.pdf
Sorry I think I did not phrase my question well. What I was not able
to understand was how / where the granted pages were being marked
dirty w.r.t EPT after the backend disconnects.

Thanks,
AP

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, Jun 24, 2011 at 1:22 PM, AP <apxeng@gmail.com> wrote:
> On Fri, Jun 24, 2011 at 1:40 AM, Tim Deegan <Tim.Deegan@citrix.com>
wrote:
> > At 11:06 -0700 on 23 Jun (1308827188), AP wrote:
> >> On Wed, Jun 22, 2011 at 1:49 AM, Tim Deegan
<Tim.Deegan@citrix.com>
> wrote:
> >> > Hi,
> >> >
> >> > At 15:43 -0700 on 21 Jun (1308671006), AP wrote:
> >> >> Consider an HVM domain with EPT. When qemu-dm, backend or
any other
> >> >> device model modifies DomU memory (like on a disk read or
packet
> >> >> received), how does Xen track the pages that have been
modified? For
> >> >> example, when an DomU is being migrated while IO is
occurring.
> >> >
> >> > Backends use grant tables to map guest memory; the granted
pages are
> >> > marked dirty when the backend disconnects.  Qemu-dm uses the
> >> > HVMOP_modified memory hypercall to tell Xen that
it''s written to
> >> > the guest''s memory.
> >>
> >> From what I see for EPT/HAP, log dirty is turned on only during
domain
> >> save.
> >
> > It''s also used to track just the virtual framebuffer so qemu
can avoid
> > having to compute which parts of the screen to send over VNC.
> >
> > But for general memory it''s only used during save, because
there''s no
> > need for it otherwise.  If you''re confused about that maybe
you should
> > read the original paper on Xen live migration:
> >
> http://www.cl.cam.ac.uk/research/srg/netos/papers/2005-nsdi-migration.pdf
>
> Sorry I think I did not phrase my question well. What I was not able
> to understand was how / where the granted pages were being marked
> dirty w.r.t EPT after the backend disconnects.
>
> I am not sure what you mean by "backend disconnects", but each
time a grant
is unmapped, log dirty bit for the page is set.
Check out the __gnttab_unmap_common function in xen/common/grant_table.c
it calls gnttab_mark_dirty(..) which is a macro for paging_mark_dirty.

Thanks,
> AP
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel