Xen devel - May 2010 - treatment grant frames during save/restore

>From what I can tell, grant frames get no special treatment by the tools
during save, and hence get treated as normal memory during restore.
If that''s correct, it would seem that there''s an accounting
issue, since
during restore a normal RAM page will be allocated by the tools for each
grant frame, hence a domain that prior to save was below its limit
could now end up crossing that limit, and thus unexpectedly fail to
restore. If that''s not a possibility, what am I missing?

Thanks, Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 27.05.10 at 17:27, "Jan Beulich"
<JBeulich@novell.com> wrote:
> From what I can tell, grant frames get no special treatment by the tools
> during save, and hence get treated as normal memory during restore.
> If that''s correct, it would seem that there''s an
accounting issue, since
> during restore a normal RAM page will be allocated by the tools for each
> grant frame, hence a domain that prior to save was below its limit
> could now end up crossing that limit, and thus unexpectedly fail to
> restore. If that''s not a possibility, what am I missing?
Ooops, just noticed that I didn''t say explicitly that this is for HVM
guests.

Thanks, Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@novell.com]
> Sent: Thursday, May 27, 2010 9:38 AM
> To: xen-devel@lists.xensource.com
> Subject: Re: [Xen-devel] treatment grant frames during save/restore
> 
> >>> On 27.05.10 at 17:27, "Jan Beulich"
<JBeulich@novell.com> wrote:
> > From what I can tell, grant frames get no special treatment by the
> tools
> > during save, and hence get treated as normal memory during restore.
> > If that''s correct, it would seem that there''s an
accounting issue,
> since
> > during restore a normal RAM page will be allocated by the tools for
> each
> > grant frame, hence a domain that prior to save was below its limit
> > could now end up crossing that limit, and thus unexpectedly fail to
> > restore. If that''s not a possibility, what am I missing?
> 
> Ooops, just noticed that I didn''t say explicitly that this is for
HVM
> guests.
There was some discussion about this last September.  Here''s one link:

http://lists.xensource.com/archives/html/xen-devel/2009-09/msg00578.html 


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 27/05/2010 16:27, "Jan Beulich" <JBeulich@novell.com> wrote:
>> From what I can tell, grant frames get no special treatment by the
tools
> during save, and hence get treated as normal memory during restore.
> If that''s correct, it would seem that there''s an
accounting issue, since
> during restore a normal RAM page will be allocated by the tools for each
> grant frame, hence a domain that prior to save was below its limit
> could now end up crossing that limit, and thus unexpectedly fail to
> restore. If that''s not a possibility, what am I missing?
Yes, it''s an issue. Fixing is tricky since in some cases dom0 *wants*
to be
able to map domU special Xen-heap pages. So we need to be able to specify
some kind of flag to say ''really map this domain''s domain-heap
RAM pages
only on this request'' and preferably tunnel that flag through existing
dom0
kernels so that it makes it unscathed down to the Xen hypercall. That''s
a
bit tricky I think, unless we do nasty things like steal bits from the
existing domid or pte.val fields to mmu_update(). Else we need dom0 kernel
mods too, which is a pain in the bum, but I suppose we could do that with
fallback to what we do currently.

 -- Keir
> Thanks, Jan
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 27.05.10 at 18:02, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:
> Yes, it''s an issue. Fixing is tricky since in some cases dom0
*wants* to be
> able to map domU special Xen-heap pages. So we need to be able to specify
> some kind of flag to say ''really map this domain''s
domain-heap RAM pages
> only on this request'' and preferably tunnel that flag through
existing dom0
> kernels so that it makes it unscathed down to the Xen hypercall.
That''s a
> bit tricky I think, unless we do nasty things like steal bits from the
> existing domid or pte.val fields to mmu_update(). Else we need dom0 kernel
> mods too, which is a pain in the bum, but I suppose we could do that with
> fallback to what we do currently.
Wouldn''t is suffice to adjust XEN_DOMCTL_getpageframeinfo{2,3} to
report these pages as special (not sure whether a new type would
be needed, or whether simply returning XEN_DOMCTL_PFINFO_XTAB
would be acceptable), thus allowing the tools to skip them as
necessary? This of course implies that the tools would need to issue
the call for HVM guests (currently I think they do so only for PV ones),
that the actual page contents is irrelevant post-restore, and that
there being a hole in the physical address space post-restore isn''t
a problem for the pv drivers.

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 28/05/2010 09:01, "Jan Beulich" <JBeulich@novell.com> wrote:
> Wouldn''t is suffice to adjust XEN_DOMCTL_getpageframeinfo{2,3} to
> report these pages as special (not sure whether a new type would
> be needed, or whether simply returning XEN_DOMCTL_PFINFO_XTAB
> would be acceptable), thus allowing the tools to skip them as
> necessary? This of course implies that the tools would need to issue
> the call for HVM guests (currently I think they do so only for PV ones),
> that the actual page contents is irrelevant post-restore, and that
> there being a hole in the physical address space post-restore
isn''t
> a problem for the pv drivers.
Yep that would probably work. And also potentially gets rid of one lot of
"if-hvm-else-pv" branched code in xc_domain_save.c. I''d take
a patch to do
that if you want to pick this item up.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 28.05.10 at 10:22, Keir Fraser <keir.fraser@eu.citrix.com>
wrote:
> On 28/05/2010 09:01, "Jan Beulich" <JBeulich@novell.com>
wrote:
> 
>> Wouldn''t is suffice to adjust XEN_DOMCTL_getpageframeinfo{2,3}
to
>> report these pages as special (not sure whether a new type would
>> be needed, or whether simply returning XEN_DOMCTL_PFINFO_XTAB
>> would be acceptable), thus allowing the tools to skip them as
>> necessary? This of course implies that the tools would need to issue
>> the call for HVM guests (currently I think they do so only for PV
ones),
>> that the actual page contents is irrelevant post-restore, and that
>> there being a hole in the physical address space post-restore
isn''t
>> a problem for the pv drivers.
> 
> Yep that would probably work. And also potentially gets rid of one lot of
> "if-hvm-else-pv" branched code in xc_domain_save.c. I''d
take a patch to do
> that if you want to pick this item up.
I''m afraid it won''t: The live_p2m table gets created for pv
guests only,
but is needed as a prerequisite to calling XEN_DOMCTL_getpageframeinfo*
(which wants MFNs as input). Hence while the hypervisor side patch is
trivial, it doesn''t get us any closer to a solution to the problem at
hand.

Unless we (re-)define the meaning of the input array to this domctl to
specify gmfn-s rather than mfn-s (at least for the hvm case; for
auto-translate pv guests, quite obviously the save code wouldn''t
work anyway, but for those passing in gmfn-s would seem the
natural thing here).

Or unless we want to add code to libxc to create a live_p2m for all
guests (which I wouldn''t want to take on).

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 31/05/2010 15:58, "Jan Beulich" <JBeulich@novell.com> wrote:
>> Yep that would probably work. And also potentially gets rid of one lot
of
>> "if-hvm-else-pv" branched code in xc_domain_save.c.
I''d take a patch to do
>> that if you want to pick this item up.
> 
> I''m afraid it won''t: The live_p2m table gets created for
pv guests only,
> but is needed as a prerequisite to calling XEN_DOMCTL_getpageframeinfo*
> (which wants MFNs as input). Hence while the hypervisor side patch is
> trivial, it doesn''t get us any closer to a solution to the problem
at hand.
> 
> Unless we (re-)define the meaning of the input array to this domctl to
> specify gmfn-s rather than mfn-s (at least for the hvm case; for
> auto-translate pv guests, quite obviously the save code wouldn''t
> work anyway, but for those passing in gmfn-s would seem the
> natural thing here).
This is fine. We can (re-)define the domctl interface as we like, and anyway
I''m pretty sure noone uses that particular domctl on HVM guests (yet).

 -- Keir
> Or unless we want to add code to libxc to create a live_p2m for all
> guests (which I wouldn''t want to take on).


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel