Xen devel - Feb 2012 - "Deprecate" order>0 allocations? (was: [PATCH 1 of 2] Global virq for low memory situations)

> -----Original Message-----
> From: Jan Beulich [mailto:JBeulich@suse.com]
> Sent: Wednesday, February 29, 2012 1:54 AM
> To: Dan Magenheimer
> Cc: ian.campbell@citrix.com; ian.jackson@citrix.com; adin@gridcentric.ca;
andres@gridcentric.ca;
> Andres Lagar-Cavilla; xen-devel; tim@xen.org
> Subject: RE: [Xen-devel] [PATCH 1 of 2] Global virq for low memory
situations
> 
> >>> On 29.02.12 at 00:50, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
> > Or maybe Jan''s recent work has eliminated all but the corner
> > cases of order>0 allocations?  (/me crosses fingers)
> 
> I''m unaware of remaining violators of this, but I''m also
not constantly
> monitoring.
> 
> Jan
As various "move memory back and forth between domains"
solutions are becoming increasingly fashionable, use
of order>0 allocations remains a ticking timebomb.

Does it make sense then now to somehow "deprecate" the
code in the allocator that allows order>0 allocations?
Not quite sure how, so just brainstorming.  Some ideas:

1) At minimum, a very big comment warning about the
   silent pitfalls due to fragmentation and a WARN_ON_ONCE
   that occurs when order>1 and debug=1 (and for xen-unstable).
2) Add a new routine for order>0 allocations (with
   a WARN_ON and a big comment) and BUG_ON order>0
   allocations attempted via the old interface.

IIRC, last I heard, some of the PCI-passthrough code
still uses order>0 allocations, so this (and any
other remaining culprits) might need to be grandfathered
in somehow.

But it would be nice to guarantee that new code isn''t
allowed to make matters worse.

>>> On 29.02.12 at 17:20, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
>>  -----Original Message-----
>> From: Jan Beulich [mailto:JBeulich@suse.com]
>> Sent: Wednesday, February 29, 2012 1:54 AM
>> To: Dan Magenheimer
>> Cc: ian.campbell@citrix.com; ian.jackson@citrix.com;
adin@gridcentric.ca;
> andres@gridcentric.ca;
>> Andres Lagar-Cavilla; xen-devel; tim@xen.org 
>> Subject: RE: [Xen-devel] [PATCH 1 of 2] Global virq for low memory
situations
>> 
>> >>> On 29.02.12 at 00:50, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
>> > Or maybe Jan''s recent work has eliminated all but the
corner
>> > cases of order>0 allocations?  (/me crosses fingers)
>> 
>> I''m unaware of remaining violators of this, but I''m
also not constantly
>> monitoring.
>> 
>> Jan
> 
> As various "move memory back and forth between domains"
> solutions are becoming increasingly fashionable, use
> of order>0 allocations remains a ticking timebomb.
> 
> Does it make sense then now to somehow "deprecate" the
> code in the allocator that allows order>0 allocations?
> Not quite sure how, so just brainstorming.  Some ideas:
> 
> 1) At minimum, a very big comment warning about the
>    silent pitfalls due to fragmentation and a WARN_ON_ONCE
>    that occurs when order>1 and debug=1 (and for xen-unstable).
> 2) Add a new routine for order>0 allocations (with
>    a WARN_ON and a big comment) and BUG_ON order>0
>    allocations attempted via the old interface.
No, that doesn''t sound feasible. When creating HVM guests, we
will want to be able to attempt large page allocations without
getting warned about, as the caller can handle the failure. That
might also be the case in other situations.
> IIRC, last I heard, some of the PCI-passthrough code
> still uses order>0 allocations, so this (and any
> other remaining culprits) might need to be grandfathered
> in somehow.
When was this "last I heard"? I thought I caught them all (largely
by storing per-interrupt information in a radix tree of per-interrupt
data structures rather than in nr_pirqs or nr_irqs sized arrays).

Jan

> From: Jan Beulich [mailto:JBeulich@suse.com]
> Subject: Re: "Deprecate" order>0 allocations? (was: [PATCH 1
of 2] Global virq for low memory
> situations)
> 
> > As various "move memory back and forth between domains"
> > solutions are becoming increasingly fashionable, use
> > of order>0 allocations remains a ticking timebomb.
> >
> > Does it make sense then now to somehow "deprecate" the
> > code in the allocator that allows order>0 allocations?
> > Not quite sure how, so just brainstorming.  Some ideas:
> >
> > 1) At minimum, a very big comment warning about the
> >    silent pitfalls due to fragmentation and a WARN_ON_ONCE
> >    that occurs when order>1 and debug=1 (and for xen-unstable).
> > 2) Add a new routine for order>0 allocations (with
> >    a WARN_ON and a big comment) and BUG_ON order>0
> >    allocations attempted via the old interface.
> 
> No, that doesn''t sound feasible. When creating HVM guests, we
> will want to be able to attempt large page allocations without
> getting warned about, as the caller can handle the failure. That
> might also be the case in other situations.
Oh, right, no bug/warning for order==9 or 18.  I''m mostly interested
in ensuring order==1,2,3 etc don''t sneak back in.  But you''re
right that those should be legal as well as long as the caller
can gracefully handle the failure.  Any thoughts on how to
enforce this?  Maybe a wrapper, or a function name like:

alloc_page_block_with_possible_failure()

with a lengthy comment?
 
> > IIRC, last I heard, some of the PCI-passthrough code
> > still uses order>0 allocations, so this (and any
> > other remaining culprits) might need to be grandfathered
> > in somehow.
> 
> When was this "last I heard"? I thought I caught them all
(largely
> by storing per-interrupt information in a radix tree of per-interrupt
> data structures rather than in nr_pirqs or nr_irqs sized arrays).
Excellent!  I think it was last summer.  (Sorry, I didn''t
follow xen-devel closely for a few months.)  When did
the last of your patches go in?
 
Dan

>>> On 01.03.12 at 15:58, Dan Magenheimer
<dan.magenheimer@oracle.com> wrote:
>>  From: Jan Beulich [mailto:JBeulich@suse.com]
>> > IIRC, last I heard, some of the PCI-passthrough code
>> > still uses order>0 allocations, so this (and any
>> > other remaining culprits) might need to be grandfathered
>> > in somehow.
>> 
>> When was this "last I heard"? I thought I caught them all
(largely
>> by storing per-interrupt information in a radix tree of per-interrupt
>> data structures rather than in nr_pirqs or nr_irqs sized arrays).
> 
> Excellent!  I think it was last summer.  (Sorry, I didn''t
> follow xen-devel closely for a few months.)  When did
> the last of your patches go in?
End of June/beginning of July last year.

Jan