Xen devel - Jan 2009 - big local array in routine in hypervisor

Is it always safe to have a large local array in a routine
in the hypervisor, say 2-3 pages (8k-12k) in size?  (I suppose
that may be 32k-48k on ia64 since the code is arch-neutral.)
Or is there some possibility the Xen stack will overflow?

If not always safe, is there any way to test to see if it safe
or how large is safe?

Thanks,
Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

It''s definitely not safe. Don''t do it! The stack is only a few
kilobytes and
needs to be shared with possibly multiple nested interrupt contexts.

 -- Keir

On 26/01/2009 22:41, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
> Is it always safe to have a large local array in a routine
> in the hypervisor, say 2-3 pages (8k-12k) in size?  (I suppose
> that may be 32k-48k on ia64 since the code is arch-neutral.)
> Or is there some possibility the Xen stack will overflow?
> 
> If not always safe, is there any way to test to see if it safe
> or how large is safe?
> 
> Thanks,
> Dan
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Rats!  I need a large contiguous "workspace" (for compression)
but can''t assume that the heap isn''t fragmented.  I can
pre-reserve
it but that will cause scaleability problems later, because
it would be too wasteful to pre-reserve one for each processor.
I guess I''ll pre-reserve and protect it with a lock for now
and worry about contention later.

Is there one stack per physical processor?  Or is there some
other reason (other than historical) for the stack to be so
limited in size?
> -----Original Message-----
> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com]
> Sent: Monday, January 26, 2009 4:24 PM
> To: Dan Magenheimer; Xen-Devel (E-mail)
> Subject: Re: [Xen-devel] big local array in routine in hypervisor
> 
> 
> It''s definitely not safe. Don''t do it! The stack is only
a
> few kilobytes and
> needs to be shared with possibly multiple nested interrupt contexts.
> 
>  -- Keir
> 
> On 26/01/2009 22:41, "Dan Magenheimer" 
> <dan.magenheimer@oracle.com> wrote:
> 
> > Is it always safe to have a large local array in a routine
> > in the hypervisor, say 2-3 pages (8k-12k) in size?  (I suppose
> > that may be 32k-48k on ia64 since the code is arch-neutral.)
> > Or is there some possibility the Xen stack will overflow?
> > 
> > If not always safe, is there any way to test to see if it safe
> > or how large is safe?
> > 
> > Thanks,
> > Dan
> > 
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xensource.com
> > http://lists.xensource.com/xen-devel
> 
> 
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 27/01/2009 00:43, "Dan Magenheimer"
<dan.magenheimer@oracle.com> wrote:
> Rats!  I need a large contiguous "workspace" (for compression)
> but can''t assume that the heap isn''t fragmented.  I can
pre-reserve
> it but that will cause scaleability problems later, because
> it would be too wasteful to pre-reserve one for each processor.
> I guess I''ll pre-reserve and protect it with a lock for now
> and worry about contention later.
Does it *really* have to be multi-page contiguous? You couldn''t for
example
build your workspace out of a radix tree with page-sized leaves?

Or are you saying that maybe you can''t allocate even a single page, or
a
sub-page? In that case it would seem memory is so tight you have little
choice but to pre-reserve.
> Is there one stack per physical processor?  Or is there some
> other reason (other than historical) for the stack to be so
> limited in size?
So you want us to wastefully pre-reserve some space for you, but call it the
''stack'' to assuage your guilt? ;-) It''s common
practice not to have very
large stacks in kernels, since pre-reservation is wateful, dynamic growth is
not necessarily feasible to implement, and kernel code doesn''t tend to
need
lots of local storage or recursion. In Linux you''d be limited to 4kB,
and
there''s a lot more code there living under that stricter regime.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 27/01/2009 08:19, "Keir Fraser" <keir.fraser@eu.citrix.com>
wrote:
>> Is there one stack per physical processor?  Or is there some
>> other reason (other than historical) for the stack to be so
>> limited in size?
> 
> So you want us to wastefully pre-reserve some space for you, but call it
the
> ''stack'' to assuage your guilt? ;-) It''s common
practice not to have very large
> stacks in kernels, since pre-reservation is wateful, dynamic growth is not
> necessarily feasible to implement, and kernel code doesn''t tend to
need lots
> of local storage or recursion. In Linux you''d be limited to 4kB,
and there''s a
> lot more code there living under that stricter regime.
I noticed that the p2m populate-on-demand code also allocates a lot (10kB)
of stack (in fact this is a bug since the stack is only 8kB!). If these new
stack users aren''t easy to implement in other ways, and are definitely
not
reentrant nor execute in interrupt context (so we know there''s only one
such
big allocation at a time) we could perhaps double the primary stack size to
16kB, or even to 32kB.

It''s a slippery slope though, determining how much stack is enough and
how
big a local array is too big. I generally end up having to check and fix big
stack frames from time to time, and I''m not sure that even doubling the
stack a few times would avoid that job!

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 27/01/2009 08:58, "Keir Fraser" <keir.fraser@eu.citrix.com>
wrote:
>> So you want us to wastefully pre-reserve some space for you, but call
it the
>> ''stack'' to assuage your guilt? ;-) It''s
common practice not to have very
>> large 
>> stacks in kernels, since pre-reservation is wateful, dynamic growth is
not
>> necessarily feasible to implement, and kernel code doesn''t
tend to need lots
>> of local storage or recursion. In Linux you''d be limited to
4kB, and there''s
>> a 
>> lot more code there living under that stricter regime.
> 
> I noticed that the p2m populate-on-demand code also allocates a lot (10kB)
of
> stack (in fact this is a bug since the stack is only 8kB!). If these new
stack
> users aren''t easy to implement in other ways, and are definitely
not reentrant
> nor execute in interrupt context (so we know there''s only one such
big
> allocation at a time) we could perhaps double the primary stack size to
16kB,
> or even to 32kB.
> 
> It''s a slippery slope though, determining how much stack is enough
and how big
> a local array is too big. I generally end up having to check and fix big
stack
> frames from time to time, and I''m not sure that even doubling the
stack a few
> times would avoid that job!
George,

In the PoD case I think only p2m_pod_zero_check_superpage() needs to be
changed. I''m not actually clear why a separate sweep needs to be done
for
guest superpage mappings? Couldn''t they get handled by
p2m_pod_zero_check()?
Can you not PoD-reclaim a subset of MFNs backing a guest super-GFN?

In any case it could map the individual pages one after the other and check
them. That would reduce pressure on map_domain_page() too (currently it
could probably fail and crash the system on x86_32). And then the 512-entry
arrays would not be needed.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> Does it *really* have to be multi-page contiguous? You 
> couldn''t for example
> build your workspace out of a radix tree with page-sized leaves?
> Or are you saying that maybe you can''t allocate even a single 
> page, or a
> sub-page? In that case it would seem memory is so tight you 
> have little
> choice but to pre-reserve.
I''m trying to use the lzo compression code now in Linux
(as of 2.6.27 I think) in linux/lib/lzo.  It uses a large
static work area... which now that I look more carefully
is actually 128kB!...  apparently ffs2 (its sole user in
Linux as of now) doesn''t care about SMP?
> So you want us to wastefully pre-reserve some space for you, 
> but call it the
> ''stack'' to assuage your guilt? ;-)
Point taken ;-)  But the stack can be used by others and
my pre-reserved space could not.
> It''s common practice not to have very
> large stacks in kernels, since pre-reservation is wateful, 
> dynamic growth is not necessarily feasible to implement,
> and kernel code doesn''t tend to need lots of local storage
> or recursion. In Linux you''d be limited to 4kB, and
> there''s a lot more code there living under that stricter regime.
True, but the history of most kernels goes back a bit farther
than Xen''s, back when memory density and cost was a factor
of 1000x different.  So... 4kb * 1000 is 4M ;-)
> It''s a slippery slope though, determining how much stack is 
> enough and how
> big a local array is too big. I generally end up having to 
> check and fix big
> stack frames from time to time, and I''m not sure that even 
> doubling the
> stack a few times would avoid that job!
Agreed.  It will never be enough.

I''ll live with the private static array for now.

Dan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel