Xen users - Dec 2006 - Could not enable shadow paging for domain?

I asked about this over in the fedora-xen list:

   https://www.redhat.com/archives/fedora-xen/2006-December/msg00089.html

Since I''ve gotten no responses there, I thought I''d try
over here:

Playing with Xen, Fedora Core 6, and 8-way Opteron system at work
with 8 gig of memory:

Xen seems to stop being able to create domains after it has been
up for a while (doing things like testing lots of guest installs
or even just using dom0 for non-xen related testing for a while).

Any attempt at this point to create a new domain winds up
with error (12, ''Cannot allocate memory''), and the
xend-debug.log shows ERROR: Could not enable shadow paging for domain.

Rebooting the machine (the real one :-) fixes the problem until
it has been up for a while again...

This sound familiar to anyone? Should I file a bug? (Against what?)

I hasten to point out that I''m not creating lots of domains and
leaving them running, so I shouldn''t actually be running out of
any memory resource (assuming resources are reclaimed properly
when a domain is destroyed).

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com 
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of 
> Tom Horsley
> Sent: 17 December 2006 19:45
> To: xen-users@lists.xensource.com
> Subject: [Xen-users] Could not enable shadow paging for domain?
> 
> I asked about this over in the fedora-xen list:
> 
>    
> https://www.redhat.com/archives/fedora-xen/2006-December/msg00089.html
> 
> Since I''ve gotten no responses there, I thought I''d try
> over here:
> 
> Playing with Xen, Fedora Core 6, and 8-way Opteron system at work
> with 8 gig of memory:
> 
> Xen seems to stop being able to create domains after it has been
> up for a while (doing things like testing lots of guest installs
> or even just using dom0 for non-xen related testing for a while).
> 
> Any attempt at this point to create a new domain winds up
> with error (12, ''Cannot allocate memory''), and the
> xend-debug.log shows ERROR: Could not enable shadow paging for domain.
The "Count not able shadow paging" is probably related to the previous
"cannot not allocate" error. If you can''t allocate the
memory, then you
can''t enable shadow paging on the memory as it''s not been
allocated...
:-(

How much memory do you give Dom0? I''d say giving Dom0 256MB would be a
good idea, rather than rely on the auto-baloon feature that is indeed a
bit more tricky to make work. 

--
Mats
> 
> Rebooting the machine (the real one :-) fixes the problem until
> it has been up for a while again...
> 
> This sound familiar to anyone? Should I file a bug? (Against what?)
> 
> I hasten to point out that I''m not creating lots of domains and
> leaving them running, so I shouldn''t actually be running out of
> any memory resource (assuming resources are reclaimed properly
> when a domain is destroyed).
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> The "Count not able shadow paging" is probably related to the
previous
> "cannot not allocate" error. If you can''t allocate the
memory, then you
> can''t enable shadow paging on the memory as it''s not been
allocated...
> :-(
I don''t think that''s a "previous" error, I think the
cannot allocate is merely
what gets reported outside the log and the shadow paging error is the
more detailed info that appears in the log for the same error.
> How much memory do you give Dom0? I''d say giving Dom0 256MB would
be a
> good idea, rather than rely on the auto-baloon feature that is indeed a
> bit more tricky to make work. 
I haven''t yet stumbled across any documentation that tells me how to
limit dom0
to a subset of the memory. What do I configure where to do that?


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> -----Original Message-----
> From: xen-users-bounces@lists.xensource.com 
> [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of 
> Tom Horsley
> Sent: 18 December 2006 13:50
> To: xen-users@lists.xensource.com
> Subject: RE: [Xen-users] Could not enable shadow paging for domain?
> 
> > The "Count not able shadow paging" is probably related to 
> the previous
> > "cannot not allocate" error. If you can''t allocate
the
> memory, then you
> > can''t enable shadow paging on the memory as it''s not
been
> allocated...
> > :-(
> 
> I don''t think that''s a "previous" error, I
think the cannot
> allocate is merely
> what gets reported outside the log and the shadow paging error is the
> more detailed info that appears in the log for the same error.
Well, to be honest, I think it''s a bit of both - the REAL error is
"Can''t allocate memory" (or something like that: I looked at
.../xen/arch/x86/mm/shadow/common.c: shadow_enable() [which is called
indirectly from xc_hvm_build_internal() that gives the "Could not enable
shadow..." message], and the shadow_enable() has several memory
allocations that can fail, resulting in ENOMEM... So the failure to
allocate memory is the "first" error, but the consequence is that
it''s
failing to enable shadow paging and in turn fails to create the domain.
It may be that there''s more to it than that, but without digging
further
down the tree of functions, I''m not able to say - and it would require
having some more information on where it fails to help here - adding
some printk''s to the shadow_enable function would be the first part,
then further down to get a precise spot (or adding BUG_ON() statement to
"crash" the system and get a stack-dump to see where it came from in
some allocation routine, but that could be dangerous as some apps try to
allocate a large chunk then smaller if it fails). 

But I would start with my original suggestion of locking down Dom0
memory and not relying on "ballooning" - it''s been known to
fail before.
See below for more details on how to do that. 

If it still fails, please provide:
1. "xm info" output
2. "xm list" output
> 
> > How much memory do you give Dom0? I''d say giving Dom0 256MB 
> would be a
> > good idea, rather than rely on the auto-baloon feature that 
> is indeed a
> > bit more tricky to make work. 
> 
> I haven''t yet stumbled across any documentation that tells me 
> how to limit dom0
> to a subset of the memory. What do I configure where to do that?
On the Xen-line in grub.conf (something like "kernel .../xen.gz ... ")
you add "dom0_mem=size". Size is by default in kilobytes, but you can
add "M" or "G" to make it megabytes or kilobytes (not sure
if "b" makes
it bytes, but I''m sure M is what you want anyways). 

So dom0_mem=256M should work well (that''s what I use!). 

--
Mats
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
> 
> 
> 


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Mon, 18 Dec 2006 15:19:52 +0100
"Petersson, Mats" <Mats.Petersson@amd.com>
wrote:
> On the Xen-line in grub.conf (something like "kernel .../xen.gz ...
")
> you add "dom0_mem=size". Size is by default in kilobytes, but you
can
> add "M" or "G" to make it megabytes or kilobytes (not
sure if "b" makes
> it bytes, but I''m sure M is what you want anyways). 
> 
> So dom0_mem=256M should work well (that''s what I use!). 
That did indeed seem to solver the problem. The system was up all day
with that today, and did lots of xm create/destroy kind of activity
and never once got the memory errors I was seeing before. I may yet
learn enough to get this machine in production come January :-).

Thanks for the new bit of info to add to my list.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users