Xen users - Mar 2011 - kernel-xen can not recognize more than 32 cores

Hi
I''ve installed xen hypervisor 3.0.3 on CentOS 5.5 from CentOS
repository.
The hardware I use is a Tyan server with 48 cores( 4*12-cores processors ).
The original CentOS kernel 2.6.18.194 can recognize 48 cores perfectly, but when
I switch to another kernel to use xen, the kernel recognize only 32 cores!
I tried to recompile the kernel by changing the maximum number of cpu in .config
file, but nothing changed!
Can anyone help me?
Regards



      

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

That''s not all that surprising as that version of xen is very very old
and afaik there were no systems like that around the time that was
current. You should benefit greatly from a newer version of xen on
that type of system

On Fri, Mar 4, 2011 at 8:13 AM, mohammad mohammadi
<m_mohammadi_dinani@yahoo.com> wrote:
> Hi
> I''ve installed xen hypervisor 3.0.3 on CentOS 5.5 from CentOS
repository.
> The hardware I use is a Tyan server with 48 cores( 4*12-cores processors ).
> The original CentOS kernel 2.6.18.194 can recognize 48 cores perfectly, but
> when I switch to another kernel to use xen, the kernel recognize only 32
> cores!
> I tried to recompile the kernel by changing the maximum number of cpu in
> .config file, but nothing changed!
> Can anyone help me?
> Regards
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi
>> I''ve installed xen hypervisor 3.0.3 on CentOS 5.5 from CentOS
repository.
>> The hardware I use is a Tyan server with 48 cores( 4*12-cores
processors ).
>> The original CentOS kernel 2.6.18.194 can recognize 48 cores perfectly,
but
>> when I switch to another kernel to use xen, the kernel recognize only
32
>> cores!
What do you mean by "another kernel" there - CentOS 5.5 supply a -xen 
kernel themselves, no?

And just to check, your "new" kernel is compiled for 64 bit, and not
32
bit - ''uname -i'' outputs x86_64 - as 32 bit kernel cannot use
more than 32
CPUs.

Last, I beleive the 3.0.3 version is a bit misleading, as CentOS follow 
RedHat and RedHat''s xen 3.0.3 will likely have a lot of patches applied
to
it and be some way on from a vanilla Xen 3.0.3.  Check the src.rpm for 
this.

Kevin


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

We saw the same thing with our Dell R815 servers.  I believe this limit
is not due to the kernel, but to the hypervisor.  Most 3.x releases of
Xen had harcoded NR_CPUS to 32.

 

Run "xm dmesg" and see how many cores the hypervisor recognizes.

 

From: xen-users-bounces@lists.xensource.com
[mailto:xen-users-bounces@lists.xensource.com] On Behalf Of mohammad
mohammadi
Sent: Friday, March 04, 2011 8:14 AM
To: xen-users@lists.xensource.com
Subject: [Xen-users] kernel-xen can not recognize more than 32 cores

 

Hi
I''ve installed xen hypervisor 3.0.3 on CentOS 5.5 from CentOS
repository. 
The hardware I use is a Tyan server with 48 cores( 4*12-cores processors
).
The original CentOS kernel 2.6.18.194 can recognize 48 cores perfectly,
but when I switch to another kernel to use xen, the kernel recognize
only 32 cores!
I tried to recompile the kernel by changing the maximum number of cpu in
.config file, but nothing changed!
Can anyone help me?
Regards

 



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

________________________________
From: Kevin Maguire <kmaguire@eso.org>
Cc: xen-users@lists.xensource.com
Sent: Fri, March 4, 2011 6:03:34 PM
Subject: Re: [Xen-users] kernel-xen can not recognize more than 32 cores

Hi
>> I''ve installed xen hypervisor 3.0.3 on CentOS 5.5 from CentOS
repository.
>> The hardware I use is a Tyan server with 48 cores( 4*12-cores
processors ).
>> The original CentOS kernel 2.6.18.194 can recognize 48 cores perfectly,
but
>> when I switch to another kernel to use xen, the kernel recognize only
32
>> cores!
What do you mean by "another kernel" there - CentOS 5.5 supply a -xen
kernel
themselves, no?

And just to check, your "new" kernel is compiled for 64 bit, and not
32 bit -
''uname -i'' outputs x86_64 - as 32 bit kernel cannot use more
than 32 CPUs.

Last, I beleive the 3.0.3 version is a bit misleading, as CentOS follow RedHat 
and RedHat''s xen 3.0.3 will likely have a lot of patches applied to it
and be
some way on from a vanilla Xen 3.0.3.  Check the src.rpm for this.

Kevin





By another kernel I mean kernel-xen from CentOS 5.5.
The kernel has been compiled for x86_64 architecture. So it is supposed to use 
more than 32 cores.
I run xm dmesg command and Surprisingly I see 48 cores. But catching 
/proc/cpuinfo and xm list gives me only 32 recognized cores.
What I need is just to make kernel and xen aware of all cores.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-user



      

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi mohammad
> By another kernel I mean kernel-xen from CentOS 5.5. The kernel has been 
> compiled for x86_64 architecture. So it is supposed to use more than 32 
> cores. I run xm dmesg command and Surprisingly I see 48 cores. But 
> catching /proc/cpuinfo and xm list gives me only 32 recognized cores. 
> What I need is just to make kernel and xen aware of all cores.
It is already aware of all the hardware, as "xm dmesg" shows you.

The kernel, dom0, is just a (special) domain.  Last time I checked Xen 
domains can support up to 32 cores.  You will not be able to see more 
within a single domain, but the Xen tools, and guests, can still use all 
your cores.

It can be argued that you should assign a much smaller number of cores to 
your dom0 anyway, via dom0-cpus in xend-config.sxp (or other ways).

My system has 48 cores, but just 2 assigned to dom0

# grep processor /proc/cpuinfo
processor       : 0
processor       : 1

# xm info | grep nr_cpus
nr_cpus                : 48

Similarly my dom0 can see only up to 32 GB of RAM, but the system actually 
has 256 GB, and anyway I limit dom0 to 10G via dom0_mem boot option.

YMMV,
Kevin

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users