Hi, What are the limits on how many CPUs and how much memory Xen supports? I am interested in this for both the host (0) kernel and the client (U) kernel. I am looking at getting some 16-way (8-way dual-core) Opteron systems with about 64 GB of RAM for prototyping, so I would like to make sure that Xen can use up all of the resources of such a machine. Best regards. Gordan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
[Resent to the xen-users list]> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Gordan Bobic > Sent: 20 September 2005 11:38 > To: xen-users@lists.xensource.com > Subject: [Xen-users] SMP and Memory Limits > > Hi, > > What are the limits on how many CPUs and how much memory Xen > supports? I am interested in this for both the host (0) > kernel and the client (U) kernel. > > I am looking at getting some 16-way (8-way dual-core) Opteron > systems with about 64 GB of RAM for prototyping, so I would > like to make sure that Xen can use up all of the resources of > such a machine.In Xen-unstable (to become 3.0) I believe there is no software limits for CPU count or memory amount, only whatever limits the hardware dictates (i.e. 40 bits of hardware address, 48 bits available in virtual space). If there are any other limitations, it''s probably fair to consider it a bug, and report such failings on the Xen-Devel mailing list. Obviously, 2.0.x, only supporting 32-bit in non-PAE mode would not be able to use more than about 3.5GB of RAM. Processor count I''m not sure of, but I would think that it would be capable of running on 16 processors... -- Mats> > Best regards. > > Gordan_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
>> What are the limits on how many CPUs and how much memory Xensupports? I am interested in this for both the host (0) kernel and the client (U) kernel. >> >> I am looking at getting some 16-way (8-way dual-core) Opteron systems with about 64 GB of RAM for prototyping, so I would like to make sure that Xen can use up all of the resources of such a machine. > > > > In Xen-unstable (to become 3.0) I believe there is no software limits > for CPU count or memory amount, only whatever limits the hardware > dictates (i.e. 40 bits of hardware address, 48 bits available in virtual > space). If there are any other limitations, it''s probably fair to > consider it a bug, and report such failings on the Xen-Devel mailing > list. > Obviously, 2.0.x, only supporting 32-bit in non-PAE mode would not be > able to use more than about 3.5GB of RAM. Is that 3.5 GB per dom-U/dom-0, or the total between dom-0 and all dom-Us put together? > Processor count I''m not sure > of, but I would think that it would be capable of running on 16 > processors... OK, thanks. Gordan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: Gordan Bobic [mailto:gordan@bobich.net] > Sent: 20 September 2005 12:34 > To: Petersson, Mats > Subject: Re: [Xen-users] SMP and Memory Limits > > >>What are the limits on how many CPUs and how much memory > Xen supports? > >>I am interested in this for both the host (0) kernel and the client > >>(U) kernel. > >> > >>I am looking at getting some 16-way (8-way dual-core) > Opteron systems > >>with about 64 GB of RAM for prototyping, so I would like to > make sure > >>that Xen can use up all of the resources of such a machine. > > > > > > In Xen-unstable (to become 3.0) I believe there is no > software limits > > for CPU count or memory amount, only whatever limits the hardware > > dictates (i.e. 40 bits of hardware address, 48 bits > available in virtual > > space). If there are any other limitations, it''s probably fair to > > consider it a bug, and report such failings on the Xen-Devel mailing > > list. > > > > Obviously, 2.0.x, only supporting 32-bit in non-PAE mode > would not be > > able to use more than about 3.5GB of RAM. > > Is that 3.5 GB per dom-U/dom-0, or the total between dom-0 and all > dom-Us put together?That would be 3.5 GB in total, since the only way to access more than this amount of memory would involve using address extended page table, which isn''t supported by Xen 2.0.x. 64-bit x86 actually uses the exisiting PAE, but with an added page-table level so that a bigger than 36-bit address can be supported.> > > Processor count I''m not sure > > of, but I would think that it would be capable of running on 16 > > processors... > > OK, thanks. > > Did you send that only to me and not the list deliberately, > or accidentally?It was meant to go to everyone, but for some reason I selected the wrong button when clicking reply... :-( -- Mats> > Gordan >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Petersson, Mats wrote:>>-----Original Message----- >>From: Gordan Bobic [mailto:gordan@bobich.net] >>Sent: 20 September 2005 12:34 >>To: Petersson, Mats >>Subject: Re: [Xen-users] SMP and Memory Limits >> >> >>>>What are the limits on how many CPUs and how much memory >> >>Xen supports? >> >>>>I am interested in this for both the host (0) kernel and the client >>>>(U) kernel. >>>> >>>>I am looking at getting some 16-way (8-way dual-core) >> >>Opteron systems >> >>>>with about 64 GB of RAM for prototyping, so I would like to >> >>make sure >> >>>>that Xen can use up all of the resources of such a machine. >>> >>> >>>In Xen-unstable (to become 3.0) I believe there is no >> >>software limits >> >>>for CPU count or memory amount, only whatever limits the hardware >>>dictates (i.e. 40 bits of hardware address, 48 bits >> >>available in virtual >> >>>space). If there are any other limitations, it''s probably fair to >>>consider it a bug, and report such failings on the Xen-Devel mailing >>>list. >>> >>>Obviously, 2.0.x, only supporting 32-bit in non-PAE mode >> >>would not be >> >>>able to use more than about 3.5GB of RAM. >> >>Is that 3.5 GB per dom-U/dom-0, or the total between dom-0 and all >>dom-Us put together? > > > That would be 3.5 GB in total, since the only way to access more than > this amount of memory would involve using address extended page table, > which isn''t supported by Xen 2.0.x. 64-bit x86 actually uses the > exisiting PAE, but with an added page-table level so that a bigger than > 36-bit address can be supported.How stable is the "unstable/64-bit" version of Xen? Is it usable? Gordan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Gordan Bobic > Sent: 20 September 2005 14:27 > To: xen-users@lists.xensource.com > Subject: Re: [Xen-users] SMP and Memory Limits > > Petersson, Mats wrote: > >>-----Original Message----- > >>From: Gordan Bobic [mailto:gordan@bobich.net] > >>Sent: 20 September 2005 12:34 > >>To: Petersson, Mats > >>Subject: Re: [Xen-users] SMP and Memory Limits > >> > >> > >>>>What are the limits on how many CPUs and how much memory > >> > >>Xen supports? > >> > >>>>I am interested in this for both the host (0) kernel and > the client > >>>>(U) kernel. > >>>> > >>>>I am looking at getting some 16-way (8-way dual-core) > >> > >>Opteron systems > >> > >>>>with about 64 GB of RAM for prototyping, so I would like to > >> > >>make sure > >> > >>>>that Xen can use up all of the resources of such a machine. > >>> > >>> > >>>In Xen-unstable (to become 3.0) I believe there is no > >> > >>software limits > >> > >>>for CPU count or memory amount, only whatever limits the hardware > >>>dictates (i.e. 40 bits of hardware address, 48 bits > >> > >>available in virtual > >> > >>>space). If there are any other limitations, it''s probably fair to > >>>consider it a bug, and report such failings on the > Xen-Devel mailing > >>>list. > >>> > >>>Obviously, 2.0.x, only supporting 32-bit in non-PAE mode > >> > >>would not be > >> > >>>able to use more than about 3.5GB of RAM. > >> > >>Is that 3.5 GB per dom-U/dom-0, or the total between dom-0 and all > >>dom-Us put together? > > > > > > That would be 3.5 GB in total, since the only way to access > more than > > this amount of memory would involve using address extended > page table, > > which isn''t supported by Xen 2.0.x. 64-bit x86 actually uses the > > exisiting PAE, but with an added page-table level so that a bigger > > than 36-bit address can be supported. > > How stable is the "unstable/64-bit" version of Xen? Is it usable?I should think that it''s usable as long as you don''t expect your system to be a "production system with 100% 24/7 availability" [or somewhere where you''d have 100 angry users to answer to if the system goes down for more than half a minute]. Unstable should really be renamed into "testing" by now, as that''s really what it is, but someone in XenSource decided that a rename of the directory was too much work... Or some such... The biggest problem would probably be obscure hardware or things that are rarely used in Xen, which I''m sure that there are some outstanding bugs and new ones to crop up in the near future. -- Mats> > Gordan > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Gordan Bobic > Sent: 20 September 2005 15:05 > To: xen-users@lists.xensource.com > Subject: Re: [Xen-users] SMP and Memory Limits > > Petersson, Mats wrote: > >>>>>>What are the limits on how many CPUs and how much memory > >>>> > >>>>Xen supports? > >>>> > >>>> > >>>>>>I am interested in this for both the host (0) kernel and > >> > >>the client > >> > >>>>>>(U) kernel. > >>>>>> > >>>>>>I am looking at getting some 16-way (8-way dual-core) > >>>> > >>>>Opteron systems > >>>> > >>>> > >>>>>>with about 64 GB of RAM for prototyping, so I would like to > >>>> > >>>>make sure > >>>> > >>>> > >>>>>>that Xen can use up all of the resources of such a machine. > >>>>> > >>>>> > >>>>>In Xen-unstable (to become 3.0) I believe there is no > >>>> > >>>>software limits > >>>> > >>>> > >>>>>for CPU count or memory amount, only whatever limits the > hardware > >>>>>dictates (i.e. 40 bits of hardware address, 48 bits > >>>> > >>>>available in virtual > >>>> > >>>> > >>>>>space). If there are any other limitations, it''s > probably fair to > >>>>>consider it a bug, and report such failings on the > >> > >>Xen-Devel mailing > >> > >>>>>list. > >>>>> > >>>>>Obviously, 2.0.x, only supporting 32-bit in non-PAE mode > >>>> > >>>>would not be > >>>> > >>>> > >>>>>able to use more than about 3.5GB of RAM. > >>>> > >>>>Is that 3.5 GB per dom-U/dom-0, or the total between > dom-0 and all > >>>>dom-Us put together? > >>> > >>> > >>>That would be 3.5 GB in total, since the only way to access > >> > >>more than > >> > >>>this amount of memory would involve using address extended > >> > >>page table, > >> > >>>which isn''t supported by Xen 2.0.x. 64-bit x86 actually uses the > >>>exisiting PAE, but with an added page-table level so that a bigger > >>>than 36-bit address can be supported. > >> > >>How stable is the "unstable/64-bit" version of Xen? Is it usable? > > > > > > I should think that it''s usable as long as you don''t expect your > > system to be a "production system with 100% 24/7 availability" [or > > somewhere where you''d have 100 angry users to answer to if > the system > > goes down for more than half a minute]. Unstable should really be > > renamed into "testing" by now, as that''s really what it is, but > > someone in XenSource decided that a rename of the directory was too > > much work... Or some such... The biggest problem would probably be > > obscure hardware or things that are rarely used in Xen, > which I''m sure > > that there are some outstanding bugs and new ones to crop > up in the near future. > > So, you are reasonably confident that when applied to a > 64GB/16-way machine it isn''t going to fall over flat on it''s face? :-)Reasonably so. But I wouldn''t bet any more than the smaller denominations of change that I''ve currently got in my pocket on it... ;-) -- Mats> > Gordan > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Petersson, Mats wrote:>>>>>>What are the limits on how many CPUs and how much memory >>>> >>>>Xen supports? >>>> >>>> >>>>>>I am interested in this for both the host (0) kernel and >> >>the client >> >>>>>>(U) kernel. >>>>>> >>>>>>I am looking at getting some 16-way (8-way dual-core) >>>> >>>>Opteron systems >>>> >>>> >>>>>>with about 64 GB of RAM for prototyping, so I would like to >>>> >>>>make sure >>>> >>>> >>>>>>that Xen can use up all of the resources of such a machine. >>>>> >>>>> >>>>>In Xen-unstable (to become 3.0) I believe there is no >>>> >>>>software limits >>>> >>>> >>>>>for CPU count or memory amount, only whatever limits the hardware >>>>>dictates (i.e. 40 bits of hardware address, 48 bits >>>> >>>>available in virtual >>>> >>>> >>>>>space). If there are any other limitations, it''s probably fair to >>>>>consider it a bug, and report such failings on the >> >>Xen-Devel mailing >> >>>>>list. >>>>> >>>>>Obviously, 2.0.x, only supporting 32-bit in non-PAE mode >>>> >>>>would not be >>>> >>>> >>>>>able to use more than about 3.5GB of RAM. >>>> >>>>Is that 3.5 GB per dom-U/dom-0, or the total between dom-0 and all >>>>dom-Us put together? >>> >>> >>>That would be 3.5 GB in total, since the only way to access >> >>more than >> >>>this amount of memory would involve using address extended >> >>page table, >> >>>which isn''t supported by Xen 2.0.x. 64-bit x86 actually uses the >>>exisiting PAE, but with an added page-table level so that a bigger >>>than 36-bit address can be supported. >> >>How stable is the "unstable/64-bit" version of Xen? Is it usable? > > > I should think that it''s usable as long as you don''t expect your system > to be a "production system with 100% 24/7 availability" [or somewhere > where you''d have 100 angry users to answer to if the system goes down > for more than half a minute]. Unstable should really be renamed into > "testing" by now, as that''s really what it is, but someone in XenSource > decided that a rename of the directory was too much work... Or some > such... The biggest problem would probably be obscure hardware or things > that are rarely used in Xen, which I''m sure that there are some > outstanding bugs and new ones to crop up in the near future.So, you are reasonably confident that when applied to a 64GB/16-way machine it isn''t going to fall over flat on it''s face? :-) Gordan _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
> > So, you are reasonably confident that when applied to a > > 64GB/16-way machine it isn''t going to fall over flat on it''s face? :-) > > Reasonably so. But I wouldn''t bet any more than the smaller > denominations of change that I''ve currently got in my pocket on it... > ;-)I don''t think *many* people are running on machines this big, but I think a few have done so (even on bigger machines). In Cambridge, we have a 8 core Opteron, I think - the Unisys guys run on much bigger CPU counts. The x86_64 is certainly passing the daily tests reliably now, so it''s more a question of whether your specific machine tickles any unexpected cases. Of course, the pre-release testing phase is a good point to scream about bugs like this ;-) Cheers, Mark _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
I have the current unstable tree running on HP DL585 (4-Socket 848) 32GB of system memory. I have nine xen-U OS(s) running with a mix of memory between 1GB and 10GB. I haven''t stressed xen-U domains too much yet, but the large memory seems to be working well. -----Original Message----- From: xen-users-bounces@lists.xensource.com [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of Petersson, Mats Sent: Tuesday, September 20, 2005 7:05 AM To: Gordan Bobic; xen-users@lists.xensource.com Subject: RE: [Xen-users] SMP and Memory Limits> -----Original Message----- > From: xen-users-bounces@lists.xensource.com > [mailto:xen-users-bounces@lists.xensource.com] On Behalf Of > Gordan Bobic > Sent: 20 September 2005 15:05 > To: xen-users@lists.xensource.com > Subject: Re: [Xen-users] SMP and Memory Limits > > Petersson, Mats wrote: > >>>>>>What are the limits on how many CPUs and how much memory > >>>> > >>>>Xen supports? > >>>> > >>>> > >>>>>>I am interested in this for both the host (0) kernel and > >> > >>the client > >> > >>>>>>(U) kernel. > >>>>>> > >>>>>>I am looking at getting some 16-way (8-way dual-core) > >>>> > >>>>Opteron systems > >>>> > >>>> > >>>>>>with about 64 GB of RAM for prototyping, so I would like to > >>>> > >>>>make sure > >>>> > >>>> > >>>>>>that Xen can use up all of the resources of such a machine. > >>>>> > >>>>> > >>>>>In Xen-unstable (to become 3.0) I believe there is no > >>>> > >>>>software limits > >>>> > >>>> > >>>>>for CPU count or memory amount, only whatever limits the > hardware > >>>>>dictates (i.e. 40 bits of hardware address, 48 bits > >>>> > >>>>available in virtual > >>>> > >>>> > >>>>>space). If there are any other limitations, it''s > probably fair to > >>>>>consider it a bug, and report such failings on the > >> > >>Xen-Devel mailing > >> > >>>>>list. > >>>>> > >>>>>Obviously, 2.0.x, only supporting 32-bit in non-PAE mode > >>>> > >>>>would not be > >>>> > >>>> > >>>>>able to use more than about 3.5GB of RAM. > >>>> > >>>>Is that 3.5 GB per dom-U/dom-0, or the total between > dom-0 and all > >>>>dom-Us put together? > >>> > >>> > >>>That would be 3.5 GB in total, since the only way to access > >> > >>more than > >> > >>>this amount of memory would involve using address extended > >> > >>page table, > >> > >>>which isn''t supported by Xen 2.0.x. 64-bit x86 actually uses the > >>>exisiting PAE, but with an added page-table level so that a bigger > >>>than 36-bit address can be supported. > >> > >>How stable is the "unstable/64-bit" version of Xen? Is it usable? > > > > > > I should think that it''s usable as long as you don''t expect your > > system to be a "production system with 100% 24/7 availability" [or > > somewhere where you''d have 100 angry users to answer to if > the system > > goes down for more than half a minute]. Unstable should really be > > renamed into "testing" by now, as that''s really what it is, but > > someone in XenSource decided that a rename of the directory was too > > much work... Or some such... The biggest problem would probably be > > obscure hardware or things that are rarely used in Xen, > which I''m sure > > that there are some outstanding bugs and new ones to crop > up in the near future. > > So, you are reasonably confident that when applied to a > 64GB/16-way machine it isn''t going to fall over flat on it''s face? :-)Reasonably so. But I wouldn''t bet any more than the smaller denominations of change that I''ve currently got in my pocket on it... ;-) -- Mats> > Gordan > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users