Hi,
I''m testing 2.6.27-rc1 and seem to have an issue when booting the domU
with >4096MB.
Xen: 3.2.1 x86_64 2.6.18.8 xen.org kernel
domU: 2.6.27-rc1 32bit PAE
# xm list test1
Name ID Mem VCPUs State Time(s)
test1 24 6000 2 -b---- 5.6
# xm console test1
free -m
total used free shared buffers cached
Mem: 1904 75 1828 0 19 15
-/+ buffers/cache: 40 1864
Swap: 2047 0 2047
testing:~# uname -a
Linux testing 2.6.27-rc1-xenU #2 SMP Thu Jul 31 22:57:42 EDT 2008 i686 GNU/Linux
(Minimal) Config: http://radium.outervoid.net/~carl/2.6.27-rc1
I see the same when using .26. Am I perhaps missing an option somewhere?
Regards,
Carl
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Carl Jones wrote:> Hi, > > I''m testing 2.6.27-rc1 and seem to have an issue when booting the domU > with >4096MB. > > Xen: 3.2.1 x86_64 2.6.18.8 xen.org kernel > domU: 2.6.27-rc1 32bit PAE > > # xm list test1 > Name ID Mem VCPUs State Time(s) > test1 24 6000 2 -b---- 5.6 > # xm console test1 > free -m > total used free shared buffers cached > Mem: 1904 75 1828 0 19 15 > -/+ buffers/cache: 40 1864 > Swap: 2047 0 2047 > testing:~# uname -a > Linux testing 2.6.27-rc1-xenU #2 SMP Thu Jul 31 22:57:42 EDT 2008 i686 GNU/Linux > > (Minimal) Config: http://radium.outervoid.net/~carl/2.6.27-rc1 > > I see the same when using .26. Am I perhaps missing an option somewhere?The config has all the obvious things (namely, HIGHMEM64G). Hm, it does look like something is truncating at 4G and wrapping. The question is whether the system really only has 2G available to it, or if it''s just mis-reporting. Can you post the full boot log, and the contents of /proc/meminfo? Thanks, J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Aug 1, 2008 at 5:13 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:> Carl Jones wrote: >> >> Hi, >> >> I''m testing 2.6.27-rc1 and seem to have an issue when booting the domU >> with >4096MB. >> >> Xen: 3.2.1 x86_64 2.6.18.8 xen.org kernel >> domU: 2.6.27-rc1 32bit PAE >> >> # xm list test1 >> Name ID Mem VCPUs State >> Time(s) >> test1 24 6000 2 -b---- >> 5.6 >> # xm console test1 >> free -m >> total used free shared buffers cached >> Mem: 1904 75 1828 0 19 15 >> -/+ buffers/cache: 40 1864 >> Swap: 2047 0 2047 >> testing:~# uname -a >> Linux testing 2.6.27-rc1-xenU #2 SMP Thu Jul 31 22:57:42 EDT 2008 i686 >> GNU/Linux >> >> (Minimal) Config: http://radium.outervoid.net/~carl/2.6.27-rc1 >> >> I see the same when using .26. Am I perhaps missing an option somewhere? > > The config has all the obvious things (namely, HIGHMEM64G). > > Hm, it does look like something is truncating at 4G and wrapping. The > question is whether the system really only has 2G available to it, or if > it''s just mis-reporting. > > Can you post the full boot log, and the contents of /proc/meminfo? > > Thanks, > JSure. testing:~# cat /proc/meminfo MemTotal: 1949928 kB MemFree: 1887372 kB Buffers: 6068 kB Cached: 15968 kB SwapCached: 0 kB Active: 17008 kB Inactive: 10184 kB HighTotal: 1204232 kB HighFree: 1181512 kB LowTotal: 745696 kB LowFree: 705860 kB SwapTotal: 2097144 kB SwapFree: 2097144 kB Dirty: 56 kB Writeback: 0 kB AnonPages: 5168 kB Mapped: 4256 kB Slab: 2872 kB SReclaimable: 804 kB SUnreclaim: 2068 kB PageTables: 500 kB NFS_Unstable: 0 kB Bounce: 0 kB WritebackTmp: 0 kB CommitLimit: 3072108 kB Committed_AS: 27176 kB VmallocTotal: 116728 kB VmallocUsed: 1208 kB VmallocChunk: 115520 kB DirectMap4k: 186366 DirectMap2M: 0 http://radium.outervoid.net/~carl/boot.log http://radium.outervoid.net/~carl/xmlist.log (''xm list test1 --long'' output, in case that is helpful) Regards, Carl _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Carl Jones wrote:> http://radium.outervoid.net/~carl/boot.log > http://radium.outervoid.net/~carl/xmlist.log (''xm list test1 --long'' > output, in case that is helpful) >Thanks. Does this fix it? iff -r 25bf2d9a2e4c arch/x86/xen/setup.c --- a/arch/x86/xen/setup.c Fri Aug 01 17:12:18 2008 -0700 +++ b/arch/x86/xen/setup.c Sat Aug 02 09:00:02 2008 -0700 @@ -42,7 +42,7 @@ e820.nr_map = 0; - e820_add_region(0, PFN_PHYS(max_pfn), E820_RAM); + e820_add_region(0, PFN_PHYS((u64)max_pfn), E820_RAM); /* * Even though this is normal, usable memory under Xen, reserve J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Sun, Aug 3, 2008 at 4:00 AM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:> Carl Jones wrote: >> >> http://radium.outervoid.net/~carl/boot.log >> http://radium.outervoid.net/~carl/xmlist.log (''xm list test1 --long'' >> output, in case that is helpful) >> > > Thanks. Does this fix it? > > iff -r 25bf2d9a2e4c arch/x86/xen/setup.c > --- a/arch/x86/xen/setup.c Fri Aug 01 17:12:18 2008 -0700 > +++ b/arch/x86/xen/setup.c Sat Aug 02 09:00:02 2008 -0700 > @@ -42,7 +42,7 @@ > > e820.nr_map = 0; > > - e820_add_region(0, PFN_PHYS(max_pfn), E820_RAM); > + e820_add_region(0, PFN_PHYS((u64)max_pfn), E820_RAM); > > /* > * Even though this is normal, usable memory under Xen, reserve > > > J >Yep works nicely now. I tested up to 15GB or so with with that patch applied and CONFIG_XEN_MAX_DOMAIN_MEMORY=32 set: testing:~# cat /proc/meminfo MemTotal: 15769832 kB Regards, Carl _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Carl Jones wrote:> On Sun, Aug 3, 2008 at 4:00 AM, Jeremy Fitzhardinge <jeremy@goop.org> wrote: > >> Carl Jones wrote: >> >>> http://radium.outervoid.net/~carl/boot.log >>> http://radium.outervoid.net/~carl/xmlist.log (''xm list test1 --long'' >>> output, in case that is helpful) >>> >>> >> Thanks. Does this fix it? >> >> iff -r 25bf2d9a2e4c arch/x86/xen/setup.c >> --- a/arch/x86/xen/setup.c Fri Aug 01 17:12:18 2008 -0700 >> +++ b/arch/x86/xen/setup.c Sat Aug 02 09:00:02 2008 -0700 >> @@ -42,7 +42,7 @@ >> >> e820.nr_map = 0; >> >> - e820_add_region(0, PFN_PHYS(max_pfn), E820_RAM); >> + e820_add_region(0, PFN_PHYS((u64)max_pfn), E820_RAM); >> >> /* >> * Even though this is normal, usable memory under Xen, reserve >> >> >> J >> >> > > Yep works nicely now. I tested up to 15GB or so with with that patch > applied and CONFIG_XEN_MAX_DOMAIN_MEMORY=32 set: > > testing:~# cat /proc/meminfo > MemTotal: 15769832 kB >Excellent, thanks. That''s 32-bit? (That''s a pretty silly amount of memory to give to a 32-bit system, but it''s nice to know it works.) Could you try this patch instead to see if it works? It''s a more general fix. Thanks, J Subject: make PFN_PHYS explicitly return 64-bit result PFN_PHYS, as its name suggests, turns a pfn into a physical address. However, it is a macro which just operates on its argument without modifying its type. pfns are typed unsigned long, but an unsigned long may not be long enough to hold a physical address (32-bit systems with more than 32 bits of physcial address). This means that the resulting address could be truncated if it doesn''t fit within an unsigned long. This isn''t generally a problem because most users end up using it for "low" memory, but there''s no reason why PFN_PHYS couldn''t be used for any possible pfn. Unfortunately there''s no univerally recognized type for holding a full physical address, so this patch makes it always return a u64 result. In theory this could generate worse code, but in practice it will make no difference: - most users end up casting the result to a pointer or unsigned long, so on 32-bit systems the compiler should be able to eliminate the 64 bit part of the expression. - most users are in init code, which is neither size or performance critical This patch also introduces PFN_LOW_PHYS which explicitly operates on an unsigned long. It is currently unused, but it could be used to document the fact that the caller expects that the result will fit into an unsigned long. Signed-off-by: Jeremy Fitzhardinge <jeremy.fitzhardinge@citrix.com> Cc: Eduardo Habkost <ehabkost@redhat.com> --- arch/m32r/mm/discontig.c | 2 +- include/asm-x86/xen/page.h | 4 ++-- include/linux/pfn.h | 3 ++- 3 files changed, 5 insertions(+), 4 deletions(-) ==================================================================--- a/arch/m32r/mm/discontig.c +++ b/arch/m32r/mm/discontig.c @@ -112,7 +112,7 @@ initrd_start, INITRD_SIZE); } else { printk("initrd extends beyond end of memory " - "(0x%08lx > 0x%08lx)\ndisabling initrd\n", + "(0x%08lx > 0x%08llx)\ndisabling initrd\n", INITRD_START + INITRD_SIZE, PFN_PHYS(max_low_pfn)); ==================================================================--- a/include/asm-x86/xen/page.h +++ b/include/asm-x86/xen/page.h @@ -76,13 +76,13 @@ static inline xmaddr_t phys_to_machine(xpaddr_t phys) { unsigned offset = phys.paddr & ~PAGE_MASK; - return XMADDR(PFN_PHYS((u64)pfn_to_mfn(PFN_DOWN(phys.paddr))) | offset); + return XMADDR(PFN_PHYS(pfn_to_mfn(PFN_DOWN(phys.paddr))) | offset); } static inline xpaddr_t machine_to_phys(xmaddr_t machine) { unsigned offset = machine.maddr & ~PAGE_MASK; - return XPADDR(PFN_PHYS((u64)mfn_to_pfn(PFN_DOWN(machine.maddr))) | offset); + return XPADDR(PFN_PHYS(mfn_to_pfn(PFN_DOWN(machine.maddr))) | offset); } /* ==================================================================--- a/include/linux/pfn.h +++ b/include/linux/pfn.h @@ -4,6 +4,7 @@ #define PFN_ALIGN(x) (((unsigned long)(x) + (PAGE_SIZE - 1)) & PAGE_MASK) #define PFN_UP(x) (((x) + PAGE_SIZE-1) >> PAGE_SHIFT) #define PFN_DOWN(x) ((x) >> PAGE_SHIFT) -#define PFN_PHYS(x) ((x) << PAGE_SHIFT) +#define PFN_PHYS(x) ((u64)(x) << PAGE_SHIFT) +#define PFN_LOW_PHYS(x) ((unsigned long)(x) << PAGE_SHIFT) #endif _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Aug 5, 2008 at 3:07 PM, Jeremy Fitzhardinge <jeremy@goop.org> wrote:> Carl Jones wrote: >> >> On Sun, Aug 3, 2008 at 4:00 AM, Jeremy Fitzhardinge <jeremy@goop.org> >> wrote: >> >>> >>> Carl Jones wrote: >>> >>>> >>>> http://radium.outervoid.net/~carl/boot.log >>>> http://radium.outervoid.net/~carl/xmlist.log (''xm list test1 --long'' >>>> output, in case that is helpful) >>>> >>>> >>> >>> Thanks. Does this fix it? >>> >>> iff -r 25bf2d9a2e4c arch/x86/xen/setup.c >>> --- a/arch/x86/xen/setup.c Fri Aug 01 17:12:18 2008 -0700 >>> +++ b/arch/x86/xen/setup.c Sat Aug 02 09:00:02 2008 -0700 >>> @@ -42,7 +42,7 @@ >>> >>> e820.nr_map = 0; >>> >>> - e820_add_region(0, PFN_PHYS(max_pfn), E820_RAM); >>> + e820_add_region(0, PFN_PHYS((u64)max_pfn), E820_RAM); >>> >>> /* >>> * Even though this is normal, usable memory under Xen, reserve >>> >>> >>> J >>> >>> >> >> Yep works nicely now. I tested up to 15GB or so with with that patch >> applied and CONFIG_XEN_MAX_DOMAIN_MEMORY=32 set: >> >> testing:~# cat /proc/meminfo >> MemTotal: 15769832 kB >> > > Excellent, thanks. That''s 32-bit? (That''s a pretty silly amount of memory > to give to a 32-bit system, but it''s nice to know it works.) > > Could you try this patch instead to see if it works? It''s a more general > fix. > > Thanks, > J > > > Subject: make PFN_PHYS explicitly return 64-bit resultHi, Guest is 32bit. That patch works fine too. Regards, Carl _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>>> Jeremy Fitzhardinge <jeremy@goop.org> 05.08.08 05:07 >>> >Subject: make PFN_PHYS explicitly return 64-bit result > >PFN_PHYS, as its name suggests, turns a pfn into a physical address. >However, it is a macro which just operates on its argument without >modifying its type. pfns are typed unsigned long, but an unsigned >long may not be long enough to hold a physical address (32-bit systems >with more than 32 bits of physcial address). This means that the >resulting address could be truncated if it doesn''t fit within an >unsigned long. This isn''t generally a problem because most users end >up using it for "low" memory, but there''s no reason why PFN_PHYS >couldn''t be used for any possible pfn. > >Unfortunately there''s no univerally recognized type for holding a full >physical address, so this patch makes it always return a u64 result.Couldn''t you use resource_size_t here? Jan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jan Beulich wrote:>>>> Jeremy Fitzhardinge <jeremy@goop.org> 05.08.08 05:07 >>> >>>> >> Subject: make PFN_PHYS explicitly return 64-bit result >> >> PFN_PHYS, as its name suggests, turns a pfn into a physical address. >> However, it is a macro which just operates on its argument without >> modifying its type. pfns are typed unsigned long, but an unsigned >> long may not be long enough to hold a physical address (32-bit systems >> with more than 32 bits of physcial address). This means that the >> resulting address could be truncated if it doesn''t fit within an >> unsigned long. This isn''t generally a problem because most users end >> up using it for "low" memory, but there''s no reason why PFN_PHYS >> couldn''t be used for any possible pfn. >> >> Unfortunately there''s no univerally recognized type for holding a full >> physical address, so this patch makes it always return a u64 result. >> > > Couldn''t you use resource_size_t here?Yeah, looks like I can. It had crossed my mind, but I''d vaguely remembered that it might no be set if you don''t have 64-bit IO devices. But it looks like it''s set for all interesting cases (64 bit machines, and 32-bit x86 PAE). J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel