Hi Ian, After enabling the UART, I have been tring to finish the rest of booting. However, the output information stops at "Placing Xen at XXXX-XXXX" like following: Starting kernel ... - UART enabled - - CPU 00000000 booting - - Machine ID 00000ec1 - - Started in Hyp mode - - Zero BSS - - Setting up control registers - - Turning on paging - - Ready - RAM: 0000000080000000 - 00000000ffffffff MODULE[1]: 00000000a0000000 - 00000000a0400000 Placing Xen at 0x00000000ffe00000-0x0000000100000000 What could be the most possible problem of this case? Thanks, Baozi
On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote:> Hi Ian, > > After enabling the UART, I have been tring to finish the rest of booting. > However, the output information stops at "Placing Xen at XXXX-XXXX" like > following: > > Starting kernel ... > > - UART enabled - > - CPU 00000000 booting - > - Machine ID 00000ec1 - > - Started in Hyp mode - > - Zero BSS - > - Setting up control registers - > - Turning on paging - > - Ready - > RAM: 0000000080000000 - 00000000ffffffff > > MODULE[1]: 00000000a0000000 - 00000000a0400000 > Placing Xen at 0x00000000ffe00000-0x0000000100000000 > > What could be the most possible problem of this case?This is a about the point where we would shift from early_printk to the proper console driver, so I expect you simply don''t have the console setup. On your hypervisor command line you need a "dtuart=<something>" where <something> is a name for the serial device you want to use (e.g. on vexpress I use "dtuart=serial3" where the dts has "serial3 &v2m_serial3" in the aliases node). You will also need a DT enabled driver for that device. If the OMAP5432 is 8250 like then xen/drivers/char/ns16550.c might be suitable but it will need the Device Tree init hooked up, e.g. a DT_DEVICE_START given the appropriate hooks, see xen/drivers/char/pl011.c for an example. The ns16550 case is interesting because it is shared with the x86 stuff, but it ought to be possible to make it work either for DT (ARM) or via the arch code (x86, hardcoded addresses or command line specification). Ian.
On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote:> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: > > Hi Ian, > > > > After enabling the UART, I have been tring to finish the rest of booting. > > However, the output information stops at "Placing Xen at XXXX-XXXX" like > > following: > > > > Starting kernel ... > > > > - UART enabled - > > - CPU 00000000 booting - > > - Machine ID 00000ec1 - > > - Started in Hyp mode - > > - Zero BSS - > > - Setting up control registers - > > - Turning on paging - > > - Ready - > > RAM: 0000000080000000 - 00000000ffffffff > > > > MODULE[1]: 00000000a0000000 - 00000000a0400000 > > Placing Xen at 0x00000000ffe00000-0x0000000100000000 > > > > What could be the most possible problem of this case? > > This is a about the point where we would shift from early_printk to the > proper console driver, so I expect you simply don''t have the console > setup. >Hi Ian, I guess that it might not all about console driver, because early_printk does not work properly just after we have updated "the copy of boot_pgtable to use the new paddrs" in setup_pagetables(). It seems that FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without being restored. I''m looking into those related codes. Cheers, Baozi
On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote:> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: > > On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: > > > Hi Ian, > > > > > > After enabling the UART, I have been tring to finish the rest of booting. > > > However, the output information stops at "Placing Xen at XXXX-XXXX" like > > > following: > > > > > > Starting kernel ... > > > > > > - UART enabled - > > > - CPU 00000000 booting - > > > - Machine ID 00000ec1 - > > > - Started in Hyp mode - > > > - Zero BSS - > > > - Setting up control registers - > > > - Turning on paging - > > > - Ready - > > > RAM: 0000000080000000 - 00000000ffffffff > > > > > > MODULE[1]: 00000000a0000000 - 00000000a0400000 > > > Placing Xen at 0x00000000ffe00000-0x0000000100000000 > > > > > > What could be the most possible problem of this case? > > > > This is a about the point where we would shift from early_printk to the > > proper console driver, so I expect you simply don''t have the console > > setup. > > > Hi Ian, > > I guess that it might not all about console driver, because early_printk > does not work properly just after we have updated "the copy of boot_pgtable > to use the new paddrs" in setup_pagetables(). It seems that > FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without > being restored.Hrm, I think the expectation is that the fixmap will be copied as part of that and doesn''t need (or receive) any additional fixups (see around "Link in the fixmap pagetable"), but it is worth checking for sure. You could probably profitably sprinkle some more early_print around the place during that sequence for debugging purposes. BTW I''ve just checked a boot here and after the "Placing Xen" message there should be two more early_printk''s which are the "Xen heap: XXX pages..." and "Looking for UART.." ones before things switch to the real console, so you are right that it isn''t making it to the real console driver. Ian.
On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: >> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: >>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: >>>> Hi Ian, >>>> >>>> After enabling the UART, I have been tring to finish the rest of booting. >>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like >>>> following: >>>> >>>> Starting kernel ... >>>> >>>> - UART enabled - >>>> - CPU 00000000 booting - >>>> - Machine ID 00000ec1 - >>>> - Started in Hyp mode - >>>> - Zero BSS - >>>> - Setting up control registers - >>>> - Turning on paging - >>>> - Ready - >>>> RAM: 0000000080000000 - 00000000ffffffff >>>> >>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 >>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 >>>> >>>> What could be the most possible problem of this case? >>> >>> This is a about the point where we would shift from early_printk to the >>> proper console driver, so I expect you simply don''t have the console >>> setup. >>> >> Hi Ian, >> >> I guess that it might not all about console driver, because early_printk >> does not work properly just after we have updated "the copy of boot_pgtable >> to use the new paddrs" in setup_pagetables(). It seems that >> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without >> being restored. > > Hrm, I think the expectation is that the fixmap will be copied as part > of that and doesn''t need (or receive) any additional fixups (see around > "Link in the fixmap pagetable"), but it is worth checking for sure. > > You could probably profitably sprinkle some more early_print around the > place during that sequence for debugging purposes.That''s exactly what I did just now. And it is interesting that early_printk doesn''t work just after the updated boot_pgtable (p = (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been accessed (e.g. access to p[i].pt.base). Baozi.> > BTW I''ve just checked a boot here and after the "Placing Xen" message > there should be two more early_printk''s which are the "Xen heap: XXX > pages..." and "Looking for UART.." ones before things switch to the real > console, so you are right that it isn''t making it to the real console > driver. > > Ian. > >
On Mon, 2013-07-01 at 17:19 +0800, Chen Baozi wrote:> On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: > >> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: > >>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: > >>>> Hi Ian, > >>>> > >>>> After enabling the UART, I have been tring to finish the rest of booting. > >>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like > >>>> following: > >>>> > >>>> Starting kernel ... > >>>> > >>>> - UART enabled - > >>>> - CPU 00000000 booting - > >>>> - Machine ID 00000ec1 - > >>>> - Started in Hyp mode - > >>>> - Zero BSS - > >>>> - Setting up control registers - > >>>> - Turning on paging - > >>>> - Ready - > >>>> RAM: 0000000080000000 - 00000000ffffffff > >>>> > >>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 > >>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 > >>>> > >>>> What could be the most possible problem of this case? > >>> > >>> This is a about the point where we would shift from early_printk to the > >>> proper console driver, so I expect you simply don''t have the console > >>> setup. > >>> > >> Hi Ian, > >> > >> I guess that it might not all about console driver, because early_printk > >> does not work properly just after we have updated "the copy of boot_pgtable > >> to use the new paddrs" in setup_pagetables(). It seems that > >> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without > >> being restored. > > > > Hrm, I think the expectation is that the fixmap will be copied as part > > of that and doesn''t need (or receive) any additional fixups (see around > > "Link in the fixmap pagetable"), but it is worth checking for sure. > > > > You could probably profitably sprinkle some more early_print around the > > place during that sequence for debugging purposes. > That''s exactly what I did just now. And it is interesting that > early_printk doesn''t work just after the updated boot_pgtable (p > (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been > accessed (e.g. access to p[i].pt.base).How odd, that set of tables isn''t even used until the call to WRITE_TTBR several lines below! I''m pretty certain it isn''t updating anything which is live. Unless perhaps the new destination overlaps the current location of Xen, I bet we don''t handle that case well... Ian.
On Jul 1, 2013, at 5:30 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Mon, 2013-07-01 at 17:19 +0800, Chen Baozi wrote: >> On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> >>> On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: >>>> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: >>>>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: >>>>>> Hi Ian, >>>>>> >>>>>> After enabling the UART, I have been tring to finish the rest of booting. >>>>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like >>>>>> following: >>>>>> >>>>>> Starting kernel ... >>>>>> >>>>>> - UART enabled - >>>>>> - CPU 00000000 booting - >>>>>> - Machine ID 00000ec1 - >>>>>> - Started in Hyp mode - >>>>>> - Zero BSS - >>>>>> - Setting up control registers - >>>>>> - Turning on paging - >>>>>> - Ready - >>>>>> RAM: 0000000080000000 - 00000000ffffffff >>>>>> >>>>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 >>>>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 >>>>>> >>>>>> What could be the most possible problem of this case? >>>>> >>>>> This is a about the point where we would shift from early_printk to the >>>>> proper console driver, so I expect you simply don''t have the console >>>>> setup. >>>>> >>>> Hi Ian, >>>> >>>> I guess that it might not all about console driver, because early_printk >>>> does not work properly just after we have updated "the copy of boot_pgtable >>>> to use the new paddrs" in setup_pagetables(). It seems that >>>> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without >>>> being restored. >>> >>> Hrm, I think the expectation is that the fixmap will be copied as part >>> of that and doesn''t need (or receive) any additional fixups (see around >>> "Link in the fixmap pagetable"), but it is worth checking for sure. >>> >>> You could probably profitably sprinkle some more early_print around the >>> place during that sequence for debugging purposes. >> That''s exactly what I did just now. And it is interesting that >> early_printk doesn''t work just after the updated boot_pgtable (p >> (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been >> accessed (e.g. access to p[i].pt.base). > > How odd, that set of tables isn''t even used until the call to WRITE_TTBR > several lines below! I''m pretty certain it isn''t updating anything which > is live.Yes. And I found it happened even if reading the p[i].pt.base. I have tried to print the p[i].pt.base before updating its content. Then, all the early_printk would fail after it.> > Unless perhaps the new destination overlaps the current location of Xen, > I bet we don''t handle that case well... > > Ian. >
On Mon, 2013-07-01 at 17:49 +0800, Chen Baozi wrote:> On Jul 1, 2013, at 5:30 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > On Mon, 2013-07-01 at 17:19 +0800, Chen Baozi wrote: > >> On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > >>> On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: > >>>> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: > >>>>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: > >>>>>> Hi Ian, > >>>>>> > >>>>>> After enabling the UART, I have been tring to finish the rest of booting. > >>>>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like > >>>>>> following: > >>>>>> > >>>>>> Starting kernel ... > >>>>>> > >>>>>> - UART enabled - > >>>>>> - CPU 00000000 booting - > >>>>>> - Machine ID 00000ec1 - > >>>>>> - Started in Hyp mode - > >>>>>> - Zero BSS - > >>>>>> - Setting up control registers - > >>>>>> - Turning on paging - > >>>>>> - Ready - > >>>>>> RAM: 0000000080000000 - 00000000ffffffff > >>>>>> > >>>>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 > >>>>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 > >>>>>> > >>>>>> What could be the most possible problem of this case? > >>>>> > >>>>> This is a about the point where we would shift from early_printk to the > >>>>> proper console driver, so I expect you simply don''t have the console > >>>>> setup. > >>>>> > >>>> Hi Ian, > >>>> > >>>> I guess that it might not all about console driver, because early_printk > >>>> does not work properly just after we have updated "the copy of boot_pgtable > >>>> to use the new paddrs" in setup_pagetables(). It seems that > >>>> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without > >>>> being restored. > >>> > >>> Hrm, I think the expectation is that the fixmap will be copied as part > >>> of that and doesn''t need (or receive) any additional fixups (see around > >>> "Link in the fixmap pagetable"), but it is worth checking for sure. > >>> > >>> You could probably profitably sprinkle some more early_print around the > >>> place during that sequence for debugging purposes. > >> That''s exactly what I did just now. And it is interesting that > >> early_printk doesn''t work just after the updated boot_pgtable (p > >> (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been > >> accessed (e.g. access to p[i].pt.base). > > > > How odd, that set of tables isn''t even used until the call to WRITE_TTBR > > several lines below! I''m pretty certain it isn''t updating anything which > > is live. > Yes. And I found it happened even if reading the p[i].pt.base. I have > tried to print the p[i].pt.base before updating its content. Then, > all the early_printk would fail after it.That''s just weird. Any chance that reading/writing p[i]...base is actually causing a fault of some description such that the following prints never even get run?
On Jul 1, 2013, at 5:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Mon, 2013-07-01 at 17:49 +0800, Chen Baozi wrote: >> On Jul 1, 2013, at 5:30 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> >>> On Mon, 2013-07-01 at 17:19 +0800, Chen Baozi wrote: >>>> On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >>>> >>>>> On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: >>>>>> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: >>>>>>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: >>>>>>>> Hi Ian, >>>>>>>> >>>>>>>> After enabling the UART, I have been tring to finish the rest of booting. >>>>>>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like >>>>>>>> following: >>>>>>>> >>>>>>>> Starting kernel ... >>>>>>>> >>>>>>>> - UART enabled - >>>>>>>> - CPU 00000000 booting - >>>>>>>> - Machine ID 00000ec1 - >>>>>>>> - Started in Hyp mode - >>>>>>>> - Zero BSS - >>>>>>>> - Setting up control registers - >>>>>>>> - Turning on paging - >>>>>>>> - Ready - >>>>>>>> RAM: 0000000080000000 - 00000000ffffffff >>>>>>>> >>>>>>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 >>>>>>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 >>>>>>>> >>>>>>>> What could be the most possible problem of this case? >>>>>>> >>>>>>> This is a about the point where we would shift from early_printk to the >>>>>>> proper console driver, so I expect you simply don''t have the console >>>>>>> setup. >>>>>>> >>>>>> Hi Ian, >>>>>> >>>>>> I guess that it might not all about console driver, because early_printk >>>>>> does not work properly just after we have updated "the copy of boot_pgtable >>>>>> to use the new paddrs" in setup_pagetables(). It seems that >>>>>> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without >>>>>> being restored. >>>>> >>>>> Hrm, I think the expectation is that the fixmap will be copied as part >>>>> of that and doesn''t need (or receive) any additional fixups (see around >>>>> "Link in the fixmap pagetable"), but it is worth checking for sure. >>>>> >>>>> You could probably profitably sprinkle some more early_print around the >>>>> place during that sequence for debugging purposes. >>>> That''s exactly what I did just now. And it is interesting that >>>> early_printk doesn''t work just after the updated boot_pgtable (p >>>> (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been >>>> accessed (e.g. access to p[i].pt.base). >>> >>> How odd, that set of tables isn''t even used until the call to WRITE_TTBR >>> several lines below! I''m pretty certain it isn''t updating anything which >>> is live. >> Yes. And I found it happened even if reading the p[i].pt.base. I have >> tried to print the p[i].pt.base before updating its content. Then, >> all the early_printk would fail after it. > > That''s just weird. Any chance that reading/writing p[i]...base is > actually causing a fault of some description such that the following > prints never even get run?Yes. But it is OK if we don''t access the its copy in "early boot misc" area. for example: 01 memcpy((void *) dest_va, _start, _end - _start); 02 03 p = (void *) boot_pgtable; 04 early_printk("p[%d].pt.base: 0x%lx\n", 0, (unsigned long)p[0].pt.base); 05 06 p = (void *) boot_pgtable + dest_va - (unsigned long)_start; 07 early_printk("hello 1?\n"); 08 early_printk("p: 0x%lx\n", (unsigned long)p); 09 early_printk("hello 2?\n"); 10 early_printk("p[%d].pt.base: 0x%lx\n", 0, (unsigned long)p[0].pt.base); 11 early_printk("hello 3?\n"); Output: p[0].pt.base: 0x802cc hello 1? p: 0x6d2000 hello 2?
On Mon, 2013-07-01 at 19:18 +0800, Chen Baozi wrote:> On Jul 1, 2013, at 5:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > On Mon, 2013-07-01 at 17:49 +0800, Chen Baozi wrote: > >> On Jul 1, 2013, at 5:30 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > >>> On Mon, 2013-07-01 at 17:19 +0800, Chen Baozi wrote: > >>>> On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >>>> > >>>>> On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: > >>>>>> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: > >>>>>>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: > >>>>>>>> Hi Ian, > >>>>>>>> > >>>>>>>> After enabling the UART, I have been tring to finish the rest of booting. > >>>>>>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like > >>>>>>>> following: > >>>>>>>> > >>>>>>>> Starting kernel ... > >>>>>>>> > >>>>>>>> - UART enabled - > >>>>>>>> - CPU 00000000 booting - > >>>>>>>> - Machine ID 00000ec1 - > >>>>>>>> - Started in Hyp mode - > >>>>>>>> - Zero BSS - > >>>>>>>> - Setting up control registers - > >>>>>>>> - Turning on paging - > >>>>>>>> - Ready - > >>>>>>>> RAM: 0000000080000000 - 00000000ffffffff > >>>>>>>> > >>>>>>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 > >>>>>>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 > >>>>>>>> > >>>>>>>> What could be the most possible problem of this case? > >>>>>>> > >>>>>>> This is a about the point where we would shift from early_printk to the > >>>>>>> proper console driver, so I expect you simply don''t have the console > >>>>>>> setup. > >>>>>>> > >>>>>> Hi Ian, > >>>>>> > >>>>>> I guess that it might not all about console driver, because early_printk > >>>>>> does not work properly just after we have updated "the copy of boot_pgtable > >>>>>> to use the new paddrs" in setup_pagetables(). It seems that > >>>>>> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without > >>>>>> being restored. > >>>>> > >>>>> Hrm, I think the expectation is that the fixmap will be copied as part > >>>>> of that and doesn''t need (or receive) any additional fixups (see around > >>>>> "Link in the fixmap pagetable"), but it is worth checking for sure. > >>>>> > >>>>> You could probably profitably sprinkle some more early_print around the > >>>>> place during that sequence for debugging purposes. > >>>> That''s exactly what I did just now. And it is interesting that > >>>> early_printk doesn''t work just after the updated boot_pgtable (p > >>>> (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been > >>>> accessed (e.g. access to p[i].pt.base). > >>> > >>> How odd, that set of tables isn''t even used until the call to WRITE_TTBR > >>> several lines below! I''m pretty certain it isn''t updating anything which > >>> is live. > >> Yes. And I found it happened even if reading the p[i].pt.base. I have > >> tried to print the p[i].pt.base before updating its content. Then, > >> all the early_printk would fail after it. > > > > That''s just weird. Any chance that reading/writing p[i]...base is > > actually causing a fault of some description such that the following > > prints never even get run? > Yes.Random stab in the dark: are you loading the Xen binary at a 2MB aligned address?> But it is OK if we don''t access the its copy in "early boot misc" area. for example: > > 01 memcpy((void *) dest_va, _start, _end - _start); > 02 > 03 p = (void *) boot_pgtable; > 04 early_printk("p[%d].pt.base: 0x%lx\n", 0, (unsigned long)p[0].pt.base); > 05 > 06 p = (void *) boot_pgtable + dest_va - (unsigned long)_start; > 07 early_printk("hello 1?\n"); > 08 early_printk("p: 0x%lx\n", (unsigned long)p); > 09 early_printk("hello 2?\n"); > 10 early_printk("p[%d].pt.base: 0x%lx\n", 0, (unsigned long)p[0].pt.base); > 11 early_printk("hello 3?\n"); > > Output: > > p[0].pt.base: 0x802cc > hello 1? > p: 0x6d2000 > hello 2?So it is more than likely faulting on the access at 10. I don''t suppose you have a hardware debugger? You device has 2GB RAM, what happens if you hack the DTS to say it only has 1GB? Ian.
On Jul 1, 2013, at 7:31 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Mon, 2013-07-01 at 19:18 +0800, Chen Baozi wrote: >> On Jul 1, 2013, at 5:57 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> >>> On Mon, 2013-07-01 at 17:49 +0800, Chen Baozi wrote: >>>> On Jul 1, 2013, at 5:30 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >>>> >>>>> On Mon, 2013-07-01 at 17:19 +0800, Chen Baozi wrote: >>>>>> On Jul 1, 2013, at 4:50 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >>>>>> >>>>>>> On Mon, 2013-07-01 at 11:54 +0800, Chen Baozi wrote: >>>>>>>> On Thu, Jun 27, 2013 at 01:10:01PM +0100, Ian Campbell wrote: >>>>>>>>> On Thu, 2013-06-27 at 19:55 +0800, Chen Baozi wrote: >>>>>>>>>> Hi Ian, >>>>>>>>>> >>>>>>>>>> After enabling the UART, I have been tring to finish the rest of booting. >>>>>>>>>> However, the output information stops at "Placing Xen at XXXX-XXXX" like >>>>>>>>>> following: >>>>>>>>>> >>>>>>>>>> Starting kernel ... >>>>>>>>>> >>>>>>>>>> - UART enabled - >>>>>>>>>> - CPU 00000000 booting - >>>>>>>>>> - Machine ID 00000ec1 - >>>>>>>>>> - Started in Hyp mode - >>>>>>>>>> - Zero BSS - >>>>>>>>>> - Setting up control registers - >>>>>>>>>> - Turning on paging - >>>>>>>>>> - Ready - >>>>>>>>>> RAM: 0000000080000000 - 00000000ffffffff >>>>>>>>>> >>>>>>>>>> MODULE[1]: 00000000a0000000 - 00000000a0400000 >>>>>>>>>> Placing Xen at 0x00000000ffe00000-0x0000000100000000 >>>>>>>>>> >>>>>>>>>> What could be the most possible problem of this case? >>>>>>>>> >>>>>>>>> This is a about the point where we would shift from early_printk to the >>>>>>>>> proper console driver, so I expect you simply don''t have the console >>>>>>>>> setup. >>>>>>>>> >>>>>>>> Hi Ian, >>>>>>>> >>>>>>>> I guess that it might not all about console driver, because early_printk >>>>>>>> does not work properly just after we have updated "the copy of boot_pgtable >>>>>>>> to use the new paddrs" in setup_pagetables(). It seems that >>>>>>>> FIXMAP_ADDR(FIXMAP_CONSOLE) has been destroyed after the updating without >>>>>>>> being restored. >>>>>>> >>>>>>> Hrm, I think the expectation is that the fixmap will be copied as part >>>>>>> of that and doesn''t need (or receive) any additional fixups (see around >>>>>>> "Link in the fixmap pagetable"), but it is worth checking for sure. >>>>>>> >>>>>>> You could probably profitably sprinkle some more early_print around the >>>>>>> place during that sequence for debugging purposes. >>>>>> That''s exactly what I did just now. And it is interesting that >>>>>> early_printk doesn''t work just after the updated boot_pgtable (p >>>>>> (void *) boot_pgtable + dest_va - (unsigned long) _start;) has been >>>>>> accessed (e.g. access to p[i].pt.base). >>>>> >>>>> How odd, that set of tables isn''t even used until the call to WRITE_TTBR >>>>> several lines below! I''m pretty certain it isn''t updating anything which >>>>> is live. >>>> Yes. And I found it happened even if reading the p[i].pt.base. I have >>>> tried to print the p[i].pt.base before updating its content. Then, >>>> all the early_printk would fail after it. >>> >>> That''s just weird. Any chance that reading/writing p[i]...base is >>> actually causing a fault of some description such that the following >>> prints never even get run? >> Yes. > > Random stab in the dark: are you loading the Xen binary at a 2MB aligned > address? > >> But it is OK if we don''t access the its copy in "early boot misc" area. for example: >> >> 01 memcpy((void *) dest_va, _start, _end - _start); >> 02 >> 03 p = (void *) boot_pgtable; >> 04 early_printk("p[%d].pt.base: 0x%lx\n", 0, (unsigned long)p[0].pt.base); >> 05 >> 06 p = (void *) boot_pgtable + dest_va - (unsigned long)_start; >> 07 early_printk("hello 1?\n"); >> 08 early_printk("p: 0x%lx\n", (unsigned long)p); >> 09 early_printk("hello 2?\n"); >> 10 early_printk("p[%d].pt.base: 0x%lx\n", 0, (unsigned long)p[0].pt.base); >> 11 early_printk("hello 3?\n"); >> >> Output: >> >> p[0].pt.base: 0x802cc >> hello 1? >> p: 0x6d2000 >> hello 2? > > So it is more than likely faulting on the access at 10. I don''t suppose > you have a hardware debugger? > > You device has 2GB RAM, what happens if you hack the DTS to say it only > has 1GB?Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: memory { device_type = "memory" - reg = <0x80000000 0x80000000>; /* 2 GB */ + reg = <0x80000000 0x40000000>; /* 1 GB */ }; I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". Cheers, Baozi.> > Ian. >
On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote:> > You device has 2GB RAM, what happens if you hack the DTS to say it only > > has 1GB? > > Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: > > memory { > device_type = "memory" > - reg = <0x80000000 0x80000000>; /* 2 GB */ > + reg = <0x80000000 0x40000000>; /* 1 GB */ > }; > > I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages".Excellent! So I guess if you keep the 2GB but put it at the right address it will also work? Ian.
On Jul 1, 2013, at 8:06 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote: > >>> You device has 2GB RAM, what happens if you hack the DTS to say it only >>> has 1GB? >> >> Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: >> >> memory { >> device_type = "memory" >> - reg = <0x80000000 0x80000000>; /* 2 GB */ >> + reg = <0x80000000 0x40000000>; /* 1 GB */ >> }; >> >> I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". > > Excellent! > > So I guess if you keep the 2GB but put it at the right address it will > also work?I''m afraid no, :-( Here is how I load and boot the image both in 1GB case and 2G case (from u-boot): OMAP5430 EVM # fatload mmc 0 0x80000000 xen-uImage reading xen-uImage 902572 bytes read in 59 ms (14.6 MiB/s) OMAP5430 EVM # bootm 0x80000000 - ## Booting kernel from Legacy Image at 80000000 ... Image Name: Image Type: ARM Linux Kernel Image (uncompressed) Data Size: 902508 Bytes = 881.4 KiB Load Address: 80200000 Entry Point: 80200000 Verifying Checksum ... OK Loading Kernel Image ... OK OK And I''ve also tried "fat load mmc 0 0x90000000 xen-uImage" in both cases. It works for 1GB but fails in 2GB case too. Something wrong? Cheers, Baozi.
On Mon, 2013-07-01 at 20:32 +0800, Chen Baozi wrote:> On Jul 1, 2013, at 8:06 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote: > > > >>> You device has 2GB RAM, what happens if you hack the DTS to say it only > >>> has 1GB? > >> > >> Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: > >> > >> memory { > >> device_type = "memory" > >> - reg = <0x80000000 0x80000000>; /* 2 GB */ > >> + reg = <0x80000000 0x40000000>; /* 1 GB */ > >> }; > >> > >> I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". > > > > Excellent! > > > > So I guess if you keep the 2GB but put it at the right address it will > > also work? > > I''m afraid no, :-( > > Here is how I load and boot the image both in 1GB case and 2G case (from u-boot): > > OMAP5430 EVM # fatload mmc 0 0x80000000 xen-uImage > reading xen-uImage > 902572 bytes read in 59 ms (14.6 MiB/s) > OMAP5430 EVM # bootm 0x80000000 - > ## Booting kernel from Legacy Image at 80000000 ... > Image Name: > Image Type: ARM Linux Kernel Image (uncompressed) > Data Size: 902508 Bytes = 881.4 KiB > Load Address: 80200000 > Entry Point: 80200000 > Verifying Checksum ... OK > Loading Kernel Image ... OK > OK> And I''ve also tried "fat load mmc 0 0x90000000 xen-uImage" in both > cases. It works for 1GB but fails in 2GB case too.What is the base address of RAM on the A31 SoC? I don''t have a DTS for an A31 system but all arch/arm/boot/dts/sun* (which covers A10 and A13 I think) have it at 0x40000000. Which means that *if* your system has 2GB then the correct memory line is: reg = <0x40000000 0x80000000>; /* 2 GB */ Ian.
On Mon, Jul 01, 2013 at 01:39:02PM +0100, Ian Campbell wrote:> On Mon, 2013-07-01 at 20:32 +0800, Chen Baozi wrote: > > On Jul 1, 2013, at 8:06 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > > > On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote: > > > > > >>> You device has 2GB RAM, what happens if you hack the DTS to say it only > > >>> has 1GB? > > >> > > >> Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: > > >> > > >> memory { > > >> device_type = "memory" > > >> - reg = <0x80000000 0x80000000>; /* 2 GB */ > > >> + reg = <0x80000000 0x40000000>; /* 1 GB */ > > >> }; > > >> > > >> I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". > > > > > > Excellent! > > > > > > So I guess if you keep the 2GB but put it at the right address it will > > > also work? > > > > I''m afraid no, :-( > > > > Here is how I load and boot the image both in 1GB case and 2G case (from u-boot): > > > > OMAP5430 EVM # fatload mmc 0 0x80000000 xen-uImage > > reading xen-uImage > > 902572 bytes read in 59 ms (14.6 MiB/s) > > OMAP5430 EVM # bootm 0x80000000 - > > ## Booting kernel from Legacy Image at 80000000 ... > > Image Name: > > Image Type: ARM Linux Kernel Image (uncompressed) > > Data Size: 902508 Bytes = 881.4 KiB > > Load Address: 80200000 > > Entry Point: 80200000 > > Verifying Checksum ... OK > > Loading Kernel Image ... OK > > OK > > > And I''ve also tried "fat load mmc 0 0x90000000 xen-uImage" in both > > cases. It works for 1GB but fails in 2GB case too. > > What is the base address of RAM on the A31 SoC?A31 SoC? I use OMAP5432 ES2.0 uEVM (OMAP5 Panda) board. The attachment is the dts file which the linux kernel uses on this board by default, which the memory line is: reg = <0x80000000 0x80000000>; /* 2 GB */> I don''t have a DTS for > an A31 system but all arch/arm/boot/dts/sun* (which covers A10 and A13 I > think) have it at 0x40000000. Which means that *if* your system has 2GB > then the correct memory line is: > reg = <0x40000000 0x80000000>; /* 2 GB */ > > Ian. > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Jul 1, 2013, at 8:43 PM, Chen Baozi <baozich@gmail.com> wrote:> On Mon, Jul 01, 2013 at 01:39:02PM +0100, Ian Campbell wrote: >> On Mon, 2013-07-01 at 20:32 +0800, Chen Baozi wrote: >>> On Jul 1, 2013, at 8:06 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >>> >>>> On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote: >>>> >>>>>> You device has 2GB RAM, what happens if you hack the DTS to say it only >>>>>> has 1GB? >>>>> >>>>> Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: >>>>> >>>>> memory { >>>>> device_type = "memory" >>>>> - reg = <0x80000000 0x80000000>; /* 2 GB */ >>>>> + reg = <0x80000000 0x40000000>; /* 1 GB */ >>>>> }; >>>>> >>>>> I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". >>>> >>>> Excellent! >>>> >>>> So I guess if you keep the 2GB but put it at the right address it will >>>> also work? >>> >>> I''m afraid no, :-( >>> >>> Here is how I load and boot the image both in 1GB case and 2G case (from u-boot): >>> >>> OMAP5430 EVM # fatload mmc 0 0x80000000 xen-uImage >>> reading xen-uImage >>> 902572 bytes read in 59 ms (14.6 MiB/s) >>> OMAP5430 EVM # bootm 0x80000000 - >>> ## Booting kernel from Legacy Image at 80000000 ... >>> Image Name: >>> Image Type: ARM Linux Kernel Image (uncompressed) >>> Data Size: 902508 Bytes = 881.4 KiB >>> Load Address: 80200000 >>> Entry Point: 80200000 >>> Verifying Checksum ... OK >>> Loading Kernel Image ... OK >>> OK >> >>> And I''ve also tried "fat load mmc 0 0x90000000 xen-uImage" in both >>> cases. It works for 1GB but fails in 2GB case too. >> >> What is the base address of RAM on the A31 SoC? > > A31 SoC? I use OMAP5432 ES2.0 uEVM (OMAP5 Panda) board. > > The attachment is the dts file which the linux kernel uses on this board by > default, which the memory line is: > > reg = <0x80000000 0x80000000>; /* 2 GB */ >However, interestingly, in omap5-evm.dts, it writes: reg = <0x80000000 0x7F000000>; /* 2032 MB */ And if I use this line, it works! So I think this might be the point.
On Mon, 2013-07-01 at 20:43 +0800, Chen Baozi wrote:> On Mon, Jul 01, 2013 at 01:39:02PM +0100, Ian Campbell wrote: > > On Mon, 2013-07-01 at 20:32 +0800, Chen Baozi wrote: > > > On Jul 1, 2013, at 8:06 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > > > > > > On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote: > > > > > > > >>> You device has 2GB RAM, what happens if you hack the DTS to say it only > > > >>> has 1GB? > > > >> > > > >> Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: > > > >> > > > >> memory { > > > >> device_type = "memory" > > > >> - reg = <0x80000000 0x80000000>; /* 2 GB */ > > > >> + reg = <0x80000000 0x40000000>; /* 1 GB */ > > > >> }; > > > >> > > > >> I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". > > > > > > > > Excellent! > > > > > > > > So I guess if you keep the 2GB but put it at the right address it will > > > > also work? > > > > > > I''m afraid no, :-( > > > > > > Here is how I load and boot the image both in 1GB case and 2G case (from u-boot): > > > > > > OMAP5430 EVM # fatload mmc 0 0x80000000 xen-uImage > > > reading xen-uImage > > > 902572 bytes read in 59 ms (14.6 MiB/s) > > > OMAP5430 EVM # bootm 0x80000000 - > > > ## Booting kernel from Legacy Image at 80000000 ... > > > Image Name: > > > Image Type: ARM Linux Kernel Image (uncompressed) > > > Data Size: 902508 Bytes = 881.4 KiB > > > Load Address: 80200000 > > > Entry Point: 80200000 > > > Verifying Checksum ... OK > > > Loading Kernel Image ... OK > > > OK > > > > > And I''ve also tried "fat load mmc 0 0x90000000 xen-uImage" in both > > > cases. It works for 1GB but fails in 2GB case too. > > > > What is the base address of RAM on the A31 SoC? > > A31 SoC? I use OMAP5432 ES2.0 uEVM (OMAP5 Panda) board.Oh, sorry, I conflated your and Bamvor''s systems in my mind after discussing combining the serial drivers last week. His is an A31 thing.> The attachment is the dts file which the linux kernel uses on this board by > default, which the memory line is: > > reg = <0x80000000 0x80000000>; /* 2 GB */OK. So I wonder why that doesn''t work for us then. Ian.
On Mon, 2013-07-01 at 20:58 +0800, Chen Baozi wrote:> On Jul 1, 2013, at 8:43 PM, Chen Baozi <baozich@gmail.com> wrote: > > > On Mon, Jul 01, 2013 at 01:39:02PM +0100, Ian Campbell wrote: > >> On Mon, 2013-07-01 at 20:32 +0800, Chen Baozi wrote: > >>> On Jul 1, 2013, at 8:06 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >>> > >>>> On Mon, 2013-07-01 at 20:00 +0800, Chen Baozi wrote: > >>>> > >>>>>> You device has 2GB RAM, what happens if you hack the DTS to say it only > >>>>>> has 1GB? > >>>>> > >>>>> Bingo! Compared to Arndale board, OMAP5432 uEVM''s RAM is not yet 0x40000000~0xbfffffff, but 0x80000000~0xffffffff. After make the following change to DTS: > >>>>> > >>>>> memory { > >>>>> device_type = "memory" > >>>>> - reg = <0x80000000 0x80000000>; /* 2 GB */ > >>>>> + reg = <0x80000000 0x40000000>; /* 1 GB */ > >>>>> }; > >>>>> > >>>>> I can reach the output "Xen heap: 131072 pages Dom heap: 131072 pages". > >>>> > >>>> Excellent! > >>>> > >>>> So I guess if you keep the 2GB but put it at the right address it will > >>>> also work? > >>> > >>> I''m afraid no, :-( > >>> > >>> Here is how I load and boot the image both in 1GB case and 2G case (from u-boot): > >>> > >>> OMAP5430 EVM # fatload mmc 0 0x80000000 xen-uImage > >>> reading xen-uImage > >>> 902572 bytes read in 59 ms (14.6 MiB/s) > >>> OMAP5430 EVM # bootm 0x80000000 - > >>> ## Booting kernel from Legacy Image at 80000000 ... > >>> Image Name: > >>> Image Type: ARM Linux Kernel Image (uncompressed) > >>> Data Size: 902508 Bytes = 881.4 KiB > >>> Load Address: 80200000 > >>> Entry Point: 80200000 > >>> Verifying Checksum ... OK > >>> Loading Kernel Image ... OK > >>> OK > >> > >>> And I''ve also tried "fat load mmc 0 0x90000000 xen-uImage" in both > >>> cases. It works for 1GB but fails in 2GB case too. > >> > >> What is the base address of RAM on the A31 SoC? > > > > A31 SoC? I use OMAP5432 ES2.0 uEVM (OMAP5 Panda) board. > > > > The attachment is the dts file which the linux kernel uses on this board by > > default, which the memory line is: > > > > reg = <0x80000000 0x80000000>; /* 2 GB */ > > > However, interestingly, in omap5-evm.dts, it writes: > > reg = <0x80000000 0x7F000000>; /* 2032 MB */ > > And if I use this line, it works! > > So I think this might be the point.Huzzah! It certainly is important to use a DTB which accurately describes the exact hardware. I bet some RAM is carved out for video RAM or secure world use or something... Ian.
On Jul 1, 2013, at 9:03 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> Huzzah! > > It certainly is important to use a DTB which accurately describes the > exact hardware. > > I bet some RAM is carved out for video RAM or secure world use or > something...Aha, I''ve found the commit log of omap5-evm.dts''s memory line: ARM: dts: omap5-evm: Update available memory to 2032 MB On OMAP5 to detect invalid/bad memory accesses, 16MB of DDR is used as a trap. Hence available memory for linux OS is 2032 MB on boards populated with 2 GB memory. So I think we''ve finally got the cause. Cheers, Baozi.