Hi! According to the ELF Spec, sh_link points to the String table. The attached patch fixes this. Christoph _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
SHT_PROGBITS sections do not use the sh_link field. It''s used by symbol-table sections, relocation-entry sections, and suchlike, of which we don''t have any. And .shstrtab section is never linked to. -- Keir On 3/1/07 09:30, "Christoph Egger" <Christoph.Egger@amd.com> wrote:> > Hi! > > According to the ELF Spec, sh_link points to the String table. > The attached patch fixes this. > > Christoph > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I am a newbie and I would like to know how to apply this patch. Can someone provide me with steps. Christoph Egger-2 wrote:> > > Hi! > > According to the ELF Spec, sh_link points to the String table. > The attached patch fixes this. > > Christoph > > diff -r 886f7312257e xen/arch/x86/boot/mkelf32.c > --- a/xen/arch/x86/boot/mkelf32.c Tue Jan 02 14:19:47 2007 +0000 > +++ b/xen/arch/x86/boot/mkelf32.c Fri Dec 22 14:22:04 2006 +0100 > @@ -72,7 +72,7 @@ static Elf32_Shdr out_shdr[] = { > DYNAMICALLY_FILLED, /* sh_addr */ > RAW_OFFSET, /* sh_offset */ > DYNAMICALLY_FILLED, /* sh_size */ > - 0, /* sh_link */ > + 2, /* sh_link */ > 0, /* sh_info */ > 64, /* sh_addralign */ > 0 /* sh_entsize */ > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >-- View this message in context: http://www.nabble.com/-PATCH--mkelf32%3A-Correct-sh_link-tf2912492.html#a8141336 Sent from the Xen - Dev mailing list archive at Nabble.com. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wednesday 03 January 2007 14:57, Keir Fraser wrote:> SHT_PROGBITS sections do not use the sh_link field. It''s used by > symbol-table sections, relocation-entry sections, and suchlike, of which we > don''t have any. And .shstrtab section is never linked to.Why do you strip the symbol table and re-add it in an unusual format when linking the bootable ELF image? Is this a historical relict? Christoph _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wednesday 03 January 2007 16:11, Christoph Egger wrote:> On Wednesday 03 January 2007 14:57, Keir Fraser wrote: > > SHT_PROGBITS sections do not use the sh_link field. It''s used by > > symbol-table sections, relocation-entry sections, and suchlike, of which > > we don''t have any. And .shstrtab section is never linked to. > > Why do you strip the symbol table and re-add it in an unusual format > when linking the bootable ELF image? Is this a historical relict?Ah, sorry! I actually meant: Why do you strip the symbol and string tables and re-add the string table in an unusal format? Is this a historical relict? Christoph _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 15:11, "Christoph Egger" <Christoph.Egger@amd.com> wrote:> On Wednesday 03 January 2007 14:57, Keir Fraser wrote: >> SHT_PROGBITS sections do not use the sh_link field. It''s used by >> symbol-table sections, relocation-entry sections, and suchlike, of which we >> don''t have any. And .shstrtab section is never linked to. > > Why do you strip the symbol table and re-add it in an unusual format > when linking the bootable ELF image? Is this a historical relict?No bootloader would load a symbol table for us (where would it load it? How would it tell us about it?), so we have to internalise it into the PROGBITS section somehow. Stealing the Linux code to do this made sense. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 15:14, "Christoph Egger" <Christoph.Egger@amd.com> wrote:>> Why do you strip the symbol table and re-add it in an unusual format >> when linking the bootable ELF image? Is this a historical relict? > > Ah, sorry! I actually meant: Why do you strip the symbol and string tables > and re-add the string table in an unusal format? > Is this a historical relict?It isn''t the symbol-name string table: look at how short it is! It''s the section-header string table. I believe all Elf images require one of those (not least because some section names have special meaning). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wednesday 03 January 2007 16:20, Keir Fraser wrote:> On 3/1/07 15:11, "Christoph Egger" <Christoph.Egger@amd.com> wrote: > > On Wednesday 03 January 2007 14:57, Keir Fraser wrote: > >> SHT_PROGBITS sections do not use the sh_link field. It''s used by > >> symbol-table sections, relocation-entry sections, and suchlike, of which > >> we don''t have any. And .shstrtab section is never linked to. > > > > Why do you strip the symbol table and re-add it in an unusual format > > when linking the bootable ELF image? Is this a historical relict? > > No bootloader would load a symbol table for us (where would it load it? How > would it tell us about it?), so we have to internalise it into the PROGBITS > section somehow. Stealing the Linux code to do this made sense.The multiboot header tells us where to find the symbol and string table. Then we can load it ourself. Look into the NetBSD code how this would work: http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/arch/i386/include/multiboot.h?annotate=1.4 http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/arch/i386/i386/multiboot.c?annotate=1.9 ;-) Would you accept a patch, that makes XEN bootable as an ELF image with ELF symbol-name and string tables? If yes, then I will work on this. Christoph _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 15:38, "Christoph Egger" <Christoph.Egger@amd.com> wrote:> Would you accept a patch, that makes XEN bootable as an ELF image with > ELF symbol-name and string tables? If yes, then I will work on this.What we have now works and produces a reasonably compressed symbol table. I don''t see any value in changing it. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, Jan 03, 2007 at 04:21:35PM +0000, Keir Fraser wrote:> On 3/1/07 15:38, "Christoph Egger" <Christoph.Egger@amd.com> wrote: > > > Would you accept a patch, that makes XEN bootable as an ELF image with > > ELF symbol-name and string tables? If yes, then I will work on this. > > What we have now works and produces a reasonably compressed symbol table. I > don''t see any value in changing it.It seems like it would simplify things. And it would certainly help us as we want to ship the binary with (compressed) debug info, which is tricky right now due to this unusual behaviour... regards john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 6:36 pm, "John Levon" <levon@movementarian.org> wrote:>> What we have now works and produces a reasonably compressed symbol table. I >> don''t see any value in changing it. > > It seems like it would simplify things. And it would certainly help us > as we want to ship the binary with (compressed) debug info, which is > tricky right now due to this unusual behaviour...You can ship xen-syms if users want extra debug info (e.g., attach a gdb client). What exactly is your requirement? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, Jan 03, 2007 at 07:14:47PM +0000, Keir Fraser wrote:> >> What we have now works and produces a reasonably compressed symbol table. I > >> don''t see any value in changing it. > > > > It seems like it would simplify things. And it would certainly help us > > as we want to ship the binary with (compressed) debug info, which is > > tricky right now due to this unusual behaviour... > > You can ship xen-syms if users want extra debug info (e.g., attach a gdb > client).Indeed, and that''s what we''re doing currently. But this is problematic as it''s very easy for other files to get lost, not be handed over etc. and working backwards to the right version is very rarely feasible... Thus we''d like to have a single binary (or more importantly dump!) that has everything in place. regards, john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 7:20 pm, "John Levon" <levon@movementarian.org> wrote:>> You can ship xen-syms if users want extra debug info (e.g., attach a gdb >> client). > > Indeed, and that''s what we''re doing currently. But this is problematic > as it''s very easy for other files to get lost, not be handed over etc. > and working backwards to the right version is very rarely feasible... > Thus we''d like to have a single binary (or more importantly dump!) that > has everything in place.That would potentially be nice, but I don''t know of any tools that would allow us to symbolically debug a dump without having a separate source of symbol information (like xen-syms). The problems you mention would typically be resolved by non-technical means anyway (offical release builds for which you keep a copy of the bits, which can then be used to augment the information from a raw dump). I wouldn''t deep-dive into a core dump for a build of dubious heritage where the builder hadn''t kept the sources and build bits! :-) -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, Jan 03, 2007 at 08:06:12PM +0000, Keir Fraser wrote:> > Indeed, and that''s what we''re doing currently. But this is problematic > > as it''s very easy for other files to get lost, not be handed over etc. > > and working backwards to the right version is very rarely feasible... > > Thus we''d like to have a single binary (or more importantly dump!) that > > has everything in place. > > That would potentially be nice, but I don''t know of any tools that would > allow us to symbolically debug a dump without having a separate source of > symbol information (like xen-syms).This is exactly what is available on Solaris both for userspace core dumps and kernel crash dumps. We''d like to be able to extend this to Xen. As far as I know, there''s nothing preventing, say, Red Hat''s ''crash'' doing the same.> The problems you mention would typically > be resolved by non-technical means anyway (offical release builds for which > you keep a copy of the bits, which can then be used to augment the > information from a raw dump).Bitter experience tends to show this doesn''t work well especially in development environments.> I wouldn''t deep-dive into a core dump for a > build of dubious heritage where the builder hadn''t kept the sources and > build bits! :-)All too common alas :) Wasting a small amount of time debugging a dodgy build is usually better than letting a real intermittent bug slip by... regards, john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 8:11 pm, "John Levon" <levon@movementarian.org> wrote:>> That would potentially be nice, but I don''t know of any tools that would >> allow us to symbolically debug a dump without having a separate source of >> symbol information (like xen-syms). > > This is exactly what is available on Solaris both for userspace core > dumps and kernel crash dumps. We''d like to be able to extend this to > Xen. As far as I know, there''s nothing preventing, say, Red Hat''s > ''crash'' doing the same.Well, extracting symbols from the existing Xen format isn''t hard. We could add that, but really it''s not as good as source-level debugging, being able look at local variables up the call stack, etc. I don''t think it is expecting much to require xen-syms to be kept around for xen images that are still being tested. Another trick might be to allow production of a xen image with the full xen-syms image appended -- then a crash dump would have all the info required to do full source-level debugging (except the source tree, but you do get a changeset number from the core dump which will suffice if you use revision control). That would be easy actually -- just get mkelf32 to append the xen-syms file and set a couple of values at a known offset in the main xen image, easily picked up by crashdump tools and by Xen itself. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, Jan 03, 2007 at 09:21:08PM +0000, Keir Fraser wrote:> >> That would potentially be nice, but I don''t know of any tools that would > >> allow us to symbolically debug a dump without having a separate source of > >> symbol information (like xen-syms). > > > > This is exactly what is available on Solaris both for userspace core > > dumps and kernel crash dumps. We''d like to be able to extend this to > > Xen. As far as I know, there''s nothing preventing, say, Red Hat''s > > ''crash'' doing the same. > > Well, extracting symbols from the existing Xen format isn''t hard.It was more of an answer to your question as to why there would be any point in shipping a standard symtab rather than the Linux thing. Essentially it makes things much easier for us as our debug info relies on a standard symtab.> suffice if you use revision control). That would be easy actually -- just > get mkelf32 to append the xen-syms file and set a couple of values at a > known offset in the main xen image, easily picked up by crashdump tools and > by Xen itself.I had in mind something very much like this, we''d just append the symtab,strtab, and our debug info on to the image and set something to point to the location. This was mostly a workaround of the difficulties with building those sections straight into the file in the normal ELF manner, though. I think the question might more be: what do we gain from being ''unusual''? A significantly smaller string size for the symbol names might be a reasonable answer, I don''t really know how much of a gain that stuff is. An entertaining logic puzzle in the makefiles, perhaps not ;) Of course it would still be a /little/ unusual because you''d want the symtab etc. as allocated PROGBITS. regards, john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3/1/07 9:32 pm, "John Levon" <levon@movementarian.org> wrote:> I had in mind something very much like this, we''d just append the > symtab,strtab, and our debug info on to the image and set something to > point to the location. This was mostly a workaround of the difficulties > with building those sections straight into the file in the normal ELF > manner, though.We could build them in as ordinary sections in our loadable xen image, but the bootloader wouldn''t load them into memory for us let alone point us at them. So I think we have to cook all the stuff you''re interested in into PROGBITS (unless you really only want symtab/strtab, but if you''re going this far why not keep stuff like the dwarf debug info as well?). One nice thing about ''dumbly'' appending xen-syms is that the whole image is then extractable to a file post-crash and can be manipulated by the developer independently of tools like RH''s crash. It''s also simple to implement and doesn''t mess with the basic structure of our loadable Xen elf image (which has acquired various tweaks and workarounds for multiboot loader bugs). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I am a newbie and I would like to know how to apply this patch. > Can someone provide me with steps.Save the patch xen-mkelf32.diff to your hard drive Get a xen repository and change into it: cd xen-unstable.hg (or whatever you called it) patch -p0 < /path/to/xen-mkelf32.diff Should work. Cheers, Mark> Christoph Egger-2 wrote: > > Hi! > > > > According to the ELF Spec, sh_link points to the String table. > > The attached patch fixes this. > > > > Christoph > > > > diff -r 886f7312257e xen/arch/x86/boot/mkelf32.c > > --- a/xen/arch/x86/boot/mkelf32.c Tue Jan 02 14:19:47 2007 +0000 > > +++ b/xen/arch/x86/boot/mkelf32.c Fri Dec 22 14:22:04 2006 +0100 > > @@ -72,7 +72,7 @@ static Elf32_Shdr out_shdr[] = { > > DYNAMICALLY_FILLED, /* sh_addr */ > > RAW_OFFSET, /* sh_offset */ > > DYNAMICALLY_FILLED, /* sh_size */ > > - 0, /* sh_link */ > > + 2, /* sh_link */ > > 0, /* sh_info */ > > 64, /* sh_addralign */ > > 0 /* sh_entsize */ > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel-- Dave: Just a question. What use is a unicyle with no seat? And no pedals! Mark: To answer a question with a question: What use is a skateboard? Dave: Skateboards have wheels. Mark: My wheel has a wheel! _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wednesday 03 January 2007 20:20, John Levon wrote:> On Wed, Jan 03, 2007 at 07:14:47PM +0000, Keir Fraser wrote: > > >> What we have now works and produces a reasonably compressed symbol > > >> table. I don''t see any value in changing it. > > > > > > It seems like it would simplify things. And it would certainly help us > > > as we want to ship the binary with (compressed) debug info, which is > > > tricky right now due to this unusual behaviour... > > > > You can ship xen-syms if users want extra debug info (e.g., attach a gdb > > client). > > Indeed, and that''s what we''re doing currently. But this is problematic > as it''s very easy for other files to get lost, not be handed over etc. > and working backwards to the right version is very rarely feasible... > Thus we''d like to have a single binary (or more importantly dump!) that > has everything in place.Further, when you boot on a serial console how do you connect gdb over a serial console? You can''t use a serial console for two things concurrently. Then a xen-kernel debugger is useful. And yeah, that needs standard symtabs and strtabs. And with a kernel debugger, you don''t need to worry about what John pointed out. regards, John _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 4/1/07 08:30, "Christoph Egger" <Christoph.Egger@amd.com> wrote:>> Indeed, and that''s what we''re doing currently. But this is problematic >> as it''s very easy for other files to get lost, not be handed over etc. >> and working backwards to the right version is very rarely feasible... >> Thus we''d like to have a single binary (or more importantly dump!) that >> has everything in place. > > Further, when you boot on a serial console how do you connect gdb > over a serial console? You can''t use a serial console for two things > concurrently. Then a xen-kernel debugger is useful. And yeah, that needs > standard symtabs and strtabs. > And with a kernel debugger, you don''t need to worry about what John pointed > out.Gdbstub and console can share a serial port. When gdbstub is entered then the gdbstub steals the serial port. Console messages are then wrapped as gdb messages. Given that this setup works and requires minimal code in Xen, I have no interest in accepting an in-kernel debugger into the tree. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thursday 04 January 2007 10:58, Keir Fraser wrote:> On 4/1/07 08:30, "Christoph Egger" <Christoph.Egger@amd.com> wrote: > >> Indeed, and that''s what we''re doing currently. But this is problematic > >> as it''s very easy for other files to get lost, not be handed over etc. > >> and working backwards to the right version is very rarely feasible... > >> Thus we''d like to have a single binary (or more importantly dump!) that > >> has everything in place. > > > > Further, when you boot on a serial console how do you connect gdb > > over a serial console? You can''t use a serial console for two things > > concurrently. Then a xen-kernel debugger is useful. And yeah, that needs > > standard symtabs and strtabs. > > And with a kernel debugger, you don''t need to worry about what John > > pointed out. > > Gdbstub and console can share a serial port. When gdbstub is entered then > the gdbstub steals the serial port. Console messages are then wrapped as > gdb messages. Given that this setup works and requires minimal code in Xen, > I have no interest in accepting an in-kernel debugger into the tree.What about xen/common/keyhandler.c ? Isn''t this a very tiny in-kernel debugger? If YOU are fine with gdb, why is it there? And it ALWAYS gets compiled in. We already have gdb, xentrace, xencrash, loglevels and more. Nonetheless, many developers still add/remove tons of printk()''s for debugging purpose. I count this as a sign, that there is a demand for better debugging facilities. gdb is a very powerful general purpose debugger. So the core question is, what is really needed and what not. _Some_ examples to think about: Is a command line useful? Should it be possible to lookup symbols for its addresses? Is it useful to reprint xen messages? Is it better to put lots of printk() to figure out which virtual cpus and interrupts are mapped to which physical ones or just tell the debugger "show me this" ? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, Jan 03, 2007 at 09:40:46PM +0000, Keir Fraser wrote:> > I had in mind something very much like this, we''d just append the > > symtab,strtab, and our debug info on to the image and set something to > > point to the location. This was mostly a workaround of the difficulties > > with building those sections straight into the file in the normal ELF > > manner, though. > > We could build them in as ordinary sections in our loadable xen image, but > the bootloader wouldn''t load them into memory for us let alone point us at > them.Unless I''m misunderstanding badly isn''t this just a case of changing the mapfile to stick them into the single program header.> One nice thing about ''dumbly'' appending xen-syms is that the whole image is > then extractable to a file post-crash and can be manipulated by the > developer independently of tools like RH''s crash. It''s also simple to > implement and doesn''t mess with the basic structure of our loadable Xen elf > image (which has acquired various tweaks and workarounds for multiboot > loader bugs).It''s perhaps a bit silly to have .text twice etc.though. I''m not familiar with the tweaks you mention so I''m sure you''re right there. john _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel