Windows has provision for a driver to get called when a bug check occurs (KeRegisterBugCheckCallback etc), and also to get called every time some data is to be written to the dump file. I would like to be able to capture that information into Dom0 somehow, as a few of the bug check''s I''ve had in GPLPV in the past have rendered the block device driver inoperable and therefore unable to write out the dump file. Is there an existing communication channel that could be used for this or do I need to invent write my own? The more low level the better (as in xenstore may not be accessible depending on the circumstances of the crash). Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 18/08/2009 07:47, "James Harper" <james.harper@bendigoit.com.au> wrote:> Windows has provision for a driver to get called when a bug check occurs > (KeRegisterBugCheckCallback etc), and also to get called every time some > data is to be written to the dump file. > > I would like to be able to capture that information into Dom0 somehow, > as a few of the bug check''s I''ve had in GPLPV in the past have rendered > the block device driver inoperable and therefore unable to write out the > dump file. > > Is there an existing communication channel that could be used for this > or do I need to invent write my own? The more low level the better (as > in xenstore may not be accessible depending on the circumstances of the > crash).Could it be as simple as write the guest physical address of a pre-allocated buffer into xenstore (from GPLPV drivers)? Then dump that area of guest memory on guest bugcheck, even as a manual operation from dom0? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On 18/08/2009 07:47, "James Harper" <james.harper@bendigoit.com.au>wrote:> > > Windows has provision for a driver to get called when a bug checkoccurs> > (KeRegisterBugCheckCallback etc), and also to get called every timesome> > data is to be written to the dump file. > > > > I would like to be able to capture that information into Dom0somehow,> > as a few of the bug check''s I''ve had in GPLPV in the past haverendered> > the block device driver inoperable and therefore unable to write outthe> > dump file. > > > > Is there an existing communication channel that could be used forthis> > or do I need to invent write my own? The more low level the better(as> > in xenstore may not be accessible depending on the circumstances ofthe> > crash). > > Could it be as simple as write the guest physical address of apre-allocated> buffer into xenstore (from GPLPV drivers)? Then dump that area ofguest> memory on guest bugcheck, even as a manual operation from dom0? >The memory in question could be anything from 64kb (minidump) to the size of DomU memory (full dump) or somewhere inbetween (kernel dump). I''m normally just interested in the kernel dump which is normally a much more respectable size, but still not the size of memory you can easily find when the system has gone belly up. James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 18/08/2009 11:42, "James Harper" <james.harper@bendigoit.com.au> wrote:>> Could it be as simple as write the guest physical address of a > pre-allocated >> buffer into xenstore (from GPLPV drivers)? Then dump that area of > guest >> memory on guest bugcheck, even as a manual operation from dom0? > > The memory in question could be anything from 64kb (minidump) to the > size of DomU memory (full dump) or somewhere inbetween (kernel dump). > I''m normally just interested in the kernel dump which is normally a much > more respectable size, but still not the size of memory you can easily > find when the system has gone belly up.Hmm, okay. Could you dump it out over the emulated serial line? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On 18/08/2009 11:42, "James Harper" <james.harper@bendigoit.com.au>wrote:> > >> Could it be as simple as write the guest physical address of a > > pre-allocated > >> buffer into xenstore (from GPLPV drivers)? Then dump that area of > > guest > >> memory on guest bugcheck, even as a manual operation from dom0? > > > > The memory in question could be anything from 64kb (minidump) to the > > size of DomU memory (full dump) or somewhere inbetween (kerneldump).> > I''m normally just interested in the kernel dump which is normally amuch> > more respectable size, but still not the size of memory you caneasily> > find when the system has gone belly up. > > Hmm, okay. Could you dump it out over the emulated serial line? >I was thinking about that. It could work, then I could just do ''xm console <domu> >dump'' and parse the dump from there or something. James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 18 Aug 2009, James Harper wrote:> > On 18/08/2009 11:42, "James Harper" <james.harper@bendigoit.com.au> > wrote: > > > > >> Could it be as simple as write the guest physical address of a > > > pre-allocated > > >> buffer into xenstore (from GPLPV drivers)? Then dump that area of > > > guest > > >> memory on guest bugcheck, even as a manual operation from dom0? > > > > > > The memory in question could be anything from 64kb (minidump) to the > > > size of DomU memory (full dump) or somewhere inbetween (kernel > dump). > > > I''m normally just interested in the kernel dump which is normally a > much > > > more respectable size, but still not the size of memory you can > easily > > > find when the system has gone belly up. > > > > Hmm, okay. Could you dump it out over the emulated serial line? > > > > I was thinking about that. It could work, then I could just do ''xm > console <domu> >dump'' and parse the dump from there or something. >Good idea, also keep in mind that qemu can emulate multiple serials if needed. Stubdoms support multiple serials too, so if you make the changes to add another serial to qemu please test with stubdoms too :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On Tue, 18 Aug 2009, James Harper wrote: > > > On 18/08/2009 11:42, "James Harper"<james.harper@bendigoit.com.au>> > wrote: > > > > > > >> Could it be as simple as write the guest physical address of a > > > > pre-allocated > > > >> buffer into xenstore (from GPLPV drivers)? Then dump that areaof> > > > guest > > > >> memory on guest bugcheck, even as a manual operation from dom0? > > > > > > > > The memory in question could be anything from 64kb (minidump) tothe> > > > size of DomU memory (full dump) or somewhere inbetween (kernel > > dump). > > > > I''m normally just interested in the kernel dump which isnormally a> > much > > > > more respectable size, but still not the size of memory you can > > easily > > > > find when the system has gone belly up. > > > > > > Hmm, okay. Could you dump it out over the emulated serial line? > > > > > > > I was thinking about that. It could work, then I could just do ''xm > > console <domu> >dump'' and parse the dump from there or something. > > > > Good idea, also keep in mind that qemu can emulate multiple serials if > needed. > Stubdoms support multiple serials too, so if you make the changes toadd> another serial to qemu please test with stubdoms too :)How fast do you think that serial port output could be? Is the baud rate limited by qemu? To put the question another way, the size of the dump could be up to the size of DomU''s memory so could be many gigabytes, although would typically be <500MB, would that dump out in reasonable time? Thinking about it some more, it would be nice to have a mechanism that could allow PV Linux/BSD/Solaris/etc to dump out similar information in the event of a complete crash. What would be involved in having a hypercall that could be called repeatedly to send data to Dom0? 1 page at a time would be fine, so the parameters could be ''pfn, offset, length''. I guess the problem wouldn''t be in adding the hypercall but in having the information end up in Dom0... would that be really difficult and/or messy? James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > On 18/08/2009 11:42, "James Harper" <james.harper@bendigoit.com.au>wrote:> > >> Could it be as simple as write the guest physical address of a > > pre-allocated > >> buffer into xenstore (from GPLPV drivers)? Then dump that area of > > guest > >> memory on guest bugcheck, even as a manual operation from dom0? > > > > The memory in question could be anything from 64kb (minidump) to the > > size of DomU memory (full dump) or somewhere inbetween (kerneldump).> > I''m normally just interested in the kernel dump which is normally amuch> > more respectable size, but still not the size of memory you caneasily> > find when the system has gone belly up. > > Hmm, okay. Could you dump it out over the emulated serial line? >The more I look into it the less fond I get of this idea... I already use the serial line for other things. What about an event channel and a page of memory that could be retrieved via the existing hvm_get_param mechanism for HVM domains and part of the start_info page for PV domains? (just like store_mfn and store_evtchn etc). James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
James Harper wrote:> > The more I look into it the less fond I get of this idea... I already > use the serial line for other things. > > What about an event channel and a page of memory that could be retrieved > via the existing hvm_get_param mechanism for HVM domains and part of the > start_info page for PV domains? (just like store_mfn and store_evtchn > etc). >What about a 2nd serial line? -- ==============================Paul Durrant, Software Engineer Citrix Systems (R&D) Ltd. First Floor, Building 101 Cambridge Science Park Milton Road Cambridge CB4 0FY United Kingdom TEL: x35957 (+44 1223 225957) ============================== _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 19 Aug 2009, Paul Durrant wrote:> James Harper wrote: > > > > The more I look into it the less fond I get of this idea... I already > > use the serial line for other things. > > > > What about an event channel and a page of memory that could be retrieved > > via the existing hvm_get_param mechanism for HVM domains and part of the > > start_info page for PV domains? (just like store_mfn and store_evtchn > > etc). > > > > What about a 2nd serial line? >This is why I was mentioning the fact that you can have multiple serials in qemu, and that stubdoms support it too. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > > The memory in question could be anything from 64kb (minidump) to the > > > size of DomU memory (full dump) or somewhere inbetween (kernel > dump). > > > I''m normally just interested in the kernel dump which is normally a > much > > > more respectable size, but still not the size of memory you can > easily > > > find when the system has gone belly up. > > > > Hmm, okay. Could you dump it out over the emulated serial line? > > > > The more I look into it the less fond I get of this idea... I already > use the serial line for other things.Not being familiar with Windows dumps, but only with Linux kdump (which is basically a mini-Linux distro running in 64MB - that includes the initrd.img and + RAM space) which can dump the entire memory on to whatever you want (NFS, disk, USB, etc). How does Windows natively/normally do its dump? Is it just dumping the entire memory space on the local disk? Or do you have to set special flags to do so from the driver? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk wrote:> > Not being familiar with Windows dumps, but only with Linux kdump (which is basically > a mini-Linux distro running in 64MB - that includes the initrd.img and + RAM space) > which can dump the entire memory on to whatever you want (NFS, disk, USB, etc). > > How does Windows natively/normally do its dump? Is it just dumping the entire memory space > on the local disk? Or do you have to set special flags to do so from the driver? >Yes, it normally dumps it''s memory to the device using a special incarnation of the driver stack for that device. Paul -- ==============================Paul Durrant, Software Engineer Citrix Systems (R&D) Ltd. First Floor, Building 101 Cambridge Science Park Milton Road Cambridge CB4 0FY United Kingdom ============================== _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > How does Windows natively/normally do its dump? Is it just dumpingthe> > entire memory space > > on the local disk? Or do you have to set special flags to do so fromthe> > driver? > > Yes, it normally dumps it''s memory to the device using a special > incarnation of the driver stack for that device. >That is correct. An additional and completely separate version of the driver is created at boot time and only activated in the event of a crash. The driver is called with all interrupts disabled and cannot allocate memory etc, so it''s a very restricted environment. The ISR is called on a timer. Hibernate is a separate copy of the driver again but similar setup. There are a few callbacks you can set in any driver to tell Windows to call you every time it has some crash dump information to write out, and the runtime environment is similar (interrupts disabled, very few system calls at your disposal). It is this that I would like to use to write out a crash dump to be captured by Dom0. The two ideas under consideration are: 1. Use the serial port 2. Create an event channel and a mapped page and dump data out that way. Add them to the start info page for PV and as HVM parameters (retrieved via hypercall like the xenstore page and pfn) for HVM I like #2 best for the following reasons: . qemu needs patching for a second serial port anyway. I don''t want to use the primary as I''m already using it for something else in the environment I''m thinking of. . much more efficient . works with PV (irrelevant though if there are no PV domains that support writing out a dump file) . easy for the DomU to tell if there is something waiting to capture the crash dump (no parameter present = no crash dump) . less code on the windows side of things, vs trying to access the serial port hardware directly for which there is no ''blessed'' mechanism to do in a crash dump environment afaik James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 20/08/2009 10:56, "James Harper" <james.harper@bendigoit.com.au> wrote:> I like #2 best for the following reasons: > . qemu needs patching for a second serial port anyway. I don''t want to > use the primary as I''m already using it for something else in the > environment I''m thinking of. > . much more efficient > . works with PV (irrelevant though if there are no PV domains that > support writing out a dump file) > . easy for the DomU to tell if there is something waiting to capture the > crash dump (no parameter present = no crash dump) > . less code on the windows side of things, vs trying to access the > serial port hardware directly for which there is no ''blessed'' mechanism > to do in a crash dump environment afaikWell, obviously you can knock up #2 yourself, with an accompanying little dom0 daemon, which you can ship with the GPLPV drivers. The two components can communicate via xenstore at start-of-day to prime themselves for later possible crash dumps. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel