Hello, I am ignorantly trying to find a way to start a second domain0 when the first domain0 crashes. My original intent was to find a way to collect a core file much in the same way the domainU core files are collected on a linux box. I managed to hack in some hooks to get a second domain0 loaded and started upon a crash. However, I am stuck without console output, ie printk() is not working. Oddly, early_printk() works fine and I can sort of track the progress of the second domain0 as it boots. So my first question is how does the serial console (not vga) magic work for a domain0 kernel? Do I have to set some internal domain bits to have the console output redirected to the serial port as opposed to userspace and xend (ala domainU kernels)? I attached my small dirty hack below, if it helps. The jist of it is I added a new parameter, "failover", to be read in from a domainU config file (ie /etc/xen/guest). The second domain0 is treated just like a domainU kernel in regards to loading and installing the userspace disk. The only thing different is "xm create -c guest" will load a domain0 kernel instead of a domainU kernel. Combined with the "failover" flag above, this triggers the hypervisor to reserve this domain as a backup for later and _not_ unpause the domain once its cpu context is loaded. Upon the first domain0 crash, the hypervisor looks up the backup domain0, mindlessly sets some interesting bits, and unpauses it. Pretty simple. Any help or tips or pointers would be great. Cheers, Don _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 27 Mar 2006, at 23:08, Don Zickus wrote:> I am ignorantly trying to find a way to start a second domain0 when the > first domain0 crashes. My original intent was to find a way to > collect a > core file much in the same way the domainU core files are collected on > a > linux box. > > I managed to hack in some hooks to get a second domain0 loaded and > started > upon a crash. However, I am stuck without console output, ie printk() > is > not working. Oddly, early_printk() works fine and I can sort of track > the > progress of the second domain0 as it boots. So my first question is > how > does the serial console (not vga) magic work for a domain0 kernel? Do > I > have to set some internal domain bits to have the console output > redirected to the serial port as opposed to userspace and xend (ala > domainU kernels)?The new domain0 must have SIF_INITDOMAIN flag set in its start_info flags. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > The new domain0 must have SIF_INITDOMAIN flag set in its start_info > flags. >Cool. Thanks. I''ll give that a try. Cheers, Don _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, Mar 27, 2006 at 11:38:10PM +0100, Keir Fraser wrote:> > On 27 Mar 2006, at 23:08, Don Zickus wrote: > > >I am ignorantly trying to find a way to start a second domain0 when the > >first domain0 crashes. My original intent was to find a way to > >collect a > >core file much in the same way the domainU core files are collected on > >a > >linux box. > > > >I managed to hack in some hooks to get a second domain0 loaded and > >started > >upon a crash. However, I am stuck without console output, ie printk() > >is > >not working. Oddly, early_printk() works fine and I can sort of track > >the > >progress of the second domain0 as it boots. So my first question is > >how > >does the serial console (not vga) magic work for a domain0 kernel? Do > >I > >have to set some internal domain bits to have the console output > >redirected to the serial port as opposed to userspace and xend (ala > >domainU kernels)? > > The new domain0 must have SIF_INITDOMAIN flag set in its start_info > flags. > > -- KeirContinuing on my project to get a domain0 to failover to a second domain0, I have stumbled upon an out of memory problem. Does anyone know what type of quirky scenario I would have had to create in order to have xen_create_contiguous_region() fail within swiotlb_init_with_default_size() of the linux x86_64 kernel? I have managed to debug it down to alloc_heap_pages(MEMZONE_DMADOM, order) returning NULL. But my problem is I couldn''t figure out how those heaps were created in the first place. This domain is supposed to be configured with 256MB of memory (and "xm list" confirms that). So for the actual domain to run out of memory seems illogical. Any thoughts, pointers, or help would be great. Cheers, Don _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
* Don Zickus (dzickus@redhat.com) wrote:> Continuing on my project to get a domain0 to failover to a second domain0, > I have stumbled upon an out of memory problem. Does anyone know what type > of quirky scenario I would have had to create in order to have > xen_create_contiguous_region() fail within > swiotlb_init_with_default_size() of the linux x86_64 kernel?Sounds like nothing other than running low on that amount of contiguous pages. CTRL-a CTRL-a CTRL-a m Typing that should show you what''s available, I suspect it''s less than the swiotlb is asking for.> I have managed to debug it down to alloc_heap_pages(MEMZONE_DMADOM, order) > returning NULL. But my problem is I couldn''t figure out how those heaps > were created in the first place.Boot time, IIRC.> This domain is supposed to be configured with 256MB of memory (and "xm > list" confirms that). So for the actual domain to run out of memory seems > illogical.It''s the swiotlb asking for too large an order allocation. You can certainly limit it to smth small like 1MB (swiotlb=1 on dom0 kernel command line) and hope that''s sufficient. Hmm, wonder if there would be a way to reuse those pages from the first dom0. thanks, -chris _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, Mar 30, 2006 at 09:48:12PM -0800, Chris Wright wrote:> * Don Zickus (dzickus@redhat.com) wrote: > > Continuing on my project to get a domain0 to failover to a second domain0, > > I have stumbled upon an out of memory problem. Does anyone know what type > > of quirky scenario I would have had to create in order to have > > xen_create_contiguous_region() fail within > > swiotlb_init_with_default_size() of the linux x86_64 kernel? > > Sounds like nothing other than running low on that amount of contiguous > pages. > > CTRL-a CTRL-a CTRL-a m > > Typing that should show you what''s available, I suspect it''s less than > the swiotlb is asking for.You know I have never been able to get this to work. It keeps wanting to initializing my modem (minicom uses the same CTRL-a interface). :) Or maybe this is because I am using the redhat version.> > > I have managed to debug it down to alloc_heap_pages(MEMZONE_DMADOM, order) > > returning NULL. But my problem is I couldn''t figure out how those heaps > > were created in the first place. > > Boot time, IIRC.I couldn''t find any code to verify that. From what I have seen, the heaps are created from the act of freeing memory (I didn''t see any initialization). So this is why I am kind of lost/stuck.> > > This domain is supposed to be configured with 256MB of memory (and "xm > > list" confirms that). So for the actual domain to run out of memory seems > > illogical. > > It''s the swiotlb asking for too large an order allocation. You can > certainly limit it to smth small like 1MB (swiotlb=1 on dom0 kernel > command line) and hope that''s sufficient. Hmm, wonder if there would > be a way to reuse those pages from the first dom0. > > thanks, > -chrisI tried this and no luck. What confuses me is that normally it wants 2MB (which isn''t to big) and works fine when running multiple domUs. I just thought out of 256MB, there ought to be at least one 2MB chunk of contig memory somewhere. Just looking for some more pointers. Cheers, Don _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 31 Mar 2006, at 16:44, Don Zickus wrote:>> Sounds like nothing other than running low on that amount of >> contiguous >> pages. >> >> CTRL-a CTRL-a CTRL-a m >> >> Typing that should show you what''s available, I suspect it''s less than >> the swiotlb is asking for. > > You know I have never been able to get this to work. It keeps wanting > to > initializing my modem (minicom uses the same CTRL-a interface). :) Or > maybe this is because I am using the redhat version.There''s a little util in our tree under tools/misc/miniterm which provides a minimal serial client for connecting to Xen boxes. It won''t eat your CTRL-a. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
* Don Zickus <dzickus@redhat.com> [2006-03-31 09:41]:> On Thu, Mar 30, 2006 at 09:48:12PM -0800, Chris Wright wrote: > > * Don Zickus (dzickus@redhat.com) wrote: > > > Continuing on my project to get a domain0 to failover to a second domain0, > > > I have stumbled upon an out of memory problem. Does anyone know what type > > > of quirky scenario I would have had to create in order to have > > > xen_create_contiguous_region() fail within > > > swiotlb_init_with_default_size() of the linux x86_64 kernel? > > > > Sounds like nothing other than running low on that amount of contiguous > > pages. > > > > CTRL-a CTRL-a CTRL-a m > > > > Typing that should show you what''s available, I suspect it''s less than > > the swiotlb is asking for. > > You know I have never been able to get this to work. It keeps wanting to > initializing my modem (minicom uses the same CTRL-a interface). :) Or > maybe this is because I am using the redhat version.The conswitch parameter to xen allows you to change the key used with control to switch between Xen and Dom0 serial input. I use ''x'' and don''t have to reconfigured minicom. title Xen 3.0-unstable / XenLinux 2.6.11 kernel /boot/xen.gz dom0_mem=524288 com1=115200,8n1 conswitch=x module /boot/vmlinuz-2.6.11-xen0 root=/dev/sda1 ro console=tty0 -- Ryan Harper Software Engineer; Linux Technology Center IBM Corp., Austin, Tx (512) 838-9253 T/L: 678-9253 ryanh@us.ibm.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Fri, Mar 31, 2006 at 04:56:41PM +0100, Keir Fraser wrote:> > On 31 Mar 2006, at 16:44, Don Zickus wrote: > > >>Sounds like nothing other than running low on that amount of > >>contiguous > >>pages. > >> > >>CTRL-a CTRL-a CTRL-a m > >> > >>Typing that should show you what''s available, I suspect it''s less than > >>the swiotlb is asking for. > > > >You know I have never been able to get this to work. It keeps wanting > >to > >initializing my modem (minicom uses the same CTRL-a interface). :) Or > >maybe this is because I am using the redhat version. > > There''s a little util in our tree under tools/misc/miniterm which > provides a minimal serial client for connecting to Xen boxes. It won''t > eat your CTRL-a. > > -- KeirAaah. Now I can extract some useful info! And sure enough DMA heap size shrinks from 14MB down to 536KB once I load the second dom0. Time to investigate. Thanks. Cheers, Don _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi, On Fri, 2006-03-31 at 10:44 -0500, Don Zickus wrote:> > CTRL-a CTRL-a CTRL-a m > > > > Typing that should show you what''s available, I suspect it''s less than > > the swiotlb is asking for. > > You know I have never been able to get this to work. It keeps wanting to > initializing my modem (minicom uses the same CTRL-a interface). :) Or > maybe this is because I am using the redhat version."ttywatch" (it''s in fedora-extras) is what I use for serial console monitoring. It logs stuff permanently in the background, with proper log file rotation; but it also lets you connect to the console live, via a per-port telnet-to-serial relay, when you need it. And it passes ^A through just fine. btw, for FC5 xen builds, it turns out that serial console input (which is needed for the HV to see the ^A''s) doesn''t work unless you specify pnpacpi=off on the kernel boot line. We need to work out why: I suspect that the ACPI pnp bios is tweaking the serial lines for mouse probing and is upsetting the HV console input in the process. --Stephen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel