Windows makes internal ''debugprint'' calls, kind of like the kernel logging that Linux does that you get via dmesg. It would make my life a heap easier if I could route those to dom0. The windows debugger allows this indirectly but it is really slow and has heaps of overhead. Is it possible to use the /local/domain/<id>/drivers/console/0 to do this? It looks like even PV linux doesn''t use this, so maybe it''s just a legacy thing? /local/domain/<id>/console appears to just be the HVM VNC thing for Windows. ''xm console'' for a HVM domain connects to the serial port if I set serial = ''pty'' in the config, but I''d rather not use a serial port if I can get around it. Any suggestions? Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Windows makes internal ''debugprint'' calls, kind of like the kernel > logging that Linux does that you get via dmesg. > > It would make my life a heap easier if I could route those to dom0. The > windows debugger allows this indirectly but it is really slow and has > heaps of overhead. > > Is it possible to use the /local/domain/<id>/drivers/console/0 to do > this? It looks like even PV linux doesn''t use this, so maybe it''s just a > legacy thing?If you boot your hypervisor with guest_loglvl=all on the command line then you should be able to log to Xen''s ring buffer by outb''ing bytes to port 0xe9. The results will show up in xm dmesg and the hypervisor serial console. I''m not sure what version of Xen you''re running, but some of the older ones had problems with rep outsb to that port, so you might be better off using a bunch of individual outb''s. Steven. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Windows makes internal ''debugprint'' calls, kind of like the kernel > > logging that Linux does that you get via dmesg. > > > > It would make my life a heap easier if I could route those to dom0.The> > windows debugger allows this indirectly but it is really slow andhas> > heaps of overhead. > > > > Is it possible to use the /local/domain/<id>/drivers/console/0 to do > > this? It looks like even PV linux doesn''t use this, so maybe it''sjust a> > legacy thing? > If you boot your hypervisor with guest_loglvl=all on the command line > then you should be able to log to Xen''s ring buffer by outb''ing bytes > to port 0xe9. The results will show up in xm dmesg and the hypervisor > serial console. > > I''m not sure what version of Xen you''re running, but some of the older > ones had problems with rep outsb to that port, so you might be better > off using a bunch of individual outb''s. >That''s interesting... so that is implemented by the hypervisor then? Is there something similar implemented via hvm/qemu that can log via hvm log files? That would make life easier. If not, I''ll implement it. Currently, ''xm save'' appears to work but after ''xm restore'' the DomU is hanging at some really strange points (eg between one KdPrint statement and the next), so I suspect that something else is going on but I''m never getting the debug output because it''s crashed so hard that even the windows debugger can''t talk to me anymore. At least if I was doing outb''s the logging would be done synchronously and I wouldn''t have the debugger connected which I think is introducing a heisenbug effect... Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
(Sorry about the slight delay responding; I''ve been travelling, and away from my usual dev machine.)> > > Windows makes internal ''debugprint'' calls, kind of like the kernel > > > logging that Linux does that you get via dmesg. > > > > > > It would make my life a heap easier if I could route those to dom0. > The > > > windows debugger allows this indirectly but it is really slow and > has > > > heaps of overhead. > > > > > > Is it possible to use the /local/domain/<id>/drivers/console/0 to do > > > this? It looks like even PV linux doesn''t use this, so maybe it''s > just a > > > legacy thing? > > If you boot your hypervisor with guest_loglvl=all on the command line > > then you should be able to log to Xen''s ring buffer by outb''ing bytes > > to port 0xe9. The results will show up in xm dmesg and the hypervisor > > serial console. > > > > I''m not sure what version of Xen you''re running, but some of the older > > ones had problems with rep outsb to that port, so you might be better > > off using a bunch of individual outb''s. > That''s interesting... so that is implemented by the hypervisor then?Yes.> Is there something similar implemented via hvm/qemu that can log via hvm > log files? That would make life easier. If not, I''ll implement it.There isn''t at the moment, at least not in the xenbits repositories. There is one in the XenServer version of qemu, though. I''ve attached the relevant patch. If you apply this and rebuild qemu then you should be able to send stuff to the qemu logfile by writing bytes to port 0x12. Note that this is rate limited, though, and if you hit the limiter the domain will be quite severely punished. I''ve checked that this applies against qemu-xen-unstable.git, and that the resulting tree at least builds, but I''ve not actually checked that it works. Upstream qemu is pretty similar to XenServer qemu at the moment, though, so it should be okay.> Currently, ''xm save'' appears to work but after ''xm restore'' the DomU is > hanging at some really strange points (eg between one KdPrint statement > and the next), so I suspect that something else is going on but I''m > never getting the debug output because it''s crashed so hard that even > the windows debugger can''t talk to me anymore. At least if I was doing > outb''s the logging would be done synchronously and I wouldn''t have the > debugger connected which I think is introducing a heisenbug effect...Can''t help you with that, sorry. Steven. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > > Is there something similar implemented via hvm/qemu that can log viahvm> > log files? That would make life easier. If not, I''ll implement it. > There isn''t at the moment, at least not in the xenbits repositories. > There is one in the XenServer version of qemu, though. I''ve attached > the relevant patch. If you apply this and rebuild qemu then you > should be able to send stuff to the qemu logfile by writing bytes to > port 0x12. Note that this is rate limited, though, and if you hit the > limiter the domain will be quite severely punished. > > I''ve checked that this applies against qemu-xen-unstable.git, and that > the resulting tree at least builds, but I''ve not actually checked that > it works. Upstream qemu is pretty similar to XenServer qemu at the > moment, though, so it should be okay.I''ll have a look at that one. I have actually just implemented the same sort of thing but trapping writes to the platform pci io space. It''s just for debugging so I haven''t done rate limiting or anything. A port 0x12 debugger would be better as I could log stuff even before the Xen PCI ioport range is mapped.> > > Currently, ''xm save'' appears to work but after ''xm restore'' the DomUis> > hanging at some really strange points (eg between one KdPrintstatement> > and the next), so I suspect that something else is going on but I''m > > never getting the debug output because it''s crashed so hard thateven> > the windows debugger can''t talk to me anymore. At least if I wasdoing> > outb''s the logging would be done synchronously and I wouldn''t havethe> > debugger connected which I think is introducing a heisenbugeffect...> Can''t help you with that, sorry. >Well hopefully you have :) Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > Is there something similar implemented via hvm/qemu that can log viahvm> > log files? That would make life easier. If not, I''ll implement it. > There isn''t at the moment, at least not in the xenbits repositories. > There is one in the XenServer version of qemu, though. I''ve attached > the relevant patch. If you apply this and rebuild qemu then you > should be able to send stuff to the qemu logfile by writing bytes to > port 0x12. Note that this is rate limited, though, and if you hit the > limiter the domain will be quite severely punished.I am using 3.2.1 so it didn''t apply for me, but it was more or less what I was doing anyway, sans the limiter. One thing I did a little differently was to implement logging on a per-cpu basis. Eg in my DomU I write to (0x10 + (cpu & 0xf)), and in qemu I maintain separate buffers for each CPU. This removes the need to maintain a global lock in the DomU, which is hard in Windows when you want to log from multiple drivers and call from any IRQL. Without separating the buffers, I was seeing all sorts of crazy stuff when two CPU''s tried to log at the same time. I have successfully tracked down the problem anyway I think, I was simply declaring too much local data in a routine that ran at DIRQL (the IRQL that hardware interrupts run at), which was (for some reason) causing a complete hard lockup that not even the debugger could break into... I''m now running save+restore in a loop via a script and will see how it goes... James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > > Is there something similar implemented via hvm/qemu that can log viahvm> > log files? That would make life easier. If not, I''ll implement it. > There isn''t at the moment, at least not in the xenbits repositories. > There is one in the XenServer version of qemu, though. I''ve attached > the relevant patch. If you apply this and rebuild qemu then you > should be able to send stuff to the qemu logfile by writing bytes to > port 0x12. Note that this is rate limited, though, and if you hit the > limiter the domain will be quite severely punished. > > I''ve checked that this applies against qemu-xen-unstable.git, and that > the resulting tree at least builds, but I''ve not actually checked that > it works. Upstream qemu is pretty similar to XenServer qemu at the > moment, though, so it should be okay. >I''d love to see this in 3.3.1... what are the chances? The only change I''d like to make is one that I did in my own version which is to keep separate log buffers per io port (with the intent that each DomU CPU writes to a different port), as I was finding the log messages from different CPU''s writing at the same time causing garbled messages. Synchronising access to the io port between different drivers in windows would require a heap of overhead. By making sure in the DomU that each CPU uses port (0x10 + (cpu# & 0x0f)) each CPU only has to disable interrupts on the local CPU while writing the log message. Obviously with #cpu''s > 16 you will get some merged log messages, but this would be less of a problem than the mess I was seeing when logging at a high rate. Basically the change is to use the full range of 16 ports, and declare log_buffer[16][4096] and use the port offset from 0x10 as the first index into the array, and obviously index all the other variables that manage the buffer. I also prefix each message with the CPU number (eg ''06: ''). The throttling would remain global. Having this in 3.3.1 would make it heaps easier to get debug info from people having problems with the GPLPV drivers! I can put my changes in and submit a patch if the above sounds reasonable... Thanks James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 23/11/08 12:32, "James Harper" <james.harper@bendigoit.com.au> wrote:> The only change I''d like to make is one that I did in my own version > which is to keep separate log buffers per io port (with the intent that > each DomU CPU writes to a different port), as I was finding the log > messages from different CPU''s writing at the same time causing garbled > messages. Synchronising access to the io port between different drivers > in windows would require a heap of overhead.Significant overhead compared with trapping to ioemu for every byte transmitted? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On 23/11/08 12:32, "James Harper" <james.harper@bendigoit.com.au>wrote:> > > The only change I''d like to make is one that I did in my own version > > which is to keep separate log buffers per io port (with the intentthat> > each DomU CPU writes to a different port), as I was finding the log > > messages from different CPU''s writing at the same time causinggarbled> > messages. Synchronising access to the io port between differentdrivers> > in windows would require a heap of overhead. > > Significant overhead compared with trapping to ioemu for every byte > transmitted? >Not so much performance overhead, but code overhead in terms of providing a synchronisation mechanism between all the bits of code that might actually write debug output. Under Linux it''s easy enough to export a global spinlock to everything, but it''s not so easy under Windows. James _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel