Howdy, This is the first patch to move the console code out of Xend and into a separate daemon. In this patch I''ve included the daemon and a front-end. I''ve also have the console code removed from Xend locally but I wanted to clarify a few things first before submitting that. Here''s how it works: 1) Daemon sits on control channel (via xcs) waiting for console messages 2) Buffers all console data received 3) Publishes the location of a tty that can be used for reading in /console/<domid>/tty 4) Reads a value from the store to determine what the limit of console data should be (/console/<domid>/limit) to prevent DoS attacks Using it is quite simple, just run consoled and use the xc_console program to connect to the console. It uses syslog for logging. My current Xend patch just removes all of the console handling code from Xend and executes /usr/libexec/xen/xc_console with the right domid for the xm console command. I choose to use an external program instead of the existing console_client.py because I already had the code and thought it would be useful to have an exec''able program so we could drop privileges. Happy to use the console_client.py code though if that''s desired. The only current issue is that I''d like to store the tty info in <home>/console/tty since it''s just another device. However, Xend uses UUIDs so figuring out the home path is difficult. Are we going to stick with this home path convention? It would be a lot nicer if we used domids in the home path so that we could easily lookup info without having to search the whole /domain directory. We can still keep UUIDs within the home path. I''ll add that to my patch if that''s agreeable. It will be easy to modify consoled to support the new kernel console driver when it''s ready too. Regards, Anthony Liguori Signed-off-by: Anthony Liguori <aliguori@us.ibm.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 7/26/05, Anthony Liguori <aliguori@us.ibm.com> wrote:> Howdy, > > This is the first patch to move the console code out of Xend and into a > separate daemon. In this patch I''ve included the daemon and a > front-end. I''ve also have the console code removed from Xend locally > but I wanted to clarify a few things first before submitting that. >Anthony, there is a small bug in xc_console.c::console_loop() -- if (len == 0 && len == -1) { -- And is there a better name than xc_console? that is great to see this code out. i am waiting to see the full patch. regards, aq _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
aq wrote:>On 7/26/05, Anthony Liguori <aliguori@us.ibm.com> wrote: > > >Anthony, there is a small bug in xc_console.c::console_loop() > >-- >if (len == 0 && len == -1) { >-- > >And is there a better name than xc_console? > >Fixed locally. Thanks aq! Regards, Anthony Liguori>that is great to see this code out. i am waiting to see the full patch. > > >regards, >aq > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 26 Jul 2005, at 05:09, Anthony Liguori wrote:> > This is the first patch to move the console code out of Xend and into > a separate daemon. In this patch I''ve included the daemon and a > front-end. I''ve also have the console code removed from Xend locally > but I wanted to clarify a few things first before submitting that.I checked in consoled. Can you send any further fixes, and code to plumb through xend? Is it possible to make it configurable where console i/o goes (e.g., tcp socket as an option instead of tty?). -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 26 Jul 2005, at 05:09, Anthony Liguori wrote:> My current Xend patch just removes all of the console handling code > from Xend and executes /usr/libexec/xen/xc_console with the right > domid for the xm console command. I choose to use an external program > instead of the existing console_client.py because I already had the > code and thought it would be useful to have an exec''able program so we > could drop privileges. Happy to use the console_client.py code though > if that''s desired.You can exec the python interpreter? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> > This is the first patch to move the console code out of > Xend and into > > a separate daemon. In this patch I''ve included the daemon and a > > front-end. I''ve also have the console code removed from > Xend locally > > but I wanted to clarify a few things first before submitting that. > > I checked in consoled. Can you send any further fixes, and > code to plumb through xend? > > Is it possible to make it configurable where console i/o goes > (e.g., tcp socket as an option instead of tty?).I guess it would be nice to have tcp sockets as an option (with a optional restriction to localhost access), but not a huge deal. In any event, xend needs to know of, or prefereable be in control of, which tty or port gets allocated. What I''d really like to see this console deamon do is to implement a FIFO buffer of the last N megabytes of console output, hence providing buffering while the tty is disconnected. Regardless of whether the tty is connected, it would be good to keep the last M megabytes stored on disk in /var/log/xen-myvmname. Thanks, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt wrote:>What I''d really like to see this console deamon do is to implement a >FIFO buffer of the last N megabytes of console output, hence providing >buffering while the tty is disconnected. Regardless of whether the tty >is connected, it would be good to keep the last M megabytes stored on >disk in /var/log/xen-myvmname. > >Today it buffers N bytes worth of data in memory regardless of whether the tty is connected (or reading from). How much it buffers is configurable via the store (by default the buffer has no limit so ensure console data is never lost). Keeping a history wouldn''t be a hard change. I''ll submit some patches. Regards, Anthony Liguori>Thanks, >Ian > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser wrote:> > On 26 Jul 2005, at 05:09, Anthony Liguori wrote: > >> >> This is the first patch to move the console code out of Xend and into >> a separate daemon. In this patch I''ve included the daemon and a >> front-end. I''ve also have the console code removed from Xend locally >> but I wanted to clarify a few things first before submitting that. > > > I checked in consoled. Can you send any further fixes, and code to > plumb through xend? > > Is it possible to make it configurable where console i/o goes (e.g., > tcp socket as an option instead of tty?).We can make it listen on tcp sockets by default or we can add another layer to expose the tty''s as tcp sockets. I prefer the later. Thoughts? Regards, Anthony Liguori> -- Keir > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> Today it buffers N bytes worth of data in memory regardless > of whether the tty is connected (or reading from). How much > it buffers is configurable via the store (by default the > buffer has no limit so ensure console data is never lost).If you connect and disconnect do you continue from where you left off or get sent the last N bytes again? I think the former behaviour is definitely preferable, or at the very least should be a configuration option. Please can you explain how tty''s get allocated. Thanks, Ian> Keeping a history wouldn''t be a hard change. I''ll submit > some patches. > > Regards, > > Anthony Liguori > > >Thanks, > >Ian > > > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt wrote:>>Today it buffers N bytes worth of data in memory regardless >>of whether the tty is connected (or reading from). How much >>it buffers is configurable via the store (by default the >>buffer has no limit so ensure console data is never lost). >> >> > >If you connect and disconnect do you continue from where you left off or >get sent the last N bytes again? > >You continue where you left off.>I think the former behaviour is definitely preferable, or at the very >least should be a configuration option. > >What I was thinking is having the following layout in the store: /domain/<uuid>/console/tty /domain/<uuid>/console/limit /domain/<uuid>/console/history tty is obviously the tty for the domain (if tty doesn''t exist, it means someone is connected already) limit is the maximum amount of data (in bytes) that will be buffered history is the amount of history (in bytes) that will be saved. When a client reconnects, they will first receive whatever''s in the history. Does this seem agreeable?>Please can you explain how tty''s get allocated. > >In the current scheme, a tty is allocated for each domain whenever data arrives for the domain or if consoled detects a new domain was created. It currently polls to see if new domains were created every second. You can avoid the race condition by signalling consoled which will break it out of it''s select loop. Not very pretty but it solves the problem of something like xm create -c. Regards, Anthony Liguori>Thanks, >Ian > > > >>Keeping a history wouldn''t be a hard change. I''ll submit >>some patches. >> >>Regards, >> >>Anthony Liguori >> >> >> >>>Thanks, >>>Ian >>> >>> >>> >>> >>> >> >> > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> What I was thinking is having the following layout in the store: > > /domain/<uuid>/console/tty > /domain/<uuid>/console/limit > /domain/<uuid>/console/history > > tty is obviously the tty for the domain (if tty doesn''t > exist, it means someone is connected already) > > limit is the maximum amount of data (in bytes) that will be buffered > > history is the amount of history (in bytes) that will be > saved. When a client reconnects, they will first receive > whatever''s in the history. > > Does this seem agreeable?Having ''history'' when you reconnect seems a bit over the top -- I''d make this a ''phase 2'' thing. However, having a parameter which indicates the amount of history that will be logged to disk might be useful. (whenever the file reaches size X, throw away the first 20%.)> >Please can you explain how tty''s get allocated. > > > In the current scheme, a tty is allocated for each domain > whenever data arrives for the domain or if consoled detects a > new domain was created. > It currently polls to see if new domains were created every > second. You can avoid the race condition by signalling > consoled which will break it out of it''s select loop. Not > very pretty but it solves the problem of something like xm create -c.We definitely need xenstore support for registering watchers for items that don''t exist yet. Having to poll is daft. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Pratt wrote:>>Does this seem agreeable? >> >> > >Having ''history'' when you reconnect seems a bit over the top -- I''d make >this a ''phase 2'' thing. > >However, having a parameter which indicates the amount of history that >will be logged to disk might be useful. (whenever the file reaches size >X, throw away the first 20%.) > >Ah, I see, I wasn''t following what the disk logging was for. Makes perfect sense now.>We definitely need xenstore support for registering watchers for items >that don''t exist yet. Having to poll is daft. > >Rusty implemented support for this already. Hopefully Christian can pull it soon so we can cleanup the balloon and console stuff to just watch non-existant nodes :-) Regards, Anthony Liguori>Ian > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Aug 02, 2005 at 06:45:25PM +0100, Ian Pratt wrote:> We definitely need xenstore support for registering watchers for items > that don''t exist yet. Having to poll is daft.I''ll pull this across from Rusty''s repo, he''s added it today/yesterday... christian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, Aug 02, 2005 at 12:49:33PM -0500, Anthony Liguori wrote:> Rusty implemented support for this already. Hopefully Christian can > pull it soon so we can cleanup the balloon and console stuff to just > watch non-existant nodes :-)Done. It''s odd that his changes appear to be 5 days old in your repository, yet they weren''t there when I checked yesterday ;-) christian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Christian Limpach wrote:>On Tue, Aug 02, 2005 at 12:49:33PM -0500, Anthony Liguori wrote: > > >>Rusty implemented support for this already. Hopefully Christian can >>pull it soon so we can cleanup the balloon and console stuff to just >>watch non-existant nodes :-) >> >> > >Done. It''s odd that his changes appear to be 5 days old in your >repository, yet they weren''t there when I checked yesterday ;-) > >Rusty forgot to hg push :-) Regards, Anthony Liguori> christian > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2005-08-02 at 11:30 -0500, Anthony Liguori wrote:> What I was thinking is having the following layout in the store: > > /domain/<uuid>/console/tty > /domain/<uuid>/console/limit > /domain/<uuid>/console/historyInteresting question. None of this is actually useful to the domain, although it''s clearly related to the domain. A better place might be: /tool/consoled/<uuid>/ I was thinking of a console rewrite which puts the list of shared pages in /domain/<uuid>/console/. I''ll be looking at this once the rest of the store stuff is merged (soon, at the rate Christian is working). Rusty. -- A bad analogy is like a leaky screwdriver -- Richard Braakman _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, 2005-08-02 at 13:58 -0500, Anthony Liguori wrote:> Christian Limpach wrote: > >Done. It''s odd that his changes appear to be 5 days old in your > >repository, yet they weren''t there when I checked yesterday ;-) > > > > > Rusty forgot to hg push :-)... and, of course, took Friday off, confident that I had done the changes required and life was good. Rusty. -- A bad analogy is like a leaky screwdriver -- Richard Braakman _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> On Tue, 2005-08-02 at 11:30 -0500, Anthony Liguori wrote: > > What I was thinking is having the following layout in the store: > > > > /domain/<uuid>/console/tty > > /domain/<uuid>/console/limit > > /domain/<uuid>/console/history > > Interesting question. None of this is actually useful to the > domain, although it''s clearly related to the domain. > > A better place might be: > > /tool/consoled/<uuid>/ > > I was thinking of a console rewrite which puts the list of > shared pages in /domain/<uuid>/console/. I''ll be looking at > this once the rest of the store stuff is merged (soon, at the > rate Christian is working).Synchornize with Robert Read, who''s working up a patch to move the console onto it''s own ring buffer. This will be set up by the domain builder so we can get immediate start of day console access, rather than being boot strapped via xenbus. Thanks, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Wed, 2005-08-03 at 10:32 +0100, Ian Pratt wrote:> > I was thinking of a console rewrite which puts the list of > > shared pages in /domain/<uuid>/console/. I''ll be looking at > > this once the rest of the store stuff is merged (soon, at the > > rate Christian is working). > > Synchornize with Robert Read, who''s working up a patch to move the > console onto it''s own ring buffer. This will be set up by the domain > builder so we can get immediate start of day console access, rather than > being boot strapped via xenbus.Hi Robert! I have some old code I wrote for the IBM Research Hypervisor which implements a console. I stole that ring buffer code for the xenstore, in fact. I don''t think externally-supplied pages are necessary. I was thinking of writing the page (or pages) into the store immediately (since we don''t need locks, or to wait for a reply). I think we want more than one page because my boot messages almost hit 8k to the first login prompt, and the idea of requiring a console daemon repulses me. Robert? Rusty. -- A bad analogy is like a leaky screwdriver -- Richard Braakman _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 3 Aug 2005, at 11:58, Rusty Russell wrote:> I have some old code I wrote for the IBM Research Hypervisor which > implements a console. I stole that ring buffer code for the xenstore, > in fact. > > I don''t think externally-supplied pages are necessary. I was thinking > of writing the page (or pages) into the store immediately (since we > don''t need locks, or to wait for a reply). I think we want more than > one page because my boot messages almost hit 8k to the first login > prompt, and the idea of requiring a console daemon repulses me.We want an initial console page initialised outside xenbus because we want to get console *early*. We don;t trust xenbus/xenstore enough just yet to enable them in the wild with no console diagnostics available. :-) We''ll add support for multiple consoles and multi-page console buffers via the xenbus later on. This is just supposed to be a really simple ring buffer of raw console bytes, with the ring page initialised same way as the xenstore page. This only differs from what Robert may be thinking about already in that I don;t think we care about muxing multiple consoles onto a single ring. We''d just set up multiple ring buffers via xenbus. But that is definitely ''phase 2''! -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> I don''t think externally-supplied pages are necessary. > I was thinking of writing the page (or pages) into the store > immediately (since we don''t need locks, or to wait for a > reply).We want to be able to get console output very early on, before xenbus has come up. However, it makes sense for xenbus to learn about the console buffer once its up, as we''ll want to use it (in future) for establishing multiple tty''s etc.> I think we want more than one page because my boot > messages almost hit 8k to the first login prompt, and the > idea of requiring a console daemon repulses me.It only really needs to be big enough to absorb all the messages before Linux''s console buffering code comes up, which isn''t very much, certainly <4KB. My view is that we should continue using the same buffer once xenbus comes up, hence there should be a simple multiplexing protocol. Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> It only really needs to be big enough to absorb all the messages before > Linux''s console buffering code comes up, which isn''t very much, > certainly <4KB. > > My view is that we should continue using the same buffer once xenbus > comes up, hence there should be a simple multiplexing protocol.The kernel will end up allocating a buffer per console anyway, usually at least a page in size. May as well just make that buffer visible via xenstore and have console daemon directly pull the bytes out of it. This also avoids the need for framing infomation, so no packet-ising of data within the console ring buffers, which also makes recovery and suspend/resume somewhat easier (no need to resync ring indexes -- the new/restarted daemon can just start reading out of the ring buffer where the previous daemon left off with no potentially fragile recover/restart protocol). Bottom line: I''d like simple, correct and highly available console ahead of features like multi-headed console driver or xenbus-enabled console. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel