A couple of people have asked about using the low-level netchannel2 ring protocol in other device classes. For their benefit, here''s a quick summary of how it works. The ring protocol itself is fairly straightforward. A ring pair consists of: -- A power-of-two set of consumer pages. -- A power-of-two set of producer pages. -- A single control page, which contains the ring indexes and so forth. -- An event channel port which is used for notifications. The consumer pages for one end of the ring will be the producer pages for the other end. All of the rings are allocated by the frontend-like end of the ring, and will be mapped into the backend-like end. For the master ring, the grant references and event channel references are communicated through xenstore in the usual way. Bypass rings are then negotiated over the master ring. A message transmitted on one of these rings consists of an 8-bit type, 8 bits of flags, and some bytes of payload. The entire message must be a multiple of eight bytes, and the payload should be padded to the appropriate size. It can wrap around the end of the ring, but a single message cannot be larger than the size of the ring. Each ring is a two-pointer ring buffer, with two event pointers in the same style as the existing netchannel1 ring pointers. An endpoint is expected to notify its peer over the event channel whenever it moves a data pointer over the matching event pointer (so if you move the producer pointer past the producer event pointer you need to notify your peer). rings.c contains a bunch of functions for manipulating the rings. The most important, if you''re looking to use this ring protocol, are: Sending messages: -- nc2_send_message() -- This copies a message to the ring, but doesn''t update the producer pointer. The message is therefore not available to the other end (so you can send a batch of messages and make them visible in one go, which makes scheduling a bit easier). -- nc2_flush_ring() -- Update the producer pointer following a batch of messages. All of the messages queued up with nc2_send_message() are made available to the other end. Returns 1 if you need to notify the remote, and 0 otherwise. -- nc2_can_send_payload_bytes() -- Check whether there''s space on the producer ring to send a message of a given size. -- nc2_reserve_payload_bytes() -- Reserve space on the ring for a given message. Calling this is entirely optional, but it can sometimes be helpful to avoid getting an error at an inconvenient time. Receiving messages is a little bit more low-level. The user is expected to implement their own message loop in a tasklet or some such which copies incoming messages out of the ring and into local storage before processing them (for netchannel2, this is done in nc2_poll()). There are a couple of helpers for this: -- nc2_copy_from_ring() -- Copy bytes from the ring into a local buffer. -- nc2_final_check_for_messages() -- Check whether there are any unconsumed messages, and set a producer event so that we get notified when new ones are produced. -- nc2_finish_messages() -- Tell the other end we''ve finished with the messages it sent, and update the consumer pointers. Returns 1 if you need to notify the remote, or 0 otherwise. This could be combined with nc2_final_check_for_messages(), but splitting them out made things a bit clearer. The main downside of this scheme, compared to netchannel1-style rings, is that it''s hard to guarantee that there''s space on the ring for response messages. This means that, when a domain receives a message, it may need to buffer up the acknowledgement (assuming that the message requires an ACK). That in turn means that the number of outstanding messages of any given type needs to be bounded somehow (to avoid potentially requiring an infinite amount of memory). The higher-level protocol is responsible for doing this. (netchannel1 doesn''t have this problem, because the space used for sending the message can later be used to send its response. This implicitly limits the number of outstanding messages to the size of the ring.) You also have to be a little bit careful that the frontend can''t consume unbounded amounts of high-priority CPU in the backend (and vice-versa, if you''re allowing untrusted backends, e.g. for bypass rings). There are two components to this: -- The interrupt needs to be disabled when you''re not expecting it to fire, so that the peer can''t just sit and spin calling notify_remote_via_irq() and make you spin in an interrupt handler. -- There needs to be some kind of rate-limiting on actual requests. If the protocol *only* exposes real hardware, it''s likely that physical limitations will be sufficient. If it occasionally processes a request without going to the hardware (e.g. interdomain traffic, error cases) then you need to be a little bit careful. These problems aren''t really specific to the ring protocol used (it''s a general weakness of putting backends in domains rather than in the hypervisor), but they''re still worth bearing in mind. Steven. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel