Xen devel - May 2006 - how can native driver response quickly for low FIFO device

Keir:
	For those devices like UART which has very few FIFOs and no DMA
support, how can scheduler (sedf?) help us to improve the response of
native device driver in driver domain? Otherwise the receive data may be
overwritten in driver domain and then makes the device unusable.
thx,eddie

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 25 May 2006, at 10:11, Dong, Eddie wrote:
> 	For those devices like UART which has very few FIFOs and no DMA
> support, how can scheduler (sedf?) help us to improve the response of
> native device driver in driver domain? Otherwise the receive data may 
> be
> overwritten in driver domain and then makes the device unusable.
You can have a receive FIFO as big as you like: it doesn''t need to be 
visible to the HVM guest. If the guest has configured the UART with an 
''N-byte FIFO'' then allocate the FIFO as big as you like and
drive the
UART device model logic such that if you have more than N bytes 
currently in the FIFO it looks to the guest (and the core of the device 
model) as if there are exactly N bytes currently in the FIFO.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 25 May 2006, at 11:30, Keir Fraser wrote:
>> 	For those devices like UART which has very few FIFOs and no DMA
>> support, how can scheduler (sedf?) help us to improve the response of
>> native device driver in driver domain? Otherwise the receive data may 
>> be
>> overwritten in driver domain and then makes the device unusable.
>
> You can have a receive FIFO as big as you like: it doesn''t need to
be
> visible to the HVM guest. If the guest has configured the UART with an 
> ''N-byte FIFO'' then allocate the FIFO as big as you like
and drive the
> UART device model logic such that if you have more than N bytes 
> currently in the FIFO it looks to the guest (and the core of the 
> device model) as if there are exactly N bytes currently in the FIFO.
The alternative is to implicitly have the extra buffering in the OS. 
This may allow also back pressure to be applied to the sender (UNIX 
pipe is full; TCP receive window is closed; etc.). This only works if 
the device model is ''pulling'' data into its receive FIFO
(e.g., read()
system call) rather than having data ''pushed'' to it.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

O, sorry for the confuse message. Here I mean the issue in native device
(e.x. dom0 UART), not HVM virtual UART.
When hardware UART device receives a byte, it will send pirq to dom0,
although dom0 is not blocking but it may not get scheduled in at
instance. Say it needs to wait for another domain (domU or domVMX) to
run out of time slice, say 10ms in default scheduler. Within this 10ms,
the hardware UART may receive another tens of bytes before dom0 UART
device driver can really read the data and copy to driver buffer. So in
this case the hardware UART device will become "receiving data
overwritten",  which means receiving data lose for us :-( 
Is there anybody ever tried to use ttyS0 succesfully when multiple
domain is running on same LP?
Eddie

Keir Fraser wrote:
> On 25 May 2006, at 10:11, Dong, Eddie wrote:
> 
>> 	For those devices like UART which has very few FIFOs and no DMA
>> support, how can scheduler (sedf?) help us to improve the response of
>> native device driver in driver domain? Otherwise the receive data
>> may be overwritten in driver domain and then makes the device
>> unusable. 
> 
> You can have a receive FIFO as big as you like: it doesn''t need to
be
> visible to the HVM guest. If the guest has configured the UART with an
> ''N-byte FIFO'' then allocate the FIFO as big as you like
and drive the
> UART device model logic such that if you have more than N bytes
> currently in the FIFO it looks to the guest (and the core of the
> device model) as if there are exactly N bytes currently in the FIFO.
> 
>   -- Keir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 25 May 2006, at 14:08, Dong, Eddie wrote:
> O, sorry for the confuse message. Here I mean the issue in native 
> device
> (e.x. dom0 UART), not HVM virtual UART.
> When hardware UART device receives a byte, it will send pirq to dom0,
> although dom0 is not blocking but it may not get scheduled in at
> instance. Say it needs to wait for another domain (domU or domVMX) to
> run out of time slice, say 10ms in default scheduler. Within this 10ms,
> the hardware UART may receive another tens of bytes before dom0 UART
> device driver can really read the data and copy to driver buffer. So in
> this case the hardware UART device will become "receiving data
> overwritten",  which means receiving data lose for us :-(
> Is there anybody ever tried to use ttyS0 succesfully when multiple
> domain is running on same LP?
There''s a 128-byte receive buffer. Bytes are greedily taken from the 
UART FIFO in interrupt context and written to the software receive 
buffer. If overflow were a problem we could obviously quite easily make 
that receive buffer substantially larger.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> 
> There''s a 128-byte receive buffer. Bytes are greedily taken from
the
> UART FIFO in interrupt context and written to the software receive
> buffer. If overflow were a problem we could obviously quite easily
> make that receive buffer substantially larger.
But this need the participate of processor in dom0. If the physical
processor is running for domU, and domU is waiting for its time slice
(descheduled), how can the bytes be taken to software receive buffer?
Eddie

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 25 May 2006, at 14:53, Dong, Eddie wrote:
>>
>> There''s a 128-byte receive buffer. Bytes are greedily taken
from the
>> UART FIFO in interrupt context and written to the software receive
>> buffer. If overflow were a problem we could obviously quite easily
>> make that receive buffer substantially larger.
> But this need the participate of processor in dom0. If the physical
> processor is running for domU, and domU is waiting for its time slice
> (descheduled), how can the bytes be taken to software receive buffer?
The bytes are taken from the UART in interrupt context inside Xen. Data 
will only build up inside the UART FIFO if interrupts are disabled for 
long periods.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> On 25 May 2006, at 14:53, Dong, Eddie wrote:
> 
> The bytes are taken from the UART in interrupt context inside Xen.
> Data will only build up inside the UART FIFO if interrupts are
> disabled for long periods.
> 
Oo, understand UART is handled by Xen, Is this because of this reason? 
or for debug/console use, and how about other devices?
If a user has an PCI card with 8 UART in the box, will we have problem?
thx,eddie

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 25 May 2006, at 15:21, Dong, Eddie wrote:
> Oo, understand UART is handled by Xen, Is this because of this reason?
> or for debug/console use, and how about other devices?
> If a user has an PCI card with 8 UART in the box, will we have problem?
> thx,eddie
Oh, now I see your point. Yes, for any serial device not handled by Xen 
you could have a scheduling problem. Other examples include telephony 
interface cards and their drivers (e.g., zaptel). Higher-rate UARTs do 
tend to support bigger FIFOs in hardware though. Best bet if it really 
matters is to dedicate a logical processor to the domain that owns the 
hardware.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> On 25 May 2006, at 15:21, Dong, Eddie wrote:
> 
>> Oo, understand UART is handled by Xen, Is this because of this
>> reason? or for debug/console use, and how about other devices?
>> If a user has an PCI card with 8 UART in the box, will we have
>> problem? thx,eddie
> 
> Oh, now I see your point. Yes, for any serial device not handled by
> Xen you could have a scheduling problem. Other examples include
> telephony interface cards and their drivers (e.g., zaptel).
> Higher-rate UARTs do tend to support bigger FIFOs in hardware though.
> Best bet if it really matters is to dedicate a logical processor to
> the domain that owns the hardware.
> 
Yes :-)
 Fortunately most devices in modern computer has either big FIFO or DMA
support.
Those kind of low FIFO device will be headache in VM environment.
Dedicated LP is a good choice, further more HT can take the advantage
here :-)
Eddie

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel