Xen devel - Sep 2004 - confusion on front end driver and allox_rx_buffers.

In the frontend ethernet driver, there is a ring buffer of buffers 
pointers -- skbs are used on linux. 

np->rx_resp_cons is I think the number that says which buffer we are 
consuming, and increases from 0 -> infinity. 
np->rx->req_prod is Xen''s count of which buffer it has produced,
and it
also increase from 0 -> infinity. Driver uses good old fashioned 
power-of-two ring sizes and masks to to turn these counts into indices in 
the ring. 

Your goal is to consume buffers from the rx ring as Xen fills them. You 
want to try to catch np->rx_resp_cons to np->rx->req_prod (I assume).
You
get interrupts as buffers become available, i.e. are filled in by Xen. 

Initially, network startup calls network_alloc_rx_buffers to set up the 
ring with a set of skbs, and hands MFNs of these skbs to Xen in a shared 
array that implements the ring buffer. 

Initially all counters are zero (cons and prod).

Pretend we get a packet and netif_poll gets called. I expect (?) that
np->rx->req_prod is 1. netif_poll will eat the buffer, and now the ring
has had a buffer consumed, so netif_poll will call
network_alloc_rx_buffers to refill the ring.

I see this:

    do {
        skb = dev_alloc_skb(RX_BUF_SIZE);
	.
	.
	.
        id = GET_ID_FROM_FREELIST(np->rx_skbs);

        np->rx_skbs[id] = skb;
        
        np->rx->ring[MASK_NETIF_RX_IDX(i)].req.id = id;
        .
	.
        nr_pfns++;
    }
    while ( (++i - np->rx_resp_cons) != NETIF_RX_RING_SIZE );

OK, the question: I only consumed one buffer, why is this code refilling 
the whole ring this way? What''s happening to me is that the id = 
GET_ID_FROM_FREELIST is failing badly because the do loop is running from 
2 to NETIF_RX_RING_SIZE, which is wrong. 

I think I''m getting caught between the differences in Plan 9 and Linux
net
interrupt handling, which I don''t fully understand yet, but some of
this
ether xen front end code has me confused. 

Comments and clarification are most welcome :-)

ron
p.s. and yes, I am at the DHCP step and it''s damn near working. 


-------------------------------------------------------
This SF.Net email is sponsored by: thawte''s Crypto Challenge Vl
Crack the code and win a Sony DCRHC40 MiniDV Digital Handycam
Camcorder. More prizes in the weekly Lunch Hour Challenge.
Sign up NOW http://ad.doubleclick.net/clk;10740251;10262165;m
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Tue, Sep 14, 2004 at 10:09:58PM -0600, Ronald G. Minnich
wrote:
> np->rx_resp_cons is I think the number that says which buffer we are 
> consuming, and increases from 0 -> infinity. 
Yes.  But note that it is not part of the interface to Xen.
> np->rx->req_prod is Xen''s count of which buffer it has
produced, and it
> also increase from 0 -> infinity.
No.  rx->req_prod is the index of the last buffer we have provided
to Xen.  rx->resp_prod is what you described, i.e. the index of the
buffer Xen has filled last.

There are two sets of indexes, one to pass buffers to Xen and one
for Xen to pass buffers back.
> Your goal is to consume buffers from the rx ring as Xen fills them. You 
> want to try to catch np->rx_resp_cons to np->rx->req_prod (I
assume). You
                                        to
np->rx->resp_prod
> get interrupts as buffers become available, i.e. are filled in by Xen. 
> 
> Pretend we get a packet and netif_poll gets called. I expect (?) that
> np->rx->req_prod is 1. netif_poll will eat the buffer, and now the
ring
np->rx->req_prod should be equal to NETIF_RX_RING_SIZE or however many
buffers you have provided to Xen initially.  If you provide less than
NETIF_RX_RING_SIZE, you have to have another exit out of the loop...

    christian



-------------------------------------------------------
This SF.Net email is sponsored by: thawte''s Crypto Challenge Vl
Crack the code and win a Sony DCRHC40 MiniDV Digital Handycam
Camcorder. More prizes in the weekly Lunch Hour Challenge.
Sign up NOW http://ad.doubleclick.net/clk;10740251;10262165;m
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> OK, the question: I only consumed one buffer, why is this code refilling 
> the whole ring this way? What''s happening to me is that the id = 
> GET_ID_FROM_FREELIST is failing badly because the do loop is running from 
> 2 to NETIF_RX_RING_SIZE, which is wrong. 
The key insight is that you won''t necessarily be getting an interrupt
callback for every packet received. e.g., under high load multiple
interrupts end up getting collapsed together into one callback.

Also, you''ll notice that at the top of the refill function we
don''t
enter the refill loop unless there is a decent batch of work to be
done -- this amortizes the cost of the MEMOP_decrease_reservation
hypercall.

 -- Keir
> I think I''m getting caught between the differences in Plan 9 and
Linux net
> interrupt handling, which I don''t fully understand yet, but some
of this
> ether xen front end code has me confused. 
> 
> Comments and clarification are most welcome :-)

-------------------------------------------------------
This SF.Net email is sponsored by: thawte''s Crypto Challenge Vl
Crack the code and win a Sony DCRHC40 MiniDV Digital Handycam
Camcorder. More prizes in the weekly Lunch Hour Challenge.
Sign up NOW http://ad.doubleclick.net/clk;10740251;10262165;m
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

On Wed, 15 Sep 2004, Christian Limpach wrote:
> No.  rx->req_prod is the index of the last buffer we have provided
> to Xen.  rx->resp_prod is what you described, i.e. the index of the
> buffer Xen has filled last.
ok, thanks, that just cleared the whole picture up. Don''t know how I 
missed that one. 

I''m now getting packets :-)

ron


-------------------------------------------------------
This SF.Net email is sponsored by: thawte''s Crypto Challenge Vl
Crack the code and win a Sony DCRHC40 MiniDV Digital Handycam
Camcorder. More prizes in the weekly Lunch Hour Challenge.
Sign up NOW http://ad.doubleclick.net/clk;10740251;10262165;m
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel