Luwei Cheng
2011-Mar-12 07:59 UTC
[Xen-devel] (Possible) Inaccurate accounting in Xen''s bandwidth control
Hi,
These days I use netperf to evaluate Xen''s network bandwidth control.
I found that somehow the bandwidth is always inaccurate (slightly less than
the promised bandwidth)
After reading the source code of Linux-2.6.3** (domain 0), I am a bit
curious about the following algorithm.
file: /linux-main-dir/drivers/xen/netback/netback.c
============================================================static void
tx_add_credit(netif_t *netif)
{
unsigned long max_burst, max_credit;
/*
* Allow a burst big enough to transmit a jumbo packet of up to
128kB.
* Otherwise the interface can seize up due to insufficient credit.
*/
max_burst = RING_GET_REQUEST(&netif->tx,
netif->tx.req_cons)->size;
max_burst = min(max_burst, 131072UL);
max_burst = max(max_burst, netif->credit_bytes);
/* Take care that adding a new chunk of credit doesn''t wrap to
zero.
*/
max_credit = netif->remaining_credit + netif->credit_bytes;
if (max_credit < netif->remaining_credit)
max_credit = ULONG_MAX; /* wrapped: clamp to ULONG_MAX */
netif->remaining_credit = min(max_credit, max_burst);
}
============================================================
Setting: rate=512Kb/s@30ms, Then,
-----------------------
"netif->credit_bytes" will be 1,920
"netif->credit_usecs" will be 30,000
============================================================Suppose that at some
moment:
-----------------------
"request size" is 1514
"netif->remaining_credit" is 406
-----------------------
Since there''s not enough credits for transmitting, the netif
will be delayed for 30,000 usecs to refill credits (using timer).
============================================================Question:
After the timer wakes up, should the netif get (406+1920) credits, or (1920)
credits?
============================================================In my mind, I think
it should be fair to let the netif get (406+1920)
credits.
However, the above algorithm will eventually give only (1920) credits.
Just wonder whether there''s some undocumented consideration to control
in
this way
(only give 1920 credits)?
Kindly please correct me if my analysis is incorrect.
Thanks for your attention.
Best Regards,
Luwei Cheng
----
Department of Computer Science
The University of Hong Kong
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Keir Fraser
2011-Mar-12 15:44 UTC
Re: [Xen-devel] (Possible) Inaccurate accounting in Xen''s bandwidth control
On 12/03/2011 07:59, "Luwei Cheng" <chengluwei@gmail.com> wrote:> Hi, > > These days I use netperf to evaluate Xen''s network bandwidth control. > I found that somehow the bandwidth is always inaccurate (slightly less than > the promised bandwidth) > > After reading the source code of Linux-2.6.3** (domain 0), I am a bit curious > about the following algorithm. > file: /linux-main-dir/drivers/xen/netback/netback.c > ============================================================> static void tx_add_credit(netif_t *netif) > { > unsigned long max_burst, max_credit; > > /* > * Allow a burst big enough to transmit a jumbo packet of up to 128kB. > * Otherwise the interface can seize up due to insufficient credit. > */ > max_burst = RING_GET_REQUEST(&netif->tx, netif->tx.req_cons)->size; > max_burst = min(max_burst, 131072UL); > max_burst = max(max_burst, netif->credit_bytes);Making the above line be max(max_burst, 2*netif->credit_bytes) would allow the interface to ''carry over'' the previous period''s credit into the following period. Hence allow the interface to get closer to its permitted bandwidth limit in the long term average, at the cost of allowing it to burst at a somewhat higher bandwidth over sub-second timescales. Basically it''s a tradeoff between giving the interface freedom over when it consumes its credits, versus keeping traffic flow smooth and unbursty. -- Keir> /* Take care that adding a new chunk of credit doesn''t wrap to zero. > */ > max_credit = netif->remaining_credit + netif->credit_bytes; > if (max_credit < netif->remaining_credit) > max_credit = ULONG_MAX; /* wrapped: clamp to ULONG_MAX */ > > netif->remaining_credit = min(max_credit, max_burst); > } > ============================================================> > > Setting: rate=512Kb/s@30ms, Then, > ----------------------- > "netif->credit_bytes" will be 1,920 > "netif->credit_usecs" will be 30,000 > ============================================================> Suppose that at some moment: > ----------------------- > "request size" is 1514 > "netif->remaining_credit" is 406 > ----------------------- > Since there''s not enough credits for transmitting, the netif > will be delayed for 30,000 usecs to refill credits (using timer). > ============================================================> Question: > After the timer wakes up, should the netif get (406+1920) credits, or (1920) > credits? > ============================================================> In my mind, I think it should be fair to let the netif get (406+1920) credits. > However, the above algorithm will eventually give only (1920) credits. > > Just wonder whether there''s some undocumented consideration to control in this > way > (only give 1920 credits)? > > Kindly please correct me if my analysis is incorrect. > > Thanks for your attention. > > Best Regards, > Luwei Cheng > ---- > Department of Computer Science > The University of Hong Kong > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel