Hi there ! I am facing a jitter issue that I will try to explain : I have the following configuration : (A) eth0 <--> br0 <--> eth1 (B) I have a small and custom kernel (I removed netfilter, for instance) and I am measuring the time between A and B (using hardware taps, etc.). My bandwidth is quite small (each packet is about 200 bytes and there is a delay of 10ms between each packet). I am measuring both the latency and the jitter (max_latency - min_latency) and this one is quite large. I added ftrace option, and after some (long) investigations, I am currently limiting my study to the bridge part. When I trace only br_handle_frame and br_forward_finish and do my test on 6500 frames I have : - average latency of 39?s - minimum latency of 34 ?s - maximum latency of 81?s Thus, my jitter is 47?s. This may seem low, but this is about two to three times what I expect and I cannot see why a bridge which is only forwarding a frame would have jitter ? If you have any idea, everything is interesting for me. I thought maybe to skb_clone ? Best Regards, JM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.linux-foundation.org/pipermail/bridge/attachments/20101012/8bc4ab8a/attachment.htm
On Tue, 12 Oct 2010 16:31:36 +0200 Jean-Michel Hautbois <jhautbois at gmail.com> wrote:> Hi there ! > > I am facing a jitter issue that I will try to explain : > I have the following configuration : > > (A) eth0 <--> br0 <--> eth1 (B) > > I have a small and custom kernel (I removed netfilter, for instance) and I > am measuring the time between A and B (using hardware taps, etc.). > My bandwidth is quite small (each packet is about 200 bytes and there is a > delay of 10ms between each packet). > I am measuring both the latency and the jitter (max_latency - min_latency) > and this one is quite large. > > I added ftrace option, and after some (long) investigations, I am currently > limiting my study to the bridge part. > > When I trace only br_handle_frame and br_forward_finish and do my test on > 6500 frames I have : > - average latency of 39?s > - minimum latency of 34 ?s > - maximum latency of 81?s > > Thus, my jitter is 47?s. > This may seem low, but this is about two to three times what I expect and I > cannot see why a bridge which is only forwarding a frame would have jitter ? > If you have any idea, everything is interesting for me. > I thought maybe to skb_clone ? > > Best Regards, > JMYou probably have NAPI and/or interrupt coalescing setting enabled on one or both of the devices. --
2010/10/12 Stephen Hemminger <shemminger at linux-foundation.org>> > Thus, my jitter is 47?s. > > This may seem low, but this is about two to three times what I expect and > I > > cannot see why a bridge which is only forwarding a frame would have > jitter ? > > If you have any idea, everything is interesting for me. > > I thought maybe to skb_clone ? > > You probably have NAPI and/or interrupt coalescing setting > enabled on one or both of the devices. > > -- >Hi Stephen, I have NAPI on my ethernet device, at least. I will check for interrupt coalescing. AFAIK, there is a skb_clone followed by a kfree in netif_receive_skb (the frame coming from eth0 to br0). There is a jitter in this call of "freeing memory". I am looking at the path, in order to see why there is a clone. BR, JM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.linux-foundation.org/pipermail/bridge/attachments/20101013/1b563c31/attachment.html
2010/10/13 Jean-Michel Hautbois <jhautbois at gmail.com>> 2010/10/12 Stephen Hemminger <shemminger at linux-foundation.org> > >> > Thus, my jitter is 47?s. >> > This may seem low, but this is about two to three times what I expect >> and I >> > cannot see why a bridge which is only forwarding a frame would have >> jitter ? >> > If you have any idea, everything is interesting for me. >> > I thought maybe to skb_clone ? >> >> You probably have NAPI and/or interrupt coalescing setting >> enabled on one or both of the devices. >> >> -- >> > > Hi Stephen, > > I have NAPI on my ethernet device, at least. > I will check for interrupt coalescing. > AFAIK, there is a skb_clone followed by a kfree in netif_receive_skb (the > frame coming from eth0 to br0). > There is a jitter in this call of "freeing memory". I am looking at the > path, in order to see why there is a clone. > > BR, > JM >I have to tell I am working on a mpc52xx and am currently only interested in the eth->br path for the moment. JM -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.linux-foundation.org/pipermail/bridge/attachments/20101013/5551e5e8/attachment.htm