search for: recvmmsg

On NetBSD 6.99.28-CURRENT, nsd 3.2.16 works fine, however nsd 4.0.0 is spinning chewing CPU. The logs show: Nov 28 23:07:00 xxx nsd[466]: sendmmsg failed: Resource temporarily unavailable ktruss shows it getting EAGAIN from sendmmsg(2) over and over again. According to the man page: [EAGAIN|EWOULDBLOCK] The socket is marked non-blocking and the requested

Hello, Linux has a recvmmsg() system call which allows to achieve several recvfrom() at a time. The patch below makes tinc use it (patch against 1.1-pre11). Basically the patch turns the handle_incoming_vpn_data variables into arrays (of size 1 when recvmmsg is not available, and thus compiled the same as before), and makes...

On Wed, Dec 02, 2015 at 12:58:26PM +0100, Samuel Thibault wrote: > Linux has a recvmmsg() system call which allows to achieve several > recvfrom() at a time. The patch below makes tinc use it (patch against > 1.1-pre11). Basically the patch turns the handle_incoming_vpn_data > variables into arrays (of size 1 when recvmmsg is not available, and > thus compiled the same as...

...eed to >> >> return to userspace to tell the guest that we've got a new packet, we >> >> don't win on latency. We might reduce syscall overhead with a >> >> multi-dimensional readv to read multiple packets at once? >> > >> > Sounds like recvmmsg(2). >> >> Could we map this to mergable rx buffers though? >> >> Regards, >> >> Anthony Liguori > > Yes because we don't have to complete buffers in order. What I meant though was for GRO, we don't know how large the received packet is going to...

...eed to >> >> return to userspace to tell the guest that we've got a new packet, we >> >> don't win on latency. We might reduce syscall overhead with a >> >> multi-dimensional readv to read multiple packets at once? >> > >> > Sounds like recvmmsg(2). >> >> Could we map this to mergable rx buffers though? >> >> Regards, >> >> Anthony Liguori > > Yes because we don't have to complete buffers in order. What I meant though was for GRO, we don't know how large the received packet is going to...

...> https://lwn.net/Articles/614348/ > > which preserve things important at high rates like GRO/GSO. Yes, FOU will probably get the highest rates, but it doesn't support even basic encryption, and is harder to set up for normal people :) > > At least for now we could commit the recvmmsg part? > > Not my call. It is a linux only thing, so far as I know. ATM yes. I wouldn't be surprised that other OSes adopt it, though. Samuel

...p. I would not just pull that as it was just some late night hacking... and it turns out the posix time calls I upgraded to to get better than second resolution are not supported on OSX. My ultimate intent was to move to not bottlenecking or dropping packets on the recv buffer ever (but I figured recvmmsg AND threads would be needed), and move tinc queue management to an fq_codel like system, so as to remove apparent network delays when it was bottlenecked on crypto or output. There are also some notes there on how to do tos encapsulation more right in the upcoming ecn'd world.... and I think I...

Guus Sliepen, on Wed 02 Dec 2015 13:53:37 +0100, wrote: > I guess in the future, we want to put a "cork" on the output until all > packets from a single recvmmsg() have been received, so that we can do > sendmmsg() on the resulting outgoing packets. Yes. > > More is yet to come: I'll have a look at extending the tun/tap interface > > to send/receive several packets at a time, and then also using sendmmsg > > will again improve per...

...e, what can we do better than readv? If we need to >> return to userspace to tell the guest that we've got a new packet, we >> don't win on latency. We might reduce syscall overhead with a >> multi-dimensional readv to read multiple packets at once? > > Sounds like recvmmsg(2). Could we map this to mergable rx buffers though? Regards, Anthony Liguori > > Stefan

...e, what can we do better than readv? If we need to >> return to userspace to tell the guest that we've got a new packet, we >> don't win on latency. We might reduce syscall overhead with a >> multi-dimensional readv to read multiple packets at once? > > Sounds like recvmmsg(2). Could we map this to mergable rx buffers though? Regards, Anthony Liguori > > Stefan

...> > > > ping. > > > > > > > Do you mean gso_size is set but gso_type is not? Looks like a bug > > elsewhere. > > > > Thanks > > > > > Yes. > > I could not trace it where it is coming from. > > I see it when doing recvmmsg on raw sockets in the UML vector network > drivers. > I think we need to find the culprit and fix it there, lots of other things can break otherwise. Just printing out skb->dev->name should do the trick, no? > -- > Anton R. Ivanov > Cambridgegreys Limited. Registered in En...

...> > > > ping. > > > > > > > Do you mean gso_size is set but gso_type is not? Looks like a bug > > elsewhere. > > > > Thanks > > > > > Yes. > > I could not trace it where it is coming from. > > I see it when doing recvmmsg on raw sockets in the UML vector network > drivers. > I think we need to find the culprit and fix it there, lots of other things can break otherwise. Just printing out skb->dev->name should do the trick, no? > -- > Anton R. Ivanov > Cambridgegreys Limited. Registered in En...

...ps that will help. Well, converting a sendto call into a sendmsg call is not really "hard" :) What will be really hard is getting multiple packet receive/send over the tap/tun device, because support for it has to be added at the kernel layer first. At least for now we could commit the recvmmsg part? Samuel

...u mean gso_size is set but gso_type is not? Looks like a bug >>>> elsewhere. >>>> >>>> Thanks >>>> >>>> >>> Yes. >>> >>> I could not trace it where it is coming from. >>> >>> I see it when doing recvmmsg on raw sockets in the UML vector network >>> drivers. >>> >> >> I think we need to find the culprit and fix it there, lots of other >> things >> can break otherwise. >> Just printing out skb->dev->name should do the trick, no? > > The pri...

...f daemon fchmod flock ftime fork get_current_dir_name gettimeofday mlockall pselect putenv random select strdup strerror strsignal strtol system unsetenv usleep vsyslog writev], +AC_CHECK_FUNCS([asprintf daemon fchmod flock ftime fork get_current_dir_name gettimeofday mlockall pselect putenv random recvmmsg select strdup strerror strsignal strtol system unsetenv usleep vsyslog writev], [], [], [#include "src/have.h"] ) diff --git a/src/net_packet.c b/src/net_packet.c index e67857c..174db34 100644 --- a/src/net_packet.c +++ b/src/net_packet.c @@ -695,31 +695,26 @@ static node_t *try_har...

..._type is not? Looks like a bug > > > > elsewhere. > > > > > > > > Thanks > > > > > > > > > > > Yes. > > > > > > I could not trace it where it is coming from. > > > > > > I see it when doing recvmmsg on raw sockets in the UML vector network > > > drivers. > > > > > > > I think we need to find the culprit and fix it there, lots of other things > > can break otherwise. > > Just printing out skb->dev->name should do the trick, no? > > The pr...

..._type is not? Looks like a bug > > > > elsewhere. > > > > > > > > Thanks > > > > > > > > > > > Yes. > > > > > > I could not trace it where it is coming from. > > > > > > I see it when doing recvmmsg on raw sockets in the UML vector network > > > drivers. > > > > > > > I think we need to find the culprit and fix it there, lots of other things > > can break otherwise. > > Just printing out skb->dev->name should do the trick, no? > > The pr...

...d and encapsulate step, and being strict FIFOs, gradually accumulate delay until finally they run out of read socket buffer space and start dropping packets. so I had a couple thoughts towards using multiple rx queues in the vtun interface, and/or trying to read more than one packet at a time (via recvmmsg) and do some level of fair queueing and queue management (codel) inside tinc itself. I think that's pretty doable without modifying the protocol any, but I'm not sure of it's value until I saturate some cpu more. (and if you thought recvmsg was complex, look at recvmmsg) C) Moving forw...

...ket processing to scale up past 100Mbit is hard without offloads even on embedded hardware considered "high end". (I was recently testing the peeling[1] feature of cake[2] only to see it cut performance by over half). Getting ANY modern platform past 10gbit is *really* hard. [3] So while recvmmsg and optimizations like that are needed, some harder thinking about the other brick walls coming up would be helpful.. (which is why I stopped) >> > At least for now we could commit the recvmmsg part? >> >> Not my call. It is a linux only thing, so far as I know. > > ATM...

...n by my headaches. Getting per packet > processing to scale up past 100Mbit is hard without offloads even on > embedded hardware considered "high end". In my tests I was getting 800Mbps with common laptop Gb ethernet devices. > >> > At least for now we could commit the recvmmsg part? > >> > >> Not my call. It is a linux only thing, so far as I know. > > > > ATM yes. I wouldn't be surprised that other OSes adopt it, though. > > Given how slow other OSes are evolving, I would not hold my breath, > and retain code paths that worke...