On Wed, Feb 10, 2016 at 02:33:20PM +0100, Giuseppe Lettieri wrote:
> Il 10/02/2016 12:59, Slawa Olhovchenkov ha scritto:
> > Can you look also on second issue?
> >
> > PS: What need from me? May be open PR?
>
> May you provide some example code that triggers the issue?
This is about 700 lines of code (not very clear), may be I can describe it?
First: this is thread code. I think it impotant point.
All sync with netmap go throw kevent() interface.
Main thread got information about two NIC (determine number of rings)
and spawn 1 balancer thread and NCPU worker thread.
All thread pinned to cores.
Balancer thread open all rings on both NIC separate (i.e. open 6+6
NETMAP fds for hardware rings) and allocate on first NIC and open
NCPU*2 master NETMAP pipes.
All fds register in kevent for read event as
EV_SET(kp, fd, EVFILT_READ, EV_ADD | EV_ENABLE | EV_CLEAR, 0, 0, kidx);
and for future write event as
EV_SET(kp, fd, EVFILT_WRITE, EV_ADD | EV_DISABLE, 0, 0, 0);
after write occur next event will be added:
EV_SET(kp, fd, EVFILT_WRITE, EV_ADD | EV_DISPATCH, 0, 0, 0);
Each worker thread open two slave NETMAP pipes in same maner as above.
Balancer got packet from inside hardware ring, dispatch (zero-copy) to inside
master pipe.
Worker got packet from inside slave pipe and zero-copy to outside slave pipe.
Balancer got packet from outside master pipe and dispatch (zero-copy) to
outside hardware ring.
And vise versa.
After program start I have deadlock instantly (with WITNESS enabled kernel).
W/o WITNESS deadloc occur only under load.
W/o EV_DISABLE/EV_DISPATCH NETAMP don't see output packets.
NETMAP don't forward incoming packet to program when open and attach to
NETMAP do in main thread.