freebsd stable - Oct 2016 - 11.0 stuck on high network load

On Thu, Oct 13, 2016 at 05:06:00PM +0200, Julien Charbon wrote:
> >> will give you that trace in the core, and without INVARIANT then
it is
> >> better to use dtrace:
> >>
> >> $ cat tcp-twstart-dropped.d
> >> fbt::tcp_twstart:entry
> >> /args[0]->t_inpcb->inp_flags & 0x04000000/
> >> {
> >>   stack();
> >>   printf("INP_DROPPED in tcp_twstart: %x",
args[0]->t_inpcb->inp_flags);
> >> }
> > 
> > Same code may be insert there too, IMHO.
> 
>  Hmm, I don't think so:
> 
>  - If you have INVARIANT, the kernel will panic in tcp_twstart() or
> tcp_detach() and you will have everything you need to debug.
>  - If you don't, dtrace is the right tool to use in all cases anyway.
dtrace don't executed in may case w/ diagnostic "dtrace: processing
aborted: Abort due to systemic unresponsiveness". This is for
tcp_close. May be tcp_twstart will be more successuful, may be not.
Also, using dtrace too complex in production (need complex startup
under screen and capture output) and for many peoples.
kdb_backtrace() have too less administrative overhead.

On 10/13/16 5:17 PM, Slawa Olhovchenkov wrote:
> On Thu, Oct 13, 2016 at 05:06:00PM +0200, Julien Charbon wrote:
> 
>>>> will give you that trace in the core, and without INVARIANT
then it is
>>>> better to use dtrace:
>>>>
>>>> $ cat tcp-twstart-dropped.d
>>>> fbt::tcp_twstart:entry
>>>> /args[0]->t_inpcb->inp_flags & 0x04000000/
>>>> {
>>>>   stack();
>>>>   printf("INP_DROPPED in tcp_twstart: %x",
args[0]->t_inpcb->inp_flags);
>>>> }
>>>
>>> Same code may be insert there too, IMHO.
>>
>>  Hmm, I don't think so:
>>
>>  - If you have INVARIANT, the kernel will panic in tcp_twstart() or
>> tcp_detach() and you will have everything you need to debug.
>>  - If you don't, dtrace is the right tool to use in all cases
anyway.
> 
> dtrace don't executed in may case w/ diagnostic "dtrace:
processing
> aborted: Abort due to systemic unresponsiveness". This is for
> tcp_close. May be tcp_twstart will be more successuful, may be not.
 It does and will.
> Also, using dtrace too complex in production (need complex startup
> under screen and capture output) and for many peoples.
> kdb_backtrace() have too less administrative overhead.
 I still think it is overkill.  The main goal of this change is to fix a
quite tricky and old TCP stack locking issue.  Let's try to do that
first, it is complex enough by itself.

 Once the fix is validated and pushed, feel free to propose your own
patch/review to add kdb_backtrace(), log(), etc.. to get other devs
point of view.

 I don't remember who said: "Never ever optimize error cases"...

--
Julien

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 496 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20161013/350f0616/attachment.sig>