nsd users - Feb 2018 - NSD database memory usage

Hi Wouter,

----- On 6 Feb, 2018, at 16:14, W.C.A. Wijngaards wouter at nlnetlabs.nl wrote:
> Hi Antti,
> 
> On 06/02/18 15:08, Antti Ristim?ki wrote:
>> Hi,
>> 
>> ----- On 6 Feb, 2018, at 15:46, Anand Buddhdev anandb at ripe.net
wrote:
>> 
>>> On 06/02/2018 13:58, W.C.A. Wijngaards wrote:
>>>
>>>> Hi Antti,
>>>>
>>>> It is a memory leak, thank you for the report!  Fixed it, code
is copied
>>>> below and in the code repository.  It happens when unknown RR
formatted
>>>> RRs are read from zonefile.
>>>
>>> Hi Wouter, thanks for the quick fix!
>>>
>>> So Antti, which unusual RRs do you have in your zone?
>> 
>> That's something I also wonder, as the zone in question contains
only very usual
>> RRs, mainly delegation NS records. There are quite a lot of IDN names
but that
>> should probably be business as usual.
>> 
>> I will try to look at the zone contents if I find anything. Wouter,
anything
>> particular in mind that I should be searching for?
> 
> RRs printed in the \#length hexadecimals format cause the problem that I
> found.  Regardless if the type was known to NSD, it was the text
> formatting in the parser that leaked.
> 
> If the problem is still there (after using patch?) then it maybe
> possible to reproduce with (a smaller subset of) the zone?
We rebuilt the package with the patch and unfortunately it seems that the memory
usage still increases after each reload like it did before the patch. We'll
try to debug this further.

Antti

Hi Antti,

On 07/02/18 13:36, Antti Ristim?ki wrote:
> Hi Wouter,
> 
> ----- On 6 Feb, 2018, at 16:14, W.C.A. Wijngaards wouter at nlnetlabs.nl
wrote:
> 
>> Hi Antti,
>>
>> On 06/02/18 15:08, Antti Ristim?ki wrote:
>>> Hi,
>>>
>>> ----- On 6 Feb, 2018, at 15:46, Anand Buddhdev anandb at ripe.net
wrote:
>>>
>>>> On 06/02/2018 13:58, W.C.A. Wijngaards wrote:
>>>>
>>>>> Hi Antti,
>>>>>
>>>>> It is a memory leak, thank you for the report!  Fixed it,
code is copied
>>>>> below and in the code repository.  It happens when unknown
RR formatted
>>>>> RRs are read from zonefile.
>>>>
>>>> Hi Wouter, thanks for the quick fix!
>>>>
>>>> So Antti, which unusual RRs do you have in your zone?
>>>
>>> That's something I also wonder, as the zone in question
contains only very usual
>>> RRs, mainly delegation NS records. There are quite a lot of IDN
names but that
>>> should probably be business as usual.
>>>
>>> I will try to look at the zone contents if I find anything. Wouter,
anything
>>> particular in mind that I should be searching for?
>>
>> RRs printed in the \#length hexadecimals format cause the problem that
I
>> found.  Regardless if the type was known to NSD, it was the text
>> formatting in the parser that leaked.
>>
>> If the problem is still there (after using patch?) then it maybe
>> possible to reproduce with (a smaller subset of) the zone?
> 
> We rebuilt the package with the patch and unfortunately it seems that the
memory usage still increases after each reload like it did before the patch.
We'll try to debug this further.
Found the issue and fixed it in the code repository (with offlist
debug).  It was NSEC3 processing and previously I fixed an NSEC signed zone.

Thanks for the report,

Best regards, Wouter
> 
> Antti
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.nlnetlabs.nl/pipermail/nsd-users/attachments/20180208/fe54a994/attachment.bin>