samba - Sep 2019 - Repacking database from v1 to v2 format: how long does it take?

hi all,

I updated a small domain with 8k object to samba-4.11.0 and the database
conversion from v1 to v2 didn't take a noticeable time.

On the other hand, in a larger domain with 67k object, where the
sudo ./bin/samba-tool dbcheck --cross-ncs --fix
takes ~40 minutes, 2 hours and half were not enough to complete the
conversion.

Is it a couple of hours something expected if dbcheck takes so much time?

Is it maybe my AD db to be pathologic?

Can you share your conversion time experiences?

thank you,

Franz

On Mon, 2019-09-23 at 11:25 +0200, Francesco Malvezzi via samba
wrote:
> hi all,
> 
> I updated a small domain with 8k object to samba-4.11.0 and the
> database
> conversion from v1 to v2 didn't take a noticeable time.
> 
> On the other hand, in a larger domain with 67k object, where the
> sudo ./bin/samba-tool dbcheck --cross-ncs --fix
> takes ~40 minutes, 2 hours and half were not enough to complete the
> conversion.
> 
> Is it a couple of hours something expected if dbcheck takes so much
> time?
> 
> Is it maybe my AD db to be pathologic?
Ouch!  We did not see such long runtimes during our testing, which is
why no major warnings were made.  Are you on TDB or LMDB?  Does the
host have enough RAM?

At your scale I would suggest say 10G, being 4GB for the main TDB, 4GB
for the transaction copy for the repack, and extra.

It was expected to be a couple of minutes at most, most likely much
less.

Sorry!

Andrew Bartlett
-- 
Andrew Bartlett                       https://samba.org/~abartlet/
Authentication Developer, Samba Team  https://samba.org
Samba Developer, Catalyst IT          
https://catalyst.net.nz/services/samba

On 23/09/2019 20:41, Andrew Bartlett wrote:
> On Mon, 2019-09-23 at 11:25 +0200, Francesco Malvezzi via samba wrote:
>> hi all,
>>
>> I updated a small domain with 8k object to samba-4.11.0 and the
>> database
>> conversion from v1 to v2 didn't take a noticeable time.
>>
>> On the other hand, in a larger domain with 67k object, where the
>> sudo ./bin/samba-tool dbcheck --cross-ncs --fix
>> takes ~40 minutes, 2 hours and half were not enough to complete the
>> conversion.
>>
>> Is it a couple of hours something expected if dbcheck takes so much
>> time?
>>
>> Is it maybe my AD db to be pathologic?
> 
> Ouch!  We did not see such long runtimes during our testing, which is
> why no major warnings were made.  Are you on TDB or LMDB?  Does the
> host have enough RAM?
It's TDB. I am unaware of the chance to convert to LMDB.
> 
> At your scale I would suggest say 10G, being 4GB for the main TDB, 4GB
> for the transaction copy for the repack, and extra.
It's close: 8GB, but while doing repack memory's usage was never higher
than 20%; it was high on cpu, one core out of two was stuck at 100%.
According to top, there was no iowait, so I supposed it was not a memory
issue.

I'm going to check more carefully,
> 
> Sorry!
no problem,

thank you,

franz