samba - Jul 2013 - memory consumption with treesize pro and cifs shares

Hi everyone. I'm looking to solve an issue with Samba on a NAS being
accessed with TreeSize Pro. Using that program to scan through millions of
files is eating up memory on swap and eventually crashing the system. It's
scanning mounted CIFS shares on the NAS running TrueNAS with samba version
3.6.9

We have a test case and have been able to replicate the issue on another
machine.

The "solution" right now is to simply not run TreeSize Pro. Not the
best of
plans.

In the meantime, I'm going to continue to check the usual manuals/google
sources to see if I can find anything. I haven't as yet and am short on
time with this. Basically looking to see if this is an actual bug that
might require a patch/upgrade, or something I can fix with some tuneables.

Thanks,
Mike

Add more memory or split the volume into smaller shares.

Gc

Cy Mike <cymike at gmail.com> wrote:
>Hi everyone. I'm looking to solve an issue with Samba on a NAS being
>accessed with TreeSize Pro. Using that program to scan through millions of
>files is eating up memory on swap and eventually crashing the system.
It's
>scanning mounted CIFS shares on the NAS running TrueNAS with samba version
>3.6.9
>
>We have a test case and have been able to replicate the issue on another
>machine.
>
>The "solution" right now is to simply not run TreeSize Pro. Not
the best of
>plans.
>
>In the meantime, I'm going to continue to check the usual manuals/google
>sources to see if I can find anything. I haven't as yet and am short on
>time with this. Basically looking to see if this is an actual bug that
>might require a patch/upgrade, or something I can fix with some tuneables.
>
>Thanks,
>Mike
>-- 
>To unsubscribe from this list go to the following URL and read the
>instructions:  https://lists.samba.org/mailman/options/samba
>

Cy Mike wrote:
> Hi everyone. I'm looking to solve an issue with Samba on a NAS being
> accessed with TreeSize Pro. Using that program to scan through millions of
> files is eating up memory on swap and eventually crashing the system.
---
Which system is crashing?  the NAS or the one running the scan?... I
think you mean the
NAS, but I'm not sure.

I ran 2 copies on different partitions at the same time but didn't
reproduce the problem.
(Never got close).

Note -- at one point I had a problem with TSP finding more disk space on
my disk(S) than
I had...

Under the Tools-Options menu, there is an option which used to default
to 'on'...

"Follow Mount Points and External Symbolic Links to Directories"


That caused it to go out for lunch, a sauna and an afternoon swim...i.e.
it ran into
loops but only TreeSize kept growing bit by bit.

Are you running smb32 or 64?

How big are the disks you are searching through?

Are you using default size INodes or larger?  How much speed to do you get
when using 'dd' to transfer a file?  (ln /dev/zero ~/zero; ln /dev/null
~/null:
dd if=/dev/null of=/dev/zero bs=16-64M seems optimal,

I'm not on the samba team, but am curious -- wanted to see if I could
repro it with
my current samba and server.  I'm running samba 3.6.16 and my server is x64.

You running 32 or 64bit?  maybe that is contributing?







>  It's
> scanning mounted CIFS shares on the NAS running TrueNAS with samba version
> 3.6.9
>
> We have a test case and have been able to replicate the issue on another
> machine.
>   
----
    Awesome...  I only have TreeSize Professional 5.4.4.707 (didn't
bother to upgrade since
this one worked for my needs at the time).

    I currently have 3 copies of it running
> The "solution" right now is to simply not run TreeSize Pro. Not
the best of
> plans.
>
> In the meantime, I'm going to continue to check the usual
manuals/google
> sources to see if I can find anything. I haven't as yet and am short on
> time with this. Basically looking to see if this is an actual bug that
> might require a patch/upgrade, or something I can fix with some tuneables.
>
> Thanks,
> Mike
>

More info on this:

The NAS running FreeBSD has 48GB RAM, same as the test NAS we are
duplicating the error on. Both machines see this error with 3.6.9 Samba.
The initial try at duplicating the error didn't produce it. It wasn't
until
we increased the amount of files in the CIFS share that we were able to
duplicate it. Number of files is in the millions. Drive freespace is large
on the test machine and the error still occurs. According to LindaW here,
the test hasn't been reproduce yet using Samba 3.6.16, so we're looking
into another test on our box using the updated version.

Has anyone else encountered an issue like with using TreeSize Pro? Does
anyone need more information to help sort this out? We'll be running
additional tests today looking for a solution and I will post back more on
this later.

Thanks,
Mike


On Tue, Jul 23, 2013 at 6:59 PM, Cy Mike <cymike at gmail.com> wrote:
> Hi everyone. I'm looking to solve an issue with Samba on a NAS being
> accessed with TreeSize Pro. Using that program to scan through millions of
> files is eating up memory on swap and eventually crashing the system.
It's
> scanning mounted CIFS shares on the NAS running TrueNAS with samba version
> 3.6.9
>
> We have a test case and have been able to replicate the issue on another
> machine.
>
> The "solution" right now is to simply not run TreeSize Pro. Not
the best
> of plans.
>
> In the meantime, I'm going to continue to check the usual
manuals/google
> sources to see if I can find anything. I haven't as yet and am short on
> time with this. Basically looking to see if this is an actual bug that
> might require a patch/upgrade, or something I can fix with some tuneables.
>
> Thanks,
> Mike
>


-- 
"that's not a bald head, that's a solar panel for a dumbass
machine" - jon
stewart 5/9/12