rsync - Mar 2013 - Best organizing hundreds of thousands files for rsync and find

Hi,

I need to organize about 100 millions small files (and the number grows 
up) on a server which should be copied to other server.

I am wondering how many files are recommended to be kept into a folder 
for optimal performance? As well, if I have a folder with only 
subfolders (not files) what number of subfolders are recommended to have?

As well, the question could be for "find" command, not just for for 
rsyncas I am doing some cleanups using find (or for - find).


I made a mistake before and I increased a lot the number of 
subfoldersfolders (having just few files within them) and rsync 
performance was decreasing considerably. Was a mistake which I will try 
to correct.

So now as the number of files is increasing constantly I need to find 
out a solution on long term to correct the current issues.

Cristian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20130328/c5531189/attachment.html>

Hi,

I need to organize about 100 millions small files (and the number grows 
up) on a server which should be copied to other server.

I am wondering how many files are recommended to be kept into a folder 
for optimal performance? As well, if I have a folder with only 
subfolders (not files) what number of subfolders are recommended to have?

As well, the question could be for "find" command, not just for for 
rsyncas I am doing some cleanups using find (or for - find).


I made a mistake before and I increased a lot the number of 
subfoldersfolders (having just few files within them) and rsync 
performance was decreasing considerably. Was a mistake which I will try 
to correct.

So now as the number of files is increasing constantly I need to find 
out a solution on long term to correct the current issues.

Cristian
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20130328/0489bbee/attachment.html>