Granzow, Doug (NCI)
2002-Apr-22 09:42 UTC
memory requirements was RE: out of memory in build_hash_table
>The key growth factor for the file construction is going to be >per-file information that's about 56 bytes (last I looked) per file. >However, the kicker is that the file storage block gets resized >exponentially as you have more files. So for 385,000 files it'll >actually be a 512,000 file block of about 30MB. (So yeah, I suppose >an ~50 byte file chunk in memory growing as a power of 2 might average >out close to 100 bytes/file as an estimate :-))Hmm... I have a filesystem that contains 3,098,119 files. That's 3,098,119 * 56 bytes or 173,494,664 bytes (about 165 MB). Allowing for the exponential resizing we end up with space for 4,096,000 files * 56 bytes 218 MB. But 'top' tells me the rsync running on this filesystem is taking up 646 MB, about 3 times what it should. Are there other factors that affect how much memory rsync takes up? I only ask because I would certainly prefer it used 218 MB instead of 646. :) Doug