search for: 100bytes

All, I have read the lists, I have read the faq - yes, rsync uses a lot of memory. That said - I need to use rsync in an environment with a lot of files, and I need to use -H and --delete options. The faq says about 100 bytes per file - at the moment I am looking at 8700000 files, which should translate to about 800MB - but it does not. In fact, running rsync not only consumes all of the ram,

Hello, I have a problem with "out of memory", I'm trying to sync around 30mil files and i get error, the sync is on the same host but different directory, this with the 100bytes per file, give me 3G of ram. I have put 12G swap (6 partitions of 2G) and my 3G ram. Last i saw the process rsync get to 2100M of SIZE and 1.8G RSS, don't know exactly at how much it gets till he get the error, but not too far after 2G. How can i solve the problem, i use 2.6.3 version. Tha...

...2003 machine to under the apache www dir. the problem is that i cannot access much of the files on win2k3 from the internet. i'll try to explain: apache seems to work fine. samba seems to work fine both windows->bsd and bsd->windows. but when apache tries to serve a file bigger than about 100bytes from win2k3 to the internet, nothing happens. the directory list comes through fine, but i cannot open files of any noteworthy size. small ones (like my documents' desktop.ini) come through fine. i'm really dazzled, this isn't like anything i've seen before. so any help is welcome.....

Our system has potentially a few thousand firewall rules and HTB classes. I need to find out the amount of memory these things consume: - iptables firewall rules - HTB classes If anyone has any easy links to this information, that would be great. Failing that, a pointer to a good place to look in the source code would be very helpful. _______________________________________________

...e? And we need a complete test to make sure >> this won't affect other use cases. > We just want <1ms ping latency, but actually we are not sure what value is reasonable. > We have some test results using netperf before this patch as follow, > > Udp payload 1byte 100bytes 1000bytes 1400bytes > Ping avg latency 25ms 10ms 2ms 1.5ms > > What is other testcases? Something like https://patchwork.kernel.org/patch/10151645/. Btw, you need use time_before() to properly handle jiffies overflow and I would also suggest you to try some...

...e? And we need a complete test to make sure >> this won't affect other use cases. > We just want <1ms ping latency, but actually we are not sure what value is reasonable. > We have some test results using netperf before this patch as follow, > > Udp payload 1byte 100bytes 1000bytes 1400bytes > Ping avg latency 25ms 10ms 2ms 1.5ms > > What is other testcases? Something like https://patchwork.kernel.org/patch/10151645/. Btw, you need use time_before() to properly handle jiffies overflow and I would also suggest you to try some...

...to draw upon. Running rsync with --progress, (v2.5.6 I believe - just downloaded it a few days ago) shows us that we reach a little over 8 million files before the server starts telling us its killing processes because its out of RAM. On the FAQ, it says that rsync should consume approximately 100bytes of memory per file, on average. So, 8 million x 100 = 800M of RAM. Why are we running out of RAM? Is there a way to tell the kernel not to use so much memory for cache and buffers, and to leave more free? Is the kernel not releasing the cache/buffer memory quick enough for rsync? I don't k...

...est to make sure > > > this won't affect other use cases. > > We just want <1ms ping latency, but actually we are not sure what value is reasonable. > > We have some test results using netperf before this patch as follow, > > > > Udp payload 1byte 100bytes 1000bytes 1400bytes > > Ping avg latency 25ms 10ms 2ms 1.5ms > > > > What is other testcases? > > Something like https://patchwork.kernel.org/patch/10151645/. > > Btw, you need use time_before() to properly handle jiffies overflow and I...

I've got an issue with remote files being deleted after the local file has been deleted. For some reason, this isn't happening. I'm running rsync 2.5.6 protocol 26 (yes, I know there are newer versions, but logistics dictates that I can't upgrade right now). I have used the --delete, --delete-after, and --ignore-errors options in all sorts of combinations. An example of the

...d based on the ceil weigth, and the unassigned bandwidth  -there are a lot more of posibilties, see DOCS Some deep explanation: The bw that you assign for each host is divided like this (this can be easy addapted if you know a litle bit of iptables): -Prio traffic - packets smallest than 100bytes (tcp ACKs, most icmp messages) - all icmp traffic (icmp messages) - all udp traffic (voip, streaming, real time gaiming, etc...) - some tcp ports (seteable in conf file, ie: http,pop3,imap,ssh...) -Default traffic (junk traffic) - all traffic that do not mach any of the above (ie:e...

After varying degrees of success with p2p detection modules, I would like to write the following rules using iptables to reliably identify p2p traffic: 1. If a host on the network has 5 or more simutaneous tcp connections to ports above 1024, mark all connections to ports 1024 and above as 60. 2. If a host has received (or sent) UDP packets from 5 different hosts'' ports above 1024 in a

I was planning to use rsync to backup to a second drive, but I ran out of swap space. No problem, I will let rsync do it a few directories at a time overnight when the computer usually sits idle except for incomming email. I have 2 question: 1. How much memory does each file to be copied need. Obvisiouly I have too many files. 2. Why does this command work: rsync -ax /usr/xx /backup/usr/

This is a continuation of my previous posts about improving write perf when trapping millions of small writes to a gluster filesystem. I was able to improve write perf by ~30x by running STDOUT thru gzip to consolidate and reduce the output stream. Today, another similar problem, having to do with yet another bioinformatics program (which these days typically handle the 'short reads' that