I have two computers: a Linux box which I use as my workstation
(day-to-day use) and an OpenBSD box which I designate for server tasks:
gateway, firewall, etc.
I use rsync to mirror many gigs worth of data from the Linux box
(running rsync in daemon mode) to the OpenBSD machine (running an rsync
client job nightly).
Since the amount of data and the number of files that I mirror is great,
most of rsync's time seems to be spent computing differences in the
files. This appears to put quite a burden on the Linux workstation.
The result, I belive, is that much of the Linux box's memory is consumed
caching all this information that it is processing.
The reason I think this is because the apps I left running (e.g. file
browser, Mozilla, gnumeric spreadsheets, etc) need a few seconds to
"wake up" when they are first used for the day (subsequent use meets
no
delays). My theory is that most of my RAM is consumed in caching the
rsync nightly job, and my actual apps are swapped out.
At one GB, I have WAY more RAM than I need for my actual work, yet it
seems like I'm not really benefitting from having this kind of memory
(at least initially).
Maybe I'm just making too big a deal out of this <grin> but is there
any
way to have rsync not allow itself to be cached, or maybe to force it to
use a disk cache instead of physical RAM? Since this jobs runs at night
when I'm not using the computer, I don't care about degrading
rsync's
performance.
Thanks,
Matt
--
Matt Garman, garman@raw-sewage.net
``I ain't never seen no whiskey, the blues made my sloppy drunk!''
-- Sleepy John Estes, ``Leaving Trunk''
On Wed, May 28, 2003 at 07:15:23PM -0500, Matt Garman wrote:> > I have two computers: a Linux box which I use as my workstation > (day-to-day use) and an OpenBSD box which I designate for server tasks: > gateway, firewall, etc. > > I use rsync to mirror many gigs worth of data from the Linux box > (running rsync in daemon mode) to the OpenBSD machine (running an rsync > client job nightly). > > Since the amount of data and the number of files that I mirror is great, > most of rsync's time seems to be spent computing differences in the > files. This appears to put quite a burden on the Linux workstation. > The result, I belive, is that much of the Linux box's memory is consumed > caching all this information that it is processing. > > The reason I think this is because the apps I left running (e.g. file > browser, Mozilla, gnumeric spreadsheets, etc) need a few seconds to > "wake up" when they are first used for the day (subsequent use meets no > delays). My theory is that most of my RAM is consumed in caching the > rsync nightly job, and my actual apps are swapped out. > > At one GB, I have WAY more RAM than I need for my actual work, yet it > seems like I'm not really benefitting from having this kind of memory > (at least initially). > > Maybe I'm just making too big a deal out of this <grin> but is there any > way to have rsync not allow itself to be cached, or maybe to force it to > use a disk cache instead of physical RAM? Since this jobs runs at night > when I'm not using the computer, I don't care about degrading rsync's > performance.Firstly, this is not an rsync question it is a kernel question. Your analysis is largely correct. Linux is trying to make the best use of available memory. If you want to look at extensive discussions of this issue i'd suggest you google for interactivity and updatedb (the most famous cache-thrasher) This has a great deal to do with the kernel version you are running. Any decent VM system will exhibit this behaviour to some degree. At a minimum the clean pages with backing store that have been unused for an extended period can be expected to be reclaimed to accommodate newer file pages. Even without rsync and updatedb this will happen some. The kernel guys are trying to find a balance but that is happening in the 2.5 tree. In 2.5 there is a swappiness tunable and there are fctl(2) hints that we might eventually use. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt