I'm copying a partition that has a bunch of hardlink based snapshots (-aPH). I think there's about 250,000 files in each backup and between 100 and 200 snapshots. Earlier today, I saw the files had completed and it was making all the hardlinks. I thought it would be "not long" but it's been making hardlinks for 12 hours (at least). There's only 36Gb in snapshot, the data didn't take too long. but now there's no progress indicator and ??? hours left. Would it be straightforward to include progress when creating hardlinks? // George -- George Georgalis, systems architect, administrator <IXOYE><
On Sat, Jan 13, 2007 at 10:25:42PM -0500, George Georgalis wrote:> hours left. Would it be straightforward to > include progress when creating hardlinks?Please upgrade -- hard-link handling is much improved in newer versions. ..wayne..
On Sat, Jan 13, 2007 at 08:43:55PM -0800, Wayne Davison wrote:>On Sat, Jan 13, 2007 at 10:25:42PM -0500, George Georgalis wrote: >> hours left. Would it be straightforward to >> include progress when creating hardlinks? > >Please upgrade -- hard-link handling is much improved in newer versions.Thanks, turns out there are closer to 500,000 files and 89 snapshots (@ ~90% of the files). The process was to install larger disks on primary host; the files went (push) from reiserfs to reiserfs with old versions of rsync (target: rsync version 2.6.3pre1 protocol version 28; source: "debian 3.1"). Now, restoring (pull) from the same old version on the (reiserfs) source to ffs2 (netbsd 3.1) with rsync version 2.6.9 protocol version 29 (target). root@target:/root $ rsync --numeric-ids -avHP source:/data/ /data/ receiving file list ... ERROR: out of memory in inode_table [receiver] rsync error: error allocating core memory buffers (code 22) at util.c(115) [receiver=2.6.9] When I tried breaking up the source data into a directory 1/4 the size, I got the same error. But I realized I cannot preserve hardlinks if I breakup the /data/ directory. There _are_ more memory slots on the target host, is there any way to adjust the rsync command to use less memory? Would it help to update the sender rsync? Or do I have to restore each snapshot, one at a time? :-} // George -- George Georgalis, systems architect, administrator <IXOYE><
On Mon, Jan 15, 2007 at 01:39:53AM -0500, George Georgalis wrote:>On Sat, Jan 13, 2007 at 08:43:55PM -0800, Wayne Davison wrote: >>On Sat, Jan 13, 2007 at 10:25:42PM -0500, George Georgalis wrote: >>> hours left. Would it be straightforward to >>> include progress when creating hardlinks? >> >>Please upgrade -- hard-link handling is much improved in newer versions. > >Thanks, turns out there are closer to 500,000 files and 89 >snapshots (@ ~90% of the files). The process was to install >larger disks on primary host; the files went (push) from reiserfs >to reiserfs with old versions of rsync (target: rsync version >2.6.3pre1 protocol version 28; source: "debian 3.1"). Now, >restoring (pull) from the same old version on the (reiserfs) >source to ffs2 (netbsd 3.1) with rsync version 2.6.9 protocol >version 29 (target). > > root@target:/root $ rsync --numeric-ids -avHP source:/data/ /data/ >receiving file list ... >ERROR: out of memory in inode_table [receiver] >rsync error: error allocating core memory buffers (code 22) at util.c(115) [receiver=2.6.9]I added more memory, same problem. any recommendations to get around this? will the old version of rsync use less memory to make all the hardlinks? // George -- George Georgalis, systems architect, administrator <IXOYE><
On Thu, Feb 22, 2007 at 10:12:17AM -0500, George Georgalis wrote:> I added more memory, same problem. > > any recommendations to get around this? will the old version of > rsync use less memory to make all the hardlinks?Newer versions should use less memory than old. Perhaps you have a ulimit set that limits the amount of memory available to the process? ..wayne..