Hello,
I'm having trouble transferring files from an rsync server on Mac OSX to a
Linux machine. I'm doing everything as root on both ends to make sure
it's
not a permissions issue, but still can't get all the files to transfer.
I have the rsync server running on OSX as root, like this:
server:/Users root# cat /etc/rsyncd.conf
[user_docs]
path = /Users
read only = true
transfer logging = no
server:/Users root# ps -alx|grep rsync
0 1658 1 0 31 0 18268 372 - Ss ?? 0:00.01
rsync --daemon
On the client, I run this:
[client:/tmp/server/Users]# rsync -av --delete server::user_docs/
/tmp/server/Users/
receiving file list ... done
wrote 197 bytes read 13348 bytes 27090.00 bytes/sec
total size is 1304385724 speedup is 96300.16
So it looks like client:/tmp/server/Users/ should now be a mirror of
server:/Users/, right? But it isn't:
server:/Users root# ls -R1a|grep [[:graph:]]|grep -vE "/|^.$|^..$"|wc
14230 18850 227942
server:/Users root# du -sh
12G .
[client:/tmp/server/Users]# ls -R1a|grep [[:graph:]]|grep -vE
"/|^.$|^..$"|wc
561 639 10926
[client:/tmp/server/Users]# du -sh
1.3G .
Also, if I re-run the rsync command once it's already been done, it gives me
this single IO error:
[client:/tmp/server/Users]# rsync -avvv --delete server::user_docs/
/tmp/server/Users/
...
recv_file_name(Shared/vida_blue/vb011231i_10_crowd.mp3)
received 568 names
done
recv_file_list done
get_local_name count=568 /tmp/server/Users/
IO error encountered - skipping file deletion
generator starting pid=4384 count=568
delta transmission enabled
recv_generator(.,0)
recv_generator(.DS_Store,1)
.DS_Store is uptodate
...
If I use the --ignore-errors option, then it doesn't print that error, but
either way, I still end up with only 1.3GB on the client, when there's 12GB
on the server.
Any ideas?
Thanks,
Anthony DiSante
http://nodivisions.com/
In an attempt to copy a particular directory or file from the "primary" server to the "backup" server - it looks as though rsync is wanting to work, but nothing is copied. primary - 10.28.123.232 backup - 10.28.123.230 rsyncd.conf file (on primary server) ------------------- uid = 0 gid = 0 use chroot = no max connections = 4 syslog facility = local5 pid file = /var/run/rsyncd.pid read only = true list = false hosts allow = 10.28.123.230 [etc] path = /etc comment = system config files [web] path = /var/www comment = web files ------------------------- While on the backup machine, I run the following command [root@fedora-backup root]# rsync -rv 10.28.123.232::web and it produces the following: -rw-r--r-- 4629 2004/05/07 02:21:08 manual/platform/index.html.en .......... -rw-r--r-- 4482 2003/10/14 16:14:08 wordtrans/wordtrans.php wrote 57 bytes read 19817 bytes 39748.00 bytes/sec total size is 8261724 speedup is 415.71 [root@fedora-backup root]# Now, I put a file in the /var/www/html directory of the Primary server - but it never seems to get transfered to the backup server. I even put that same file in the /var/www directory thinking it was only looking there - to no avail. Can someone tell me what I am doing wrong? I'd just like to get the new or changed information from the primary server to the backup server. Thanks, Scott -- Outgoing mail is certified Virus Free. Checked by AVG Anti-Virus (http://www.grisoft.com). Version: 7.0.261 / Virus Database: 263.4.11 - Release Date: 7/27/2004
On Wed, Jul 28, 2004 at 02:44:34PM -0400, Anthony DiSante wrote:> I'm doing everything as root on both ends to make sure it's > not a permissions issue, but still can't get all the files to transfer.First off, look for error messages in the log file that the daemon is logging into (it will be one of your syslog files since you did not override it). Also, your config file didn't specify a uid, so it is defaulting to running the daemon as user "nobody", so it may be a permissions problem after all. See the uid and gid settings in the rsyncd.conf manpage. ..wayne..
I have done a lot of testing with single files as large as 85Gig and havent seen any significant issues. My testbed(s) are Debian Linux and Redhat Linux. The tests run as root. I have had performance issues with large files but nothing "broke" because of the size of the files. That is assuming that the basic OS will handle huge files correctly. You do have to do some work with Debian and Redhat but our business is huge files so all of our testbeds are correctly configured for them. I tested with rsync 2.6.2 and with 2.5.7. I can't testify to any other os or operating environment. But rsync works correctly (if somewhat slower) with huge files. 2.5.7 is slower than 2.6.2 but I didnt see a huge difference in their performance. What I did see was that the rolling checksum isnt much use with huge files and you spend a lot of time computing strong checksums. I am not being critical of rsync. It is designed for large directories of reasonable sized files and using it for a single huge file is at the extreme limits of what it is intended to be used for. We have a special case with large proprietary tarballs (ie. Veritas) that we can't trust to untar correctly so we occasionally have to handle them as a single large file. Unfortunately, we loose many of the advantages that rsync has in this environment and are now working on a proprietary solution to these exceptions. wally -----Original Message----- From: Wayne Davison [mailto:wayned@samba.org] Sent: Wednesday, July 28, 2004 6:21 PM To: Anthony DiSante Cc: rsync@lists.samba.org Subject: Re: only 1.3 of 12 GB will transfer from OSX to Linux On Wed, Jul 28, 2004 at 02:44:34PM -0400, Anthony DiSante wrote:> I'm doing everything as root on both ends to make sure it's > not a permissions issue, but still can't get all the files to transfer.First off, look for error messages in the log file that the daemon is logging into (it will be one of your syslog files since you did not override it). Also, your config file didn't specify a uid, so it is defaulting to running the daemon as user "nobody", so it may be a permissions problem after all. See the uid and gid settings in the rsyncd.conf manpage. ..wayne.. -- To unsubscribe or change options: http://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html