Dear experts,
I would like get your help for the issue with --partial-dir=*absolute path.*
I’m trying to use rsync to send big file to server and config
--partial-dir=*absolute
path* since I want to store temp partial file in /tmp/ directory and keep
the destination folder clear.
My server is joe28, client is joe27. And here is the config on server.
root at joe28:~# cat /etc/rsyncd.conf
# Global settings
address = 10.148.34.28
numeric ids = yes
uid = 0
read only = no
reverse lookup = no
dont compress = *
timeout = 10
[JOE]
path = /home/joe/rsync
comment = Joe's rsync test
Sending big file from joe27 to joe28:
root at joe27:~# ls -l /home/joe/rsync/flash_image
-rw-r--r-- 1 root root 766028824 Aug 15 15:23 /home/joe/rsync/flash_image
root at joe27:~# rsync -a --delete --delay-updates --no-relative
--partial-dir=/tmp /home/joe/rsync/ rsync://10.148.34.28:873/JOE -P
building file list ...
2 files to consider
./
flash_image
766028824 100% 108.69MB/s 0:00:06 (xfer#1, to-check=0/2)
sent 766122429 bytes received 30 bytes 61289796.72 bytes/sec
total size is 766028824 speedup is 1.00
However, I still see the partial file in the local directory, but not in
/tmp which I configured.
root at joe28:~# watch --interval=1 ls -al /home/joe/rsync/ /tmp/flash*
Every 1.0s: ls -al /home/joe/rsync/ /tmp/flash*
Tue Aug 15 15:35:55 2017
ls: cannot access /tmp/flash*: No such file or directory
/home/joe/rsync/:
total 77064
drwxr-xr-x 2 root root 4096 Aug 15 15:35 .
drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 ..
-rw------- 1 root nogroup 78905344 Aug 15 15:35 .flash_image.xlIBqo
And finally I saw the “tmp” folder is created on server. That is not my
expected.
Every 1.0s: ls -al /home/joe/rsync/ /tmp/flash*
Tue Aug 15 15:36:05 2017
ls: cannot access /tmp/flash*: No such file or directory
/home/joe/rsync/:
total 748092
drwxr-xr-x 3 root root 4096 Aug 15 15:14 .
drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 ..
-rw-r--r-- 1 root root 766028824 Aug 15 15:23 flash_image
drwx------ 2 root nogroup 4096 Aug 15 15:36 tmp
Would you please give some suggestions? Appreciate the help!
BTW. I tried with --temp-dir but same result. Rsync version is 3.1.2.
Regards,
-Joe
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20170815/2410e575/attachment.html>
With rsyncd you can't. You would have to switch to rsync over ssh... rsync -a --delete --delay-updates --partial-dir=/tmp -P /home/joe/rsync/ 10.148.34.28:/home/joe/rsync/ Also, when you use --partial-dir you probably want to also use --temp-dir. On 08/15/2017 07:19 PM, Joe Qiao via rsync wrote:> Dear experts, > > > > I would like get your help for the issue with --partial-dir=_absolute path._ > > > > I’m trying to use rsync to send big file to server and config > --partial-dir=_absolute path_ since I want to store temp partial file in > /tmp/ directory and keep the destination folder clear. > > > > My server is joe28, client is joe27. And here is the config on server. > > > > root at joe28:~# cat /etc/rsyncd.conf > > # Global settings > > address = 10.148.34.28 > > numeric ids = yes > > uid = 0 > > read only = no > > reverse lookup = no > > dont compress = * > > timeout = 10 > > [JOE] > > path = /home/joe/rsync > > comment = Joe's rsync test > > > > Sending big file from joe27 to joe28: > > > > root at joe27:~# ls -l /home/joe/rsync/flash_image > > -rw-r--r-- 1 root root 766028824 Aug 15 15:23 /home/joe/rsync/flash_image > > > > root at joe27:~# rsync -a --delete --delay-updates > --no-relative --partial-dir=/tmp /home/joe/rsync/ > rsync://10.148.34.28:873/JOE <http://10.148.34.28:873/JOE> -P > > building file list ... > > 2 files to consider > > ./ > > flash_image > > 766028824 100% 108.69MB/s 0:00:06 (xfer#1, to-check=0/2) > > > > sent 766122429 bytes received 30 bytes 61289796.72 bytes/sec > > total size is 766028824 speedup is 1.00 > > > > However, I still see the partial file in the local directory, but not in > /tmp which I configured. > > > > root at joe28:~# watch --interval=1 ls -al /home/joe/rsync/ /tmp/flash* > > Every 1.0s: ls -al /home/joe/rsync/ > /tmp/flash* Tue Aug 15 15:35:55 2017 > > > > ls: cannot access /tmp/flash*: No such file or directory > > /home/joe/rsync/: > > total 77064 > > drwxr-xr-x 2 root root 4096 Aug 15 15:35 . > > drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 .. > > -rw------- 1 root nogroup 78905344 Aug 15 15:35 .flash_image.xlIBqo > > > > And finally I saw the “tmp” folder is created on server. That is not my > expected. > > > > Every 1.0s: ls -al /home/joe/rsync/ > /tmp/flash* Tue Aug 15 15:36:05 2017 > > > > ls: cannot access /tmp/flash*: No such file or directory > > /home/joe/rsync/: > > total 748092 > > drwxr-xr-x 3 root root 4096 Aug 15 15:14 . > > drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 .. > > -rw-r--r-- 1 root root 766028824 Aug 15 15:23 flash_image > > drwx------ 2 root nogroup 4096 Aug 15 15:36 tmp > > > > Would you please give some suggestions? Appreciate the help! > > > > BTW. I tried with --temp-dir but same result. Rsync version is 3.1.2. > > > > Regards, > > -Joe > > >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: http://www.sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 224 bytes Desc: OpenPGP digital signature URL: <http://lists.samba.org/pipermail/rsync/attachments/20170815/5b9e17a0/signature.sig>
Thanks so much for the quick reply, Kevin!
I tried with ssh and --partial-dir, it looks the partial file still will be
stored in local dir, but not in /tmp.
Every 1.0s: ls -al /home/joe/rsync/ /tmp/
Tue Aug 15 17:29:30 2017
/home/joe/rsync/:
total 408840
drwxr-xr-x 2 root root 4096 Aug 15 17:29 .
drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 ..
*-rw------- 1 root root 418643968 Aug 15 17:29 .flash_image.14WoMV
<<<<<<<<<<<<<<<<<<*
/tmp/:
total 32
drwxrwxrwt 7 root root 4096 Aug 15 17:28 .
drwxr-xr-x 24 root root 4096 Mar 27 15:05 ..
drwxrwxrwt 2 lightdm lightdm 4096 Aug 7 09:23 at-spi2
drwxrwxrwt 2 root root 4096 Aug 7 09:23 .ICE-unix
drwx------ 2 lightdm lightdm 4096 Aug 7 09:24 pulse-bYV3V1IM0vBC
drwx------ 2 lightdm lightdm 4096 Aug 7 09:24 pulse-PKdhtXMmr18n
-rw-rw-r-- 1 lightdm lightdm 0 Aug 7 09:24 unity_support_test.1
-r--r--r-- 1 root root 11 Aug 7 09:23 .X0-lock
drwxrwxrwt 2 root root 4096 Aug 7 09:23 .X11-unix
And tried again with ssh and --temp-dir, it works.
Every 1.0s: ls -al /home/joe/rsync/ /tmp/
Tue Aug 15 17:28:15 2017
/home/joe/rsync/:
total 8
drwxr-xr-x 2 root root 4096 Aug 15 15:14 .
drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 ..
/tmp/:
total 581664
drwxrwxrwt 7 root root 4096 Aug 15 17:28 .
drwxr-xr-x 24 root root 4096 Mar 27 15:05 ..
drwxrwxrwt 2 lightdm lightdm 4096 Aug 7 09:23 at-spi2
*-rw------- 1 root root 595591168 Aug 15 17:28 .flash_image.AX9F1y
<<<<<<<<<<<<<<<<<<<<<*
drwxrwxrwt 2 root root 4096 Aug 7 09:23 .ICE-unix
drwx------ 2 lightdm lightdm 4096 Aug 7 09:24 pulse-bYV3V1IM0vBC
drwx------ 2 lightdm lightdm 4096 Aug 7 09:24 pulse-PKdhtXMmr18n
-rw-rw-r-- 1 lightdm lightdm 0 Aug 7 09:24 unity_support_test.1
-r--r--r-- 1 root root 11 Aug 7 09:23 .X0-lock
drwxrwxrwt 2 root root 4096 Aug 7 09:23 .X11-unix
Anyway, at least the ssh + temp-dir works for me. Thank you!
Regards,
-Joe
On Tue, Aug 15, 2017 at 5:14 PM, Kevin Korb via rsync <rsync at
lists.samba.org> wrote:
> With rsyncd you can't. You would have to switch to rsync over ssh...
>
> rsync -a --delete --delay-updates --partial-dir=/tmp -P /home/joe/rsync/
> 10.148.34.28:/home/joe/rsync/
>
> Also, when you use --partial-dir you probably want to also use --temp-dir.
>
>
> On 08/15/2017 07:19 PM, Joe Qiao via rsync wrote:
> > Dear experts,
> >
> >
> >
> > I would like get your help for the issue with --partial-dir=_absolute
> path._
> >
> >
> >
> > I’m trying to use rsync to send big file to server and config
> > --partial-dir=_absolute path_ since I want to store temp partial file
in
> > /tmp/ directory and keep the destination folder clear.
> >
> >
> >
> > My server is joe28, client is joe27. And here is the config on server.
> >
> >
> >
> > root at joe28:~# cat /etc/rsyncd.conf
> >
> > # Global settings
> >
> > address = 10.148.34.28
> >
> > numeric ids = yes
> >
> > uid = 0
> >
> > read only = no
> >
> > reverse lookup = no
> >
> > dont compress = *
> >
> > timeout = 10
> >
> > [JOE]
> >
> > path = /home/joe/rsync
> >
> > comment = Joe's rsync test
> >
> >
> >
> > Sending big file from joe27 to joe28:
> >
> >
> >
> > root at joe27:~# ls -l /home/joe/rsync/flash_image
> >
> > -rw-r--r-- 1 root root 766028824 Aug 15 15:23
/home/joe/rsync/flash_image
> >
> >
> >
> > root at joe27:~# rsync -a --delete --delay-updates
> > --no-relative --partial-dir=/tmp /home/joe/rsync/
> > rsync://10.148.34.28:873/JOE <http://10.148.34.28:873/JOE> -P
> >
> > building file list ...
> >
> > 2 files to consider
> >
> > ./
> >
> > flash_image
> >
> > 766028824 100% 108.69MB/s 0:00:06 (xfer#1, to-check=0/2)
> >
> >
> >
> > sent 766122429 bytes received 30 bytes 61289796.72 bytes/sec
> >
> > total size is 766028824 speedup is 1.00
> >
> >
> >
> > However, I still see the partial file in the local directory, but not
in
> > /tmp which I configured.
> >
> >
> >
> > root at joe28:~# watch --interval=1 ls -al /home/joe/rsync/
/tmp/flash*
> >
> > Every 1.0s: ls -al /home/joe/rsync/
> > /tmp/flash* Tue Aug 15
15:35:55
> 2017
> >
> >
> >
> > ls: cannot access /tmp/flash*: No such file or directory
> >
> > /home/joe/rsync/:
> >
> > total 77064
> >
> > drwxr-xr-x 2 root root 4096 Aug 15 15:35 .
> >
> > drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 ..
> >
> > -rw------- 1 root nogroup 78905344 Aug 15 15:35 .flash_image.xlIBqo
> >
> >
> >
> > And finally I saw the “tmp” folder is created on server. That is not
my
> > expected.
> >
> >
> >
> > Every 1.0s: ls -al /home/joe/rsync/
> > /tmp/flash* Tue Aug 15
15:36:05
> 2017
> >
> >
> >
> > ls: cannot access /tmp/flash*: No such file or directory
> >
> > /home/joe/rsync/:
> >
> > total 748092
> >
> > drwxr-xr-x 3 root root 4096 Aug 15 15:14 .
> >
> > drwxr-xr-x 32 joe joe 4096 Aug 15 15:01 ..
> >
> > -rw-r--r-- 1 root root 766028824 Aug 15 15:23 flash_image
> >
> > drwx------ 2 root nogroup 4096 Aug 15 15:36 tmp
> >
> >
> >
> > Would you please give some suggestions? Appreciate the help!
> >
> >
> >
> > BTW. I tried with --temp-dir but same result. Rsync version is 3.1.2.
> >
> >
> >
> > Regards,
> >
> > -Joe
> >
> >
> >
>
> --
>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
> Kevin Korb Phone: (407) 252-6853
> Systems Administrator Internet:
> FutureQuest, Inc. Kevin at FutureQuest.net (work)
> Orlando, Florida kmk at sanitarium.net (personal)
> Web page: http://www.sanitarium.net/
> PGP public key available on web site.
>
~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
>
>
> --
> Please use reply-all for most replies to avoid omitting the mailing list.
> To unsubscribe or change options: https://lists.samba.org/
> mailman/listinfo/rsync
> Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.samba.org/pipermail/rsync/attachments/20170815/f1a054d8/attachment.html>
Possibly Parallel Threads
- How to set absolute path for rsync?
- How to set absolute path for rsync?
- Suggestion/opinions: add a `absolute` param to `normalizePath()` to force returning an absolute path
- Suggestion/opinions: add a `absolute` param to `normalizePath()` to force returning an absolute path
- [LLVMdev] Tight overlapping loops and performance