Kevin Korb
2021-Nov-28 19:41 UTC
Copying a large file from an USB 3 disk (NTFS or EXT4) to another one (UFS/FreeBSD) is a very slow process....
rsync is terribly slow at local copies. Also, it doesn't do its normal optimizing (see --whole-file). Just use cp. On 11/28/21 13:38, Mario Marietto via rsync wrote:> Hello to everyone. > > I'm copying a large file from a NTFS formatted disk to another > one,UFS/FreeBSD disk,both are removable disks attached to the USB 3 > port. The file is 200 GB large and it is copied very slowly. Why it is > so slow ? I don't know where to store my virtual machines. I tried to > save them on the ext4 disk because I wanted to share them easily between > Linux and FreeBSD but I've realized that when I mount the disk in > FreeBSD after some time it corrupts. I tried to store it on the NTFS > disk but it happens the same. So,now I'm on FreeBSD and I'm copying them > to a dedicated UFS/FreeBSD style disk,but as I said,the speed is very > slow. How can I increase the speed ?. Actually I'm using this > command,because I want to resume the uploading if it breaks at some point : > > > root at marietto:/mnt/da3p2/bhyve/Ubuntu # rsync -avAXEWSlHh > /mnt/da0p1/Backups/OS/bhyve/Ubuntu/im* . --no-compress --info=progress2 > > sending incremental file list > impish-cuda-11-4-nvidia-470.img > > 2.13M 0% 9.49kB/s 6284:55:38 > > > and : where do you save large files ? what's the procedure that you use > to copy large files with a decent speed ? Unfortunately under Linux is > not safe to use a RW ufs disk access. So,I'm out of solutions. > > - > Mario. >-- ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., Kevin Korb Phone: (407) 252-6853 Systems Administrator Internet: FutureQuest, Inc. Kevin at FutureQuest.net (work) Orlando, Florida kmk at sanitarium.net (personal) Web page: https://sanitarium.net/ PGP public key available on web site. ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,
Mario Marietto
2021-Nov-28 20:11 UTC
Copying a large file from an USB 3 disk (NTFS or EXT4) to another one (UFS/FreeBSD) is a very slow process....
Hello. thanks. Can you tell me the most appropriate parameters to use with cp ? U will save me a lot of time. Thanks very much. Il giorno dom 28 nov 2021 alle ore 20:43 Kevin Korb via rsync < rsync at lists.samba.org> ha scritto:> rsync is terribly slow at local copies. Also, it doesn't do its normal > optimizing (see --whole-file). Just use cp. > > On 11/28/21 13:38, Mario Marietto via rsync wrote: > > Hello to everyone. > > > > I'm copying a large file from a NTFS formatted disk to another > > one,UFS/FreeBSD disk,both are removable disks attached to the USB 3 > > port. The file is 200 GB large and it is copied very slowly. Why it is > > so slow ? I don't know where to store my virtual machines. I tried to > > save them on the ext4 disk because I wanted to share them easily between > > Linux and FreeBSD but I've realized that when I mount the disk in > > FreeBSD after some time it corrupts. I tried to store it on the NTFS > > disk but it happens the same. So,now I'm on FreeBSD and I'm copying them > > to a dedicated UFS/FreeBSD style disk,but as I said,the speed is very > > slow. How can I increase the speed ?. Actually I'm using this > > command,because I want to resume the uploading if it breaks at some > point : > > > > > > root at marietto:/mnt/da3p2/bhyve/Ubuntu # rsync -avAXEWSlHh > > /mnt/da0p1/Backups/OS/bhyve/Ubuntu/im* . --no-compress --info=progress2 > > > > sending incremental file list > > impish-cuda-11-4-nvidia-470.img > > > > 2.13M 0% 9.49kB/s 6284:55:38 > > > > > > and : where do you save large files ? what's the procedure that you use > > to copy large files with a decent speed ? Unfortunately under Linux is > > not safe to use a RW ufs disk access. So,I'm out of solutions. > > > > - > > Mario. > > > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: https://sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-- Mario. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20211128/fbd90b60/attachment.htm>
Harry Mangalam
2021-Nov-28 21:44 UTC
Copying a large file from an USB 3 disk (NTFS or EXT4) to another one (UFS/FreeBSD) is a very slow process....
Can you elaborate on why this is? I wrote a parallel rsync wrapper that works very well over networks but is similarly very slow over local disks. I thought it was a bug in my code but didn't get around to tracking it down since my use cases were all network/ parallel file systems. Harry On Sun, Nov 28, 2021, 11:43 AM Kevin Korb via rsync <rsync at lists.samba.org> wrote:> rsync is terribly slow at local copies. Also, it doesn't do its normal > optimizing (see --whole-file). Just use cp. > > On 11/28/21 13:38, Mario Marietto via rsync wrote: > > Hello to everyone. > > > > I'm copying a large file from a NTFS formatted disk to another > > one,UFS/FreeBSD disk,both are removable disks attached to the USB 3 > > port. The file is 200 GB large and it is copied very slowly. Why it is > > so slow ? I don't know where to store my virtual machines. I tried to > > save them on the ext4 disk because I wanted to share them easily between > > Linux and FreeBSD but I've realized that when I mount the disk in > > FreeBSD after some time it corrupts. I tried to store it on the NTFS > > disk but it happens the same. So,now I'm on FreeBSD and I'm copying them > > to a dedicated UFS/FreeBSD style disk,but as I said,the speed is very > > slow. How can I increase the speed ?. Actually I'm using this > > command,because I want to resume the uploading if it breaks at some > point : > > > > > > root at marietto:/mnt/da3p2/bhyve/Ubuntu # rsync -avAXEWSlHh > > /mnt/da0p1/Backups/OS/bhyve/Ubuntu/im* . --no-compress --info=progress2 > > > > sending incremental file list > > impish-cuda-11-4-nvidia-470.img > > > > 2.13M 0% 9.49kB/s 6284:55:38 > > > > > > and : where do you save large files ? what's the procedure that you use > > to copy large files with a decent speed ? Unfortunately under Linux is > > not safe to use a RW ufs disk access. So,I'm out of solutions. > > > > - > > Mario. > > > > -- > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > Kevin Korb Phone: (407) 252-6853 > Systems Administrator Internet: > FutureQuest, Inc. Kevin at FutureQuest.net (work) > Orlando, Florida kmk at sanitarium.net (personal) > Web page: https://sanitarium.net/ > PGP public key available on web site. > ~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._.,-*~'`^`'~*-,._., > > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.samba.org/pipermail/rsync/attachments/20211128/988da0ed/attachment.htm>