search for: gridftp

Are there any plans to add support for multiple stream copying (BBFTP and GridFTP do this, though only for a single file) to rsync? Thanks, Jason

...; http://psc.edu/index.php/hpn-ssh, or rsync to an nfs mount. I'm not certain what the problem could be. But enabling jumbo packets would be the fist thing I would try, after turning off firewall to test if it isn't involved. Joining "other advises" move: we usually use bbftp and gridftp for massive data transfers. Bbftp: http://doc.in2p3.fr/bbftp/ gridftp is available with globus installation. Just my $0.02 Valeri ++++++++++++++++++++++++++++++++++++++++ Valeri Galtsev Sr System Administrator Department of Astronomy and Astrophysics Kavli Institute for Cosmological Physics Un...

...ourceforge.net/ For a while the UDT API was available, developed by the Laboratory for Advanced Computing at the University of Chicago, but there were little development around it for actually developing a software suite to allow for high speed WAN transfers, such as can be achieved by Aspera, GridFTP, FDT, and a couple others. The problem with those often is: Aspera: Great but *crazy* expensive GridFTP: Not bad but non-trivial to set up FDT: Java (Eh...) But the awesome thing here is that UDR is a lightweight wrapper for rsync that allows for rsync functionality, but uses UDT as the underl...

...width), but the total interconnection to the customer is higher (let's say 10Gb/s). Each front node shares a distributed file system on an internal high bandwidth network. So the contention point is the 1Gb/s limit of a connection. If the customer wants to use more than 1Gb/s, he currently uses GridFTP. We want to provide a solution based on ssh to our customers. 2. The solution I made some changes in the sftp client. The new option "-n" (defaults to 0) sets the number of extra channels. There is one main ssh channel, and n extra channels. The main ssh channel does everything, except...

...kspace Service but can also be used standalone. Cumulus is compatible with the Amazon Web Services S3 REST API and has a pluggable backend that allows it to support multiple storage systems used by the scientific community. It also includes support for quota management. Cumulus replaces the current GridFTP-based upload and download of VM images. The second new feature is the Zero -> Cloud installation process, which significantly simplifies Nimbus installation and also includes user management tools. In addition, this release also contains new scheduling and network configuration options, new pro...

This must get asked all the time, but I didn't see any results from a search in the archive, so... Are there any plans to integrate features such as using configurable, multiple streams, or configurable window sizes? Something like BBFTP or GridFTP features would be nice - is anyone working on this? Rsync is much more feature-rich than either, but they do have a few nice features in terms of bandwidth management. Thanks, Jason

...gain, and remount the disk 325 vi /etc/fstab 326 mount -av I was quite pleasantly surprised by the speed of the reformatted volume when I used lfs_migrate to repopulate it from the file list previously removed. The volume seemed fine. Then a user reported that his newly created files (a gridftp variant was used), that were written to this volume, were corrupted, whereas if they copied to a different volume, then they were just fine. "md5sum" shows they are, indeed, different despite showing the same size via "ls". Can anyone tell me what may have gone wrong here? Is...

Hi, I do have two centos 6.6 servers. With a "performance optimized" rsync I get an speed of 15 - 20 MB/s The options I use are: rsync -aHAXxv --numeric-ids --progress -e "ssh -T -c arcfour -o Compression=no -x" If I copy files by smb to/from the servers I do get 60 - 80 MB/s, a dd (r/w) on the storages attached gives 90 MB/s on the 1Gbit ISCSI (Source Server) and up to

Hello, I know this has come up before; but is the HPN patch (or elements thereof) currently being considered for integration in to the OpenSSH code base? Are there pending issues (buffer management, none cipher, etc) which still need to be addressed? We have been using HPN-SSH for over a year now, and like others, have observed significant performance improvement over standard OpenSSH. I can

...n >> to the customer is higher (let's say 10Gb/s). Each front node shares a >> distributed file system on an internal high bandwidth network. So the contention >> point is the 1Gb/s limit of a connection. If the customer wants to use more than >> 1Gb/s, he currently uses GridFTP. We want to provide a solution based on ssh to >> our customers. >> >> 2. The solution >> >> I made some changes in the sftp client. The new option "-n" (defaults to 0) sets >> the number of extra channels. There is one main ssh channel, and n extra &...

Hi, I''ve been looking at Btrfs and have a couple of naive questions that don''t seem to be answered on the wiki or in the articles I''ve read on the filesystem. First: discovering a file''s checksum value. Here''s the scenario: software is writing some data as a fresh file. This software happens to know (a priori) the checksum of this data; for

...t now?) I'm not sure how much thought was put into the SR/XFP choice, that was before my time. Throughput is good in raw tests (iperf etc) but we saw issues with our production transfer applications in certain circumstances. Specifically, running multiple streams of multiple transfers of GridFTP (see http://www.globus.org/toolkit/data/gridftp/). I think 10 transfers with 10 streams (not my department) would cause the network card to "lock up" and connectivity was completely lost. Generally, this took a matter of minutes to accomplish. Using the RSA interface on the x3650 w...