C Bergström via llvm-dev
2016-Aug-19 15:22 UTC
[llvm-dev] OT: Cloning llvm repo over low speed connection != fun
Hah! Woo is me you say... Have you ever stayed in a non-hilton hotel while traveling? I don't mean couch surfing, but just average place.. I'd even add some above average places to the list and when you get 100 people all on the same line.. it can get "slow"... Lets also factor in networking policies that favor normal internet traffic.. So they may allow youtube to stream like normal, but ssh or something else gets throttled a lot.. (which is what's happening in my case) I wish I was in Tibet.. I'll have to add that to my things to try if I visit some day.. ;) You may mock me and think this can't ever happen, but I'm happy to benchmark real world connection speeds while traveling. On Fri, Aug 19, 2016 at 11:14 PM, James Molloy <james at jamesmolloy.co.uk> wrote:> Hi, > > No matter what we do, I hardly think it likely we can optimize the LLVM > clone speed for a 20KB/s connection. I agree that corporate internet > connections being gigantic pipes means we run the risk of forgetting what > home infrastructure looks like, but 20KBps is... around dial-up. Where are > you, rural Tibet? > > Cheers, > > James > > On Fri, 19 Aug 2016 at 15:55 C Bergström <llvm-dev at lists.llvm.org> wrote: >> >> I remember reading all those examples about how a big repo isn't an >> issue and when sitting at some ${LOCATION} I have 1010101010Gbps and >> it takes like 30 seconds.. Well.. that may be true when I'm at home, >> but I'm not.. and this laptop doesn't have llvm sources installed.. >> >> So real world feedback of the pita it can be.. >> /* Side rant - I wish I didn't even need the llvm sources. I just want >> to build libcxxrt */ >> >> /* Yes I'm about to do the smarter thing and fetch a shallow clone or >> try to get the tarball snapshot from github.. I did however want to >> show an example of how big repo on non-corporate infrastructure can be >> a challenge. I'd hate for some ambitious community dev to run into the >> same thing and turn away as a result */ >> ------------ >> CMake Error at CMakeLists.txt:53 (message): >> llvm-config not found and LLVM_MAIN_SRC_DIR not defined. Reconfigure >> with >> -DLLVM_CONFIG=path/to/llvm-config or >> -DLLVM_PATH=path/to/llvm-source-root. >> >> >> -- Configuring incomplete, errors occurred! >> See also "/usr/local/src/libcxxabi/build/CMakeFiles/CMakeOutput.log". >> asura at asura-X3:/usr/local/src/libcxxabi/build$ cd ../.. >> asura at asura-X3:/usr/local/src$ git clone >> git at github.com:llvm-mirror/llvm.git >> Cloning into 'llvm'... >> remote: Counting objects: 1228082, done. >> remote: Compressing objects: 100% (245/245), done. >> Receiving objects: 13% (167280/1228082), 68.86 MiB | 20.00 KiB/s >> >> ^Cfatal: The remote end hung up unexpectedly >> >> asura at asura-X3:/usr/local/src$ rm -rf llv^C >> asura at asura-X3:/usr/local/src$ git clone >> git at github.com:llvm-mirror/llvm.git >> Cloning into 'llvm'... >> remote: Counting objects: 1228082, done. >> remote: Compressing objects: 100% (245/245), done. >> ^Cfatal: The remote end hung up unexpectedly1 KiB | 20.00 KiB/s >> >> asura at asura-X3:/usr/local/src$ git clone >> https://github.com/llvm-mirror/llvm.git >> Cloning into 'llvm'... >> remote: Counting objects: 1228082, done. >> remote: Compressing objects: 100% (245/245), done. >> error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull >> function. >> fatal: The remote end hung up unexpectedly >> fatal: early EOF >> fatal: index-pack failed >> asura at asura-X3:/usr/local/src$ cd lib^C >> asura at asura-X3:/usr/local/src$ git clone >> https://github.com/llvm-mirror/llvm.git >> Cloning into 'llvm'... >> remote: Counting objects: 1228183, done. >> remote: Compressing objects: 100% (346/346), done. >> ^Cceiving objects: 0% (1069/1228183), 388.01 KiB | 1024 bytes/s >> asura at asura-X3:/usr/local/src$ git clone >> https://github.com/llvm-mirror/llvm.git >> Cloning into 'llvm'... >> remote: Counting objects: 1228183, done. >> error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull >> function. >> fatal: The remote end hung up unexpectedly >> fatal: protocol error: bad pack header >> asura at asura-X3:/usr/local/src$ git clone >> https://github.com/llvm-mirror/llvm.git >> Cloning into 'llvm'... >> remote: Counting objects: 1228271, done. >> remote: Compressing objects: 100% (11/11), done. >> error: RPC failed; curl 56 GnuTLS recv error (-54): Error in the pull >> function. >> fatal: The remote end hung up unexpectedly >> fatal: early EOF >> fatal: index-pack failed >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Renato Golin via llvm-dev
2016-Aug-19 15:30 UTC
[llvm-dev] OT: Cloning llvm repo over low speed connection != fun
On 19 August 2016 at 16:22, C Bergström <llvm-dev at lists.llvm.org> wrote:> Have you ever stayed in a non-hilton hotel while traveling? I don't > mean couch surfing, but just average place.. I'd even add some above > average places to the list and when you get 100 people all on the same > line.. it can get "slow"...I've been to many places while going to Connects, events and sprints, and I can attest that hotels, no matter how many stars, have horrible Internet connection. One hotel we had to tell the admin to stop throttling SSH and also to allow connections to more ports than just 25, 80 and 443. But that affected *everybody*. Using Git or SVN, using Windows or Linux, trying to download Linux, LLVM, GCC, Linaro stuff. So, while I do literally feel your pain around 5 weeks per year, so does every one else in the same situation. There are no solutions to that problem other than "not going there", which for me, really isn't an option. Bottom line is, clone your repos before going away and always keep an updated repo in a backup, so you can use it to considerably reduce the download time on remote places. cheers, --renato
Konstantin Tokarev via llvm-dev
2016-Aug-19 15:34 UTC
[llvm-dev] OT: Cloning llvm repo over low speed connection != fun
19.08.2016, 18:30, "Renato Golin via llvm-dev" <llvm-dev at lists.llvm.org>:> On 19 August 2016 at 16:22, C Bergström <llvm-dev at lists.llvm.org> wrote: >> Have you ever stayed in a non-hilton hotel while traveling? I don't >> mean couch surfing, but just average place.. I'd even add some above >> average places to the list and when you get 100 people all on the same >> line.. it can get "slow"... > > I've been to many places while going to Connects, events and sprints, > and I can attest that hotels, no matter how many stars, have horrible > Internet connection. > > One hotel we had to tell the admin to stop throttling SSH and also to > allow connections to more ports than just 25, 80 and 443. > > But that affected *everybody*. Using Git or SVN, using Windows or > Linux, trying to download Linux, LLVM, GCC, Linaro stuff.But with SVN you don't fetch complete history.> > So, while I do literally feel your pain around 5 weeks per year, so > does every one else in the same situation. There are no solutions to > that problem other than "not going there", which for me, really isn't > an option. > > Bottom line is, clone your repos before going away and always keep an > updated repo in a backup, so you can use it to considerably reduce the > download time on remote places. > > cheers, > --renato > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-- Regards, Konstantin