freebsd stable - Apr 2021 - freebsd-update and speed

On 2021-04-18 03:12:35 (+0800), Rainer Duffner wrote:
>> Am 16.04.2021 um 10:17 schrieb Ferdinand Goldmann 
>> <ferdinand.goldmann at jku.at>:
>>
>> On Thu, 15 Apr 2021, Rainer Duffner wrote:
>>
>>>
>>>
>>> It?s OK-ish most of the time here (CH).
>>>
>>> It does *NOT* work through a proxy, due to the use of pipelined 
>>> http-requests.
>>>
>>> What?s your internet-connection?
>>
>> The 10Gbit uplink of my university, directly connected to the 
>> internet, not
>> behind a proxy. I don't think that's the problem. When update3
was
>> still online
>> I'd always use that and updates were really fast back then.
>>
>> Now that update3 is gone all update servers seem to be in the US or 
>> Australia.
>>
>> After waiting for nearly one hour:
>>
>>
..8530....8540....8550....8560....8570....8580....8590....8600....8610....8620....8630....8640....8650....8660....8670....8680....8690....8700....
>>  done.
>> Applying patches... done.
>> Fetching 9628 files... gunzip: (stdin): unexpected end of file
>> 0a4626107f3700cf5f87bd9c123bf427bd5a8561aadc2eca1d1605465c090935 has 
>> incorrect hash.
>>
>> This is getting kind of tiresome. :(
>>
>> Regards
>> Ferdinand
>
>
>
>
> There seems to be a problem with update4.
>
> I now have this, too.
>
>
> I?m cc-ing clusteradm and dnsadmin, in hope that there?s somebody 
> there who can either fix it or take update4 out of the srv record?
I can take update4 out of the DNS if it's misbehaving consistently.  If 
at all possible though, I'd prefer to fix the actual problem rather than 
simply make it disappear from the DNS.

It looks like there were at least experiments with pointing 
freebsd-update at AWS, similar to how portsnap currently works.  I will 
check if these experiments went anywhere and possibly point 
freebsd-update there instead.

I believe the problem with update4 is load-related.
> :-(
>
> I would rather just mirror the update server but I think this is not 
> supposed to be done?
I think you can set up your own freebsd-update servers, but that won't 
fix this problem.

I'll see what can be done to fix this.

Watch this space.

Philip [hat: clusteradm firefighter]


-- 
Philip Paeps
Senior Reality Engineer
Alternative Enterprises

On 2021-04-18 08:51:05 (+0800), Philip Paeps wrote:
> On 2021-04-18 03:12:35 (+0800), Rainer Duffner wrote:
>> I?m cc-ing clusteradm and dnsadmin, in hope that there?s somebody 
>> there who can either fix it or take update4 out of the srv record?
>
> I can take update4 out of the DNS if it's misbehaving consistently.  
> If at all possible though, I'd prefer to fix the actual problem rather 
> than simply make it disappear from the DNS.
I've taken update4 out of the SRV record for the time being since it was 
doing more harm than good.

Philip

-- 
Philip Paeps
Senior Reality Engineer
Alternative Enterprises

On Sun, 18 Apr 2021 at 10:51, Philip Paeps <philip at freebsd.org> wrote:
>
>
> It looks like there were at least experiments with pointing
> freebsd-update at AWS, similar to how portsnap currently works.  I will
> check if these experiments went anywhere and possibly point
> freebsd-update there instead.
>
The AWS freebsd-update has been working fine for quite a while.  All
project mirrors are slow in Australia so we have been using the AWS one
since Colin brought it online to make it a better experience for our team
to update the fleet.

FWIW adjust update.FreeBSD.org to aws.update.FreeBSD.org in
/etc/freebsd-update.conf and you are good to go.

Can the project look at offering traditional mirrors for base and pkgs
rather than the current offering?  Those that want to stand up un-official
mirrors can do so by pointing rsync to a Tier 2 mirror for updating
purposes so they can provide faster, localised access.  From Melbourne to
our closest geo mirror is 240ms, this latency really drags out updates, so
having a mirror out of a Melbourne DC would be beneficial.

Cheers,

Jason.