freebsd stable - Dec 2015 - freebsd-update incorrect hashes

I've had problems with freebsd-update for many years now. It is by far the
least reliable component of FreeBSD since I started with the operating system
back at 3.4 in 1999.

Anyhow, I'm usually able to get past the exceedingly slow downloads and
errors to the upgrade process, but this time nothing I do will get me to the
end. I've tried deleting /var/db/freebsd-update but several hours later I
was at the same place again. The internet link is fast, but with a web proxy in
this location, some downloads are slightly delayed while the virus scanner on
the proxy does its thing. Perhaps 3-5 seconds delayed.

I've run the update maybe a dozen times, progressing a few patches each
time. But it will always fetch 64 patches and then the number of files to fetch
will drop by 5-25


# freebsd-update upgrade -r 10.2 -s update.freebsd.org
Looking up update.freebsd.org mirrors... 4 mirrors found.
Fetching metadata signature for 9.3-RELEASE from update6.freebsd.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system... done.

The following components of FreeBSD seem to be installed:
kernel/generic world/base world/doc world/lib32

The following components of FreeBSD do not seem to be installed:
world/games

Does this look reasonable (y/n)? y

Fetching metadata signature for 10.2-RELEASE from update6.freebsd.org... done.
Fetching metadata index... done.
Fetching 1 metadata patches. done.
Applying metadata patches... done.
Fetching 1 metadata files... done.
Inspecting system... done.
Fetching files from 9.3-RELEASE for merging... done.
Preparing to download files... done.
Fetching 64 patches.....10....20....30....40....50....60.. done.
Applying patches... done.
Fetching 1834 files...
109e9b1e3e8719aa81bc06e4c4c8dc642db7137ea8330f11f70b8e91524afef7 has incorrect
hash.


Different file each time with an incorrect hash. So it will make very slow
progress. Oddly, it had no issue downloading the 10,000 patches it needed,
except for the last 64. No idea why it downloads them again and again each time
I attempt this.


Are there any ways to manually dump the right files in the right place? When I
run phttpget by hand, I have no trouble very quickly downloading the files it
seems to want. But how do I trick the system into skipping downloading them
again?

phttpget has no man pages, so I've been unable to get it to spit out any
more verbose options.


Thanks
Ari




-- 
-------------------------->
Aristedes Maniatis
ish
http://www.ish.com.au
Level 1, 30 Wilson Street Newtown 2042 Australia
phone +61 2 9550 5001   fax +61 2 9550 4001
GPG fingerprint CBFB 84B4 738D 4E87 5E5C  5EFA EF6A 7D2E 3E49 102A

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 163 bytes
Desc: OpenPGP digital signature
URL:
<http://lists.freebsd.org/pipermail/freebsd-stable/attachments/20151223/b60fb2cf/attachment.sig>

Am 2015-12-23 13:25, schrieb Aristedes Maniatis:
> I've had problems with freebsd-update for many years now. It is by far
> the least reliable component of FreeBSD since I started with the
> operating system back at 3.4 in 1999.
> 
> Anyhow, I'm usually able to get past the exceedingly slow downloads
> and errors to the upgrade process, but this time nothing I do will get
> me to the end. I've tried deleting /var/db/freebsd-update but several
> hours later I was at the same place again. The internet link is fast,
> but with a web proxy in this location, some downloads are slightly
> delayed while the virus scanner on the proxy does its thing. Perhaps
> 3-5 seconds delayed.


The problem is phttpget or the proxy, depending on the point of view.

Some proxies have (had) problems with the pipelined http requests that 
phttpget seems to use.

apt (Debian/Ubuntu) has, too - but they can be disabled altogether 
there.

http://webcache.googleusercontent.com/search?q=cache:OwcOVJamJOoJ:https://www.astaro.org/gateway-products/web-protection-web-filtering-application-visibility-control/55213-http-pipelining-broken-after-upgrade-utm-9-3-a.html+&cd=1&hl=de&ct=clnk&gl=ch

IMO, there should be an option to use wget instead of phttpget. Or at 
least disable the request-pipelining.
There was a PR with patches floating around to make freebsd-update use 
wget, but it never gained traction.

Also, didn't phttpget have problems with proxies needing authentication?
I usually have authentication at the proxy disabled for *.freebsd.org 
for this reason.