On 2021-May-3, at 21:27, Mark Millard <marklmi at yahoo.com> wrote:
> On 2021-May-3, at 19:26, Mark Millard <marklmi at yahoo.com> wrote:
>
>> On 2021-May-3, at 10:51, Mark Millard <marklmi at yahoo.com>
wrote:
>>
>>> On 2021-May-3, at 07:47, Ed Maste <emaste at freebsd.org>
wrote:
>>>
>>>> On Thu, 29 Apr 2021 at 02:50, Mark Millard via freebsd-current
>>>> <freebsd-current at freebsd.org> wrote:
>>>>>
>>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping and
/usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping differ
>>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/sbin/ping6 and
/usr/obj/DESTDIRs/13_0R-CA7-poud/sbin/ping6 differ
>>>>> Files /usr/obj/DESTDIRs/13_0R-CA7-chroot/usr/bin/ntpq and
/usr/obj/DESTDIRs/13_0R-CA7-poud/usr/bin/ntpq differ...
>>>>
>>>> This is unexpected. Unfortunately I haven't looked at
reproducibility
>>>> in a while, and my work was all on x86. This could be a
regression or
>>>> a longstanding issue with arm64.
>>>>
>>>> If you install the diffoscope package (py37-diffoscope) and run
it on
>>>> the two directories / files it should give a more convenient
view of
>>>> the differences. (Or, if you can make a tarball of the
differing files
>>>> I can take a look.)
>>>
>>> I no longer have the same content in those directory
>>> trees: newer rebuild and the same buildworld used to
>>> installworld to both places, instead of 2 different
>>> buildworld's. I'm also unsure how reproducible getting
>>> differences was.
>>>
>>> I can eventually do experiments to test multiple separate
>>> buildworld's and installworld's, but the machine is busy
>>> building ports and the llvm builds involved means it
>>> will be some time before I'd switch activities. And the
>>> buildworld's involve llvm builds as well and take notable
>>> time themselves. So my next comparison will not be any
>>> time soon.
>>>
>>> I'll let you know if I manage to generate another example,
>>> this time being sure to keep the data. If I try multiple
>>> times without finding any differences, I'll eventually
>>> decide "enough is enough" and let you know.
>>
>> I've still got a long ways to go to do the first
>> actual comparison of builds.
>>
>> But I'll note that I've built and stalled py37-diffoscope
>> (new to me). A basic quick test showed that it reports:
>>
>> W: diffoscope.main: Fuzzy-matching is currently disabled as the
"tlsh" module is unavailable.
>>
>> As I'm not familiar with the tool, you might need to send
>> notes about how you want me to use the tool to get the
>> output that you would want. (And, so, I get to learn . . .)
>
> I've tried another experiment (* in the path matches "28" and
"30"):
>
> # diffoscope /.zfs/snapshot/2021-04-*-01:40:48-0/bin/sh
> $<3/>2021-05-03 21:08:48 W: diffoscope.main: Fuzzy-matching is
currently disabled as the "tlsh" module is unavailable.
> $<3/>Traceback (most recent call last):
> File
"/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line 745,
in main
> sys.exit(run_diffoscope(parsed_args))
> File
"/usr/local/lib/python3.7/site-packages/diffoscope/main.py", line 677,
in run_diffoscope
> difference = load_diff_from_path(path1)
> File
"/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py",
line 31, in load_diff_from_path
> return load_diff(codecs.getreader("utf-8")(fp), path)
> File
"/usr/local/lib/python3.7/site-packages/diffoscope/readers/__init__.py",
line 35, in load_diff
> return JSONReaderV1().load(fp, path)
> File
"/usr/local/lib/python3.7/site-packages/diffoscope/readers/json.py",
line 33, in load
> raw = json.load(fp)
> File "/usr/local/lib/python3.7/json/__init__.py", line 293, in
load
> return loads(fp.read(),
> File "/usr/local/lib/python3.7/codecs.py", line 504, in read
> newchars, decodedbytes = self.decode(data, self.errors)
> UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb7 in
position 18: invalid start byte
>
> The two older snapshots of a Boot Environment have
> bin/sh files that compare equal. But every program I
> tried the above sort of thing against on got the same
> UnicodeDecodeError result from diffoscope, byte value
> and position matching.
>
> These snapshots have more than an installworld in them
> and so are messy to compare overall. But the
> installworld (and installkernel) content show similar
> differences to what I reported before as far as
> example files with differences go. But this is aarch64,
> not armv7.
>
> It will still be notable time before I have simple
> installworld tree's to compare.
While waiting for the 2nd buildworld to complete, I used
the 1st buildworld's materials to installworld twice (to
different, empty directory trees) and then did a diff.
The diff reported:
# diff -rq /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/
/usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm2/ | more
diff: /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local: No such file
or directory
diff:
/usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/sys/pjdfstest/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests/tests:
Too many levels of symbolic links
FYI:
# ls -Tld /usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local
lrwxr-xr-x 1 root wheel 14 May 3 23:18:22 2021
/usr/obj/DESTDIRs/13_0R-CA72-instwrld-norm/usr/tests/local -> ../local/tests
So this looks like the expected result and suggests that
the problem(s) are at the buildworld stage. (No surprise.)
I should be sleeping by the time the 2nd buildworld for
cortex-a72 (aarch64) is done. (That is what I decided
to test initially.) So it will be more hours until I have
basic comparison results from 2 distinct buildworlds.
==Mark Millard
marklmi at yahoo.com
( dsl-only.net went
away in early 2018-Mar)