openssh unix dev - Jan 2017 - sshd 7.4p1 with ssl 1.0.2j seg faults, MacOSX 10.12.2/3, clang-800.0.42.1

Never had much trouble building on the Mac until this round.

Trying to build 7.4p1 with openssl 1.0.2j on a MacOSX 10.12.2/3
machine.  gcc --version returns clang-800.0.42.1.  This is the
latest Xcode.

Builds fine.  Upon running sshd, it seg faults with this in the logs:

assertion failed 16C67:  libsystem_trace.dynlib+76912
[5BD4ECD4-75CA-38EA-AF5C-B481C15955F8]: 0x0

If I run the tests, it fails in:

test_utf8

regress/unittests/utf8/tests.c:  51
test #9  "utf8_inv_badbyt"

ASSERT_INT_EQ (len, wantlen) failed:
  len = 2
  wantlen = 5

/bin/sh:  line 1:  739  Abort trap: 6.

Thank you for your time and for any clues.

Cheers,
Zube

On Thu, Jan 26, 2017 at 10:49 AM, Zube <Zube at stat.colostate.edu> wrote:
[...]
> regress/unittests/utf8/tests.c:  51
> test #9  "utf8_inv_badbyt"
>
> ASSERT_INT_EQ (len, wantlen) failed:
>   len = 2
>   wantlen = 5
That's not a segfault, it's an assertion failure in a UTF8 unit test,
most likely because it's not escaping something that the tests think
should be.
You can skip these tests by setting the environment variable
TEST_SSH_UTF8=no to see if there are other problems.

The test in question is:

  one("inv_badbyte", "\377x", -2, -2, -2,
"\\377x");

which passes it through OpenSSH's snmprintf which passes it through a
handful of multibyte and wide character functions, so it's not
immediately obvious what's going on.  It passes here on a mac mini
running 11.4.2, though, so it'd be interesting to see what's different
between them.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA (new)
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

On Thu Jan 26 11:41:50 AM, Darren Tucker wrote:
> On Thu, Jan 26, 2017 at 10:49 AM, Zube <Zube at stat.colostate.edu>
wrote:
> [...]
> > regress/unittests/utf8/tests.c:  51
> > test #9  "utf8_inv_badbyt"
> >
> > ASSERT_INT_EQ (len, wantlen) failed:
> >   len = 2
> >   wantlen = 5
> 
> That's not a segfault, it's an assertion failure in a UTF8 unit
test,
> most likely because it's not escaping something that the tests think
> should be.
> You can skip these tests by setting the environment variable
> TEST_SSH_UTF8=no to see if there are other problems.
> 
> The test in question is:
> 
>   one("inv_badbyte", "\377x", -2, -2, -2,
"\\377x");
> 
> which passes it through OpenSSH's snmprintf which passes it through a
> handful of multibyte and wide character functions, so it's not
> immediately obvious what's going on.  It passes here on a mac mini
> running 11.4.2, though, so it'd be interesting to see what's
different
> between them.
Thank you for your reply.

Sorry for the delay in getting back to this.  For the record, I do
see a segfault if I try to run sshd as a non-root user.  Not sure
if that is relevant, though.

Let me take it from the top and add additional information.

openssl 1.0.2k is configured with:

./Configure shared darwin64-x86_64-cc

openssh 7.4p1 is configured with:

./configure --prefix=/usr/local/ssh --with-ssl-dir=/usr/local/ssl 
--with-ldflags=-ldl --with-md5-passwords --with-pam --with-sandbox=rlimit 
--without-pie

On a 10.12.2 machine using Apple LLVM version 7.3.0 (clang-703.0.31),
this builds and runs fine.

On a 10.12.2 machine using Apple LLVM version 8.0.0 (clang-800.0.42.1),
it builds fine, but when executed, I get these two entries in the system
logs:

com.apple.xpc.launchd[1] (com.apple.ReportCrash.Root[5419]): Endpoint 
has been activated through legacy launch(3) APIs. Please switch to XPC 
or bootstrap_check_in(): com.apple.ReportCrash.DirectoryService

assertion failed 16C67:  libsystem_trace.dynlib+76912 
[5BD4ECD4-75CA-38EA-AF5C-B481C15955F8]: 0x0

and nothing else.  sshd does not run.

Then we have the UTF8 failure from the tests noted above.  If I set
TEST_SSH_UTF8=no and rerun the tests, it fails later on when it tries
to run connect.sh:

Fatal:  no sshd running on port 4242
make[1]: *** [t-exec] Error 1

So, again, sshd built with the latter compiler falls over when executed.

Thanks for any additional clues.  Perhaps it's time to build or brew
gcc and be done with it.

Cheers,
Zube