openssh unix dev - Nov 2016 - one host only: ssh_dispatch_run_fatal

all but one lan host (gv)can ssh connect to host 2x

Included windows 10, several linux.  2x is a solaris host.

That problem host (gv) can connect to all others but 2x.

2x can connect to all other encluding the problem host (gv)

So gv ssh to 2x throws this error:

ssh_dispatch_run_fatal: Connection to 192.168.1.42 port 22: incorrect
signature.

192.168.1.42 is 2x.  So incoming ssh from gv to 2x has the problem.

So far I have removed all ssh config and files on both hosts and
reinstalled openssh.

The problem remains

Below is the full output of ssh -vv

gv (a gentoo host) ssh to 2x (a solaris <openindian> host).

gv harry> ssh -vv 2x

OpenSSH_7.3p1-hpn14v11, OpenSSL 1.0.2j  26 Sep 2016
debug1: Reading configuration data /etc/ssh/ssh_config
Pseudo-terminal will not be allocated because stdin is not a terminal.
debug2: resolving "2x" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 2x [192.168.1.42] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file /home/harry/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/harry/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/harry/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/harry/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/harry/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/harry/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3p1-hpn14v11
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6
debug1: match: OpenSSH_6.6 pat OpenSSH_6.5*,OpenSSH_6.6* compat 0x14000000
debug1: Remote is NON-HPN aware
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to 2x:22 as 'harry'
debug2: compat_kex_proposal: original KEX proposal: curve25519-sha256 at
libssh.org,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: Compat: skipping algorithm "curve25519-sha256 at libssh.org"
debug2: compat_kex_proposal: compat KEX proposal:
diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: AUTH STATE IS 0
debug2: local client KEXINIT proposal
debug2: KEX algorithms:
diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,ext-info-c
debug2: host key algorithms: ssh-ed25519-cert-v01 at
openssh.com,ssh-ed25519,ssh-rsa-cert-v01 at
openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug2: ciphers ctos: chacha20-poly1305 at
openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm at
openssh.com,aes256-gcm at openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc
debug2: ciphers stoc: chacha20-poly1305 at
openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm at
openssh.com,aes256-gcm at openssh.com,aes128-cbc,aes192-cbc,aes256-cbc,3des-cbc
debug2: MACs ctos: umac-64-etm at openssh.com,umac-128-etm at
openssh.com,hmac-sha2-256-etm at openssh.com,hmac-sha2-512-etm at
openssh.com,hmac-sha1-etm at openssh.com,umac-64 at openssh.com,umac-128 at
openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm at openssh.com,umac-128-etm at
openssh.com,hmac-sha2-256-etm at openssh.com,hmac-sha2-512-etm at
openssh.com,hmac-sha1-etm at openssh.com,umac-64 at openssh.com,umac-128 at
openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib at openssh.com,zlib
debug2: compression stoc: none,zlib at openssh.com,zlib
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256 at
libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ssh-ed25519
debug2: ciphers ctos:
aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm at
openssh.com,aes256-gcm at openssh.com,chacha20-poly1305 at
openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc
at lysator.liu.se
debug2: ciphers stoc:
aes128-ctr,aes192-ctr,aes256-ctr,arcfour256,arcfour128,aes128-gcm at
openssh.com,aes256-gcm at openssh.com,chacha20-poly1305 at
openssh.com,aes128-cbc,3des-cbc,blowfish-cbc,cast128-cbc,aes192-cbc,aes256-cbc,arcfour,rijndael-cbc
at lysator.liu.se
debug2: MACs ctos: hmac-md5-etm at openssh.com,hmac-sha1-etm at
openssh.com,umac-64-etm at openssh.com,umac-128-etm at
openssh.com,hmac-sha2-256-etm at openssh.com,hmac-sha2-512-etm at
openssh.com,hmac-ripemd160-etm at openssh.com,hmac-sha1-96-etm at
openssh.com,hmac-md5-96-etm at openssh.com,hmac-md5,hmac-sha1,umac-64 at
openssh.com,umac-128 at
openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160 at
openssh.com,hmac-sha1-96,hmac-md5-96
debug2: MACs stoc: hmac-md5-etm at openssh.com,hmac-sha1-etm at
openssh.com,umac-64-etm at openssh.com,umac-128-etm at
openssh.com,hmac-sha2-256-etm at openssh.com,hmac-sha2-512-etm at
openssh.com,hmac-ripemd160-etm at openssh.com,hmac-sha1-96-etm at
openssh.com,hmac-md5-96-etm at openssh.com,hmac-md5,hmac-sha1,umac-64 at
openssh.com,umac-128 at
openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-ripemd160,hmac-ripemd160 at
openssh.com,hmac-sha1-96,hmac-md5-96
debug2: compression ctos: none,zlib at openssh.com
debug2: compression stoc: none,zlib at openssh.com
debug2: languages ctos: 
debug2: languages stoc: 
debug2: first_kex_follows 0 
debug2: reserved 0 
debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: REQUESTED ENC.NAME is 'chacha20-poly1305 at openssh.com'
debug1: kex: server->client cipher: chacha20-poly1305 at openssh.com MAC:
<implicit> compression: none
debug1: REQUESTED ENC.NAME is 'chacha20-poly1305 at openssh.com'
debug1: kex: client->server cipher: chacha20-poly1305 at openssh.com MAC:
<implicit> compression: none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(2048<8192<8192) sent
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug2: bits set: 4070/8192
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-ed25519
SHA256:fDVJdFDNtnMrobEPdtuoLTTTXju1T+V+aKRGdI5Yook
debug1: Host '2x' is known and matches the ED25519 host key.
debug1: Found key in /home/harry/.ssh/known_hosts:2
debug2: bits set: 4134/8192
debug2: ssh_ed25519_verify: crypto_sign_ed25519_open failed: -1
ssh_dispatch_run_fatal: Connection to 192.168.1.42 port 22: incorrect signature

On Tue, Nov 8, 2016 at 1:02 PM, Harry Putnam <reader at newsguy.com>
wrote:
[...]
> gv harry> ssh -vv 2x
>
> OpenSSH_7.3p1-hpn14v11, OpenSSL 1.0.2j  26 Sep 2016
this is a third-party modified version of OpenSSH.  Can you reproduce
the problem with a stock OpenSSH from the source from openssh.com?
> debug1: match: OpenSSH_6.6 pat OpenSSH_6.5*,OpenSSH_6.6* compat 0x14000000
OpenSSH 6.6 has a bug in curve25519-sha256 at libssh.org, which is the
kex method later selected.

Quoting the 6.7 release notes: https://www.openssh.com/releasenotes.html#6.7
"""
 * OpenSSH 6.5 and 6.6 have a bug that causes ~0.2% of connections
   using the curve25519-sha256 at libssh.org KEX exchange method to fail
   when connecting with something that implements the specification
   correctly. OpenSSH 6.7 disables this KEX method when speaking to
   one of the affected versions.
"""
> debug1: kex: host key algorithm: ssh-ed25519
[...]
> debug1: Found key in /home/harry/.ssh/known_hosts:2
> debug2: bits set: 4134/8192
> debug2: ssh_ed25519_verify: crypto_sign_ed25519_open failed: -1
> ssh_dispatch_run_fatal: Connection to 192.168.1.42 port 22: incorrect
signature
Maybe the same bug also affects ed25519 as a host key algorithm?  If
so, setting HostKeyAlgorithms in ssh_config on the client to something
that doesn't include ssh-ed25519 might help.

-- 
Darren Tucker (dtucker at zip.com.au)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA (new)
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

Darren Tucker <dtucker at zip.com.au> writes:
> On Tue, Nov 8, 2016 at 1:02 PM, Harry Putnam <reader at newsguy.com>
wrote:
> [...]
>> gv harry> ssh -vv 2x
>>
>> OpenSSH_7.3p1-hpn14v11, OpenSSL 1.0.2j  26 Sep 2016
>
> this is a third-party modified version of OpenSSH.  Can you reproduce
> the problem with a stock OpenSSH from the source from openssh.com?
Well, since I'm not on a BSD system... <on gentoo linux> and since the
instructions in the README file:

To extract and install this release on your OpenBSD system use:

      # cd /usr/src/usr.bin
      # tar xvfz .../openssh-x.y.tgz
      # cd ssh
      # make obj
      # make cleandir
      # make depend
      # make
      # make install
      # cp ssh_config sshd_config /etc/ssh

And since there appear to be no other helpful files... I guess mucking
around with it and pounding thru google for a hour or two are a little
above my paygrade... not to mention skill level.... I guess not.

What I am doing is backup down to version 7.2 and compiling on gentoo
system. ... will report back.

I can say that 3 Debian systems running verion 6.7 have no problem
connecting to that same host.

Darren Tucker <dtucker at zip.com.au> writes:
> On Tue, Nov 8, 2016 at 1:02 PM, Harry Putnam <reader at newsguy.com>
wrote:
> [...]
>> gv harry> ssh -vv 2x
>>
>> OpenSSH_7.3p1-hpn14v11, OpenSSL 1.0.2j  26 Sep 2016
>
> this is a third-party modified version of OpenSSH.  Can you reproduce
> the problem with a stock OpenSSH from the source from openssh.com?
I can report that gentoo's version 7.2 probably also with some kind of
3rd party changes.... also fails in the same way.

Darren Tucker <dtucker at zip.com.au> writes:
> On Tue, Nov 8, 2016 at 1:02 PM, Harry Putnam <reader at newsguy.com>
wrote:
> [...]
>> gv harry> ssh -vv 2x
>>
>> OpenSSH_7.3p1-hpn14v11, OpenSSL 1.0.2j  26 Sep 2016
>
> this is a third-party modified version of OpenSSH.  Can you reproduce
> the problem with a stock OpenSSH from the source from openssh.com?
I found a comment somewhere that explained that I needed the portable
version. ... got that building right now so may have something shortly