openssh unix dev - Aug 2023 - Host key verification (known_hosts) with ProxyJump/ProxyCommand

On 18.08.23 07:39, Darren Tucker wrote:
> On Fri, 18 Aug 2023 at 15:25, Stuart Longland VK4MSL <me at
vk4msl.com> wrote:
> [...]
>> The crux of this is that we cannot assume the local IPv4 address is
>> unique, since it's not (and in many cases, not even static).
> 
> If the IP address is not significant, you can tell ssh to not record
> them ("CheckHostIP no").
If I understand correctly, you need to *know* the target system's local 
172-ish IP to be able to log in. If so, and your DNS admin frowns at 
setting up 16 million RRs to cover 172.0.0.0/8 in preparation, sslip.io 
might be helpful.

https://sslip.io/

Otherwise, and assuming a *manageable* (mainly, enumerable) population 
of remote sites, I wonder whether this approach might work, too?

Host	Perth-47
	HostName		172.23.45.47
	ProxyJump		Perth-GW
	GlobalKnownHostsFile	/dev/null
	UserKnownHostsFile	~/.ssh/known-in-Perth
Host	Adelaide-11
	HostName		172.45.67.11
	ProxyJump		Adelaide-GW
	GlobalKnownHostsFile	/dev/null
	UserKnownHostsFile	~/.ssh/known-in-Adelaide

(Yes, I realize that with target IPs being *potentially dynamic* per 
DHCP, having known hostkeys indexed by site *and IP* might still turn 
out to be bothersome.)

Kind regards,
-- 
Jochen Bern
Systemingenieur

Binect GmbH
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3449 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20230818/2275b97c/attachment-0001.p7s>

On Fri, 18 Aug 2023 at 18:38, Jochen Bern <Jochen.Bern at binect.de>
wrote:
[...]
> If I understand correctly, you need to *know* the target system's local
> 172-ish IP to be able to log in.
No, the client doesn't need to know the server's address.   By default
non-fully-qualified hostnames will be passed to the Jumphost to be
resolved there (subject to the CanonicalDomains and Canonicalized.*
settings) and you also have the option of setting "ProxyCommand ssh -W
..." yourself.

See https://github.com/openssh/openssh-portable/blob/master/ssh.c#L461

-- 
Darren Tucker (dtucker at dtucker.net)
GPG key 11EAA6FA / A86E 3E07 5B19 5880 E860  37F4 9357 ECEF 11EA A6FA
    Good judgement comes with experience. Unfortunately, the experience
usually comes from bad judgement.

On 18/8/23 18:37, Jochen Bern wrote:
> On 18.08.23 07:39, Darren Tucker wrote:
>> On Fri, 18 Aug 2023 at 15:25, Stuart Longland VK4MSL <me at
vk4msl.com>
>> wrote:
>> [...]
>>> The crux of this is that we cannot assume the local IPv4 address is
>>> unique, since it's not (and in many cases, not even static).
>>
>> If the IP address is not significant, you can tell ssh to not record
>> them ("CheckHostIP no").
> 
> If I understand correctly, you need to *know* the target system's local
> 172-ish IP to be able to log in. If so, and your DNS admin frowns at 
> setting up 16 million RRs to cover 172.0.0.0/8 in preparation, sslip.io 
> might be helpful.
> 
> https://sslip.io/
That's a handy little service? not sure of its long-term stability 
though for production use, but one to have a closer look at and keep in 
the memory bank.

It's not so much the DNS admin frowning on its use.  I think the subnets 
involved are /24s and our public DNS infrastructure is Amazon AWS 
managed via Terraform, so it could be scripted if we wanted such detail 
to be publicly visible.  (And we do have a couple of private IPs visible 
on our domain -- mostly so Let's Encrypt can validate the host exists.)

The biggest impediment is the constrained nature of the routers that 
we're using as bastion hosts on site.  We'd have to deploy the DNS 
server either on the router itself, or at a static address within reach 
of it (and configure the router to use that resolver).

 From what I understand of ProxyJump:

	ssh -J proxyuser at proxyhost targetuser at targethost.domain

targethost.domain would need to be resolved by proxyhost, not the local 
client.

Another approach would be to set up /etc/hosts on the bastion, if it 
were a conventional Linux machine I'd have little issue with this.  I'm 
not sure OpenWRT (or at least Teltonica's flavour of it, which is an 
older release) would maintain /etc/hosts changes persistently.
> Otherwise, and assuming a *manageable* (mainly, enumerable) population 
> of remote sites, I wonder whether this approach might work, too?
> 
> Host??? Perth-47
>  ????HostName??????? 172.23.45.47
>  ????ProxyJump??????? Perth-GW
>  ????GlobalKnownHostsFile??? /dev/null
>  ????UserKnownHostsFile??? ~/.ssh/known-in-Perth
> Host??? Adelaide-11
>  ????HostName??????? 172.45.67.11
>  ????ProxyJump??????? Adelaide-GW
>  ????GlobalKnownHostsFile??? /dev/null
>  ????UserKnownHostsFile??? ~/.ssh/known-in-Adelaide
> 
> (Yes, I realize that with target IPs being *potentially dynamic* per 
> DHCP, having known hostkeys indexed by site *and IP* might still turn 
> out to be bothersome.)
Ahh okay, so you can have a separate `UserKnownHostsFile` per host entry.

The situation we have is our workstations' .ssh/config actually imports 
config files from elsewhere (git repo):

Include /home/me/workplace/ops/config/ssh/prod/*
Include /home/me/workplace/ops/config/ssh/dev/*
Include /home/me/workplace/ops/eng-ssh/*-config

So assuming one of those files was 
/home/me/workplace/ops/eng-ssh/bigcust-config

# Bastion router on the site, VPNing back to the office
Host bigcustomer-00123-bne-md01
     HostName	10.20.34.5
     UserKnownHostsFile bigcustomer-00123-bne-md01-hosts

Host bigcustomer-00123-bne-br01
     HostName    172.30.0.100
     ProxyJump   user at bigcustomer-00123-bne-md01
     UserKnownHostsFile bigcustomer-00123-bne-md01-hosts

Host bigcustomer-00123-bne-md02
     HostName	10.20.34.6
     UserKnownHostsFile bigcustomer-00123-bne-md02-hosts

Host bigcustomer-00123-bne-br02
     HostName    172.30.0.100
     ProxyJump   user at bigcustomer-00123-bne-md02
     UserKnownHostsFile bigcustomer-00123-bne-md02-hosts

Would the UserKnownHostsFile be relative to the current working 
directory of the `ssh` process at the time of its call, or would it 
figure out that these files are relative to 
/home/me/workplace/ops/eng-ssh/bigcust-config?

If the latter, I could then store that in the git repository (as a 
*signed* git commit, so it can be authenticated later) which would offer 
similar benefits to the `ExpectHostKey` I made earlier.
-- 
Stuart Longland (aka Redhatter, VK4MSL)

I haven't lost my mind...
   ...it's backed up on a tape somewhere.