openssh unix dev - Feb 2023 - ssh host keys on cloned virtual machines

On 24.02.23 12:58, Keine Eile wrote:
> does any one of you have a best practice on renewing ssh host keys on 
> cloned machines?
> I have a customer who never thought about that, while cloning all VMs 
> from one template. Now all machines have the exact same host key.
> My approach would be to store a machines MAC address(es). Then when 
> starting the sshd.service, check if this MAC has changed. If so, remove 
> all host keys, let sshd create new ones.
Strictly speaking, *if* you have an interest to make sure that *every* 
VM gets unique host keypairs, then you should implement a cleanup 
routine that takes care of "everything"? that matters to you.

Once you have *that*, the decision whether to trigger it as the last 
step on the template before making a new image, or as the first step on 
a VM created "to stay" from the template, and by what means and 
mechanisms, becomes somewhat secondary.?

I'd be wary of having it triggered automatically whenever the MAC or 
some other "hardware ID" changes, though, as that can happen when you 
move VMs between hypervisors, add or remove virtual devices, etc..

? Erase host keypairs, erase keypairs of local users (so that access to 
elsewhere doesn't get copied along), generate individual moduli, erase 
shell histories, remove local non-system users' crontabs, empty the mail 
spools of same, empty system and application logfiles, hunt for 
passwords set in whatever config files, rename LVM VGs so as to have 
names as unique as the VM itself, .......

? Personally, I like to add color coding to shell prompts to signal 
platform, test vs. prod etc.. Blinking black-on-yellow works quite well 
as a reminder that a VM freshly created from a template might still need 
some finalizing touches. ;-)

Kind regards,
-- 
Jochen Bern
Systemingenieur

Binect GmbH
-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3449 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20230224/e7616c75/attachment-0001.p7s>

On Fri, Feb 24, 2023 at 10:01 AM Jochen Bern <Jochen.Bern at binect.de>
wrote:
>
> On 24.02.23 12:58, Keine Eile wrote:
> > does any one of you have a best practice on renewing ssh host keys on
> > cloned machines?
> > I have a customer who never thought about that, while cloning all VMs
> > from one template. Now all machines have the exact same host key.
> > My approach would be to store a machines MAC address(es). Then when
> > starting the sshd.service, check if this MAC has changed. If so,
remove
> > all host keys, let sshd create new ones.
>
> Strictly speaking, *if* you have an interest to make sure that *every*
> VM gets unique host keypairs, then you should implement a cleanup
> routine that takes care of "everything"? that matters to you.
These vagaries are why many environments simply disable the validation
of hostkeys in their .ssh/config settings and move on to work that is
of some more effective use to their workplace. I've encountered,
several times, when sites relied on extensive use of SSH key managed
git access and shattered their deployment systems when the git server
got moved and hostkeys were either incorrectly migrated or the IP was
a re-used IP of a previously accessed SSH target. Hilarity ensued.
This kind of hand-tuning of every deployment rapidly becomes a waste
of admin time and serves little purpose without very tight control of
the "known_hosts", which can be overridden by local users anyway.