openssh unix dev - Sep 2018 - add keys and certificate to forwarded agent on remote host

Apologies if this post is inappropriate to this list; please redirect me
if so.

Our team uses ssh extensively for server access and maintenance
(Debian). An issue is acting as root when operating, for example, over
ansible and keeping a record of who performed the actions, something ssh
certificates solves well. 

The problem is then to automate certificate issuance since it would be
pretty arduous for someone to keep issuing short-lived user
certificates.

I was intrigued to read Uber's ussh announcement page and wondered if
this suggests a route for doing so:
(https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc)

    An employee gets a ussh certificate when they run the ussh command.
    This connects to the USSHCA, performs the pam conversation and
    forwards the client?s ssh agent to the CA. If the client
    successfully authenticates, the CA generates a new ssh key,
    populates the associated cert with the configured information
    (validity period, the user it?s valid for, the options permitted,
    etc.) and adds both the key and the certificate to the remote agent.
    The certificates are added to the agent with a timeout telling the
    agent to remove the keys when the certificate expires.

Assuming we write a program to generate a new key pair and associated
user certificate, how would one go about adding this to a forwarded
agent on the remote server hosting the program? Can ssh-add work on the
remote socket file? Is such an operation advisable?

Apologies for this rather naive questions, and also if I'm covering
well-worn territory.

Thanks
Rory

Rory Campbell-Lange wrote:
> Can ssh-add work on the remote socket file?
I expect that it will just work<tm>. The local socket is just a
socket, and the protocol[1] message SSH_AGENT_ADD_KEY is the same.

> Is such an operation advisable?
That's up to you. ssh-add decrypts the private key locally where invoked
and transfers the key in a form immediately usable to the agent.

Once the agent has the key, it's not really possible to force the agent
to remove it.


//Peter

[1] https://tools.ietf.org/html/draft-miller-ssh-agent-02

On 17/09/18, Peter Stuge (peter at stuge.se) wrote:
> Rory Campbell-Lange wrote:
> > Can ssh-add work on the remote socket file?
> 
> I expect that it will just work<tm>. The local socket is just a
> socket, and the protocol[1] message SSH_AGENT_ADD_KEY is the same.
Local:

	$ ssh-agent > /tmp/agent.env
    $ source /tmp/agent.env
    $ ssh-add ~/.ssh/id_user
    $ ssh -A remote

Remote:

	$ SSH_AUTH_SOCK=/tmp/ssh-1rVbCSbuDP/agent.3145 
	$ ssh-add newkey
	Identity added: newkey (newkey)

Local:

    $ source /tmp/agent.env
	$ ssh-add -l
	2048 SHA256:32C...qYBs /home/user/.ssh/id_user (RSA)
	2048 SHA256:32C...qYBs /home/user/.ssh/id_user (RSA-CERT)
	2048 SHA256:SZG...5hUQ newkey (RSA)

That worked perfectly, it seems.
> > Is such an operation advisable?
> 
> That's up to you. ssh-add decrypts the private key locally where
invoked
> and transfers the key in a form immediately usable to the agent.
> 
> Once the agent has the key, it's not really possible to force the agent
> to remove it.
I guess one could set a short life on the remotely added key, such as:

Remote:

	SSH_AUTH_SOCK=/tmp/ssh-X85qP7jRtG/agent.4079 
	$ ssh-add -t 300 shortlifekey
	Identity added: shortlifekey (shortlifekey)
	Lifetime set to 300 seconds

Local:

	$ ssh-add -l
	2048 SHA256:32C...qYBs /home/user/.ssh/id_user (RSA)
	2048 SHA256:32C...qYBs /home/user/.ssh/id_user (RSA-CERT)
	2048 SHA256:SZG...5hUQ newkey (RSA)
	2048 SHA256:7IS...JRi8 shortlifekey (RSA)

	wait 5 minutes...

	2048 SHA256:32Cv...qYBs /home/user/.ssh/id_user (RSA)
	2048 SHA256:32Cv...qYBs /home/user/.ssh/id_user (RSA-CERT)
	2048 SHA256:SZGf...5hUQ newkey (RSA)


Thanks for the great pointers
Rory

Why not just use Yubikeys ?  SSH keys (at least the RSA type, the SSH's
developers failure to adopt other supported key types after many years is
something of an un-necessary frustration to the greater SSH community).

So issue your users with Yubikeys.  You can enforce the Yubikey so it requires
the user to enter a PIN *and* touch the Yubikey.  This means there's an
incredibly high degree of confidence that it was the user who performed the
actiion (i.e. two-factor authentication of physical Yubikey and PIN, plus
anti-keylogger because of the mandatory touching of the Yubikey).

You can use Yubikeys with ssh-add too, if you want.  Or you can just use it for
ad-hoc individual logins.

On 9/18/18 9:45 AM, Tim Jones wrote:
> Why not just use Yubikeys ?
Because signature generation is slow.

The original poster mentioned ansible. If you're doing ansible with
thousands of hosts a delay of 800+ ms for each new connection is a major
PITA.

Ciao, Michael.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3829 bytes
Desc: S/MIME Cryptographic Signature
URL:
<http://lists.mindrot.org/pipermail/openssh-unix-dev/attachments/20180918/3e9a19b0/attachment.p7s>

On 18/09/18, Tim Jones (b631093f-779b-4d67-9ffe-5f6d5b1d3f8a at protonmail.ch)
wrote:
...
> So issue your users with Yubikeys.  You can enforce the Yubikey so it
> requires the user to enter a PIN *and* touch the Yubikey.  This means
> there's an incredibly high degree of confidence that it was the user
> who performed the actiion (i.e. two-factor authentication of physical
> Yubikey and PIN, plus anti-keylogger because of the mandatory touching
> of the Yubikey).
I've been meaning to try a Yubikeys. As I understand it that would help
ensure that the user is the person they should be.

What is nice about runtime certificate issuance is that certificates can
be tuned for particular per-user, per-instance use cases, such as "root
on all DC1 webservers".

Unless I've misunderstood, verification of the user and the permissions
they have for potentially many roles on many servers are quite different
things.

Thanks very much 
Rory

On Mon, Sep 17, 2018 at 6:13 AM, Rory Campbell-Lange
<rory at campbell-lange.net> wrote:
> Apologies if this post is inappropriate to this list; please redirect me
> if so.
>
> Our team uses ssh extensively for server access and maintenance
> (Debian). An issue is acting as root when operating, for example, over
> ansible and keeping a record of who performed the actions, something ssh
> certificates solves well.
>
> The problem is then to automate certificate issuance since it would be
> pretty arduous for someone to keep issuing short-lived user
> certificates.
>
> I was intrigued to read Uber's ussh announcement page and wondered if
> this suggests a route for doing so:
>
(https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc)
>
>     An employee gets a ussh certificate when they run the ussh command.
>     This connects to the USSHCA, performs the pam conversation and
>     forwards the client?s ssh agent to the CA. If the client
>     successfully authenticates, the CA generates a new ssh key,
>     populates the associated cert with the configured information
>     (validity period, the user it?s valid for, the options permitted,
>     etc.) and adds both the key and the certificate to the remote agent.
>     The certificates are added to the agent with a timeout telling the
>     agent to remove the keys when the certificate expires.
>
> Assuming we write a program to generate a new key pair and associated
> user certificate, how would one go about adding this to a forwarded
> agent on the remote server hosting the program? Can ssh-add work on the
> remote socket file? Is such an operation advisable?
Hi,

It's unclear if I'm ever going to be able to opensource usshca at this
point, but I can assure that you it's definitely possible to add a
private key (and certificate) to a forwarded ssh-agent. It might not
be clear from the blog post, but usshca is itself an ssh server,
albeit one that doesn't drop an authenticated user into a shell.
openssh's sshd makes a remotely forwarded agent available locally via
unix domain socket, but usshca just takes the remote agent and,
assuming everything's a-ok, adds the key and cert.

there should be examples online of what this looks like, at least in go.

In the example below, if 'conn' is an ssh.ServerConn (ie. what you get
from calling ssh.NewServerConn [1] on an incoming connection), you can
run something like

  agentChan, reqs, err := conn.OpenChannel("auth-agent at
openssh.com", nil)
  if err != nil {
    panic(err)
  }
  go ssh.DiscardRequests(reqs)
  agentConn := agent.NewClient(agentChan)

and now agentConn is an agent interface [2] connecting you to a remote
ssh-agent. You can add your newly generated ssh keys and certificates,
you can fetch public keys, request data be signed, verify signatures,
etc.

Hope this helps.

Cheers,
peter

[1] https://godoc.org/golang.org/x/crypto/ssh#NewServerConn
[2] https://godoc.org/golang.org/x/crypto/ssh/agent#Agent
> Apologies for this rather naive questions, and also if I'm covering
> well-worn territory.
>
> Thanks
> Rory
>
>
> _______________________________________________
> openssh-unix-dev mailing list
> openssh-unix-dev at mindrot.org
> https://lists.mindrot.org/mailman/listinfo/openssh-unix-dev

On 18/09/18, Peter Moody (mindrot at hda3.com) wrote:
> On Mon, Sep 17, 2018 at 6:13 AM, Rory Campbell-Lange
> <rory at campbell-lange.net> wrote:
...
> > The problem is then to automate certificate issuance since it would be
> > pretty arduous for someone to keep issuing short-lived user
> > certificates.
> >
> > I was intrigued to read Uber's ussh announcement page and wondered
if
> > this suggests a route for doing so:
> >
(https://medium.com/uber-security-privacy/introducing-the-uber-ssh-certificate-authority-4f840839c5cc)
...
> > Assuming we write a program to generate a new key pair and associated
> > user certificate, how would one go about adding this to a forwarded
> > agent on the remote server hosting the program? Can ssh-add work on
the
> > remote socket file? Is such an operation advisable?
...
> It's unclear if I'm ever going to be able to opensource usshca at
this
> point, but I can assure that you it's definitely possible to add a
> private key (and certificate) to a forwarded ssh-agent. It might not
> be clear from the blog post, but usshca is itself an ssh server,
> albeit one that doesn't drop an authenticated user into a shell.
> openssh's sshd makes a remotely forwarded agent available locally via
> unix domain socket, but usshca just takes the remote agent and,
> assuming everything's a-ok, adds the key and cert.
Hi Peter

Thanks for the response, and the Go code snippets (unfortunately I'm not
a Go programmer). It would be great if usshca could be open-sourced in
future.

I've verified what you've written about adding keys and certificate to a
forwarded agent following Peter Stuge's helpful comments. Darren Tucker
pointed out that there is nothing to stop someone hacking their agent to
copy a key (even with a timeout), so the use of time-limited
certificates seems important.

The missing piece in the puzzle for our use case is extracting the user
from the connection by pairing their connection key to one in a user
database without having to create a local user for each remote ssh user
on the authenticating server. I assume the usshca ssh server deals with
this by allowing "username at usshca" connections for all known users?

Regards
Rory