Jakub Jelen
2020-Feb-24 10:24 UTC
Re-adding PKCS#11 key in ssh-agent produces "agent refused operation" error.
On Sat, 2020-02-22 at 10:50 -0600, Douglas E Engert wrote:> As a side note, OpenSC is looking at issues with using tokens vs > separate > readers and smart cards. The code paths in PKCS#11 differ. Removing a > card > from a reader leaves the pkcs#11 slot still available. Removing a > token (Yubikey) > removes both the reader and and its builtin smart card. Firefox has a > similar > problem. > > See > https://github.com/OpenSC/OpenSC/pull/1947 and #1945, #1935 and #1908 > https://bugzilla.mozilla.org/show_bug.cgi?id=1613632 > > #1947 may be withdrawn and replaced with a different approach.Right, I tried to address transparent smart card/yubikey removal in the OpenSSH before [1], but it still had some issues inside of OpenSC [2] that should be hopefully addressed by this time (though some more referenced by previous mail might still be present). I would like to get back to this in coming months as it is popping up as a common pain point over and over. Rebasing/testing would be welcomed. [1] https://bugzilla.mindrot.org/show_bug.cgi?id=2890 [2] https://github.com/OpenSC/OpenSC/issues/1822 On 2/21/2020 9:53 PM, Jacob Hoffman-Andrews wrote:> > Hi all, > > > > Thanks for all your hard work! I was particularly excited to see > > FIDO/U2F support in the latest release. > > > > I'd like to make the following bug report in ssh-agent's PKCS#11 > > support: > > > > Steps to reproduce: > > > > 1. Configure a smart card (e.g. Yubikey in PIV mode) as an SSH key. > > 2. Add that key to ssh-agent. > > 3. Remove that key from ssh-agent. > > 4. Add that key to ssh-agent. > > > > Expected results: > > > > Key is successfully added to ssh-agent. > > > > Actual results: > > > > ssh-add fails with "agent refused operation". > > > > I've looked at the code, and it appears that > > register_pkcs11_provider > > ( > > https://github.com/openssh/openssh-portable/blob/master/ssh-pkcs11.c#L1470 > > ) > > fails if a PKCS#11 provider already exists. However, PKCS#11 > > providers > > are never unloaded. There is a pkcs11_del_provider but it is never > > called. > > > > That means that after deleting a key, there is no way to re-add it. > > Also, since > > removing a USB smartcard reader results in ssh-agent losing its > > session, the > > user must call ssh-add again after reinserting the USB card reader, > > and that > > second ssh-add will fail in the same way. > > > > I think the best fix here is to treat "provider already exists" as > > a non-error. > > There is no need to unload providers when they become unused > > because they > > don't use very much memory, and because it is uncommon to have more > > than one > > provider on any given system. Also, a user is likely to reuse a > > provider they > > have previously used. > > > > If a maintainer can confirm that this is an acceptable fix, I may > > be able to > > write a patch. > > > > Environments reproduced on: Ubuntu 19.10, Fedora > > Version of OpenSSH: git commit b2491c28, latest at time of writing. > > > > Example output demonstrating the problem (with a Yubikey in PIV > > mode inserted): > > > > $ SSH_AUTH_SOCK=/tmp/ssh-dhfNCpXwSk8B/agent.21022; export > > SSH_AUTH_SOCK; > > $ ssh-add -s /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so > > Enter passphrase for PKCS#11: > > Could not add card "/usr/lib/x86_64-linux-gnu/opensc-pkcs11.so": > > agent > > refused operation > > $ SSH_AUTH_SOCK=/tmp/ssh-RORElJeiiHBc/agent.21116; export > > SSH_AUTH_SOCK; > > $ ssh-add -s /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so > > Enter passphrase for PKCS#11: > > Card added: /usr/lib/x86_64-linux-gnu/opensc-pkcs11.so > > $ ssh-add -D > > All identities removed.I think the problem here is that the -D switch is not smartcards aware. PKCS#11 modules should be removed using -e switch, which works fine to my testing. The correct fix would be for the -D switch to remove the pkcs11 providers too. Regards, -- Jakub Jelen Senior Software Engineer Security Technologies Red Hat, Inc.
Jacob Hoffman-Andrews
2020-Feb-24 20:41 UTC
Re-adding PKCS#11 key in ssh-agent produces "agent refused operation" error.
On Mon, Feb 24, 2020 at 2:29 AM Jakub Jelen <jjelen at redhat.com> wrote:> I think the problem here is that the -D switch is not smartcards aware. > PKCS#11 modules should be removed using -e switch, which works fine to > my testing.Aha, thanks for pointing this flag out to me. I had missed it. Indeed, `ssh-add -e` does fix this issue for me on the latest release (though on the release that ships with Ubuntu 19.10, "OpenSSH_8.0p1", it fails). I realized there's a similar problem with the `-d` flag: If you delete an identity backed by a PKCS#11 device, it will remove the identity and report success but not remove the provider. Is it desirable in the future to have multiple identities offered by the same provider? For instance, multiple instances of the same smartcard reader? If so, we would need to have some facility to keep track of already-loaded providers and reuse them, as well as do reference counting for removed identities. That's why I was suggesting it would be more straightforward to never unload providers (or in other words, require a restart of ssh-agent if user requires that provider to be non-resident, which I think is quite rare). FWIW, I maintain a signing library in Go that uses PKCS#11, and it uses the approach I describe above, keeping the PKCS#11 module loaded until end of process: https://github.com/letsencrypt/pkcs11key/blob/master/key.go#L113.
Jakub Jelen
2020-Feb-25 09:09 UTC
Re-adding PKCS#11 key in ssh-agent produces "agent refused operation" error.
On Mon, 2020-02-24 at 12:41 -0800, Jacob Hoffman-Andrews wrote:> On Mon, Feb 24, 2020 at 2:29 AM Jakub Jelen <jjelen at redhat.com> > wrote: > > I think the problem here is that the -D switch is not smartcards > > aware. > > PKCS#11 modules should be removed using -e switch, which works fine > > to > > my testing. > > Aha, thanks for pointing this flag out to me. I had missed it. > Indeed, > `ssh-add -e` does fix this issue for me on the latest release (though > on the > release that ships with Ubuntu 19.10, "OpenSSH_8.0p1", it fails). > > I realized there's a similar problem with the `-d` flag: If you > delete > an identity > backed by a PKCS#11 device, it will remove the identity and report > success > but not remove the provider.Thank you for pointing that. It is certainly something that should be fixed. Can you open a new bug in so it will not get lost: https://bugzilla.mindrot.org/ Hopefully I will be able to look in to it in coming weeks.> Is it desirable in the future to have multiple identities offered by > the same > provider? For instance, multiple instances of the same smartcard > reader? > If so, we would need to have some facility to keep track of already- > loaded > providers and reuse them, as well as do reference counting for > removed > identities. That's why I was suggesting it would be more > straightforward > to never unload providers (or in other words, require a restart of > ssh-agent > if user requires that provider to be non-resident, which I think is > quite rare). > > FWIW, I maintain a signing library in Go that uses PKCS#11, and it > uses the > approach I describe above, keeping the PKCS#11 module loaded until > end > of process: > https://github.com/letsencrypt/pkcs11key/blob/master/key.go#L113.Never unloading pkcs11 modules can have unexpected results for users of for example long running ssh-agents and updates -- if you update pkcs11 module, you expect that if you remove it and add it back, it will load the new one. I implemented a way of adding different keys from single or different pkcs11 modules using PKCS #11 URIs, which is in use in Fedora: https://github.com/Jakuje/openssh-portable/commits/jjelen-pkcs11 Regards, -- Jakub Jelen Senior Software Engineer Security Technologies Red Hat, Inc.
Jacob Hoffman-Andrews
2020-Apr-04 23:19 UTC
Re-adding PKCS#11 key in ssh-agent produces "agent refused operation" error.
On Mon, Feb 24, 2020 at 2:29 AM Jakub Jelen <jjelen at redhat.com> wrote:> I tried to address transparent smart card/yubikey removal in the > OpenSSH before [1], but it still had some issues inside of OpenSC [2] > that should be hopefully addressed by this time (though some more > referenced by previous mail might still be present). > > [1] https://bugzilla.mindrot.org/show_bug.cgi?id=2890 > [2] https://github.com/OpenSC/OpenSC/issues/1822An update on these: I've rebased the patch in [1] and tested with the latest OpenSC. It works great. Is there anything else I can contribute towards merging that patch? Thanks, Jacob
Jakub Jelen
2020-Apr-27 09:50 UTC
(Was: Re-adding PKCS#11 key in ssh-agent produces "agent refused operation" error.)
On Sat, 2020-04-04 at 16:19 -0700, Jacob Hoffman-Andrews wrote:> On Mon, Feb 24, 2020 at 2:29 AM Jakub Jelen <jjelen at redhat.com> > wrote: > > I tried to address transparent smart card/yubikey removal in the > > OpenSSH before [1], but it still had some issues inside of OpenSC > > [2] > > that should be hopefully addressed by this time (though some more > > referenced by previous mail might still be present). > > > > [1] https://bugzilla.mindrot.org/show_bug.cgi?id=2890 > > [2] https://github.com/OpenSC/OpenSC/issues/1822 > > An update on these: I've rebased the patch in [1] and tested with the > latest > OpenSC. It works great. Is there anything else I can contribute > towards > merging that patch?Darren, Damien, what would it take to consider the following patch to be included in openssh? I believe this is the only painful issue of using smart cards with ssh-agent and why many people still rather use the gpg-agent, which handles this use case correctly. Best Regards, -- Jakub Jelen Senior Software Engineer Security Technologies Red Hat, Inc.