openssh bugs - Jun 2020 - [Bug 3188] New: Problems creating a second ecdsa-sk key for a second Yubikey

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

            Bug ID: 3188
           Summary: Problems creating a second ecdsa-sk key for a second
                    Yubikey
           Product: Portable OpenSSH
           Version: 8.3p1
          Hardware: Other
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: ssh-keygen
          Assignee: unassigned-bugs at mindrot.org
          Reporter: David at WalkerStreet.info

I've been playing around with ecdsa-sk type keys since 8.3p1 came out
in a recent openSUSE Tumbleweed snapshot. It works fine for me, except
when I try to add a second Yubikey. My first key (a Yubikey 5c Nano)
was set up with "ssh-keygen -t ecdsa-sk" using the default key files
(~/.ssh/id_ecdsa_sk*), but when I try to do the same for a second key
(a Yubikey 5 NFC, using USB), the light doesn't flash on the Yubikey
when I'm prompted to press the Yubikey's button, so pressing the button
has no effect, I'm not prompted for where to store the new key pair,
and no key is created.  I have to use ctrl-C to get out of ssh-keygen.

After this happens, the first key will not work for ssh authentication
for a while (a few hours to a couple of days), even if I reboot the
system. Both keys continue to work with a browser (Vivaldi), though.

Any ideas of how to diagnose what's going on?

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org

--- Comment #1 from Damien Miller <djm at mindrot.org> ---
Please attach debug logs for a successful case and a failed case - use
"ssh-keygen -vvv ..." to increase the verbosity.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #2 from David Walker <David at WalkerStreet.info> ---
Created attachment 3422
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3422&action=edit
ssh-keygen with Yubikey 5c Nano

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #3 from David Walker <David at WalkerStreet.info> ---
Created attachment 3423
  --> https://bugzilla.mindrot.org/attachment.cgi?id=3423&action=edit
ssh-keygen with Yubikey 5 NFC

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #4 from David Walker <David at WalkerStreet.info> ---
(In reply to Damien Miller from comment #1)
> Please attach debug logs for a successful case and a failed case -
> use "ssh-keygen -vvv ..." to increase the verbosity.
I've attached two logs; here's how I created them:

1. Using my "first" Yubikey (the 5c Nano), I generated a test key with
"ssh-keygen -vvv -t ecdsa-sk". The log is in the "ssh-keygen log
with
Yubikey 5c Nano" attachment.

2. I removed the "first" key and inserted the "second" key
(the 5 NFC).

3. I tried to generate a test key with "ssh-keygen -vvv -t ecdsa-sk";
the log is in the "ssh-keygen log with Yubikey 5 NFC" attachment. The
light on the 5 NFC never started flashing, and when I pressed its
button anyway, it appears to have sent an HOTP string.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #3422|application/octet-stream    |text/plain
          mime type|                            |

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #3423|application/octet-stream    |text/plain
          mime type|                            |

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #5 from Damien Miller <djm at mindrot.org> ---
Thanks, it looks like libfido is unable to communicate with your
yk5nfc, but by the OTP it dumped into your keyboard buffer it does seem
to be attached as far as the system is concerned.

Can you try to talk to the card using a tool like ykman? E.g.

$ ykman info 
Device type: YubiKey 5 Nano
Serial number: 8331229
Firmware version: 5.1.2
Form factor: Nano (USB-A)
Enabled USB interfaces: FIDO+CCID

Applications
OTP             Disabled
FIDO U2F        Enabled 
OpenPGP         Enabled 
PIV             Enabled 
OATH            Disabled
FIDO2           Enabled

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #6 from David Walker <David at WalkerStreet.info> ---
(In reply to Damien Miller from comment #5)
> Can you try to talk to the card using a tool like ykman? E.g.
Here it is with the 5 NFC inserted:

  > ykman info
  Device type: YubiKey 5 NFC
  Serial number: 13377198
  Firmware version: 5.2.6
  Form factor: Keychain (USB-A)
  Enabled USB interfaces: OTP+FIDO+CCID
  NFC interface is enabled.

  Applications  USB     NFC    
  OTP           Enabled Enabled 
  FIDO U2F      Enabled Enabled 
  OpenPGP       Enabled Enabled 
  PIV           Enabled Enabled 
  OATH          Enabled Enabled 
  FIDO2         Enabled Enabled 

And here's the 5c Nano:

  > ykman info
  Device type: YubiKey 5C Nano
  Serial number: 11541414
  Firmware version: 5.2.4
  Form factor: Nano (USB-C)
  Enabled USB interfaces: OTP+FIDO+CCID

  Applications
  OTP           Enabled 
  FIDO U2F      Enabled 
  OpenPGP       Enabled 
  PIV           Enabled 
  OATH          Enabled 
  FIDO2         Enabled

Note that *neither* Yubikey works with ssh (and its associated tools)
for a period of time after the ssh-keygen failure, but both continue to
work with browsers (Vivaldi, in particular). Does ssh-sk-helper have
some kind of cache? The fact that things start working after a period
of time is suspicious to me.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Ismail Donmez <ismail at i10z.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ismail at i10z.com

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Tyson Moore <tyson at tyson.me> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |tyson at tyson.me

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #7 from Damien Miller <djm at mindrot.org> ---
No, ssh-keygen doesn't cache anything. The persistent failure could be
explained by the USB bus getting confused or something hogging it.

I think the next thing to try is to put libfido2 in debugging mode. It
might be possible to do this by setting the FIDO_DEBUG environment
variable, but it's also possible to do it by editing sk-usbhid.c in the
OpenSSH source and uncommenting the "/* #define SK_DEBUG 1 */" line.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

--- Comment #8 from David Walker <David at WalkerStreet.info> ---
(In reply to Damien Miller from comment #7)
> I think the next thing to try is to put libfido2 in debugging mode.
> It might be possible to do this by setting the FIDO_DEBUG
> environment variable, but it's also possible to do it by editing
> sk-usbhid.c in the OpenSSH source and uncommenting the "/* #define
> SK_DEBUG 1 */" line.
Since my previous comment, I've installed an openSUSE Tumbleweed
patched version of libfido2 that doesn't have this problem. (See
https://github.com/Yubico/libfido2/issues/190.) Would you still like me
to test? I could downgrade libfido2 and do that, if it's still useful
now that the libfido2 project is working on the issue.

By the way, now that I'm able to generate keys for multiple Yubikeys,
I've run into another issue. I'll open another bug for that.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |WORKSFORME

--- Comment #9 from Damien Miller <djm at mindrot.org> ---
no need to retest, it looks like problem is in libfido2

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3188

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #10 from Damien Miller <djm at mindrot.org> ---
close bugs that were resolved in OpenSSH 8.5 release cycle

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.