openssh unix dev - Apr 2024 - [PATCH] sshsig: check hashalg before selecting the RSA signature algorithm

`ssh-keygen -Y sign` only selects the signing algorithm `rsa-sha2-512`
and this prevents ssh-agent implementations that can't support sha512
from signing messages.

An example of this is TPMs which mostly only really supports sha256
widely.

This change enables `ssh-keygen -Y sign` to honor the `hashalg` option
for the signing algorithm.

Signed-off-by: Morten Linderud <morten at linderud.pw>
---
 sshsig.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

diff --git a/sshsig.c b/sshsig.c
index 470b286a3..033b43353 100644
--- a/sshsig.c
+++ b/sshsig.c
@@ -190,8 +190,14 @@ sshsig_wrap_sign(struct sshkey *key, const char *hashalg,
 	}
 
 	/* If using RSA keys then default to a good signature algorithm */
-	if (sshkey_type_plain(key->type) == KEY_RSA)
-		sign_alg = RSA_SIGN_ALG;
+	if (sshkey_type_plain(key->type) == KEY_RSA){
+		if (hashalg == NULL)
+			sign_alg = RSA_SIGN_ALG;
+		else if (strcmp(hashalg, "sha256") == 0)
+			sign_alg = "rsa-sha2-256";
+		else if (strcmp(hashalg, "sha512") == 0)
+			sign_alg = "rsa-sha2-512";
+	}
 
 	if (signer != NULL) {
 		if ((r = signer(key, &sig, &slen,
-- 
2.44.0