openssh bugs - Nov 2016 - [Bug 2643] New: Can not ssh with tr_TR.UTF-8 locale (Bad configuration options)

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

            Bug ID: 2643
           Summary: Can not ssh with tr_TR.UTF-8 locale (Bad configuration
                    options)
           Product: Portable OpenSSH
           Version: 7.3p1
          Hardware: amd64
                OS: Linux
            Status: NEW
          Severity: major
          Priority: P5
         Component: ssh
          Assignee: unassigned-bugs at mindrot.org
          Reporter: if.gnu.linux at gmail.com

I can not ssh to any host with Turkish locale (tr_TR.UTF-8). I reported
this bug to Launchpad[1] and I was asked to report it to upstream[2].

[1] https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1638338
[2]
https://bugs.launchpad.net/ubuntu/+source/openssh/+bug/1638338/comments/16 

What I expected to happen:

    I expected that 'ssh user at some_host' command would successfully
run.

What actually happened

    'ssh user at some_host' command failed with an error.

Steps to produce:

1. Open a terminal
2. Run LANG=tr_TR.UTF-8 ssh user at some_host

If I run ssh with tr_TR.UTF-8 locale, the first error I get is:

    $HOME/.ssh/config: line 7: Bad configuration option: Identityfile
    $HOME/.ssh/config: terminating, 1 bad configuration options

If I commend out IdentityFile option from $HOME/.ssh/config file and
re-run the command, I get this:

    debug1: Reading configuration data ~/.ssh/config
    debug1: Reading configuration data /etc/ssh/ssh_config
    debug1: /etc/ssh/ssh_config line 19: Applying options for *
    /etc/ssh/ssh_config: line 55: Bad configuration option:
gssapIauthentication
    /etc/ssh/ssh_config: line 56: Bad configuration option:
gssapIdelegatecredentials
    /etc/ssh/ssh_config: terminating, 2 bad configuration options

If I commend aut GSSAPIAuthentication and GSSAPIDelegateCredentials
options, I can ssh to a host.

So to ssh to a host with tr_TR.UTF-8 locale, one must commend out
IdentityFile, if it is used, GSSAPIAuthentication and
GSSAPIDelegateCredentials optons.

Workaround:

    LC_ALL=C ssh some_host

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Colin Watson <cjwatson at debian.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |cjwatson at debian.org

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org
             Blocks|                            |2594

--- Comment #1 from Damien Miller <djm at mindrot.org> ---
This is the infamous Turkish dotless i confusion and will break stuff
wherever we naively tolower a compare a string and compare it.

We lowercase strings in:

auth.c
match.c
misc.c
misc.h
pkcs11.h
readconf.c
ssh.c
sshconnect.c

Given that this is a behaviour change, I propose that we replace the
handful of calls with tolower() with one that uses the C locale always.
This best matches what openssh <7.3 did.


Referenced Bugs:

https://bugzilla.mindrot.org/show_bug.cgi?id=2594
[Bug 2594] Tracking bug for OpenSSH 7.4 release
-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

--- Comment #2 from Damien Miller <djm at mindrot.org> ---
Created attachment 2908
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2908&action=edit
use an ASCII tolower replacement for lowercasing strings

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2908|                            |ok-
              Flags|                            |

--- Comment #3 from Darren Tucker <dtucker at zip.com.au> ---
Comment on attachment 2908
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2908
use an ASCII tolower replacement for lowercasing strings
>+int
>+tolowerc(int c)
>+{
>+	if (c < (int)'A' || c > (int)'Z')
That's wrong for EBCDIC systems since the alphabet is not contiguous.
>+		return c;
>+	return c - ((int)'A' - (int)'a');
>+}
Does the C standard guarantee that the offset between upper and
lowercase characters is constant?  It is in ASCII and EBCDIC.  What
else is out there?

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2908|0                           |1
        is obsolete|                            |
           Assignee|unassigned-bugs at mindrot.org |djm at mindrot.org
                 CC|                            |dtucker at zip.com.au
             Status|NEW                         |ASSIGNED
   Attachment #2910|                            |ok?(dtucker at zip.com.au)
              Flags|                            |

--- Comment #4 from Damien Miller <djm at mindrot.org> ---
Created attachment 2910
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2910&action=edit
use ctype functions instead of roll-ur-own

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2910|ok?(dtucker at zip.com.au)     |
              Flags|                            |

--- Comment #5 from Damien Miller <djm at mindrot.org> ---
Comment on attachment 2910
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2910
use ctype functions instead of roll-ur-own

Not sure I want to go with the approach

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

--- Comment #6 from Damien Miller <djm at mindrot.org> ---
Created attachment 2911
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2911&action=edit
fall back to C locale for LC_CTYPE

I think this is less terrible - it falls back to the C locale for tr*
locales, and tries to preserve UTF-8 if the system supports C.UTF-8

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2911|0                           |1
        is obsolete|                            |

--- Comment #7 from Damien Miller <djm at mindrot.org> ---
Created attachment 2912
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2912&action=edit
fall back to C locale for LC_CTYPE

Helps if I attach the correct diff...

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

--- Comment #8 from Numan Demird??en <if.gnu.linux at gmail.com> ---
Thank you for your work Damien Miller.

Do I have to compile ssh with the last patch you wrote in order to test
whether patched ssh fixes the bug or not?

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

--- Comment #9 from Colin Watson <cjwatson at debian.org> ---
Wow, that's really pretty ugly, and I'm not certain that Turkish is the
only locale with this problem.

How about a simple char[256] table lookup instead?  That shouldn't be
too ugly (still less code than the complicated setlocale fiddling) and
it ought to deal with EBCDIC.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2910|0                           |1
        is obsolete|                            |

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

--- Comment #10 from Damien Miller <djm at mindrot.org> ---
(In reply to Numan Demird????en from comment #8)
> Do I have to compile ssh with the last patch you wrote in order to
> test whether patched ssh fixes the bug or not?
Yes, you'd need to recompile. I've already reproduced and tested the C
locale fix though.

(In reply to Colin Watson from comment #9)
> Wow, that's really pretty ugly, and I'm not certain that Turkish is
> the only locale with this problem.
> 
> How about a simple char[256] table lookup instead?  That shouldn't
> be too ugly (still less code than the complicated setlocale
> fiddling) and it ought to deal with EBCDIC.
I think falling back to C locale is preferable. It matches the
behaviour for OpenSSH <7.3 best while giving us a chance at preserving
UTF-8 output. I'll tidy up that patch - there are a couple of minor
problems with it.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2912|0                           |1
        is obsolete|                            |
   Attachment #2913|                            |ok?(dtucker at zip.com.au)
              Flags|                            |

--- Comment #11 from Damien Miller <djm at mindrot.org> ---
Created attachment 2913
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2913&action=edit
Cleaned-up fallback diff

Previous fallback diff with a few fixes:

1) Correct precedence order for environment variables

2) Correctly terminate environment search if we find a non-TR locale

3) Check for a couple more variants of UTF-8 (lowercase and without
hyphen)

4) Also try POSIX.UTF-8 locale if C.UTF-8 doesn't exist.

5) Stick the while mess in utf8.c where it better belongs. I preferred
this rather than doing a xsetlocale, because I expect we'll upstream
something like the final version in future.

6) Add a longwinded XXX comment describing how to fix this better in
the future :)

Thanks to Ingo Schwarze for valuable feedback.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2913|ok?(dtucker at zip.com.au)     |ok+
              Flags|                            |

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|ASSIGNED                    |RESOLVED
         Resolution|---                         |FIXED

--- Comment #12 from Damien Miller <djm at mindrot.org> ---
Patch applied - this will be in openssh-7.4

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Bj?rn Jacke <bjoern at j3e.de> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |bjoern at j3e.de

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2643

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #13 from Damien Miller <djm at mindrot.org> ---
Close all resolved bugs after release of OpenSSH 7.7.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.