openssh bugs - Mar 2023 - [Bug 3547] New: sftp crash with 'invalid multibyte character' when pressing Tab to complete specific Chinese filenames

https://bugzilla.mindrot.org/show_bug.cgi?id=3547

            Bug ID: 3547
           Summary: sftp crash with 'invalid multibyte character' when
                    pressing Tab to complete specific Chinese filenames
           Product: Portable OpenSSH
           Version: 8.4p1
          Hardware: amd64
                OS: Linux
            Status: NEW
          Severity: trivial
          Priority: P5
         Component: sftp
          Assignee: unassigned-bugs at mindrot.org
          Reporter: nebclllo0444 at gmail.com

I'm using sftp bundled with OpenSSH 8.4p1-150300.3.15.4 on openSUSE
Leap 15.4. I encountered a problem that when there are some files with
specific Chinese filename. For example:
(using bash with LANG=zh_CN.utf8)
touch ???
touch ???
# Create two file that contains character that represents number. They
have the same prefix and suffix. These parentheses are also CJK
characters(Not ASCII).
sftp 127.0.0.1
ls ?
# Input the first character(The common prefix of the two filenames)
# And press the Tab to auto-complete
And the client immediately after printing the filenames, leaving the
terminal not usable without a `reset`:
??? ???
invalid multibyte character

It seems that only a few combinations will make the client crash. If
these are "???" and "???", the client will not crash.

Prefixes and suffixes other than CJK parentheses will make the client
crash, though: 

Input 'ls ?' and press Tab. "???" "???" will crash
the client. "???"
"???" will not.

Switching sftp subsystem from external sftp server to 'internal-sftp'
in sshd_config will not fix it.

The same problem also exists on Debian Bullseye (with OpenSSH
1:8.4p1-5+deb11u1)

This is my first bug report for OpenSSH, and I'm not good at English.
Sorry for any inconvenience.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3547

Darren Tucker <dtucker at dtucker.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dtucker at dtucker.net

--- Comment #1 from Darren Tucker <dtucker at dtucker.net> ---
Can you reproduce the problem with a a stock openssh built from the
source from openssh.com?  We cannot help you with vendor-supplied
(usually modified) binaries, you will need to seek help from the vendor
that supplied them.

A couple of suggestions:
 - run sftp under gdb and see where it crashes ("gdb --args
/usr/bin/sftp localhost", "run", then when it crashes use
"bt" to get a
backtrace).
 - I vaguely recall utf8 crashing bugs with some versions of libedit,
maybe check if there's an update for that.  

I made a quick attempt to reproduce here on Fedora
(libedit-3.1-41.20210910cvs.fc36.x86_64) with both -current and 8.4p1
but it worked as expected (although it's possible I'm doing something
wrong, I don't normally used non-ascii).

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3547

--- Comment #2 from Darren Tucker <dtucker at dtucker.net> ---
I did manage to reproduce this on an ARM SBC running a Debian 10.13
derivative.

$ gdb -q --args /usr/bin/sftp localhost
Reading symbols from /usr/bin/sftp...(no debugging symbols
found)...done.
(gdb) run
[...]
Connected to localhost.
sftp> cd /tmp/t
sftp> ls ?
???  ???  
invalid multibyte character
[Inferior 1 (process 26809) exited with code 0377]

So it's not a crash as in segfault, it prints an error then exits.

The system has libedit 3.1-20181209-1.  If I build OpenSSH against a
locally-built libedit-20221030-3.1 it doesn't do this (it doesn't
autocomplete either, but that seems reasonable).  I think you're
looking at a libedit bug.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3547

--- Comment #3 from Darren Tucker <dtucker at dtucker.net> ---
(In reply to Darren Tucker from comment #2)
> The system has libedit 3.1-20181209-1.
... and if I build OpenSSH 8.4p1 against the system libedit, it also
fails, but if I build 8.4p1 against the newest libedit it doesn't.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3547

--- Comment #4 from nebclllo0444 at gmail.com ---
libedit on linux is ported from NetBSD, so I installed NetBSD 9.3 but
found that there are few documentations about localization, and CJK
characters in terminal(both in ssh and Xfce4 Terminal) will be
displayed in '?' but Xfce4 it self seems normal.
Should I keep on trying to reproduce the problem on NetBSD?

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=3547

--- Comment #5 from Darren Tucker <dtucker at dtucker.net> ---
(In reply to nebclllo0444 from comment #4)
> Should I keep on trying to reproduce the problem on NetBSD?
Based on the libedit version numbers, my guess is that the problem was
fixed in NetBSD sometime between 2018 and 2020.  If that's correct, you
won't be able to reproduce it on any modern NetBSD version, so not
being able to reproduce on 9.3 is not surprising.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.