openssh bugs - Sep 2011 - [Bug 1935] New: ls -l with wildcards produces incorrect output

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

             Bug #: 1935
           Summary: ls -l with wildcards produces incorrect output
    Classification: Unclassified
           Product: Portable OpenSSH
           Version: 5.8p1
          Platform: All
        OS/Version: Linux
            Status: NEW
          Severity: normal
          Priority: P2
         Component: sftp
        AssignedTo: unassigned-bugs at mindrot.org
        ReportedBy: jma at freedomcircle.com


This is probably best demonstrated with some sample output.  Issuing
the command:

sftp> ls -l localhost.access.log.?.gz

gives the following (abbreviated) output:

29787 Aug 31 02:24 localhost.access.log.2.gz
43664 Sep  1 02:23 localhost.access.log.3.gz
37122 Aug 29 02:23 localhost.access.log.4.gz
31291 Aug 26 02:24 localhost.access.log.5.gz
33117 Aug 27 02:22 localhost.access.log.6.gz
30870 Aug 30 02:24 localhost.access.log.7.gz
26787 Sep  2 02:23 localhost.access.log.8.gz
38128 Aug 28 02:24 localhost.access.log.9.gz

whereas ls -l reports the following (correct) data:

26787 Sep  2 06:23 localhost.access.log.2.gz
43664 Sep  1 06:23 localhost.access.log.3.gz
29787 Aug 31 06:24 localhost.access.log.4.gz
30870 Aug 30 06:24 localhost.access.log.5.gz
37122 Aug 29 06:23 localhost.access.log.6.gz
38128 Aug 28 06:24 localhost.access.log.7.gz
33117 Aug 27 06:22 localhost.access.log.8.gz
31291 Aug 26 06:24 localhost.access.log.9.gz

So it seems that the wildcard is munging the file size and timestamp in
some random manner.  The bad output also happens with an asterisk
instead of a question mark.

Client is 5.8p1 on Debian-7 x86.  Server is 5.5p1 on Debian-6 amd64.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org

--- Comment #1 from Damien Miller <djm at mindrot.org> 2011-09-11 08:13:02
EST ---
Can you replicate this with a more recent server version? It might be a
bug in 5.5 that has been fixed since.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #2 from Joe <jma at freedomcircle.com> 2011-09-11 08:40:04 EST
---
Yes, I tried against my localhost, i.e., server also running 5.8p1, and
got similar results:

ls -l (on localhost)

43690 Sep  3 11:35 localhost.access.log.0901.gz
26813 Sep  3 11:35 localhost.access.log.0902.gz
24932 Sep 10 08:44 localhost.access.log.0903.gz

sftp> ls -l localhost.access.log.*.gz
26193 Sep 10 08:45 localhost.access.log.0901.gz
43690 Sep  3 11:35 localhost.access.log.0902.gz
35457 Sep 10 08:46 localhost.access.log.0903.gz

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #3 from Damien Miller <djm at mindrot.org> 2011-09-11 08:49:05
EST ---
One more thing to try: could you please try to recreate on localhost
after cutting ssh/sshd out of the picture. Can you replicate using
"sftp -D /usr/lib/openssh/sftp-server" to directly connect sftp to
sftp-server (you might need a different path - AFAIK this one is
correct for Debian/Ubuntu).

Also, what filesystem are these logs hosted on?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #4 from Joe <jma at freedomcircle.com> 2011-09-11 09:09:41 EST
---
Same results using "sftp -D /usr/lib/openssh/sftp-server".  On my
local
host, the files are on ext3.  BTW, if I'm not mistaken this started
happening when I upgraded to 5.8p1 on the client.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #5 from Damien Miller <djm at mindrot.org> 2011-09-12 10:43:10
EST ---
okay, I can replicate it now if I copy your filenames. Using
single-character wildcards doesn't seem enough, there has to be a
preamble.

Also, the screwup persists when sorting is turned off (ls -lf) so it
might be in the server or glob code.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #6 from Damien Miller <djm at mindrot.org> 2011-09-21 12:50:10
EST ---
Created attachment 2093
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2093
glob.diff

fix glob GLOB_KEEPSTAT vs sorting

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #7 from Damien Miller <djm at mindrot.org> 2011-09-21 12:53:39
EST ---
Created attachment 2094
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2094
sftp.diff

don't bother sorting remote ls glob output

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

--- Comment #8 from Damien Miller <djm at mindrot.org> 2011-09-21 12:57:19
EST ---
okay, either of these diffs will fix your bug. I intend to commit both.

The problem is that glob()'s GLOB_KEEPSTAT support (retain the results
of stat() operations in an array) didn't take into account that glob()
sorts its results by default. The first patch implements a sort that
takes the separate array of stat information into account so it doesn't
get out of sync with the pathnames.

Furthermore, "ls" shouldn't bother with glob's internal
sorting anyway
because it implements its own sort which might be different. The second
patch disables glob's sort for remote ls operations.

If you are applying the first patch, then please "cd openbsd-compat"
first.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1935

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |1930
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #9 from Damien Miller <djm at mindrot.org> 2011-09-22 21:53:52
EST ---
These patches have been applied and will be in OpenSSH-6.0. Thanks!

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.