openssh bugs - Jul 2008 - [Bug 1495] New: sftp performance of "ls -l" cmd is slow on network-authed servers

https://bugzilla.mindrot.org/show_bug.cgi?id=1495

           Summary: sftp performance of "ls -l" cmd is slow on
                    network-authed servers
    Classification: Unclassified
           Product: Portable OpenSSH
           Version: -current
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: normal
          Priority: P2
         Component: sftp-server
        AssignedTo: unassigned-bugs at mindrot.org
        ReportedBy: masmitty at us.ibm.com


when sftp client connects to sftp-server and does "ls -l" command on a
large directory (many files), performance is unacceptably slow when the
sftp-server node used ldap/kerberos/NIS/...(etc) to look up EVERY
uid/gid on every file.

a simple fix is to cache/buffer the uid/gid of the previous file from
the stat()/fstat() & chk to see if they have changed before issuing the
lookup of the same uid/gid (ergo inhibiting a wait-for-network to get
info that we already have)

same issue true for v4.1p1 through current.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1495


Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org




--- Comment #1 from Damien Miller <djm at mindrot.org>  2009-01-21
21:27:35 ---
Isn't this the role of nscd (or similar)?

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1495


Michael Rhyner <michael.rhyner at stepping-stone.ch> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |michael.rhyner at stepping-sto
                   |                            |ne.ch




--- Comment #2 from Michael Rhyner <michael.rhyner at stepping-stone.ch> 
2009-07-06 18:52:28 ---
nscd helped on this issue, issuing e.g. 'ls -l' is now as fast as
expected. Cache times within nscd.conf could be quite small, e.g.
between 20 and 60 seconds, it you want to avoid that services rely on
outdated information.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1495

--- Comment #3 from Darren Tucker <dtucker at zip.com.au> 2010-01-14
16:24:08 EST ---
Created an attachment (id=1779)
 --> (https://bugzilla.mindrot.org/attachment.cgi?id=1779)
sftp-namecache.patch

Add a small LRU to cache user and group names.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1495

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dtucker at zip.com.au
             Blocks|                            |1626
             Status|NEW                         |RESOLVED
         Resolution|                            |FIXED

--- Comment #4 from Darren Tucker <dtucker at zip.com.au> 2010-01-15
10:46:31 EST ---
deraadt points out that similar functions (user_from_uid and
group_from_gid) exist in OpenBSD's libc ( both keep a 64-entry hash
table as a cache)  We have just commited a change with that.

We'll add these functions to the compat library for any platform that
doesn't have them.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=1495

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #5 from Darren Tucker <dtucker at zip.com.au> 2010-03-26
10:50:53 EST ---
With the release of 5.4p1, this bug is now considered closed.

-- 
Configure bugmail: https://bugzilla.mindrot.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.