openssh bugs - Jul 2015 - [Bug 2428] New: realpath command doesn't work

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

            Bug ID: 2428
           Summary: realpath command doesn't work
           Product: Portable OpenSSH
           Version: 6.9p1
          Hardware: amd64
                OS: Linux
            Status: NEW
          Severity: normal
          Priority: P5
         Component: Build system
          Assignee: unassigned-bugs at mindrot.org
          Reporter: david.kahles96 at gmail.com

Created attachment 2665
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2665&action=edit
The configure command after running 'autoreconf -fi'

I use libssh to access a server via sftp, but the issue is not related
to libssh according to the logs.

How to reproduce:
"/home/david/test" exists, but "/home/david/test/test"
doesn't exist.
When I canonicalize "/home/david/test/test" using the SSH_FXP_REALPATH
command it doesn't work and reports "no such file".
I did also compile OpenSSH from git, see the attached files.

Expected result:
Canonicalizing "/home/david/test/test" works, according to [1]

Notes:
Further investigation showed that the libc realpath function is linked,
and not the included OpenBSD one. The included realpath function works
fine, but the libc one doesn't. As far as I understand the build
system, the OpenBSD realpath function should not be used on linux, but
the libc realpath function seems not to work.
Or is the realpath function not supposed to work if the file doesn't
exist?

System: Up-to-date Archlinux
GCC 5.1.0
make 4.1
glibc 2.21

[1]: http://www.openssh.com/txt/draft-ietf-secsh-filexfer-02.txt, page
17: "...canonicalize any given path name..."

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #1 from David Kahles <david.kahles96 at gmail.com> ---
Created attachment 2666
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2666&action=edit
The ouput of 'make'

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #2 from David Kahles <david.kahles96 at gmail.com> ---
Created attachment 2667
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2667&action=edit
The logged output of the sftp-server when the problem occurs

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #3 from David Kahles <david.kahles96 at gmail.com> ---
Created attachment 2668
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2668&action=edit
My sshd_config file

-- 
You are receiving this mail because:
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |dtucker at zip.com.au

--- Comment #4 from Darren Tucker <dtucker at zip.com.au> ---
The only platform that sets BROKEN_REALPATH is AIX, not Linux, and I
can't remember what the reason for that one was offhand.

The OpenBSD man page says ("All but the last component of pathname must
exist when realpath() is called."

The POSIX spec
(http://pubs.opengroup.org/onlinepubs/009695399/functions/realpath.html)
says that it can return ENOENT if "a component of file_name does not
name an existing file".

but it also says:

"IEEE Std 1003.1-2001/Cor 2-2004, item XSH/TC2/D6/110 is applied,
updating the ERRORS section to refer to the file_name argument, rather
than a nonexistent path argument."

so it looks like earlier versions of the spec allowed (or in fact
specified, depending on how you read it) the OpenBSD behaviour.

The filexfer (v3) spec is silent on the question of what to do in the
case of a non-existent file.  Later versions[1] punt on this by adding
an optional flag to select either behaviour, but the default is to not
check  for the existence of the file, so you could argue that this is
the behaviour openssh should have.

[1] https://tools.ietf.org/html/draft-ietf-secsh-filexfer-13

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #5 from Darren Tucker <dtucker at zip.com.au> ---
Created attachment 2669
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2669&action=edit
Check if realpath works with non-existent files.

Here's a patch to check for this behaviour at build time.   If you try
it you'll need to run "autoreconf" to rebuild configure before
running
it.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #6 from David Kahles <david.kahles96 at gmail.com> ---
Thank you for the patch, it changes config.h to contain
#define BROKEN_REALPATH 1
but unfortunately it still doesn't work. There seems to be a linker
problem. The openbsd realpath function gets build (I checked it with a
#pragma message), but it seems as the libc version gets linked. I
copied the openbsd function, renamed it and executed it when the libc
function failed, and the result was as expected: the openbsd function
didn't fail.
But I wonder why the libc version gets linked, as the openbsd one
should overwrite it? I attach the Makefile, but it seems to be right.
I tried it also with gcc 4.9.2 - no difference.
Do you have a clue what's going wrong?

PS: of course I reconfigured and rebuild the whole project after
applying the patch.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #7 from David Kahles <david.kahles96 at gmail.com> ---
Created attachment 2671
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2671&action=edit
Makefile after patch applied

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2671|application/octet-stream    |text/plain
          mime type|                            |

--- Comment #8 from Darren Tucker <dtucker at zip.com.au> ---
Comment on attachment 2671
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2671
Makefile after patch applied

That's weird, we do that kind of thing all the time.  I'll see if I can
figure out what's going on.

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #9 from Darren Tucker <dtucker at zip.com.au> ---
Ah, I see what happened:

$ nm sftp-server | grep realpath
00003060 t process_realpath
         U __realpath_chk@@GLIBC_2.4

That symbol is the checked version of realpath enabled by
-DFORTIFY_SOURCE=2.  If I remove that from the Makefile:

$ nm sftp-server | grep realpath00003030 t process_realpath
0000e280 T realpath

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2669|0                           |1
        is obsolete|                            |

--- Comment #10 from Darren Tucker <dtucker at zip.com.au> ---
Created attachment 2672
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2672&action=edit
Use compat realpath if native one does not work with non-existent
files.

This seems to behave as expected:

$ nm sftp-server | grep realpath
00003090 t process_realpath
0000e610 T _ssh_compat_realpath

-- 
You are receiving this mail because:
You are watching the assignee of the bug.
You are watching someone on the CC list of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #11 from David Kahles <david.kahles96 at gmail.com> ---
Thank you, this works well :-)
Will it be included in the next OpenSSH release?

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |djm at mindrot.org
   Attachment #2672|                            |ok?(djm at mindrot.org)
              Flags|                            |

--- Comment #12 from Darren Tucker <dtucker at zip.com.au> ---
Comment on attachment 2672
  --> https://bugzilla.mindrot.org/attachment.cgi?id=2672
Use compat realpath if native one does not work with non-existent
files.

Probably.  Damien, what do you think?

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
   Attachment #2672|ok?(djm at mindrot.org)        |ok+
              Flags|                            |

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Darren Tucker <dtucker at zip.com.au> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Blocks|                            |2403
         Resolution|---                         |FIXED
             Status|NEW                         |RESOLVED

--- Comment #13 from Darren Tucker <dtucker at zip.com.au> ---
Patch applied, it will be in the 7.0p1 release.  Thanks.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

--- Comment #14 from David Kahles <david.kahles96 at gmail.com> ---
Thank you too.

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.

https://bugzilla.mindrot.org/show_bug.cgi?id=2428

Damien Miller <djm at mindrot.org> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|RESOLVED                    |CLOSED

--- Comment #15 from Damien Miller <djm at mindrot.org> ---
Set all RESOLVED bugs to CLOSED with release of OpenSSH 7.1

-- 
You are receiving this mail because:
You are watching someone on the CC list of the bug.
You are watching the assignee of the bug.