R devel - Jul 2006 - R uses private function in libc (PR#9107)

Full_Name: Quanah Gibson-Mount
Version: 2.3.0, 2.3.1
OS: Linux
Submission from: (NULL) (171.66.155.86)


libc used to export the symbol:

__libc_stack_end;

however, newer versions of libc no longer export it.  R has some serious
problems around this, because the code has:

# ifdef linux
extern void * __libc_stack_end;
# endif


and

#if defined(linux)
    R_CStackStart = (uintptr_t) __libc_stack_end;
#elif defined(HAVE_KERN_USRSTACK)


in r-2.3.1/src/unix/system.c

This code needs to be fixed.  It is causing all sorts of problems when trying to
run R across multiple systems with different libc versions.

--Quanah

On Fri, 28 Jul 2006, quanah at stanford.edu wrote:
> Full_Name: Quanah Gibson-Mount
> Version: 2.3.0, 2.3.1
> OS: Linux
That's not adequate information: see below.
> Submission from: (NULL) (171.66.155.86)
> 
> 
> libc used to export the symbol:
> 
> __libc_stack_end;
> 
> however, newer versions of libc no longer export it.  R has some serious
Please tell us what these `newer versions of libc'
are.
>g<libc does on e.g. 2.3.6 and 2.4, which is AFAICS current.
> problems around this, because the code has:
> 
> # ifdef linux
> extern void * __libc_stack_end;
> # endif
> 
> 
> and
> 
> #if defined(linux)
>     R_CStackStart = (uintptr_t) __libc_stack_end;
> #elif defined(HAVE_KERN_USRSTACK)
> 
> 
> in r-2.3.1/src/unix/system.c
> 
> This code needs to be fixed.  It is causing all sorts of problems when
trying to
> run R across multiple systems with different libc versions.
What are the 'all sorts of problems'?  It seems they are affecting no
one
else, and R is very widely used on Linux systems.  So there is nothing 
here we can reproduce.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595

--On Friday, July 28, 2006 9:23 AM +0100 Prof Brian Ripley 
<ripley at stats.ox.ac.uk> wrote:
> On Fri, 28 Jul 2006, quanah at stanford.edu wrote:
>
>> Full_Name: Quanah Gibson-Mount
>> Version: 2.3.0, 2.3.1
>> OS: Linux
>
> That's not adequate information: see below.
>
>> Submission from: (NULL) (171.66.155.86)
>>
>>
>> libc used to export the symbol:
>>
>> __libc_stack_end;
>>
>> however, newer versions of libc no longer export it.  R has some
serious
>
> Please tell us what these `newer versions of libc' are.
>> g<libc does on e.g. 2.3.6 and 2.4, which is AFAICS current.
>
>> problems around this, because the code has:
>>
>> # ifdef linux
>> extern void * __libc_stack_end;
>> # endif
>>
>>
>> and
>>
>> # if defined(linux)
>>     R_CStackStart = (uintptr_t) __libc_stack_end;
>> # elif defined(HAVE_KERN_USRSTACK)
>>
>>
>> in r-2.3.1/src/unix/system.c
>>
>> This code needs to be fixed.  It is causing all sorts of problems when
>> trying to run R across multiple systems with different libc versions.
>
> What are the 'all sorts of problems'?  It seems they are affecting
no one
> else, and R is very widely used on Linux systems.  So there is nothing
> here we can reproduce.
Well, here is a little background:

At Stanford, we build software and then make it available to campus via a 
distributed file system (AFS in our case).  In building "R" for our
new
64-bit Linux tree, we encountered this issue after building R on our 64-bit 
Debian Sarge system, which has libc version 2.3.2, and does export 
__libc_stack_end, although it is marked private, as it should be:

readelf -s /lib/libc.so.6 | grep __libc_stack_end
  1144: 0000000000000000     8 OBJECT  GLOBAL DEFAULT  UND 
__libc_stack_end at GLIBC_PRIVATE (9)

After I released the build to campus, we immediately got reports that it 
didn't work on other systems, and finally tracked that down to the newer 
versions of libc on those systems:

bramble01:~> readelf -s /lib/libc.so.6 | grep __libc_stack_end
bramble01:~> ldd --version
ldd (GNU libc) 2.3.6

yen4:~> readelf -s /lib/libc.so.6 | grep __libc_stack_end
  2148: 00000000     4 OBJECT  GLOBAL DEFAULT  UND 
__libc_stack_end at GLIBC_2.1 (22)
  6010: 00000000     4 OBJECT  GLOBAL DEFAULT  UND __libc_stack_end@@GLIBC_2
yen4:~> ldd --version
ldd (GNU libc) 2.3.4


When running "R" on either the bramble or yen systems, the error one
gets
is:

yen4:~> R
/usr/pubsw/lib/R/2.3.1/R/bin/exec/R: relocation error:
/usr/pubsw/lib/R/2.3.1/R/bin/exec/R: symbol __libc_stack_end, version
GLIBC_PRIVATE not defined in file ld-linux-x86-64.so.2 with link time
reference


bramble01:~> R
/usr/pubsw/lib/R/2.3.1/R/bin/exec/R: relocation error: 
/usr/pubsw/lib/R/2.3.1/R/bin/exec/R: symbol __libc_stack_end, version 
GLIBC_PRIVATE not defined in file ld-linux-x86-64.so.2 with link time 
reference


For now, I've managed to get around this by patching R thusly:

cat libc
--- r-2.3.1/src/unix/system.c.orig      2006-07-27 17:42:10.000000000 -0700
+++ r-2.3.1/src/unix/system.c   2006-07-27 17:46:42.000000000 -0700
@@ -117,9 +117,9 @@
 #  include <sys/time.h>
 # endif
 # include <sys/resource.h>
-# ifdef linux
+/* # ifdef linux
 extern void * __libc_stack_end;
-# endif
+# endif */
 # ifdef HAVE_KERN_USRSTACK
 #  include <unistd.h>
 #  include <sys/types.h>
@@ -157,9 +157,10 @@
        lim2 = (unsigned long) rlim.rlim_max; /* Usually unlimited */
         R_CStackLimit = lim1 < lim2 ? lim1 : lim2;
     }
-#if defined(linux)
+/* #if defined(linux)
     R_CStackStart = (uintptr_t) __libc_stack_end;
-#elif defined(HAVE_KERN_USRSTACK)
+#elif defined(HAVE_KERN_USRSTACK) */
+#if defined(HAVE_KERN_USRSTACK)
     {
        /* Borrowed from mzscheme/gc/os_dep.c */
        int nm[2] = {CTL_KERN, KERN_USRSTACK};


In any case, as you can see, __libc_stack_end goes away completely by libc 
2.3.6, and as noted in the follow-up I sent to this bug yesterday, the 
reasons one shouldn't use this symbol are quite clearly stated here:

<http://www.mail-archive.com/debian-glibc at
lists.debian.org/msg28253.html>

--Quanah

--
Quanah Gibson-Mount
Principal Software Developer
ITS/Shared Application Services
Stanford University
GnuPG Public Key: http://www.stanford.edu/~quanah/pgp.html

On Fri, 28 Jul 2006, Thomas Lumley wrote, quoting someone else:
> > In any case, as you can see, __libc_stack_end goes away completely by
libc
> > 2.3.6, 
I don't see this: my FC3 AMD64 *g*libc 2.3.6 system gives

  2148: 00000000     4 OBJECT  GLOBAL DEFAULT  UND 
__libc_stack_end at GLIBC_2.1 (22)
  9516: 00000000     4 OBJECT  GLOBAL DEFAULT  UND 
__libc_stack_end@@GLIBC_2

so the claim is not a general property of glibc 2.3.6.  Indeed, it seems 
rather to be a question of visibility: the symbol has NOT `gone away 
completely' but been hidden in the .so (it is visible in libc.a in vanilla 
glibc 2.4).

It had already been said that many systems using glibc 2.3.6 or 2.4 do 
export the symbol, as evidenced by the RPMs that exist for FC3 (at 2.3.6) 
and FC5 (at 2.4), the runs on the Debian test farm, and the absence of 
reported problems from anywhere else on any Linux system.  (Not only that, 
the code detects C stack overflows at the intended stack size limit, so 
the symbol linked to has the expected value.)

I had checked the glibc Changelogs and found nothing relevant, and it 
seems this is an undocumented change that the major distros have reverted 
and hence not much of a problem.
> > and as noted in the follow-up I sent to this bug yesterday, the
> > reasons one shouldn't use this symbol are quite clearly stated
here:
> >
> > <http://www.mail-archive.com/debian-glibc at
lists.debian.org/msg28253.html>
> >
> 
> Only up to a point.  We know that __libc_stack_end is not portable, but it 
> is being used to do something that is impossible to do portably: check 
> whether the C stack is about to overflow. Since there is no way to recover 
> from a C stack overflow, being able to prevent it is valuable.
> 
> Your fix disables this facility on all Linux systems, which may be 
> appropriate for your systems but is clearly undesirable as a change to R. 
> The real question is how to detect systems that do provide 
> __libc_stack_end and whether there is another non-portable way to do the 
> same thing on glibc versions that don't provide it.
I've added a test for the symbol in R-devel.  I did originally expect to 
need one for non-glibc Linuxen (e.g. using Intel and PG compilers) but 
they do find the symbol.  Hopefully this will continue to be the case for 
the major distros.

-- 
Brian D. Ripley,                  ripley at stats.ox.ac.uk
Professor of Applied Statistics,  http://www.stats.ox.ac.uk/~ripley/
University of Oxford,             Tel:  +44 1865 272861 (self)
1 South Parks Road,                     +44 1865 272866 (PA)
Oxford OX1 3TG, UK                Fax:  +44 1865 272595