Nouveau - Jan 2014 - [Bug 73473] New: Potential crash bug in src/gallium/auxiliary/rtasm/rtasm_execmem.c

https://bugs.freedesktop.org/show_bug.cgi?id=73473

          Priority: medium
            Bug ID: 73473
          Assignee: nouveau at lists.freedesktop.org
           Summary: Potential crash bug in
                    src/gallium/auxiliary/rtasm/rtasm_execmem.c
          Severity: critical
    Classification: Unclassified
                OS: Linux (All)
          Reporter: jaak at ristioja.ee
          Hardware: x86-64 (AMD64)
            Status: NEW
           Version: unspecified
         Component: Drivers/DRI/nouveau
           Product: Mesa

glxgears[4186]: segfault at ffffffffffffffff ip 000078805fc4b901 sp
00007ce9598e21c0 error 7 in nouveau_dri.so[78805f7d1000+136c000]

Stracing it revealed that the crash happens after a mmap(NULL, 10485760,
PROT_READ|PROT_WRITE|PROT_EXEC, MAP_PRIVATE,MAP_ANONYMOUS, -1, 0) syscall
returns -1.

I think it might be caused by the return value of mmap not being checked in
src/gallium/auxiliary/rtasm/rtasm_execmem.c, leading to the the memory being
accessed somewhere else.

So it probably needs some

  if (exec_mem == MAP_FAILED)

check somewhere.

PS: Sorry if this is not the correct component.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140110/32137324/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #1 from Emil Velikov <emil.l.velikov at gmail.com> ---
Hi Jaak

If you're getting consistent crashes in glxgears I would recommend using gdb
to
get a backtrace of the problem.

With that said, I suspect that the problems is elsewhere for a few reasons
* There are more than a handful cases when mmap fails and I have yet to notice
after 3+ years constant use of nouveau any glxgears segfaults.
* I would suspect other mesa users will be affected and this problem would be
well know/resolved by now.

Apart from the backtrace would you mind attaching your dmesg output after the
problem/segfault ?
Can you reproduce with the swrast driver ?
$ LIBGL_ALWAYS_SOFTWARE=1 glxgears

Cheers

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140110/1340f386/attachment-0001.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #2 from Jaak Ristioja <jaak at ristioja.ee> ---
(In reply to comment #1)
> If you're getting consistent crashes in glxgears I would recommend
using gdb
> to get a backtrace of the problem.
I wish I could but gdb only shows ?? in backtrace:

(gdb) thread apply all bt full

Thread 1 (process 2782):
#0  0x00006a0a172d4901 in ?? ()
No symbol table info available.
#1  0x0000000000000000 in ?? ()
No symbol table info available.
> With that said, I suspect that the problems is elsewhere for a few reasons
> * There are more than a handful cases when mmap fails and I have yet to
> notice after 3+ years constant use of nouveau any glxgears segfaults.
> * I would suspect other mesa users will be affected and this problem would
> be well know/resolved by now.
> 
> Apart from the backtrace would you mind attaching your dmesg output after
> the problem/segfault ?
[17407.732321] grsec: From 5.4.2.83: denied RWX mmap of <anonymous
mapping> by
/usr/bin/glxgears[glxgears:2866] uid/euid:1000/1000 gid/egid:100/100, parent
/bin/bash[bash:2860] uid/euid:1000/1000 gid/egid:100/100
[17407.732328] glxgears[2866]: segfault at ffffffffffffffff ip 0000685b13c99901
sp 000077a968f66e50 error 7 in nouveau_dri.so[685b1381f000+136c000]
[17407.732342] grsec: From 5.4.2.83: Segmentation fault occurred at
ffffffffffffffff in /usr/bin/glxgears[glxgears:2866] uid/euid:1000/1000
gid/egid:100/100, parent /bin/bash[bash:2860] uid/euid:1000/1000
gid/egid:100/100
[17407.732356] grsec: From 5.4.2.83: denied resource overstep by requesting
4096 for RLIMIT_CORE against limit 0 for /usr/bin/glxgears[glxgears:2866]
uid/euid:1000/1000 gid/egid:100/100, parent /bin/bash[bash:2860]
uid/euid:1000/1000 gid/egid:100/100

So it appears to try to read memory at ffffffffffffffff (i.e. -1 alias
MAP_FAILED). It doesn't matter why mmap fails (EPERM). What matters is that
it
is documented that it may fail, but errors do not appear to be handled in code.

I don't think Gentoo should be expected to mark every single application
using
OpenGL to be allowed mmap RWX memory. I use Intel + OpenGL on my Gentoo
Hardened laptop with no problems.
> Can you reproduce with the swrast driver ?
> $ LIBGL_ALWAYS_SOFTWARE=1 glxgears
$ DISPLAY=:0.0 LIBGL_ALWAYS_SOFTWARE=1 glxgears; echo $?
LLVM ERROR: Allocation failed when allocating new memory in the JIT
Can't allocate RWX Memory: Operation not permitted
1

I'm using Hardened Gentoo (kernel is vanilla-3.12.6 + genpatches-3.12-7 +
grsecurity-3.0-3.12.6-201401021726; gcc --version is "gcc (Gentoo Hardened
4.7.3-r1 p1.4, pie-0.5.5) 4.7.3").

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140110/7dfd7e46/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #3 from Emil Velikov <emil.l.velikov at gmail.com> ---
Created attachment 91834
  --> https://bugs.freedesktop.org/attachment.cgi?id=91834&action=edit
gallium/rtasm: add support for SELinux

You're absolutely right here. Seems like there aren't many
hardened/selinux
users of the gallium drivers.

The attached patch should do the job, can you give it a try ?

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140110/54e28b7f/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #4 from Jaak Ristioja <jaak at ristioja.ee> ---
(In reply to comment #3)
> Created attachment 91834 [details] [review]
> gallium/rtasm: add support for SELinux
> 
> You're absolutely right here. Seems like there aren't many
hardened/selinux
> users of the gallium drivers.
> 
> The attached patch should do the job, can you give it a try ?
I tried the patch on Mesa 5.1.9 under Gentoo.

patching file src/gallium/auxiliary/rtasm/rtasm_execmem.c
Hunk #1 succeeded at 60 (offset -1 lines).
Hunk #2 succeeded at 72 (offset -1 lines).
Hunk #3 succeeded at 90 (offset -1 lines).
Hunk #4 succeeded at 103 (offset -1 lines).
Hunk #5 succeeded at 115 (offset -1 lines).

Running headless over SSH I could not verify today that the glxgears were
actually shown in X, but here are the results I got:

$ DISPLAY=:0.0 glxgears
Running synchronized to the vertical refresh.  The framerate should be
approximately the same as the monitor refresh rate.
29168 frames in 5.0 seconds = 5833.411 FPS
29463 frames in 5.0 seconds = 5892.528 FPS
...

This still gives the kernel message of RWX mmap being denied which is normal.

$ DISPLAY=:0.0 LIBGL_ALWAYS_SOFTWARE=1 glxgears
LLVM ERROR: Allocation failed when allocating new memory in the JIT
Can't allocate RWX Memory: Operation not permitted

This also gives the kernel message, but exits immediately after with exit
status 1.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140111/86d87b57/attachment-0001.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #5 from Jaak Ristioja <jaak at ristioja.ee> ---
PS: I'm not using selinux. So I guess I'm unable to test that part of
the
patch.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140111/eca1334c/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #6 from Emil Velikov <emil.l.velikov at gmail.com> ---
(In reply to comment #4)
> (In reply to comment #3)
> > Created attachment 91834 [details] [review] [review]
> > gallium/rtasm: add support for SELinux
> > 
> > You're absolutely right here. Seems like there aren't many
hardened/selinux
> > users of the gallium drivers.
> > 
> > The attached patch should do the job, can you give it a try ?
> 
> I tried the patch on Mesa 5.1.9 under Gentoo.
> 
I hope you mean 9.1.5 here :)
> patching file src/gallium/auxiliary/rtasm/rtasm_execmem.c
> Hunk #1 succeeded at 60 (offset -1 lines).
> Hunk #2 succeeded at 72 (offset -1 lines).
> Hunk #3 succeeded at 90 (offset -1 lines).
> Hunk #4 succeeded at 103 (offset -1 lines).
> Hunk #5 succeeded at 115 (offset -1 lines).
> 
> Running headless over SSH I could not verify today that the glxgears were
> actually shown in X, but here are the results I got:
> 
> $ DISPLAY=:0.0 glxgears
> Running synchronized to the vertical refresh.  The framerate should be
> approximately the same as the monitor refresh rate.
> 29168 frames in 5.0 seconds = 5833.411 FPS
> 29463 frames in 5.0 seconds = 5892.528 FPS
> ...
> 
> This still gives the kernel message of RWX mmap being denied which is
normal.
> 
Great, thanks.
> $ DISPLAY=:0.0 LIBGL_ALWAYS_SOFTWARE=1 glxgears
> LLVM ERROR: Allocation failed when allocating new memory in the JIT
> Can't allocate RWX Memory: Operation not permitted
> 
> This also gives the kernel message, but exits immediately after with exit
> status 1.
I'm guessing that some work may be needed for LLVM.

(In reply to comment #5)
> PS: I'm not using selinux. So I guess I'm unable to test that part
of the
> patch.
I thought that was the case. Either way big thanks for the report.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140111/a3110be7/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #7 from Jaak Ristioja <jaak at ristioja.ee> ---
(In reply to comment #4)
> Running headless over SSH I could not verify today that the glxgears were
> actually shown in X, but here are the results I got:
Confirmed over SSH and VNC that glxgears works properly.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140111/ae6e9dfc/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #8 from Amadeusz <amade at asmblr.net> ---
Created attachment 92300
  --> https://bugs.freedesktop.org/attachment.cgi?id=92300&action=edit
patch adding check for PaX mprotect

As I said on #gentoo-hardened channel, I wouldn't like to see the SELinux
part
of this patch to be merged. Provided SELinux check effectively requires
allowing all applications to be allowed access to write|exec memory regardless
of if it is needed or not.

I tested patch without the SELinux part and it worked fine on my PaX &
SELinux
enabled system. Starting glxgears didn't bring down whole X server as was
the
case before applying patch ;) .


If one wants to avoid "grsec: denied RWX mmap" messages probably
something
along the lines of patch I attached (based on checks from the SELinux one and
http://sources.gentoo.org/cgi-bin/viewvc.cgi/gentoo-x86/dev-libs/libffi/files/libffi-3.0.13-emutramp_pax_proc.patch?revision=1.2&view=markup)
would be needed.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140117/42f6b0af/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #9 from Jaak Ristioja <jaak at ristioja.ee> ---
Personally, I'd keep both the selinux and PaX parts out of the patch.

I suggest simply checking the return value for the mmap call. I think this
would suffice. Keep it simple. There are also other reasons why mmap can fail
(e.g. due to ulimit) and this should be checked. Please fix this first, add
PaX/selinux/whatever support later. :)

PS: While in the selinux patch the selinux code can be conditionally #ifdef
enabled, in the PaX patch the PaX code is not, meaning this results in a minor
performance impact for all users.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140117/a9e5d48b/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

--- Comment #10 from Emil Velikov <emil.l.velikov at gmail.com> ---
Yay the crash should happen no more

commit 4dd445f1cf80292f10eda53665cefc2a674d838d
Author: Emil Velikov <emil.l.velikov at gmail.com>
Date:   Fri Jan 10 18:00:17 2014 +0000

    gallium/rtasm: handle mmap failures appropriately

    For a variety of reasons mmap (selinux and pax to name
    a few) and can fail and with current code. This will
    result in a crash in the driver, if not worse.

    This has been the case since the inception of the
    gallium copy of rtasm.

    Cc: 9.1 9.2 10.0 <mesa-stable at lists.freedesktop.org>
    Bugzilla: https://bugs.freedesktop.org/show_bug.cgi?id=73473

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140127/3d08c40a/attachment.html>

https://bugs.freedesktop.org/show_bug.cgi?id=73473

Emil Velikov <emil.l.velikov at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |RESOLVED
         Resolution|---                         |FIXED

--- Comment #11 from Emil Velikov <emil.l.velikov at gmail.com> ---
Jaak Ristioja

Considering how long it took for people to review this patch it may be worth
opening another bug about
"[gallium] Add selinux/pax support to auxiliary/rtasm" and select
"Mesa core"
as the component. This will hopefully get some more people looking in this
direction.

Closing this bug as the crash should be resolved.

-- 
You are receiving this mail because:
You are the assignee for the bug.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.freedesktop.org/archives/nouveau/attachments/20140127/c0edfb72/attachment.html>