Alexander Kapshuk
2021-Jan-03 08:18 UTC
[Nouveau] [mesa-20.3.2] NULL pointer dereference in vl_compositor_yuv_deint_full
NVIDIA chip affected:
01:00.0 VGA compatible controller: NVIDIA Corporation GT216 [GeForce
210] (rev a1)
The null pointer dereference occurs here:
Thread 27 "vlc" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fff8f7c1640 (LWP 79292)]
0x00007fff8d59d1da in vl_compositor_yuv_deint_full (s=0x7fff980e8518,
c=0x7fff980e83d8, src=0x7fff98670030, dst=0x0,
src_rect=0x7fff8f7c0470,
dst_rect=0x7fff8f7c0460, deinterlace=VL_COMPOSITOR_WEAVE) at
../mesa-20.3.2/src/gallium/auxiliary/vl/vl_compositor.c:689
689 dst_surfaces = dst->get_surfaces(dst); //dst==NULL
=> 0x00007fff8d5981da <+42>: call *0x38(%rcx) //rcx is dst
(gdb) i r rcx
rcx 0x0 0
(gdb) bt
#0 0x00007fff8d59d1da in vl_compositor_yuv_deint_full
(s=0x7fff980e8518, c=0x7fff980e83d8, src=0x7fff98670030, dst=0x0,
src_rect=0x7fff8f7c0470, dst_rect=0x7fff8f7c0460,
deinterlace=VL_COMPOSITOR_WEAVE) at
../mesa-20.3.2/src/gallium/auxiliary/vl/vl_compositor.c:689
#1 0x00007fff8d58a29b in vlVaDeriveImage (ctx=0x7fff980c1590,
surface=<optimized out>, image=0x7fff8f7c05e0)
at ../mesa-20.3.2/src/gallium/frontends/va/image.c:321
#2 0x00007fff91485799 in vaDeriveImage () at /usr/lib/libva.so.2
#3 0x00007fff8e2256d2 in () at
/usr/lib/vlc/plugins/video_output/libglconv_vaapi_x11_plugin.so
#4 0x00007fff8e224189 in () at
/usr/lib/vlc/plugins/video_output/libglconv_vaapi_x11_plugin.so
#5 0x00007fff8f6b1896 in () at
/usr/lib/vlc/plugins/video_output/libgl_plugin.so
#6 0x00007fff8f6b86db in () at
/usr/lib/vlc/plugins/video_output/libgl_plugin.so
#7 0x00007ffff7d07cee in () at /usr/lib/libvlccore.so.9
#8 0x00007ffff7cfa019 in () at /usr/lib/libvlccore.so.9
#9 0x00007ffff7cfbf9e in () at /usr/lib/libvlccore.so.9
#10 0x00007ffff7f623e9 in start_thread () at /usr/lib/libpthread.so.0
#11 0x00007ffff7e8a293 in clone () at /usr/lib/libc.so.6
mesa-20.3.2/src/gallium/frontends/va/image.c:312,313
VAStatus
vlVaDeriveImage(VADriverContextP ctx, VASurfaceID surface, VAImage *image)
{
...
new_template.interlaced = false; //create_video_buffer
returns NULL if new_template.interlaced is set to false See below.
new_buffer = drv->pipe->create_video_buffer(drv->pipe,
&new_template);
...
vl_compositor_yuv_deint_full(&drv->cstate,
&drv->compositor,
surf->buffer, new_buffer,
&src_rect, &dst_rect,
VL_COMPOSITOR_WEAVE);
...
}
[Note]
mesa-20.3.2/src/gallium/drivers/nouveau/nv50/nv84_video.c:618,621
struct pipe_video_buffer *
nv84_video_buffer_create(struct pipe_context *pipe,
const struct pipe_video_buffer *template)
{
...
if (!template->interlaced) { //setting this to true in
vlVaDeriveImage returns valid buffer
debug_printf("Require interlaced video buffers\n");
return NULL;
}
...
}
Please advise on the further course of action.
Ilia Mirkin
2021-Jan-03 17:19 UTC
[Nouveau] [mesa-20.3.2] NULL pointer dereference in vl_compositor_yuv_deint_full
On Sun, Jan 3, 2021 at 3:18 AM Alexander Kapshuk <alexander.kapshuk at gmail.com> wrote:> > NVIDIA chip affected: > 01:00.0 VGA compatible controller: NVIDIA Corporation GT216 [GeForce > 210] (rev a1) > > The null pointer dereference occurs here: > Thread 27 "vlc" received signal SIGSEGV, Segmentation fault. > [Switching to Thread 0x7fff8f7c1640 (LWP 79292)] > 0x00007fff8d59d1da in vl_compositor_yuv_deint_full (s=0x7fff980e8518, > c=0x7fff980e83d8, src=0x7fff98670030, dst=0x0, > src_rect=0x7fff8f7c0470, > dst_rect=0x7fff8f7c0460, deinterlace=VL_COMPOSITOR_WEAVE) at > ../mesa-20.3.2/src/gallium/auxiliary/vl/vl_compositor.c:689 > 689 dst_surfaces = dst->get_surfaces(dst); //dst==NULL > > => 0x00007fff8d5981da <+42>: call *0x38(%rcx) //rcx is dst > (gdb) i r rcx > rcx 0x0 0 > > (gdb) bt > #0 0x00007fff8d59d1da in vl_compositor_yuv_deint_full > (s=0x7fff980e8518, c=0x7fff980e83d8, src=0x7fff98670030, dst=0x0, > src_rect=0x7fff8f7c0470, dst_rect=0x7fff8f7c0460, > deinterlace=VL_COMPOSITOR_WEAVE) at > ../mesa-20.3.2/src/gallium/auxiliary/vl/vl_compositor.c:689 > #1 0x00007fff8d58a29b in vlVaDeriveImage (ctx=0x7fff980c1590, > surface=<optimized out>, image=0x7fff8f7c05e0) > at ../mesa-20.3.2/src/gallium/frontends/va/image.c:321 > #2 0x00007fff91485799 in vaDeriveImage () at /usr/lib/libva.so.2 > #3 0x00007fff8e2256d2 in () at > /usr/lib/vlc/plugins/video_output/libglconv_vaapi_x11_plugin.so > #4 0x00007fff8e224189 in () at > /usr/lib/vlc/plugins/video_output/libglconv_vaapi_x11_plugin.so > #5 0x00007fff8f6b1896 in () at > /usr/lib/vlc/plugins/video_output/libgl_plugin.so > #6 0x00007fff8f6b86db in () at > /usr/lib/vlc/plugins/video_output/libgl_plugin.so > #7 0x00007ffff7d07cee in () at /usr/lib/libvlccore.so.9 > #8 0x00007ffff7cfa019 in () at /usr/lib/libvlccore.so.9 > #9 0x00007ffff7cfbf9e in () at /usr/lib/libvlccore.so.9 > #10 0x00007ffff7f623e9 in start_thread () at /usr/lib/libpthread.so.0 > #11 0x00007ffff7e8a293 in clone () at /usr/lib/libc.so.6 > > mesa-20.3.2/src/gallium/frontends/va/image.c:312,313 > VAStatus > vlVaDeriveImage(VADriverContextP ctx, VASurfaceID surface, VAImage *image) > { > ... > new_template.interlaced = false; //create_video_buffer > returns NULL if new_template.interlaced is set to false See below. > new_buffer = drv->pipe->create_video_buffer(drv->pipe, &new_template); > ... > vl_compositor_yuv_deint_full(&drv->cstate, &drv->compositor, > surf->buffer, new_buffer, > &src_rect, &dst_rect, > VL_COMPOSITOR_WEAVE); > ... > } > > [Note] > mesa-20.3.2/src/gallium/drivers/nouveau/nv50/nv84_video.c:618,621 > struct pipe_video_buffer * > nv84_video_buffer_create(struct pipe_context *pipe, > const struct pipe_video_buffer *template) > { > ... > if (!template->interlaced) { //setting this to true in > vlVaDeriveImage returns valid buffer > debug_printf("Require interlaced video buffers\n"); > return NULL; > } > ... > } > > Please advise on the further course of action.Figure out what change in mesa broke it, send a revert (or fix, if you're able). There has been a bunch of activity in st/vl lately, and the developers never take NVIDIA into account, only AMD (well, they're AMD developers, so not entirely surprising). Cheers, -ilia