search for: nvcopynv12colorplan

https://bugs.freedesktop.org/show_bug.cgi?id=63263 Priority: medium Bug ID: 63263 Assignee: nouveau at lists.freedesktop.org Summary: X server crash in nouveau_xv.c:NVPutImage (NVCopyNV12ColorPlanes) QA Contact: xorg-team at lists.x.org Severity: normal Classification: Unclassified OS: Linux (All) Reporter: imirkin at alum.mit.edu Hardware: x86-64 (AMD64) Status: NEW Version: unspecified Component: Driv...

The 'w' argument to NVCopyNV12ColorPlanes is used to index into the source image. line_len is rounded up to 8 on NV50+, so if the source image (+ left offset) is not rounded to 8, NVCopyNV12ColorPlanes could read past the end of the array and crash X. This change can cause the last few horizontal pixels of dst to not be initialized, but...

Signed-off-by: Ilia Mirkin <imirkin at alum.mit.edu> --- src/nouveau_xv.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/src/nouveau_xv.c b/src/nouveau_xv.c index 8eafcf0..567e30c 100644 --- a/src/nouveau_xv.c +++ b/src/nouveau_xv.c @@ -552,8 +552,11 @@ NVCopyNV12ColorPlanes(unsigned char *src1, unsigned char *src2, if (e) { unsigned short *vud = (unsigned short *) vuvud; - +#if X_BYTE_ORDER == X_BIG_ENDIAN + *vud = us[0] | (vs[0]<<8); +#else *vud = vs[0] | (us[0]<<8); +#endif } dst += dstPitch; -- 1.8.1.5

.../nouveau_xv.c index 567e30c..5569b7c 100644 --- a/src/nouveau_xv.c +++ b/src/nouveau_xv.c @@ -25,6 +25,8 @@ #include "config.h" #endif +#include <immintrin.h> + #include "xf86xv.h" #include <X11/extensions/Xv.h> #include "exa.h" @@ -532,30 +534,47 @@ NVCopyNV12ColorPlanes(unsigned char *src1, unsigned char *src2, w >>= 1; h >>= 1; +#ifdef __SSE2__ + l = w >> 3; + e = w & 7; +#else l = w >> 1; e = w & 1; +#endif for (j = 0; j < h; j++) { unsigned char *us = src1; unsigned char *vs = src2; unsigned int *vuvud...