Greg Stark via llvm-dev
2015-Nov-12 02:18 UTC
[llvm-dev] Inexplicable ASAN report. Code generation bug?
I'm struggling to explain an ASAN report I'm now getting that I
didn't
get previously on the same code. In fact the report only happens with
-O2 and not when I remove the -O flags which makes it hard to debug
and makes me suspect it's dependent on exactly which instructions the
code generation decides to access the bytes involved. Afaict the C
code shouldn't be accessing the poisoned bytes.
The report looks like:
init_var_from_num: NUMERIC (short) w=0 d=1 POS 6 bytes: 18 00 00 00 80 80
==================================================================28714==ERROR:
AddressSanitizer: unknown-crash on address
0x62900003d7ac at pc 0x00000144a6d6 bp 0x7ffed6680db0 sp
0x7ffed6680da8
READ of size 4 at 0x62900003d7ac thread T0
#0 0x144a6d5 in init_var_from_num
/home/stark/src/pg/postgresql-master/src/backend/utils/adt/numeric.c:4764:17
...
SUMMARY: AddressSanitizer: unknown-crash
/home/stark/src/pg/postgresql-master/src/backend/utils/adt/numeric.c:4764:17
in init_var_from_num
Shadow bytes around the buggy address:
...
=>0x0c527ffffaf0: f7 f7 00 00 f7[06]00 00 f7 00 00 00 00 f7 00 00
The "18 00 00 00 80 80" is a hex dump of the bytes in the 6-byte
object. The preceding and following bytes were poisoned explicitly by
my code which agrees with the shadow bytes listed.
It looks like asan is complaining that the code is doing a 4-byte read
of the last 2 bytes of the object. But in fact the code is accessing
those last two bytes through a 2-byte short which an expression that
looks like (n)->choice.n_short.n_header with the structure and union
looking like these:
struct NumericData
{
int32 vl_len_; /* varlena header (do not touch directly!) */
union NumericChoice choice; /* choice of format */
};
union NumericChoice
{
uint16 n_header; /* Header word */
struct NumericLong n_long; /* Long form (4-byte header) */
struct NumericShort n_short; /* Short form (2-byte header) */
};
struct NumericShort
{
uint16 n_header; /* Sign + display scale + weight */
NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */
};
struct NumericLong
{
uint16 n_sign_dscale; /* Sign + display scale */
int16 n_weight; /* Weight of 1st digit */
NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */
};
So it looks to me like -O2 is causing the optimizer to turn the 2-byte
read into a 4-byte read and overrun the allocated object. But I
haven't tried looking at the assembly yet.
--
greg
Greg Stark via llvm-dev
2015-Nov-12 02:43 UTC
[llvm-dev] Inexplicable ASAN report. Code generation bug?
On Thu, Nov 12, 2015 at 2:18 AM, Greg Stark <stark at mit.edu> wrote:> So it looks to me like -O2 is causing the optimizer to turn the 2-byte > read into a 4-byte read and overrun the allocated object. But I > haven't tried looking at the assembly yet.Fwiw the assembly is obviously a 4-byte load but identifying the mapping from the C code reading two of those bytes to the assembly is a bit beyond my level of familiarity with x86 assembly: #define NUMERIC_WEIGHT(n) (NUMERIC_HEADER_IS_SHORT((n)) ? \ (((n)->choice.n_short.n_header & NUMERIC_SHORT_WEIGHT_SIGN_MASK ? \ ~NUMERIC_SHORT_WEIGHT_MASK : 0) \ | ((n)->choice.n_short.n_header & NUMERIC_SHORT_WEIGHT_MASK)) \ : ((n)->choice.n_long.n_weight)) ... 0x000000000144a6ba <+1050>: mov %r15d,%ecx 0x000000000144a6bd <+1053>: and $0x7,%ecxn 0x000000000144a6c0 <+1056>: add $0x3,%ecx 0x000000000144a6c3 <+1059>: movsbl %al,%eax 0x000000000144a6c6 <+1062>: cmp %eax,%ecx 0x000000000144a6c8 <+1064>: jl 0x144a441 <numeric_out+417> 0x000000000144a6ce <+1070>: mov %r15,%rdi 0x000000000144a6d1 <+1073>: callq 0x526350 <__asan_report_load4> ==> 0x000000000144a6d6 <+1078>: mov $0x2e02680,%edi 0x000000000144a6db <+1083>: mov %r14,0x10(%rbx) 0x000000000144a6df <+1087>: mov %rsi,%r14 0x000000000144a6e2 <+1090>: callq 0x53b1b0 <__sanitizer_cov()> 0x000000000144a6e7 <+1095>: mov %r14,%rsi 0x000000000144a6ea <+1098>: mov 0x10(%rbx),%r14 0x000000000144a6ee <+1102>: jmpq 0x144a4a1 <numeric_out+513> 0x000000000144a6f3 <+1107>: mov %edi,%ecx 0x000000000144a6f5 <+1109>: and $0x7,%ecx 0x000000000144a6f8 <+1112>: add $0x3,%ecx 0x000000000144a6fb <+1115>: movsbl %al,%eax ... So I'm guessing the logic is that the struct as that VLA so the compiler sees that the struct size is at least two more bytes so it assumes memory will always be allocated for the whole object and feels free to reference those extra bytes? In practice this is a false positive since the object will always be aligned so those two extra bytes will always be on the same page. We already have another code site with a similar false positive but it's a much narrower circumstance. If this can happen anywhere there's a non 4-byte access then that will remove a lot of the value in asan (fwiw msan doesn't complain about this) -- greg
Kostya Serebryany via llvm-dev
2015-Nov-12 20:42 UTC
[llvm-dev] Inexplicable ASAN report. Code generation bug?
2 questions: - Do you see this with the fresh llvm trunk? - Can you prepare a minimized example? On Wed, Nov 11, 2015 at 6:18 PM, Greg Stark via llvm-dev < llvm-dev at lists.llvm.org> wrote:> I'm struggling to explain an ASAN report I'm now getting that I didn't > get previously on the same code. In fact the report only happens with > -O2 and not when I remove the -O flags which makes it hard to debug > and makes me suspect it's dependent on exactly which instructions the > code generation decides to access the bytes involved. Afaict the C > code shouldn't be accessing the poisoned bytes. > > The report looks like: > > init_var_from_num: NUMERIC (short) w=0 d=1 POS 6 bytes: 18 00 00 00 80 80 > ================================================================> ==28714==ERROR: AddressSanitizer: unknown-crash on address > 0x62900003d7ac at pc 0x00000144a6d6 bp 0x7ffed6680db0 sp > 0x7ffed6680da8 > READ of size 4 at 0x62900003d7ac thread T0 > #0 0x144a6d5 in init_var_from_num > > /home/stark/src/pg/postgresql-master/src/backend/utils/adt/numeric.c:4764:17 > ... > SUMMARY: AddressSanitizer: unknown-crash > > /home/stark/src/pg/postgresql-master/src/backend/utils/adt/numeric.c:4764:17 > in init_var_from_num > Shadow bytes around the buggy address: > ... > =>0x0c527ffffaf0: f7 f7 00 00 f7[06]00 00 f7 00 00 00 00 f7 00 00 > > > The "18 00 00 00 80 80" is a hex dump of the bytes in the 6-byte > object. The preceding and following bytes were poisoned explicitly by > my code which agrees with the shadow bytes listed. > > It looks like asan is complaining that the code is doing a 4-byte read > of the last 2 bytes of the object. But in fact the code is accessing > those last two bytes through a 2-byte short which an expression that > looks like (n)->choice.n_short.n_header with the structure and union > looking like these: > > struct NumericData > { > int32 vl_len_; /* varlena header (do not touch directly!) */ > union NumericChoice choice; /* choice of format */ > }; > > union NumericChoice > { > uint16 n_header; /* Header word */ > struct NumericLong n_long; /* Long form (4-byte header) */ > struct NumericShort n_short; /* Short form (2-byte header) */ > }; > > struct NumericShort > { > uint16 n_header; /* Sign + display scale + weight */ > NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */ > }; > > struct NumericLong > { > uint16 n_sign_dscale; /* Sign + display scale */ > int16 n_weight; /* Weight of 1st digit */ > NumericDigit n_data[FLEXIBLE_ARRAY_MEMBER]; /* Digits */ > }; > > > So it looks to me like -O2 is causing the optimizer to turn the 2-byte > read into a 4-byte read and overrun the allocated object. But I > haven't tried looking at the assembly yet. > > > -- > greg > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151112/6866a32e/attachment.html>
Greg Stark via llvm-dev
2015-Nov-14 17:09 UTC
[llvm-dev] Inexplicable ASAN report. Code generation bug?
On Thu, Nov 12, 2015 at 8:42 PM, Kostya Serebryany <kcc at google.com> wrote:> 2 questions: > - Do you see this with the fresh llvm trunk? > - Can you prepare a minimized example?Pretty recent, I updated a couple days ago. I tried to minimize the attached but at the same time I didn't want to lose too many unions and casts in case it didn't trigger any more. $ clang -fsanitize=address -Wall numeric-asan-test.c $ ./a.out VARSIZE 6 NDIGITS 0 WEIGHT 0 SIGN 0 DSCALE 0 $ clang -fsanitize=address -O2 -Wall numeric-asan-test.c $ ./a.out VARSIZE 6 ==================================================================19982==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x60200000eff4 at pc 0x0000004d4986 bp 0x7ffe14e2cb90 sp 0x7ffe14e2cb88 READ of size 4 at 0x60200000eff4 thread T0 #0 0x4d4985 in main (/home/stark/src/a.out+0x4d4985) #1 0x7f6360f40b44 in __libc_start_main /tmp/buildd/glibc-2.19/csu/libc-start.c:287 #2 0x41a935 in _start (/home/stark/src/a.out+0x41a935) 0x60200000eff6 is located 0 bytes to the right of 6-byte region [0x60200000eff0,0x60200000eff6) allocated by thread T0 here: #0 0x4abceb in __interceptor_malloc /home/stark/src/llvm/projects/compiler-rt/lib/asan/asan_malloc_linux.cc:40:3 #1 0x4d479e in main (/home/stark/src/a.out+0x4d479e) SUMMARY: AddressSanitizer: heap-buffer-overflow (/home/stark/src/a.out+0x4d4985) in main Shadow bytes around the buggy address: 0x0c047fff9da0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9db0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9dc0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9dd0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9de0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa =>0x0c047fff9df0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa[06]fa 0x0c047fff9e00: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9e10: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9e20: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9e30: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa 0x0c047fff9e40: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa Shadow byte legend (one shadow byte represents 8 application bytes): Addressable: 00 Partially addressable: 01 02 03 04 05 06 07 Heap left redzone: fa Heap right redzone: fb Freed heap region: fd Stack left redzone: f1 Stack mid redzone: f2 Stack right redzone: f3 Stack partial redzone: f4 Stack after return: f5 Stack use after scope: f8 Global redzone: f9 Global init order: f6 Poisoned by user: f7 Container overflow: fc Array cookie: ac Intra object redzone: bb ASan internal: fe Left alloca redzone: ca Right alloca redzone: cb ==19982==ABORTING -- greg -------------- next part -------------- A non-text attachment was scrubbed... Name: numeric-asan-test.c Type: text/x-csrc Size: 4464 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20151114/1c7d8dbe/attachment.c>