flac dev - Sep 2007 - Re: 1.2.0: Test suite failures on LP64 archs?

Running the basic (--disable-thorough-tests) test suite, I get these
failures

round-trip test (rt-1-24-111.raw) encode... Segmentation fault (core
dumped) ERROR
FAIL: ./test_flac.sh

fsd24-01 (--channels=1 --bps=24 -0 -l 16 --lax -m -e -p): encode...ERROR during
encode of fsd24-01
FAIL: ./test_streams.sh

on alpha and amd64.  By contrast, i386 is fine.  (All OpenBSD/4.2.)
Could be a generic LP64 bug...

-- 
Christian "naddy" Weisgerber                         
naddy@mips.inka.de

Christian Weisgerber <naddy@mips.inka.de> wrote:
> round-trip test (rt-1-24-111.raw) encode... Segmentation fault (core
> dumped) ERROR
> FAIL: ./test_flac.sh
> 
> fsd24-01 (--channels=1 --bps=24 -0 -l 16 --lax -m -e -p): encode...ERROR
> during encode of fsd24-01
> FAIL: ./test_streams.sh
(The latter is also a segfault, btw.)

I've poked around a bit with gdb.  Both crashes happen in the same
place:

#0  0x0000000040d18810 in FLAC__lpc_compute_residual_from_qlp_coefficients_wide
    (data=0x49e4c014, data_len=110, qlp_coeff=0x7f7ffffece70, order=1, 
    lp_quantization=14, residual=0x4fced000) at lpc.c:745
745                                                     residual[i] = data[i] -
(FLAC__int32)((qlp_coeff[0] * (FLAC__int64)data[i-1]) >> lp_quantization);
(gdb) i loc
i = 0
sum = 70368744161280

Alas, I don't see anything wrong there.  Also, I can evaluate that
expression in gdb just fine.

Hmm.

-- 
Christian "naddy" Weisgerber                         
naddy@mips.inka.de

Christian Weisgerber <naddy@mips.inka.de> wrote:
> #0  0x0000000040d18810 in
FLAC__lpc_compute_residual_from_qlp_coefficients_wide
>     (data=0x49e4c014, data_len=110, qlp_coeff=0x7f7ffffece70, order=1, 
>     lp_quantization=14, residual=0x4fced000) at lpc.c:745
> 745                                                     residual[i] >
data[i] - (FLAC__int32)((qlp_coeff[0] * (FLAC__int64)data[i-1]) >>
> lp_quantization);
> (gdb) i loc
> i = 0
I have figured out the problem now.  The index variable i is defined
as unsigned int.  Underflowing it causes the value to go to 0xffffffff
which is _not_ the same as -1 when added to a pointer on 64-bit
architectures.

Specifically in the case of the crash above, the generated code
performs an access at offset 0x3fffffffc (0xffffffff * sizeof(int)).
This problem is hidden on 32-bit archs due to truncation.

The bug was introduced in 1.2.0 with the fully unrolled functions
in libFLAC/lpc.c.

-- 
Christian "naddy" Weisgerber                         
naddy@mips.inka.de