freebsd stable - Nov 2012 - dc(1) fails with "big number failure" on 2^64

Hi,

I am seeing this in dc:

janm at gray: dc $ dc
18446744073709551616 18446744073709551616 / ps
dc: big number failure 306b06b: No such file or directory

That number is 2^64. The error is coming from BN_check in bdiv(), which is
complaining about the number at the top of the stack being uninitialised.
Looking at the data, after the second pop in bdiv() in bdata.c,
b->number->d[b->number->top - 1] == 0. After a while poking around
in a debugger, it looks like the first word of the second number
(a->number->d) is being allocated at the same location as the last word of
the second number, it gets zeroed, and then looks uninitialised.

All of this seems to be happening in the BN_* routines in openssl.

I am seeing this on my builds for 9.1-RC3 and 9.0-p3, as well as the CDROM shell
on the 9.1-RC3 ISO, so I'm pretty sure it isn't my build process or
compiler flags. I have checked an OpenBSD 5.2 installation, and it works fine.

Can anyone confirm this? Am I just seeing things? Is there an obvious fix?

Thanks,

Jan Mikkelsen

I can confirm something similar.

$ dc
18446744073709551616 18446744073709551616 / ps
dc: big number failure 306b06b: No error: 0

FreeBSD 9.1-PRERELEASE #0 r242513 amd64, clang 



--
View this message in context:
http://freebsd.1045724.n5.nabble.com/dc-1-fails-with-big-number-failure-on-2-64-tp5759049p5759215.html
Sent from the freebsd-stable mailing list archive at Nabble.com.

On 11/08/2012 03:06, Jan Mikkelsen wrote:
> Hi,
>
> I am seeing this in dc:
>
> janm at gray: dc $ dc
> 18446744073709551616 18446744073709551616 / ps
> dc: big number failure 306b06b: No such file or directory
>
> That number is 2^64. The error is coming from BN_check in bdiv(), which is
complaining about the number at the top of the stack being uninitialised.
Looking at the data, after the second pop in bdiv() in bdata.c,
b->number->d[b->number->top - 1] == 0. After a while poking around
in a debugger, it looks like the first word of the second number
(a->number->d) is being allocated at the same location as the last word of
the second number, it gets zeroed, and then looks uninitialised.
>
> All of this seems to be happening in the BN_* routines in openssl.
>
> I am seeing this on my builds for 9.1-RC3 and 9.0-p3, as well as the CDROM
shell on the 9.1-RC3 ISO, so I'm pretty sure it isn't my build process
or compiler flags. I have checked an OpenBSD 5.2 installation, and it works
fine.
>
> Can anyone confirm this? Am I just seeing things? Is there an obvious fix?
No fix, but I see a problem in the BN_add_word function in
/usr/src/crypto/openssl/crypto/bn/bn_word.c

         /* Only expand (and risk failing) if it's possibly necessary */
         if (((BN_ULONG)(a->d[a->top - 1] + 1) == 0) &&
                         (bn_wexpand(a,a->top+1) == NULL))
                 return(0);

This can't possibly work, since it never calls bn_wexpand if, for example,
you
add 6 to the number 18446744073709551610

Cheers
Michiel