llvm dev - Apr 2020 - Segfault after compiling wget with dfsan

Hi all,

I was trying to compile dfsan with wget. (Just enabling the dfsan feature,
without actually making changes to the source code) Without dfsan, I am
able to compile and run wget 1.19.5 (available at
https://ftp.gnu.org/gnu/wget/wget-1.19.5.tar.gz). But when compiled with
dfsan, it encountered a null pointer dereference error.

Following an old post:
http://lists.llvm.org/pipermail/cfe-dev/2014-May/037160.html . I was trying
to use a blacklist for openssl functions.

My commands are:
export CC="clang -g -fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt
export LDFLAGS=" -fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt
./configure --with-ssl=openssl
make

I've tried with clang 9,10, and the nightly build of 11 today.

The error is:
when trying to run "src/wget www.google.com"

Program received signal SIGSEGV, Segmentation fault.
0x00005555556de113 in url_parse (url=0x555555c47550
"http://www.google.com",

    error=0x7fffffffdb30, iri=0x555555c442f0 <dummy_iri>,
percent_encode=true)
    at url.c:837
837  if (*p == ':')
and p is a null pointer.

I am not sure if this is because I misused the dfsan or for some other
reason, any ideas?

Attached is the  fsanitize-blacklist I've used.

Thanks,
Tianyi
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200403/2a8dfb54/attachment.html>
-------------- next part --------------
fun:SSL*=uninstrumented
fun:ssl*=uninstrumented
fun:RAND*=uninstrumented
fun:DES*=uninstrumented
fun:ERR*=uninstrumented
fun:MD5*=uninstrumented
fun:MD4*=uninstrumented
fun:*X509*=uninstrumented
fun:OPENSSL*=uninstrumented
fun:CRYPTO*=uninstrumented
fun:TLS*=uninstrumented
fun:BIO_*=uninstrumented
fun:gzdopen=uninstrumented
fun:gzclose=uninstrumented
fun:gzwrite=uninstrumented
fun:a2i_IPADDRESS=uninstrumented
fun:ENGINE_load_builtin_engines=uninstrumented
fun:inflateInit2_=uninstrumented
fun:inflate=uninstrumented
fun:inflateEnd=uninstrumented
fun:inflateInit2_=uninstrumented
fun:ASN1_*=uninstrumented
fun:GENERAL_NAME_free=uninstrumented
fun:CONF_modules_load_file=uninstrumented

On Fri, Apr 3, 2020 at 7:54 PM Tianyi Chen via llvm-dev
<llvm-dev at lists.llvm.org> wrote:
>
> Hi all,
>
> I was trying to compile dfsan with wget. (Just enabling the dfsan feature,
without actually making changes to the source code) Without dfsan, I am able to
compile and run wget 1.19.5 (available at
https://ftp.gnu.org/gnu/wget/wget-1.19.5.tar.gz). But when compiled with dfsan,
it encountered a null pointer dereference error.
>
> Following an old
post:http://lists.llvm.org/pipermail/cfe-dev/2014-May/037160.html . I was trying
to use a blacklist for openssl functions.
>
> My commands are:
> export CC="clang -g -fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt
> export LDFLAGS=" -fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt
> ./configure --with-ssl=openssl
> make
>
> I've tried with clang 9,10, and the nightly build of 11 today.
>
> The error is:
> when trying to run "src/wget www.google.com"
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x00005555556de113 in url_parse (url=0x555555c47550
"http://www.google.com",
>     error=0x7fffffffdb30, iri=0x555555c442f0 <dummy_iri>,
percent_encode=true)
>     at url.c:837
> 837  if (*p == ':')
> and p is a null pointer.
If `p` is a null pointer, then reading *p should segfault.  To debug
this, you need to discover why p is null.
> I am not sure if this is because I misused the dfsan or for some other
reason, any ideas?
Does this happen if you build without `-fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt` in CC and LDFLAGS?
>
> Attached is the  fsanitize-blacklist I've used.
>
> Thanks,
> Tianyi
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

No. Without these sanitizer flags, running the built binary with "wget
www.google.com " successfully downloaded the html file without any errors.
My OS is Ubuntu 18.04.4 LTS (64 bit), in case this info may help.

On Mon, Apr 6, 2020 at 8:42 AM Sam Kerner <skerner at chromium.org> wrote:
> On Fri, Apr 3, 2020 at 7:54 PM Tianyi Chen via llvm-dev
> <llvm-dev at lists.llvm.org> wrote:
> >
> > Hi all,
> >
> > I was trying to compile dfsan with wget. (Just enabling the dfsan
> feature, without actually making changes to the source code) Without dfsan,
> I am able to compile and run wget 1.19.5 (available at
>
https://urldefense.com/v3/__https://ftp.gnu.org/gnu/wget/wget-1.19.5.tar.gz__;!!LIr3w8kk_Xxm!4FISewdn_YnoZgAHPlovgXkVlsZGYeOykrOoifTELfbDARRzYHBtw8Ls9J9S468NSA$
> ). But when compiled with dfsan, it encountered a null pointer dereference
> error.
> >
> > Following an old post:
>
https://urldefense.com/v3/__http://lists.llvm.org/pipermail/cfe-dev/2014-May/037160.html__;!!LIr3w8kk_Xxm!4FISewdn_YnoZgAHPlovgXkVlsZGYeOykrOoifTELfbDARRzYHBtw8Ls9J832bNqXQ$
> . I was trying to use a blacklist for openssl functions.
> >
> > My commands are:
> > export CC="clang -g -fsanitize=dataflow
> -fsanitize-blacklist=/tmp/openssl-list.txt
> > export LDFLAGS=" -fsanitize=dataflow
> -fsanitize-blacklist=/tmp/openssl-list.txt
> > ./configure --with-ssl=openssl
> > make
> >
> > I've tried with clang 9,10, and the nightly build of 11 today.
> >
> > The error is:
> > when trying to run "src/wget
>
https://urldefense.com/v3/__http://www.google.com__;!!LIr3w8kk_Xxm!4FISewdn_YnoZgAHPlovgXkVlsZGYeOykrOoifTELfbDARRzYHBtw8Ls9J-IP4CUvA$
> "
> >
> > Program received signal SIGSEGV, Segmentation fault.
> > 0x00005555556de113 in url_parse (url=0x555555c47550 "
>
https://urldefense.com/v3/__http://www.google.com__;!!LIr3w8kk_Xxm!4FISewdn_YnoZgAHPlovgXkVlsZGYeOykrOoifTELfbDARRzYHBtw8Ls9J-IP4CUvA$
> ",
> >     error=0x7fffffffdb30, iri=0x555555c442f0 <dummy_iri>,
> percent_encode=true)
> >     at url.c:837
> > 837  if (*p == ':')
> > and p is a null pointer.
>
> If `p` is a null pointer, then reading *p should segfault.  To debug
> this, you need to discover why p is null.
>
> > I am not sure if this is because I misused the dfsan or for some other
> reason, any ideas?
>
> Does this happen if you build without `-fsanitize=dataflow
> -fsanitize-blacklist=/tmp/openssl-list.txt` in CC and LDFLAGS?
>
> >
> > Attached is the  fsanitize-blacklist I've used.
> >
> > Thanks,
> > Tianyi
> >
> > _______________________________________________
> > LLVM Developers mailing list
> > llvm-dev at lists.llvm.org
> >
>
https://urldefense.com/v3/__https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev__;!!LIr3w8kk_Xxm!4FISewdn_YnoZgAHPlovgXkVlsZGYeOykrOoifTELfbDARRzYHBtw8Ls9J_rhBfirg$
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200406/ed67ed6c/attachment.html>

I concluded my investigation and the reason looks like dfsan breaks the
strchr function, as reported previously here
https://bugs.llvm.org/show_bug.cgi?id=22392 .

The following is the gdb log, for the build without dfsan, strchr enters
the implementation, for the build with dfsan, the function was just skipped.

=================================(gdb) b strpbrk_or_eos
Breakpoint 1 at 0x440f00: file url.c, line 633.
(gdb) r www.google.com
Starting program: /tmp/wget-1.19.5/src/wget www.google.com
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, strpbrk_or_eos (s=0x68f547 "www.google.com",
    accept=0x684bc8 <init_seps.seps> ":/?#") at url.c:633
633  char *p = strpbrk (s, accept);
(gdb) n
634  if (!p)
(gdb) p p
$2 = 0x0
(gdb) step
635    p = strchr (s, '\0');
(gdb) p p
$3 = 0x0
(gdb) step
__strchr_sse2 () at ../sysdeps/x86_64/multiarch/../strchr.S:24
24 ../sysdeps/x86_64/multiarch/../strchr.S: No such file or directory.
(gdb)

============== With dfsan =================
(gdb) b strpbrk_or_eos
Breakpoint 1 at 0x1938ec: file url.c, line 633.
(gdb) r www.google.com
Starting program: /tmp/wget-1.19.5-dfsan/src/wget www.google.com
[Thread debugging using libthread_db enabled]
Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".

Breakpoint 1, strpbrk_or_eos (s=0x555555c6e577 "www.google.com",
    accept=0x555555a525a8 <init_seps.seps> ":/?#") at url.c:633
633  char *p = strpbrk (s, accept);
(gdb) n
634  if (!p)
(gdb) p p
$1 = 0x0
(gdb) step
635    p = strchr (s, '\0');
(gdb) p p
$2 = 0x0
(gdb) step
636  return p;
(gdb)

On Fri, Apr 3, 2020 at 4:53 PM Tianyi Chen <tchen025 at usc.edu> wrote:
> Hi all,
>
> I was trying to compile dfsan with wget. (Just enabling the dfsan feature,
> without actually making changes to the source code) Without dfsan, I am
> able to compile and run wget 1.19.5 (available at
> https://ftp.gnu.org/gnu/wget/wget-1.19.5.tar.gz). But when compiled with
> dfsan, it encountered a null pointer dereference error.
>
> Following an old post:
> http://lists.llvm.org/pipermail/cfe-dev/2014-May/037160.html . I was
> trying to use a blacklist for openssl functions.
>
> My commands are:
> export CC="clang -g -fsanitize=dataflow
> -fsanitize-blacklist=/tmp/openssl-list.txt
> export LDFLAGS=" -fsanitize=dataflow
> -fsanitize-blacklist=/tmp/openssl-list.txt
> ./configure --with-ssl=openssl
> make
>
> I've tried with clang 9,10, and the nightly build of 11 today.
>
> The error is:
> when trying to run "src/wget www.google.com"
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x00005555556de113 in url_parse (url=0x555555c47550
"http://www.google.com",
>
>     error=0x7fffffffdb30, iri=0x555555c442f0 <dummy_iri>,
> percent_encode=true)
>     at url.c:837
> 837  if (*p == ':')
> and p is a null pointer.
>
> I am not sure if this is because I misused the dfsan or for some other
> reason, any ideas?
>
> Attached is the  fsanitize-blacklist I've used.
>
> Thanks,
> Tianyi
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20200412/cf8bfca1/attachment.html>

On Sun, Apr 12, 2020 at 3:00 PM Tianyi Chen <tchen025 at usc.edu>
wrote:
>
> I concluded my investigation and the reason looks like dfsan breaks the
strchr function, as reported previously here  
https://bugs.llvm.org/show_bug.cgi?id=22392 .
Tianyi,
    Thanks for reducing the problem to a specific issue.  I created a
patch to fix it:

   https://reviews.llvm.org/D77996

Sam
>
> The following is the gdb log, for the build without dfsan, strchr enters
the implementation, for the build with dfsan, the function was just skipped.
>
> =================================> (gdb) b strpbrk_or_eos
> Breakpoint 1 at 0x440f00: file url.c, line 633.
> (gdb) r www.google.com
> Starting program: /tmp/wget-1.19.5/src/wget www.google.com
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
>
> Breakpoint 1, strpbrk_or_eos (s=0x68f547 "www.google.com",
>     accept=0x684bc8 <init_seps.seps> ":/?#") at url.c:633
> 633  char *p = strpbrk (s, accept);
> (gdb) n
> 634  if (!p)
> (gdb) p p
> $2 = 0x0
> (gdb) step
> 635    p = strchr (s, '\0');
> (gdb) p p
> $3 = 0x0
> (gdb) step
> __strchr_sse2 () at ../sysdeps/x86_64/multiarch/../strchr.S:24
> 24 ../sysdeps/x86_64/multiarch/../strchr.S: No such file or directory.
> (gdb)
>
> ============== With dfsan =================>
> (gdb) b strpbrk_or_eos
> Breakpoint 1 at 0x1938ec: file url.c, line 633.
> (gdb) r www.google.com
> Starting program: /tmp/wget-1.19.5-dfsan/src/wget www.google.com
> [Thread debugging using libthread_db enabled]
> Using host libthread_db library
"/lib/x86_64-linux-gnu/libthread_db.so.1".
>
> Breakpoint 1, strpbrk_or_eos (s=0x555555c6e577 "www.google.com",
>     accept=0x555555a525a8 <init_seps.seps> ":/?#") at
url.c:633
> 633  char *p = strpbrk (s, accept);
> (gdb) n
> 634  if (!p)
> (gdb) p p
> $1 = 0x0
> (gdb) step
> 635    p = strchr (s, '\0');
> (gdb) p p
> $2 = 0x0
> (gdb) step
> 636  return p;
> (gdb)
>
> On Fri, Apr 3, 2020 at 4:53 PM Tianyi Chen <tchen025 at usc.edu>
wrote:
>>
>> Hi all,
>>
>> I was trying to compile dfsan with wget. (Just enabling the dfsan
feature, without actually making changes to the source code) Without dfsan, I am
able to compile and run wget 1.19.5 (available at
https://ftp.gnu.org/gnu/wget/wget-1.19.5.tar.gz). But when compiled with dfsan,
it encountered a null pointer dereference error.
>>
>> Following an old
post:http://lists.llvm.org/pipermail/cfe-dev/2014-May/037160.html . I was trying
to use a blacklist for openssl functions.
>>
>> My commands are:
>> export CC="clang -g -fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt
>> export LDFLAGS=" -fsanitize=dataflow
-fsanitize-blacklist=/tmp/openssl-list.txt
>> ./configure --with-ssl=openssl
>> make
>>
>> I've tried with clang 9,10, and the nightly build of 11 today.
>>
>> The error is:
>> when trying to run "src/wget www.google.com"
>>
>> Program received signal SIGSEGV, Segmentation fault.
>> 0x00005555556de113 in url_parse (url=0x555555c47550
"http://www.google.com",
>>     error=0x7fffffffdb30, iri=0x555555c442f0 <dummy_iri>,
percent_encode=true)
>>     at url.c:837
>> 837  if (*p == ':')
>> and p is a null pointer.
>>
>> I am not sure if this is because I misused the dfsan or for some other
reason, any ideas?
>>
>> Attached is the  fsanitize-blacklist I've used.
>>
>> Thanks,
>> Tianyi
>>