llvm dev - Sep 2017 - libFuzzer: issue with weak symbols on Mac

I'd like to discuss the following change: https://reviews.llvm.org/D37526

For the context, there is a comment
in compiler-rt/lib/fuzzer/FuzzerExtFunctionsWeak.cpp:

// Implementation for Linux. This relies on the linker's support for weak
// symbols. We don't use this approach on Apple platforms because it
requires
// clients of LibFuzzer to pass ``-U _<symbol_name>`` to the linker to
allow
// weak symbols to be undefined. That is a complication we don't want to
expose
// to clients right now.

That makes sense, but with current implementation, you cannot use
libFuzzer's interface functions other than LLVMFuzzerTestOneInput. Below is
a small example to verify that LLVMFuzzerInitialize is not being called on
Mac:

#include <stddef.h>
#include <stdint.h>
#include <stdio.h>

extern "C" int LLVMFuzzerInitialize(int* argc, char*** argv) {
  printf("Hello from LLVMFuzzerInitialize, argc: %i\n", *argc);
  return *argc;
}

extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t
size) {
  printf("Hello from LLVMFuzzerTestOneInput, size: %zu\n", size);
  if (size) {
  	return data[0];
  }
  return size;
}


Assuming that there are libFuzzer customers who don't mind to specify
"-U,_%function_name%" explicitly (e.g.
https://chromium-review.googlesource.com/c/chromium/src/+/653846/1/testing/libfuzzer/BUILD.gn),
we need to have a way to use FuzzerExtFunctionsWeak.cpp instead
of FuzzerExtFunctionsDlsym.cpp on Mac.

The CL I've uploaded feels a bit hacky to me, but I don't see any less
intrusive solution that would still comply with existing implementation and
would also support weak symbols to be explicitly allowed if needed.

Thanks!

--
Max
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20170906/5af0f91c/attachment.html>

Max Moroz via llvm-dev <llvm-dev at lists.llvm.org>
writes:
> I'd like to discuss the following change:
https://reviews.llvm.org/D37526
>
> For the context, there is a comment
> in compiler-rt/lib/fuzzer/FuzzerExtFunctionsWeak.cpp:
>
> // Implementation for Linux. This relies on the linker's support for
weak
> // symbols. We don't use this approach on Apple platforms because it
> requires
> // clients of LibFuzzer to pass ``-U _<symbol_name>`` to the linker
to allow
> // weak symbols to be undefined. That is a complication we don't want
to
> expose
> // to clients right now.
>
> That makes sense, but with current implementation, you cannot use
> libFuzzer's interface functions other than LLVMFuzzerTestOneInput.
Below is
> a small example to verify that LLVMFuzzerInitialize is not being called on
> Mac:
>
> #include <stddef.h>
> #include <stdint.h>
> #include <stdio.h>
>
> extern "C" int LLVMFuzzerInitialize(int* argc, char*** argv) {
>   printf("Hello from LLVMFuzzerInitialize, argc: %i\n", *argc);
>   return *argc;
> }
>
> extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data, size_t
size) {
>   printf("Hello from LLVMFuzzerTestOneInput, size: %zu\n", size);
>   if (size) {
>   	return data[0];
>   }
>   return size;
> }
I suspect you might be mistaken about the problem, and what's actually
happening is that the linker is dead stripping your hook functions. At
least, I've had plenty of success with fuzzers on macOS with
LLVMFuzzerInitialize and LLVMFuzzerCustomMutator.

Try adding __attribute__((__used__)) to LLVMFuzzerInitialize and see if
that fixes the problem for you:

  extern "C" __attribute__((__used__)) int LLVMFuzzerInitialize(...)
> Assuming that there are libFuzzer customers who don't mind to specify
> "-U,_%function_name%" explicitly (e.g.
>
https://chromium-review.googlesource.com/c/chromium/src/+/653846/1/testing/libfuzzer/BUILD.gn),
> we need to have a way to use FuzzerExtFunctionsWeak.cpp instead
> of FuzzerExtFunctionsDlsym.cpp on Mac.
All of this seems unnecessarily awkward - the correct way to use weak
symbols on macOS is just to provide a default implementation that does
nothing. The function call overhead isn't that much worse than the
branch overhead to avoid calling it.
> The CL I've uploaded feels a bit hacky to me, but I don't see any
less
> intrusive solution that would still comply with existing implementation and
> would also support weak symbols to be explicitly allowed if needed.
>
> Thanks!
>
> --
> Max
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

> On Sep 6, 2017, at 1:40 PM, Justin Bogner <mail at justinbogner.com>
wrote:
> 
> Max Moroz via llvm-dev <llvm-dev at lists.llvm.org> writes:
>> I'd like to discuss the following change:
https://reviews.llvm.org/D37526
>> 
>> For the context, there is a comment
>> in compiler-rt/lib/fuzzer/FuzzerExtFunctionsWeak.cpp:
>> 
>> // Implementation for Linux. This relies on the linker's support
for weak
>> // symbols. We don't use this approach on Apple platforms because
it
>> requires
>> // clients of LibFuzzer to pass ``-U _<symbol_name>`` to the
linker to allow
>> // weak symbols to be undefined. That is a complication we don't
want to
>> expose
>> // to clients right now.
>> 
>> That makes sense, but with current implementation, you cannot use
>> libFuzzer's interface functions other than LLVMFuzzerTestOneInput.
Below is
>> a small example to verify that LLVMFuzzerInitialize is not being called
on
>> Mac:
>> 
>> #include <stddef.h>
>> #include <stdint.h>
>> #include <stdio.h>
>> 
>> extern "C" int LLVMFuzzerInitialize(int* argc, char*** argv)
{
>>  printf("Hello from LLVMFuzzerInitialize, argc: %i\n",
*argc);
>>  return *argc;
>> }
>> 
>> extern "C" int LLVMFuzzerTestOneInput(const uint8_t* data,
size_t size) {
>>  printf("Hello from LLVMFuzzerTestOneInput, size: %zu\n",
size);
>>  if (size) {
>>  	return data[0];
>>  }
>>  return size;
>> }
> 
> I suspect you might be mistaken about the problem, and what's actually
> happening is that the linker is dead stripping your hook functions. At
> least, I've had plenty of success with fuzzers on macOS with
> LLVMFuzzerInitialize and LLVMFuzzerCustomMutator.
> 
> Try adding __attribute__((__used__)) to LLVMFuzzerInitialize and see if
> that fixes the problem for you:
> 
>  extern "C" __attribute__((__used__)) int LLVMFuzzerInitialize(…)
Moreover, libFuzzer tests do run on Mac,
and they do use other interface functions.

Another way to avoid that is simply not to request dead stripping.
> 
>> Assuming that there are libFuzzer customers who don't mind to
specify
>> "-U,_%function_name%" explicitly (e.g.
>>
https://chromium-review.googlesource.com/c/chromium/src/+/653846/1/testing/libfuzzer/BUILD.gn),
>> we need to have a way to use FuzzerExtFunctionsWeak.cpp instead
>> of FuzzerExtFunctionsDlsym.cpp on Mac.
> 
> All of this seems unnecessarily awkward - the correct way to use weak
> symbols on macOS is just to provide a default implementation that does
> nothing. The function call overhead isn't that much worse than the
> branch overhead to avoid calling it.
> 
>> The CL I've uploaded feels a bit hacky to me, but I don't see
any less
>> intrusive solution that would still comply with existing implementation
and
>> would also support weak symbols to be explicitly allowed if needed.
>> 
>> Thanks!
>> 
>> --
>> Max
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev