Johannes Doerfert via llvm-dev
2021-Mar-08 20:20 UTC
[llvm-dev] Applying for GSoC 2021(Fuzzing LLVM-IR Passes)
Hi Chibin, apologies for the late reply. There have been multiple people that expressed interest in this project, though it is open ended and we might want to tackle it from different angles. Truth be told, anything towards testing LLVM would be OK with me. To name a few areas: mutation testing inputs (C/C++/LLVM-IR/...), reordering or skipping passes, creating an IR database for testing but maybe also other purposes. I'd also be interested in fuzzing the OpenMP frontend and runtime (both on the host and GPU) if that is something you might want to do. I think here is a plethora of crashes and hangs to be found, not to mention the correctness issues if we manage to do test generation for which we can verify the result. Given that you have experience building and extending LLVM already, I think a good next would be to narrow it down so you can start looking at the infrastructure we want to test, e.g., the pass manager builder if we want to swap passes to find hidden dependences between them. The existing fuzzer capabilities and the C++ mutation testing developed outside of LLVM (https://github.com/mull-project/mull) are also good places to take a look. Let me know what of the above areas, or areas of your choosing, might be most interesting to you. We can also schedule a 30min chat, just send me an email with times that would work for you. ~ Johannes On 3/1/21 1:29 PM, Stefan Stipanovic wrote:> +Johannes Doerfert <johannesdoerfert at gmail.com> > > Hi Chibin, > > Johannes will give you more information, but you can always start by > familiarizing yourself with the Attributor. > > Stefan > > On Mon, Mar 1, 2021 at 6:11 PM 张驰斌 via llvm-dev <llvm-dev at lists.llvm.org> > wrote: > >> Hi LLVM developers, >> >> I am a junior student majoring in Computer Science at ShanghaiTech >> University. I’ve been browsing through >> https://llvm.org/OpenProjects.html#gsoc21 and found out that LLVM is >> participating in Google Summer of Code 2021. I wanted to be sure to inquire >> about it and get in touch soon. >> >> One project idea really caught my eyes, ‘Fuzzing LLVM-IR Passes,’ >> striking chord with my experience. Currently, I am working with Professor >> Hao Chen (Contact author of the fuzzer Angora) and his fellow graduate >> students researching fuzzing. My main contribution lies in the >> experimental evaluation part. I’ve written many scripts to automate the >> benchmarking of different fuzzers, e.g., building libraries with fuzzer >> instrumentation (afl-clang-fast…), running fuzzers, and triaging and >> analyzing fuzz-results with afl-cov. Our most recent paper is under review >> at USENIX Security 2021. I’ve also written a dozen LLVM passes following >> USCD’s advanced compiler course (Open-sourced code and notes at >> https://github.com/chibinz/CSE231) as practice and contributed to the >> basic block stubbing pass for coverage feedback in the prior research >> project. If applicable, I see this as a golden opportunity to exercise what >> I’ve learned about LLVM and fuzzing in real-world application and at LLVM >> scale. This also a chance for me, as an LLVM user, to contribute back to >> LLVM, following the FOSS spirit. >> >> I wonder if this project is already occupied or still available? Are >> there any ‘good first issue’ that I can start working on or code of >> interest worth reading? I am aware the project description directs me to >> reach out to mentor Johannes Doerfert on IRC, but somehow trying to connect >> to the LLVM IRC channel using different clients always complains that the >> server refused the connection…(Sorry, but I’m not familiar with IRC, this >> is the first time trying…) Are there any other way to get in touch with the >> mentor? I feel really excited and hope to hear from you soon. >> >> >> >> Sincerely, >> >> Chibin Zhang >> >> 2021.3.1 >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>
John Regehr via llvm-dev
2021-Mar-08 20:26 UTC
[llvm-dev] Applying for GSoC 2021(Fuzzing LLVM-IR Passes)
Hi folks, an angle related to IR fuzzing that I would be happy to help out with is using Alive2 as a test oracle. Using Alive2 incurs a set of problems (not all IR features supported, can be very slow) but has corresponding advantages (considers all inputs at once, handles UB gracefully). John On 3/8/21 1:20 PM, Johannes Doerfert via llvm-dev wrote:> Hi Chibin, > > apologies for the late reply. > > There have been multiple people that expressed interest in this project, > though it is open ended and we might want to tackle it from different > angles. > > Truth be told, anything towards testing LLVM would be OK with me. To name a > few areas: mutation testing inputs (C/C++/LLVM-IR/...), reordering or > skipping > passes, creating an IR database for testing but maybe also other purposes. > > I'd also be interested in fuzzing the OpenMP frontend and runtime (both > on the > host and GPU) if that is something you might want to do. I think here is a > plethora of crashes and hangs to be found, not to mention the > correctness issues > if we manage to do test generation for which we can verify the result. > > Given that you have experience building and extending LLVM already, I > think a > good next would be to narrow it down so you can start looking at the > infrastructure > we want to test, e.g., the pass manager builder if we want to swap > passes to find > hidden dependences between them. The existing fuzzer capabilities and > the C++ > mutation testing developed outside of LLVM > (https://github.com/mull-project/mull) > are also good places to take a look. > > Let me know what of the above areas, or areas of your choosing, might be > most > interesting to you. We can also schedule a 30min chat, just send me an > email with > times that would work for you. > > ~ Johannes > > > On 3/1/21 1:29 PM, Stefan Stipanovic wrote: >> +Johannes Doerfert <johannesdoerfert at gmail.com> >> >> Hi Chibin, >> >> Johannes will give you more information, but you can always start by >> familiarizing yourself with the Attributor. >> >> Stefan >> >> On Mon, Mar 1, 2021 at 6:11 PM 张驰斌 via llvm-dev >> <llvm-dev at lists.llvm.org> >> wrote: >> >>> Hi LLVM developers, >>> >>> I am a junior student majoring in Computer Science at >>> ShanghaiTech >>> University. I’ve been browsing through >>> https://llvm.org/OpenProjects.html#gsoc21 and found out that LLVM is >>> participating in Google Summer of Code 2021. I wanted to be sure to >>> inquire >>> about it and get in touch soon. >>> >>> One project idea really caught my eyes, ‘Fuzzing LLVM-IR Passes,’ >>> striking chord with my experience. Currently, I am working with >>> Professor >>> Hao Chen (Contact author of the fuzzer Angora) and his fellow graduate >>> students researching fuzzing. My main contribution lies in the >>> experimental evaluation part. I’ve written many scripts to automate the >>> benchmarking of different fuzzers, e.g., building libraries with fuzzer >>> instrumentation (afl-clang-fast…), running fuzzers, and triaging and >>> analyzing fuzz-results with afl-cov. Our most recent paper is under >>> review >>> at USENIX Security 2021. I’ve also written a dozen LLVM passes following >>> USCD’s advanced compiler course (Open-sourced code and notes at >>> https://github.com/chibinz/CSE231) as practice and contributed to the >>> basic block stubbing pass for coverage feedback in the prior research >>> project. If applicable, I see this as a golden opportunity to >>> exercise what >>> I’ve learned about LLVM and fuzzing in real-world application and at >>> LLVM >>> scale. This also a chance for me, as an LLVM user, to contribute back to >>> LLVM, following the FOSS spirit. >>> >>> I wonder if this project is already occupied or still >>> available? Are >>> there any ‘good first issue’ that I can start working on or code of >>> interest worth reading? I am aware the project description directs me to >>> reach out to mentor Johannes Doerfert on IRC, but somehow trying to >>> connect >>> to the LLVM IRC channel using different clients always complains that >>> the >>> server refused the connection…(Sorry, but I’m not familiar with IRC, >>> this >>> is the first time trying…) Are there any other way to get in touch >>> with the >>> mentor? I feel really excited and hope to hear from you soon. >>> >>> >>> >>> Sincerely, >>> >>> Chibin Zhang >>> >>> 2021.3.1 >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
Arthur Eubanks via llvm-dev
2021-Mar-08 20:27 UTC
[llvm-dev] Applying for GSoC 2021(Fuzzing LLVM-IR Passes)
https://llvm.org/docs/FuzzingLLVM.html looks like good background (if you haven't already seen it). On Mon, Mar 8, 2021 at 12:21 PM Johannes Doerfert via llvm-dev < llvm-dev at lists.llvm.org> wrote:> Hi Chibin, > > apologies for the late reply. > > There have been multiple people that expressed interest in this project, > though it is open ended and we might want to tackle it from different > angles. > > Truth be told, anything towards testing LLVM would be OK with me. To name a > few areas: mutation testing inputs (C/C++/LLVM-IR/...), reordering or > skipping > passes, creating an IR database for testing but maybe also other purposes. > > I'd also be interested in fuzzing the OpenMP frontend and runtime (both > on the > host and GPU) if that is something you might want to do. I think here is a > plethora of crashes and hangs to be found, not to mention the > correctness issues > if we manage to do test generation for which we can verify the result. > > Given that you have experience building and extending LLVM already, I > think a > good next would be to narrow it down so you can start looking at the > infrastructure > we want to test, e.g., the pass manager builder if we want to swap > passes to find > hidden dependences between them. The existing fuzzer capabilities and > the C++ > mutation testing developed outside of LLVM > (https://github.com/mull-project/mull) > are also good places to take a look. > > Let me know what of the above areas, or areas of your choosing, might be > most > interesting to you. We can also schedule a 30min chat, just send me an > email with > times that would work for you. > > ~ Johannes > > > On 3/1/21 1:29 PM, Stefan Stipanovic wrote: > > +Johannes Doerfert <johannesdoerfert at gmail.com> > > > > Hi Chibin, > > > > Johannes will give you more information, but you can always start by > > familiarizing yourself with the Attributor. > > > > Stefan > > > > On Mon, Mar 1, 2021 at 6:11 PM 张驰斌 via llvm-dev <llvm-dev at lists.llvm.org > > > > wrote: > > > >> Hi LLVM developers, > >> > >> I am a junior student majoring in Computer Science at > ShanghaiTech > >> University. I’ve been browsing through > >> https://llvm.org/OpenProjects.html#gsoc21 and found out that LLVM is > >> participating in Google Summer of Code 2021. I wanted to be sure to > inquire > >> about it and get in touch soon. > >> > >> One project idea really caught my eyes, ‘Fuzzing LLVM-IR Passes,’ > >> striking chord with my experience. Currently, I am working with > Professor > >> Hao Chen (Contact author of the fuzzer Angora) and his fellow graduate > >> students researching fuzzing. My main contribution lies in the > >> experimental evaluation part. I’ve written many scripts to automate the > >> benchmarking of different fuzzers, e.g., building libraries with fuzzer > >> instrumentation (afl-clang-fast…), running fuzzers, and triaging and > >> analyzing fuzz-results with afl-cov. Our most recent paper is under > review > >> at USENIX Security 2021. I’ve also written a dozen LLVM passes following > >> USCD’s advanced compiler course (Open-sourced code and notes at > >> https://github.com/chibinz/CSE231) as practice and contributed to the > >> basic block stubbing pass for coverage feedback in the prior research > >> project. If applicable, I see this as a golden opportunity to exercise > what > >> I’ve learned about LLVM and fuzzing in real-world application and at > LLVM > >> scale. This also a chance for me, as an LLVM user, to contribute back to > >> LLVM, following the FOSS spirit. > >> > >> I wonder if this project is already occupied or still available? > Are > >> there any ‘good first issue’ that I can start working on or code of > >> interest worth reading? I am aware the project description directs me to > >> reach out to mentor Johannes Doerfert on IRC, but somehow trying to > connect > >> to the LLVM IRC channel using different clients always complains that > the > >> server refused the connection…(Sorry, but I’m not familiar with IRC, > this > >> is the first time trying…) Are there any other way to get in touch with > the > >> mentor? I feel really excited and hope to hear from you soon. > >> > >> > >> > >> Sincerely, > >> > >> Chibin Zhang > >> > >> 2021.3.1 > >> _______________________________________________ > >> LLVM Developers mailing list > >> llvm-dev at lists.llvm.org > >> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >> > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20210308/10549e8f/attachment.html>