Marco Leogrande
2015-Jun-17 03:32 UTC
[LLVMdev] Contributing a buildbot for the BPF backend
Galina, thanks again for your help. I have cooked a new patch that should make a bit more sense. I have added the new builder under _get_experimental_scheduled_builders(), as the comment above that function suggested me it would be a good idea. :) Does it look a reasonable start? Regards, Marco Leogrande Sent by a carbon-based life form; hence, it may contain repetitions, inaccuracies, logical fallacies and repetitions. On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com> wrote:> Hi Galina, > > thanks for the info! > > I think we will go for the "LLVM+clang with cmake" approach, since the > BPF target has been marked non-experimental in the meantime, and we > are interested in building clang as well. > I will provide the info requested in step #10 separately and experiment a bit. > > Thanks. > > > Regards, > > Marco Leogrande > > Sent by a carbon-based life form; hence, it may contain repetitions, > inaccuracies, logical fallacies and repetitions. > > > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova <gkistanova at gmail.com> wrote: >> Hello Marco, >> >> Welcome aboard! >> >> slaves.py keeps the build slaves definitions, builders.py keeps the builders >> definitions. You have to have both. >> The steps of adding a new slave is here: >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you done the >> step # 10 before bringing your slave up, otherwise it wouldn't be authorized >> by the master and will be blacklisted after multiple failures. >> >> To make sure the slave is ready, you can start from manually building and >> testing everything you want the build to do on that slave in context of the >> account the build slave will be running in. >> If you succeed, then the slave is likely ready. >> >> You patch doesn't look right. >> getLLVMBuildFactory does not support cmake, so it wouldn't do what you think >> it will. For the same reason your slave properties (jobs and loadaverage) >> wouldn't work. >> See zorg/buildbot/builders/LLVMBuilder.py for more details. >> >> You either build LLVM with configure (this is what the existing LLVM factory >> does), add a new CMakeBuildFactory to support cmake, or build LLVM+clang >> with cmake by using a clang build factory. >> >> If you use any of the existing build factories, you do not really need a >> local master. You can add your builder and your slave to the lab.llvm.org >> master and move from there. >> As the first step you will be adding your builder as experimental anyway, so >> you will have a chance to work out possible issues and instabilities before >> promoting it to a stable one. >> >> You do need a local master to be able to debug and test if you develop zorg, >> i.e. write a new CMakeBuildFactory and such, though. >> >> Hope this helps. >> >> Thanks >> >> Galina >> >> >> >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande <marcol at plumgrid.com> wrote: >>> >>> Hi all, >>> I was trying to make some progress on this front and I spent some more >>> time on it. >>> >>> * I think I managed to configure the buildslave correctly; any chance >>> there is a way to test that before the buildmaster is configured? >>> >>> * I was trying to figure out where/how to add the configuration in >>> zorg. slaves.py looks reasonably easy to configure, but builders.py >>> looks more convoluted. I think the attached patch is a start for the >>> build part, but of course I have no idea if it makes any sense :) >>> The idea would be to replicate the following workflow (build >>> experimental backend + tests): >>> >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/ >>> make -jX >>> ./bin/llvm-lit test/CodeGen/BPF/ >>> >>> >>> Any suggestion would be appreciated. >>> >>> Thanks! >>> >>> Regards, >>> >>> Marco Leogrande >>> >>> Sent by a carbon-based life form; hence, it may contain repetitions, >>> inaccuracies, logical fallacies and repetitions. >>> >>> >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande <marcol at plumgrid.com> >>> wrote: >>> > Hi everyone, >>> > I am working with Alexei Starovoitov to contribute an LLVM buildbot >>> > for the experimental BPF backend. >>> > I am following the steps at [1] to setup a buildbot and I was mostly >>> > successful: I was able to setup a slave and a temporary master to >>> > check its base config. >>> > >>> > Now I'm not sure about the next step: patching the "slaves.py" and >>> > "builders.py" files in zorg. >>> > >>> > * Is there a guide I can follow to understand the format of these >>> > files? For example, being BPF an experimental backend, I need to >>> > provide extra parameters to cmake at build time. >>> > >>> > * Also, is there a way to test these changes in zorg in a local >>> > environment before submitting the patch for review? >>> > >>> > Thank you! >>> > >>> > >>> > [1] http://llvm.org/docs/HowToAddABuilder.html >>> > >>> > Regards, >>> > >>> > Marco Leogrande >>> > >>> > Sent by a carbon-based life form; hence, it may contain repetitions, >>> > inaccuracies, logical fallacies and repetitions. >> >>-------------- next part -------------- A non-text attachment was scrubbed... Name: 0001-Add-builder-and-slave-for-experimental-BPF-builder.patch Type: text/x-patch Size: 2017 bytes Desc: not available URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150616/3d3daae9/attachment.bin>
Galina Kistanova
2015-Jun-17 21:52 UTC
[LLVMdev] Contributing a buildbot for the BPF backend
LGTM. Please commit. Thanks Galina On Tue, Jun 16, 2015 at 8:32 PM, Marco Leogrande <marcol at plumgrid.com> wrote:> Galina, > thanks again for your help. > > I have cooked a new patch that should make a bit more sense. > I have added the new builder under > _get_experimental_scheduled_builders(), as the comment above that > function suggested me it would be a good idea. :) > Does it look a reasonable start? > > > Regards, > > Marco Leogrande > > Sent by a carbon-based life form; hence, it may contain repetitions, > inaccuracies, logical fallacies and repetitions. > > > On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com> > wrote: > > Hi Galina, > > > > thanks for the info! > > > > I think we will go for the "LLVM+clang with cmake" approach, since the > > BPF target has been marked non-experimental in the meantime, and we > > are interested in building clang as well. > > I will provide the info requested in step #10 separately and experiment > a bit. > > > > Thanks. > > > > > > Regards, > > > > Marco Leogrande > > > > Sent by a carbon-based life form; hence, it may contain repetitions, > > inaccuracies, logical fallacies and repetitions. > > > > > > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova <gkistanova at gmail.com> > wrote: > >> Hello Marco, > >> > >> Welcome aboard! > >> > >> slaves.py keeps the build slaves definitions, builders.py keeps the > builders > >> definitions. You have to have both. > >> The steps of adding a new slave is here: > >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you done > the > >> step # 10 before bringing your slave up, otherwise it wouldn't be > authorized > >> by the master and will be blacklisted after multiple failures. > >> > >> To make sure the slave is ready, you can start from manually building > and > >> testing everything you want the build to do on that slave in context of > the > >> account the build slave will be running in. > >> If you succeed, then the slave is likely ready. > >> > >> You patch doesn't look right. > >> getLLVMBuildFactory does not support cmake, so it wouldn't do what you > think > >> it will. For the same reason your slave properties (jobs and > loadaverage) > >> wouldn't work. > >> See zorg/buildbot/builders/LLVMBuilder.py for more details. > >> > >> You either build LLVM with configure (this is what the existing LLVM > factory > >> does), add a new CMakeBuildFactory to support cmake, or build LLVM+clang > >> with cmake by using a clang build factory. > >> > >> If you use any of the existing build factories, you do not really need a > >> local master. You can add your builder and your slave to the > lab.llvm.org > >> master and move from there. > >> As the first step you will be adding your builder as experimental > anyway, so > >> you will have a chance to work out possible issues and instabilities > before > >> promoting it to a stable one. > >> > >> You do need a local master to be able to debug and test if you develop > zorg, > >> i.e. write a new CMakeBuildFactory and such, though. > >> > >> Hope this helps. > >> > >> Thanks > >> > >> Galina > >> > >> > >> > >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande <marcol at plumgrid.com> > wrote: > >>> > >>> Hi all, > >>> I was trying to make some progress on this front and I spent some more > >>> time on it. > >>> > >>> * I think I managed to configure the buildslave correctly; any chance > >>> there is a way to test that before the buildmaster is configured? > >>> > >>> * I was trying to figure out where/how to add the configuration in > >>> zorg. slaves.py looks reasonably easy to configure, but builders.py > >>> looks more convoluted. I think the attached patch is a start for the > >>> build part, but of course I have no idea if it makes any sense :) > >>> The idea would be to replicate the following workflow (build > >>> experimental backend + tests): > >>> > >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/ > >>> make -jX > >>> ./bin/llvm-lit test/CodeGen/BPF/ > >>> > >>> > >>> Any suggestion would be appreciated. > >>> > >>> Thanks! > >>> > >>> Regards, > >>> > >>> Marco Leogrande > >>> > >>> Sent by a carbon-based life form; hence, it may contain repetitions, > >>> inaccuracies, logical fallacies and repetitions. > >>> > >>> > >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande <marcol at plumgrid.com> > >>> wrote: > >>> > Hi everyone, > >>> > I am working with Alexei Starovoitov to contribute an LLVM buildbot > >>> > for the experimental BPF backend. > >>> > I am following the steps at [1] to setup a buildbot and I was mostly > >>> > successful: I was able to setup a slave and a temporary master to > >>> > check its base config. > >>> > > >>> > Now I'm not sure about the next step: patching the "slaves.py" and > >>> > "builders.py" files in zorg. > >>> > > >>> > * Is there a guide I can follow to understand the format of these > >>> > files? For example, being BPF an experimental backend, I need to > >>> > provide extra parameters to cmake at build time. > >>> > > >>> > * Also, is there a way to test these changes in zorg in a local > >>> > environment before submitting the patch for review? > >>> > > >>> > Thank you! > >>> > > >>> > > >>> > [1] http://llvm.org/docs/HowToAddABuilder.html > >>> > > >>> > Regards, > >>> > > >>> > Marco Leogrande > >>> > > >>> > Sent by a carbon-based life form; hence, it may contain repetitions, > >>> > inaccuracies, logical fallacies and repetitions. > >> > >> >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150617/951ddd5f/attachment.html>
Marco Leogrande
2015-Jun-18 00:38 UTC
[LLVMdev] Contributing a buildbot for the BPF backend
Thank you. Alexei just committed the change to SVN. Regards, Marco Leogrande Sent by a carbon-based life form; hence, it may contain repetitions, inaccuracies, logical fallacies and repetitions. On Wed, Jun 17, 2015 at 2:52 PM, Galina Kistanova <gkistanova at gmail.com> wrote:> LGTM. > Please commit. > > Thanks > > Galina > > On Tue, Jun 16, 2015 at 8:32 PM, Marco Leogrande <marcol at plumgrid.com> > wrote: >> >> Galina, >> thanks again for your help. >> >> I have cooked a new patch that should make a bit more sense. >> I have added the new builder under >> _get_experimental_scheduled_builders(), as the comment above that >> function suggested me it would be a good idea. :) >> Does it look a reasonable start? >> >> >> Regards, >> >> Marco Leogrande >> >> Sent by a carbon-based life form; hence, it may contain repetitions, >> inaccuracies, logical fallacies and repetitions. >> >> >> On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com> >> wrote: >> > Hi Galina, >> > >> > thanks for the info! >> > >> > I think we will go for the "LLVM+clang with cmake" approach, since the >> > BPF target has been marked non-experimental in the meantime, and we >> > are interested in building clang as well. >> > I will provide the info requested in step #10 separately and experiment >> > a bit. >> > >> > Thanks. >> > >> > >> > Regards, >> > >> > Marco Leogrande >> > >> > Sent by a carbon-based life form; hence, it may contain repetitions, >> > inaccuracies, logical fallacies and repetitions. >> > >> > >> > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova <gkistanova at gmail.com> >> > wrote: >> >> Hello Marco, >> >> >> >> Welcome aboard! >> >> >> >> slaves.py keeps the build slaves definitions, builders.py keeps the >> >> builders >> >> definitions. You have to have both. >> >> The steps of adding a new slave is here: >> >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you done >> >> the >> >> step # 10 before bringing your slave up, otherwise it wouldn't be >> >> authorized >> >> by the master and will be blacklisted after multiple failures. >> >> >> >> To make sure the slave is ready, you can start from manually building >> >> and >> >> testing everything you want the build to do on that slave in context of >> >> the >> >> account the build slave will be running in. >> >> If you succeed, then the slave is likely ready. >> >> >> >> You patch doesn't look right. >> >> getLLVMBuildFactory does not support cmake, so it wouldn't do what you >> >> think >> >> it will. For the same reason your slave properties (jobs and >> >> loadaverage) >> >> wouldn't work. >> >> See zorg/buildbot/builders/LLVMBuilder.py for more details. >> >> >> >> You either build LLVM with configure (this is what the existing LLVM >> >> factory >> >> does), add a new CMakeBuildFactory to support cmake, or build >> >> LLVM+clang >> >> with cmake by using a clang build factory. >> >> >> >> If you use any of the existing build factories, you do not really need >> >> a >> >> local master. You can add your builder and your slave to the >> >> lab.llvm.org >> >> master and move from there. >> >> As the first step you will be adding your builder as experimental >> >> anyway, so >> >> you will have a chance to work out possible issues and instabilities >> >> before >> >> promoting it to a stable one. >> >> >> >> You do need a local master to be able to debug and test if you develop >> >> zorg, >> >> i.e. write a new CMakeBuildFactory and such, though. >> >> >> >> Hope this helps. >> >> >> >> Thanks >> >> >> >> Galina >> >> >> >> >> >> >> >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande <marcol at plumgrid.com> >> >> wrote: >> >>> >> >>> Hi all, >> >>> I was trying to make some progress on this front and I spent some more >> >>> time on it. >> >>> >> >>> * I think I managed to configure the buildslave correctly; any chance >> >>> there is a way to test that before the buildmaster is configured? >> >>> >> >>> * I was trying to figure out where/how to add the configuration in >> >>> zorg. slaves.py looks reasonably easy to configure, but builders.py >> >>> looks more convoluted. I think the attached patch is a start for the >> >>> build part, but of course I have no idea if it makes any sense :) >> >>> The idea would be to replicate the following workflow (build >> >>> experimental backend + tests): >> >>> >> >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/ >> >>> make -jX >> >>> ./bin/llvm-lit test/CodeGen/BPF/ >> >>> >> >>> >> >>> Any suggestion would be appreciated. >> >>> >> >>> Thanks! >> >>> >> >>> Regards, >> >>> >> >>> Marco Leogrande >> >>> >> >>> Sent by a carbon-based life form; hence, it may contain repetitions, >> >>> inaccuracies, logical fallacies and repetitions. >> >>> >> >>> >> >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande <marcol at plumgrid.com> >> >>> wrote: >> >>> > Hi everyone, >> >>> > I am working with Alexei Starovoitov to contribute an LLVM buildbot >> >>> > for the experimental BPF backend. >> >>> > I am following the steps at [1] to setup a buildbot and I was mostly >> >>> > successful: I was able to setup a slave and a temporary master to >> >>> > check its base config. >> >>> > >> >>> > Now I'm not sure about the next step: patching the "slaves.py" and >> >>> > "builders.py" files in zorg. >> >>> > >> >>> > * Is there a guide I can follow to understand the format of these >> >>> > files? For example, being BPF an experimental backend, I need to >> >>> > provide extra parameters to cmake at build time. >> >>> > >> >>> > * Also, is there a way to test these changes in zorg in a local >> >>> > environment before submitting the patch for review? >> >>> > >> >>> > Thank you! >> >>> > >> >>> > >> >>> > [1] http://llvm.org/docs/HowToAddABuilder.html >> >>> > >> >>> > Regards, >> >>> > >> >>> > Marco Leogrande >> >>> > >> >>> > Sent by a carbon-based life form; hence, it may contain repetitions, >> >>> > inaccuracies, logical fallacies and repetitions. >> >> >> >> > >