Marco Leogrande
2015-Jun-18 00:38 UTC
[LLVMdev] Contributing a buildbot for the BPF backend
Thank you. Alexei just committed the change to SVN. Regards, Marco Leogrande Sent by a carbon-based life form; hence, it may contain repetitions, inaccuracies, logical fallacies and repetitions. On Wed, Jun 17, 2015 at 2:52 PM, Galina Kistanova <gkistanova at gmail.com> wrote:> LGTM. > Please commit. > > Thanks > > Galina > > On Tue, Jun 16, 2015 at 8:32 PM, Marco Leogrande <marcol at plumgrid.com> > wrote: >> >> Galina, >> thanks again for your help. >> >> I have cooked a new patch that should make a bit more sense. >> I have added the new builder under >> _get_experimental_scheduled_builders(), as the comment above that >> function suggested me it would be a good idea. :) >> Does it look a reasonable start? >> >> >> Regards, >> >> Marco Leogrande >> >> Sent by a carbon-based life form; hence, it may contain repetitions, >> inaccuracies, logical fallacies and repetitions. >> >> >> On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com> >> wrote: >> > Hi Galina, >> > >> > thanks for the info! >> > >> > I think we will go for the "LLVM+clang with cmake" approach, since the >> > BPF target has been marked non-experimental in the meantime, and we >> > are interested in building clang as well. >> > I will provide the info requested in step #10 separately and experiment >> > a bit. >> > >> > Thanks. >> > >> > >> > Regards, >> > >> > Marco Leogrande >> > >> > Sent by a carbon-based life form; hence, it may contain repetitions, >> > inaccuracies, logical fallacies and repetitions. >> > >> > >> > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova <gkistanova at gmail.com> >> > wrote: >> >> Hello Marco, >> >> >> >> Welcome aboard! >> >> >> >> slaves.py keeps the build slaves definitions, builders.py keeps the >> >> builders >> >> definitions. You have to have both. >> >> The steps of adding a new slave is here: >> >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you done >> >> the >> >> step # 10 before bringing your slave up, otherwise it wouldn't be >> >> authorized >> >> by the master and will be blacklisted after multiple failures. >> >> >> >> To make sure the slave is ready, you can start from manually building >> >> and >> >> testing everything you want the build to do on that slave in context of >> >> the >> >> account the build slave will be running in. >> >> If you succeed, then the slave is likely ready. >> >> >> >> You patch doesn't look right. >> >> getLLVMBuildFactory does not support cmake, so it wouldn't do what you >> >> think >> >> it will. For the same reason your slave properties (jobs and >> >> loadaverage) >> >> wouldn't work. >> >> See zorg/buildbot/builders/LLVMBuilder.py for more details. >> >> >> >> You either build LLVM with configure (this is what the existing LLVM >> >> factory >> >> does), add a new CMakeBuildFactory to support cmake, or build >> >> LLVM+clang >> >> with cmake by using a clang build factory. >> >> >> >> If you use any of the existing build factories, you do not really need >> >> a >> >> local master. You can add your builder and your slave to the >> >> lab.llvm.org >> >> master and move from there. >> >> As the first step you will be adding your builder as experimental >> >> anyway, so >> >> you will have a chance to work out possible issues and instabilities >> >> before >> >> promoting it to a stable one. >> >> >> >> You do need a local master to be able to debug and test if you develop >> >> zorg, >> >> i.e. write a new CMakeBuildFactory and such, though. >> >> >> >> Hope this helps. >> >> >> >> Thanks >> >> >> >> Galina >> >> >> >> >> >> >> >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande <marcol at plumgrid.com> >> >> wrote: >> >>> >> >>> Hi all, >> >>> I was trying to make some progress on this front and I spent some more >> >>> time on it. >> >>> >> >>> * I think I managed to configure the buildslave correctly; any chance >> >>> there is a way to test that before the buildmaster is configured? >> >>> >> >>> * I was trying to figure out where/how to add the configuration in >> >>> zorg. slaves.py looks reasonably easy to configure, but builders.py >> >>> looks more convoluted. I think the attached patch is a start for the >> >>> build part, but of course I have no idea if it makes any sense :) >> >>> The idea would be to replicate the following workflow (build >> >>> experimental backend + tests): >> >>> >> >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/ >> >>> make -jX >> >>> ./bin/llvm-lit test/CodeGen/BPF/ >> >>> >> >>> >> >>> Any suggestion would be appreciated. >> >>> >> >>> Thanks! >> >>> >> >>> Regards, >> >>> >> >>> Marco Leogrande >> >>> >> >>> Sent by a carbon-based life form; hence, it may contain repetitions, >> >>> inaccuracies, logical fallacies and repetitions. >> >>> >> >>> >> >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande <marcol at plumgrid.com> >> >>> wrote: >> >>> > Hi everyone, >> >>> > I am working with Alexei Starovoitov to contribute an LLVM buildbot >> >>> > for the experimental BPF backend. >> >>> > I am following the steps at [1] to setup a buildbot and I was mostly >> >>> > successful: I was able to setup a slave and a temporary master to >> >>> > check its base config. >> >>> > >> >>> > Now I'm not sure about the next step: patching the "slaves.py" and >> >>> > "builders.py" files in zorg. >> >>> > >> >>> > * Is there a guide I can follow to understand the format of these >> >>> > files? For example, being BPF an experimental backend, I need to >> >>> > provide extra parameters to cmake at build time. >> >>> > >> >>> > * Also, is there a way to test these changes in zorg in a local >> >>> > environment before submitting the patch for review? >> >>> > >> >>> > Thank you! >> >>> > >> >>> > >> >>> > [1] http://llvm.org/docs/HowToAddABuilder.html >> >>> > >> >>> > Regards, >> >>> > >> >>> > Marco Leogrande >> >>> > >> >>> > Sent by a carbon-based life form; hence, it may contain repetitions, >> >>> > inaccuracies, logical fallacies and repetitions. >> >> >> >> > >
Galina Kistanova
2015-Jun-18 17:31 UTC
[LLVMdev] Contributing a buildbot for the BPF backend
Hi Marco, Your slave should be able to attach to llvm master now. Please let me know if there is any problems with this. Thanks Galina On Wed, Jun 17, 2015 at 5:38 PM, Marco Leogrande <marcol at plumgrid.com> wrote:> Thank you. > Alexei just committed the change to SVN. > > Regards, > > Marco Leogrande > > Sent by a carbon-based life form; hence, it may contain repetitions, > inaccuracies, logical fallacies and repetitions. > > > On Wed, Jun 17, 2015 at 2:52 PM, Galina Kistanova <gkistanova at gmail.com> > wrote: > > LGTM. > > Please commit. > > > > Thanks > > > > Galina > > > > On Tue, Jun 16, 2015 at 8:32 PM, Marco Leogrande <marcol at plumgrid.com> > > wrote: > >> > >> Galina, > >> thanks again for your help. > >> > >> I have cooked a new patch that should make a bit more sense. > >> I have added the new builder under > >> _get_experimental_scheduled_builders(), as the comment above that > >> function suggested me it would be a good idea. :) > >> Does it look a reasonable start? > >> > >> > >> Regards, > >> > >> Marco Leogrande > >> > >> Sent by a carbon-based life form; hence, it may contain repetitions, > >> inaccuracies, logical fallacies and repetitions. > >> > >> > >> On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com> > >> wrote: > >> > Hi Galina, > >> > > >> > thanks for the info! > >> > > >> > I think we will go for the "LLVM+clang with cmake" approach, since the > >> > BPF target has been marked non-experimental in the meantime, and we > >> > are interested in building clang as well. > >> > I will provide the info requested in step #10 separately and > experiment > >> > a bit. > >> > > >> > Thanks. > >> > > >> > > >> > Regards, > >> > > >> > Marco Leogrande > >> > > >> > Sent by a carbon-based life form; hence, it may contain repetitions, > >> > inaccuracies, logical fallacies and repetitions. > >> > > >> > > >> > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova < > gkistanova at gmail.com> > >> > wrote: > >> >> Hello Marco, > >> >> > >> >> Welcome aboard! > >> >> > >> >> slaves.py keeps the build slaves definitions, builders.py keeps the > >> >> builders > >> >> definitions. You have to have both. > >> >> The steps of adding a new slave is here: > >> >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you > done > >> >> the > >> >> step # 10 before bringing your slave up, otherwise it wouldn't be > >> >> authorized > >> >> by the master and will be blacklisted after multiple failures. > >> >> > >> >> To make sure the slave is ready, you can start from manually building > >> >> and > >> >> testing everything you want the build to do on that slave in context > of > >> >> the > >> >> account the build slave will be running in. > >> >> If you succeed, then the slave is likely ready. > >> >> > >> >> You patch doesn't look right. > >> >> getLLVMBuildFactory does not support cmake, so it wouldn't do what > you > >> >> think > >> >> it will. For the same reason your slave properties (jobs and > >> >> loadaverage) > >> >> wouldn't work. > >> >> See zorg/buildbot/builders/LLVMBuilder.py for more details. > >> >> > >> >> You either build LLVM with configure (this is what the existing LLVM > >> >> factory > >> >> does), add a new CMakeBuildFactory to support cmake, or build > >> >> LLVM+clang > >> >> with cmake by using a clang build factory. > >> >> > >> >> If you use any of the existing build factories, you do not really > need > >> >> a > >> >> local master. You can add your builder and your slave to the > >> >> lab.llvm.org > >> >> master and move from there. > >> >> As the first step you will be adding your builder as experimental > >> >> anyway, so > >> >> you will have a chance to work out possible issues and instabilities > >> >> before > >> >> promoting it to a stable one. > >> >> > >> >> You do need a local master to be able to debug and test if you > develop > >> >> zorg, > >> >> i.e. write a new CMakeBuildFactory and such, though. > >> >> > >> >> Hope this helps. > >> >> > >> >> Thanks > >> >> > >> >> Galina > >> >> > >> >> > >> >> > >> >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande <marcol at plumgrid.com > > > >> >> wrote: > >> >>> > >> >>> Hi all, > >> >>> I was trying to make some progress on this front and I spent some > more > >> >>> time on it. > >> >>> > >> >>> * I think I managed to configure the buildslave correctly; any > chance > >> >>> there is a way to test that before the buildmaster is configured? > >> >>> > >> >>> * I was trying to figure out where/how to add the configuration in > >> >>> zorg. slaves.py looks reasonably easy to configure, but builders.py > >> >>> looks more convoluted. I think the attached patch is a start for the > >> >>> build part, but of course I have no idea if it makes any sense :) > >> >>> The idea would be to replicate the following workflow (build > >> >>> experimental backend + tests): > >> >>> > >> >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/ > >> >>> make -jX > >> >>> ./bin/llvm-lit test/CodeGen/BPF/ > >> >>> > >> >>> > >> >>> Any suggestion would be appreciated. > >> >>> > >> >>> Thanks! > >> >>> > >> >>> Regards, > >> >>> > >> >>> Marco Leogrande > >> >>> > >> >>> Sent by a carbon-based life form; hence, it may contain repetitions, > >> >>> inaccuracies, logical fallacies and repetitions. > >> >>> > >> >>> > >> >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande < > marcol at plumgrid.com> > >> >>> wrote: > >> >>> > Hi everyone, > >> >>> > I am working with Alexei Starovoitov to contribute an LLVM > buildbot > >> >>> > for the experimental BPF backend. > >> >>> > I am following the steps at [1] to setup a buildbot and I was > mostly > >> >>> > successful: I was able to setup a slave and a temporary master to > >> >>> > check its base config. > >> >>> > > >> >>> > Now I'm not sure about the next step: patching the "slaves.py" and > >> >>> > "builders.py" files in zorg. > >> >>> > > >> >>> > * Is there a guide I can follow to understand the format of these > >> >>> > files? For example, being BPF an experimental backend, I need to > >> >>> > provide extra parameters to cmake at build time. > >> >>> > > >> >>> > * Also, is there a way to test these changes in zorg in a local > >> >>> > environment before submitting the patch for review? > >> >>> > > >> >>> > Thank you! > >> >>> > > >> >>> > > >> >>> > [1] http://llvm.org/docs/HowToAddABuilder.html > >> >>> > > >> >>> > Regards, > >> >>> > > >> >>> > Marco Leogrande > >> >>> > > >> >>> > Sent by a carbon-based life form; hence, it may contain > repetitions, > >> >>> > inaccuracies, logical fallacies and repetitions. > >> >> > >> >> > > > > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20150618/33667d72/attachment.html>
Marco Leogrande
2015-Jun-18 18:14 UTC
[LLVMdev] Contributing a buildbot for the BPF backend
Hi Galina, I can see the buildslave in the waterfall display, and one build was already issued and completed successfully. I will give it a deeper look at the output and monitor in the next few days. Thanks! Regards, Marco Leogrande Sent by a carbon-based life form; hence, it may contain repetitions, inaccuracies, logical fallacies and repetitions. On Thu, Jun 18, 2015 at 10:31 AM, Galina Kistanova <gkistanova at gmail.com> wrote:> Hi Marco, > > Your slave should be able to attach to llvm master now. > Please let me know if there is any problems with this. > > Thanks > > Galina > > On Wed, Jun 17, 2015 at 5:38 PM, Marco Leogrande <marcol at plumgrid.com> > wrote: >> >> Thank you. >> Alexei just committed the change to SVN. >> >> Regards, >> >> Marco Leogrande >> >> Sent by a carbon-based life form; hence, it may contain repetitions, >> inaccuracies, logical fallacies and repetitions. >> >> >> On Wed, Jun 17, 2015 at 2:52 PM, Galina Kistanova <gkistanova at gmail.com> >> wrote: >> > LGTM. >> > Please commit. >> > >> > Thanks >> > >> > Galina >> > >> > On Tue, Jun 16, 2015 at 8:32 PM, Marco Leogrande <marcol at plumgrid.com> >> > wrote: >> >> >> >> Galina, >> >> thanks again for your help. >> >> >> >> I have cooked a new patch that should make a bit more sense. >> >> I have added the new builder under >> >> _get_experimental_scheduled_builders(), as the comment above that >> >> function suggested me it would be a good idea. :) >> >> Does it look a reasonable start? >> >> >> >> >> >> Regards, >> >> >> >> Marco Leogrande >> >> >> >> Sent by a carbon-based life form; hence, it may contain repetitions, >> >> inaccuracies, logical fallacies and repetitions. >> >> >> >> >> >> On Wed, Jun 10, 2015 at 7:06 PM, Marco Leogrande <marcol at plumgrid.com> >> >> wrote: >> >> > Hi Galina, >> >> > >> >> > thanks for the info! >> >> > >> >> > I think we will go for the "LLVM+clang with cmake" approach, since >> >> > the >> >> > BPF target has been marked non-experimental in the meantime, and we >> >> > are interested in building clang as well. >> >> > I will provide the info requested in step #10 separately and >> >> > experiment >> >> > a bit. >> >> > >> >> > Thanks. >> >> > >> >> > >> >> > Regards, >> >> > >> >> > Marco Leogrande >> >> > >> >> > Sent by a carbon-based life form; hence, it may contain repetitions, >> >> > inaccuracies, logical fallacies and repetitions. >> >> > >> >> > >> >> > On Wed, Jun 10, 2015 at 4:05 PM, Galina Kistanova >> >> > <gkistanova at gmail.com> >> >> > wrote: >> >> >> Hello Marco, >> >> >> >> >> >> Welcome aboard! >> >> >> >> >> >> slaves.py keeps the build slaves definitions, builders.py keeps the >> >> >> builders >> >> >> definitions. You have to have both. >> >> >> The steps of adding a new slave is here: >> >> >> http://llvm.org/docs/HowToAddABuilder.html. Please make sure you >> >> >> done >> >> >> the >> >> >> step # 10 before bringing your slave up, otherwise it wouldn't be >> >> >> authorized >> >> >> by the master and will be blacklisted after multiple failures. >> >> >> >> >> >> To make sure the slave is ready, you can start from manually >> >> >> building >> >> >> and >> >> >> testing everything you want the build to do on that slave in context >> >> >> of >> >> >> the >> >> >> account the build slave will be running in. >> >> >> If you succeed, then the slave is likely ready. >> >> >> >> >> >> You patch doesn't look right. >> >> >> getLLVMBuildFactory does not support cmake, so it wouldn't do what >> >> >> you >> >> >> think >> >> >> it will. For the same reason your slave properties (jobs and >> >> >> loadaverage) >> >> >> wouldn't work. >> >> >> See zorg/buildbot/builders/LLVMBuilder.py for more details. >> >> >> >> >> >> You either build LLVM with configure (this is what the existing LLVM >> >> >> factory >> >> >> does), add a new CMakeBuildFactory to support cmake, or build >> >> >> LLVM+clang >> >> >> with cmake by using a clang build factory. >> >> >> >> >> >> If you use any of the existing build factories, you do not really >> >> >> need >> >> >> a >> >> >> local master. You can add your builder and your slave to the >> >> >> lab.llvm.org >> >> >> master and move from there. >> >> >> As the first step you will be adding your builder as experimental >> >> >> anyway, so >> >> >> you will have a chance to work out possible issues and instabilities >> >> >> before >> >> >> promoting it to a stable one. >> >> >> >> >> >> You do need a local master to be able to debug and test if you >> >> >> develop >> >> >> zorg, >> >> >> i.e. write a new CMakeBuildFactory and such, though. >> >> >> >> >> >> Hope this helps. >> >> >> >> >> >> Thanks >> >> >> >> >> >> Galina >> >> >> >> >> >> >> >> >> >> >> >> On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande >> >> >> <marcol at plumgrid.com> >> >> >> wrote: >> >> >>> >> >> >>> Hi all, >> >> >>> I was trying to make some progress on this front and I spent some >> >> >>> more >> >> >>> time on it. >> >> >>> >> >> >>> * I think I managed to configure the buildslave correctly; any >> >> >>> chance >> >> >>> there is a way to test that before the buildmaster is configured? >> >> >>> >> >> >>> * I was trying to figure out where/how to add the configuration in >> >> >>> zorg. slaves.py looks reasonably easy to configure, but builders.py >> >> >>> looks more convoluted. I think the attached patch is a start for >> >> >>> the >> >> >>> build part, but of course I have no idea if it makes any sense :) >> >> >>> The idea would be to replicate the following workflow (build >> >> >>> experimental backend + tests): >> >> >>> >> >> >>> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/ >> >> >>> make -jX >> >> >>> ./bin/llvm-lit test/CodeGen/BPF/ >> >> >>> >> >> >>> >> >> >>> Any suggestion would be appreciated. >> >> >>> >> >> >>> Thanks! >> >> >>> >> >> >>> Regards, >> >> >>> >> >> >>> Marco Leogrande >> >> >>> >> >> >>> Sent by a carbon-based life form; hence, it may contain >> >> >>> repetitions, >> >> >>> inaccuracies, logical fallacies and repetitions. >> >> >>> >> >> >>> >> >> >>> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande >> >> >>> <marcol at plumgrid.com> >> >> >>> wrote: >> >> >>> > Hi everyone, >> >> >>> > I am working with Alexei Starovoitov to contribute an LLVM >> >> >>> > buildbot >> >> >>> > for the experimental BPF backend. >> >> >>> > I am following the steps at [1] to setup a buildbot and I was >> >> >>> > mostly >> >> >>> > successful: I was able to setup a slave and a temporary master to >> >> >>> > check its base config. >> >> >>> > >> >> >>> > Now I'm not sure about the next step: patching the "slaves.py" >> >> >>> > and >> >> >>> > "builders.py" files in zorg. >> >> >>> > >> >> >>> > * Is there a guide I can follow to understand the format of these >> >> >>> > files? For example, being BPF an experimental backend, I need to >> >> >>> > provide extra parameters to cmake at build time. >> >> >>> > >> >> >>> > * Also, is there a way to test these changes in zorg in a local >> >> >>> > environment before submitting the patch for review? >> >> >>> > >> >> >>> > Thank you! >> >> >>> > >> >> >>> > >> >> >>> > [1] http://llvm.org/docs/HowToAddABuilder.html >> >> >>> > >> >> >>> > Regards, >> >> >>> > >> >> >>> > Marco Leogrande >> >> >>> > >> >> >>> > Sent by a carbon-based life form; hence, it may contain >> >> >>> > repetitions, >> >> >>> > inaccuracies, logical fallacies and repetitions. >> >> >> >> >> >> >> > >> > > >