llvm dev - May 2015 - [LLVMdev] Contributing a buildbot for the BPF backend

Hi everyone,
I am working with Alexei Starovoitov to contribute an LLVM buildbot
for the experimental BPF backend.
I am following the steps at [1] to setup a buildbot and I was mostly
successful: I was able to setup a slave and a temporary master to
check its base config.

Now I'm not sure about the next step: patching the "slaves.py" and
"builders.py" files in zorg.

* Is there a guide I can follow to understand the format of these
files? For example, being BPF an experimental backend, I need to
provide extra parameters to cmake at build time.

* Also, is there a way to test these changes in zorg in a local
environment before submitting the patch for review?

Thank you!


[1] http://llvm.org/docs/HowToAddABuilder.html

Regards,

Marco Leogrande

Sent by a carbon-based life form; hence, it may contain repetitions,
inaccuracies, logical fallacies and repetitions.

Hi all,
I was trying to make some progress on this front and I spent some more
time on it.

* I think I managed to configure the buildslave correctly; any chance
there is a way to test that before the buildmaster is configured?

* I was trying to figure out where/how to add the configuration in
zorg. slaves.py looks reasonably easy to configure, but builders.py
looks more convoluted. I think the attached patch is a start for the
build part, but of course I have no idea if it makes any sense :)
The idea would be to replicate the following workflow (build
experimental backend + tests):

cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/
make -jX
./bin/llvm-lit test/CodeGen/BPF/


Any suggestion would be appreciated.

Thanks!

Regards,

Marco Leogrande

Sent by a carbon-based life form; hence, it may contain repetitions,
inaccuracies, logical fallacies and repetitions.


On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande <marcol at plumgrid.com>
wrote:
> Hi everyone,
> I am working with Alexei Starovoitov to contribute an LLVM buildbot
> for the experimental BPF backend.
> I am following the steps at [1] to setup a buildbot and I was mostly
> successful: I was able to setup a slave and a temporary master to
> check its base config.
>
> Now I'm not sure about the next step: patching the
"slaves.py" and
> "builders.py" files in zorg.
>
> * Is there a guide I can follow to understand the format of these
> files? For example, being BPF an experimental backend, I need to
> provide extra parameters to cmake at build time.
>
> * Also, is there a way to test these changes in zorg in a local
> environment before submitting the patch for review?
>
> Thank you!
>
>
> [1] http://llvm.org/docs/HowToAddABuilder.html
>
> Regards,
>
> Marco Leogrande
>
> Sent by a carbon-based life form; hence, it may contain repetitions,
> inaccuracies, logical fallacies and repetitions.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: 01-zorg-bpf.patch
Type: text/x-patch
Size: 1770 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150608/1393f33e/attachment.bin>

Hello Marco,

Welcome aboard!

slaves.py keeps the build slaves definitions, builders.py keeps the
builders definitions. You have to have both.
The steps of adding a new slave is here:
http://llvm.org/docs/HowToAddABuilder.html. Please make sure you done the
step # 10 before bringing your slave up, otherwise it wouldn't be
authorized by the master and will be blacklisted after multiple failures.

To make sure the slave is ready, you can start from manually building and
testing everything you want the build to do on that slave in context of the
account the build slave will be running in.
If you succeed, then the slave is likely ready.

You patch doesn't look right.
getLLVMBuildFactory does not support cmake, so it wouldn't do what you
think it will. For the same reason your slave properties (jobs and
loadaverage) wouldn't work.
See zorg/buildbot/builders/LLVMBuilder.py for more details.

You either build LLVM with configure (this is what the existing LLVM
factory does), add a new CMakeBuildFactory to support cmake, or build
LLVM+clang with cmake by using a clang build factory.

If you use any of the existing build factories, you do not really need a
local master. You can add your builder and your slave to the lab.llvm.org
master and move from there.
As the first step you will be adding your builder as experimental anyway,
so you will have a chance to work out possible issues and instabilities
before promoting it to a stable one.

You do need a local master to be able to debug and test if you develop
zorg, i.e. write a new CMakeBuildFactory and such, though.

Hope this helps.

Thanks

Galina



On Mon, Jun 8, 2015 at 9:04 PM, Marco Leogrande <marcol at plumgrid.com>
wrote:
> Hi all,
> I was trying to make some progress on this front and I spent some more
> time on it.
>
> * I think I managed to configure the buildslave correctly; any chance
> there is a way to test that before the buildmaster is configured?
>
> * I was trying to figure out where/how to add the configuration in
> zorg. slaves.py looks reasonably easy to configure, but builders.py
> looks more convoluted. I think the attached patch is a start for the
> build part, but of course I have no idea if it makes any sense :)
> The idea would be to replicate the following workflow (build
> experimental backend + tests):
>
> cmake -DLLVM_EXPERIMENTAL_TARGETS_TO_BUILD=BPF ../path/to/src/
> make -jX
> ./bin/llvm-lit test/CodeGen/BPF/
>
>
> Any suggestion would be appreciated.
>
> Thanks!
>
> Regards,
>
> Marco Leogrande
>
> Sent by a carbon-based life form; hence, it may contain repetitions,
> inaccuracies, logical fallacies and repetitions.
>
>
> On Thu, May 7, 2015 at 8:56 PM, Marco Leogrande <marcol at
plumgrid.com>
> wrote:
> > Hi everyone,
> > I am working with Alexei Starovoitov to contribute an LLVM buildbot
> > for the experimental BPF backend.
> > I am following the steps at [1] to setup a buildbot and I was mostly
> > successful: I was able to setup a slave and a temporary master to
> > check its base config.
> >
> > Now I'm not sure about the next step: patching the
"slaves.py" and
> > "builders.py" files in zorg.
> >
> > * Is there a guide I can follow to understand the format of these
> > files? For example, being BPF an experimental backend, I need to
> > provide extra parameters to cmake at build time.
> >
> > * Also, is there a way to test these changes in zorg in a local
> > environment before submitting the patch for review?
> >
> > Thank you!
> >
> >
> > [1] http://llvm.org/docs/HowToAddABuilder.html
> >
> > Regards,
> >
> > Marco Leogrande
> >
> > Sent by a carbon-based life form; hence, it may contain repetitions,
> > inaccuracies, logical fallacies and repetitions.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20150610/ca47ff3e/attachment.html>