Naturally I want some sort of continuous integration system. But what makes my situation unusual is that what I want to build is structured in an unusual way, and it''s not clear to me if CruiseControl.rb can gracefully handle that structure. I want to independently build half a dozen "projects", each of which (in CruiseControl.NET terms) is its own "queue". Each corresponds to one deliverable set of files. (One project represents our web site; another represents a Windows installer that we build ... etc). Now, those projects don''t simply correspond to a single Subversion URL; instead, each is composed of a number of "kits", each of which has its own Subversion URL. We have about 50 separate kits; most are basically a bunch of C-sharp files and a .csproj file. Each kit typically depends on some others; the kits thus form a directed acyclic graph of dependencies. We have a tool that will build a kit, and all its dependencies; and will also check them out of Subversion if needed. I wrote this tool myself and am perfectly comfortable hacking it. For each project, I want CruiseControl.rb to monitor the bits of the subversion repository that correspond to the constituent kits, and trigger a build whenever anything changes in those bits. Note that this is not the same as monitoring the entire repository, since I don''t want a change in an unrelated kit to trigger a build for this project; nor is it the same as monitoring just one kit, since the project depends on many kits. Our building tool already knows how to determine whether a kit needs to be updated, so I''m imagining that CruiseControl.rb would simply invoke the build tool as a subprocess. At first I thought I would be able to teach it to do this by suitable fiddling of the "SourceControl" class, but I quickly confused myself -- partly because I haven''t yet figured out how CruiseControl.rb works, and partly because I barely know the fundamentals of Ruby. I also have a very vague suspicion that, fundamentally, CruiseControl.rb assumes that there''s a one-to-one-to-one correspondence between projects, Subversion URLs, and working copies; and that I''d need to change that assumption in order to get things working the way I want. Anyway: does sound doable -- for someone who''s a decent programmer but new to Ruby -- and if so, do you have any advice on how to start?
You can probably use SVN externals to achieve what you want :) Structure: /project -> http://yourdomain.com/svn/project /lib -> SVN External pointing to http://yourdomain.com/svn/kits/lib201 /config -> SVN External pointing to http://yourdomain.com/svn/kits/ config991 /docs -> SVN External pointing to http://yourdomain.com/svn/kits/ docs2093 In this scenario, if you add http://yourdomain.com/svn/project to CC, it will trigger any time anything in ANY of the kits is committed. If you add just the kit''s svn repository location to CC, it will only trigger for that particular kit. Does this help? Tag a kit and just add that kit to the repository, or On Apr 21, 2008, at 1:20 PM, Eric Hanchrow wrote:> Naturally I want some sort of continuous integration system. But what > makes my situation unusual is that what I want to build is structured > in an unusual way, and it''s not clear to me if CruiseControl.rb can > gracefully handle that structure. > > I want to independently build half a dozen "projects", each of which > (in CruiseControl.NET terms) is its own "queue". Each corresponds to > one deliverable set of files. (One project represents our web site; > another represents a Windows installer that we build ... etc). Now, > those projects don''t simply correspond to a single Subversion URL; > instead, each is composed of a number of "kits", each of which has its > own Subversion URL. We have about 50 separate kits; most are > basically a bunch of C-sharp files and a .csproj file. Each kit > typically depends on some others; the kits thus form a directed > acyclic graph of dependencies. We have a tool that will build a kit, > and all its dependencies; and will also check them out of Subversion > if needed. I wrote this tool myself and am perfectly comfortable > hacking it. > > For each project, I want CruiseControl.rb to monitor the bits of the > subversion repository that correspond to the constituent kits, and > trigger a build whenever anything changes in those bits. Note that > this is not the same as monitoring the entire repository, since I > don''t want a change in an unrelated kit to trigger a build for this > project; nor is it the same as monitoring just one kit, since the > project depends on many kits. Our building tool already knows how to > determine whether a kit needs to be updated, so I''m imagining that > CruiseControl.rb would simply invoke the build tool as a subprocess. > > At first I thought I would be able to teach it to do this by suitable > fiddling of the "SourceControl" class, but I quickly confused myself > -- partly because I haven''t yet figured out how CruiseControl.rb > works, and partly because I barely know the fundamentals of Ruby. > > I also have a very vague suspicion that, fundamentally, > CruiseControl.rb assumes that there''s a one-to-one-to-one > correspondence between projects, Subversion URLs, and working copies; > and that I''d need to change that assumption in order to get things > working the way I want. > > Anyway: does sound doable -- for someone who''s a decent programmer but > new to Ruby -- and if so, do you have any advice on how to start? > _______________________________________________ > Cruisecontrolrb-users mailing list > Cruisecontrolrb-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean.-- This message has been scanned for viruses and dangerous content by MailScanner, and is believed to be clean.
Thanks for the quick response, but I don''t think externals will work, because I don''t want to change the structure of our source tree; doing so would force the developers to change the way they work, and one of my goals is to avoid that. On Mon, Apr 21, 2008 at 1:28 PM, Billy Kimble <bkimble at inc21.com> wrote:> You can probably use SVN externals to achieve what you want :) > > Structure: > > /project -> http://yourdomain.com/svn/project > /lib -> SVN External pointing to http://yourdomain.com/svn/kits/lib201 > /config -> SVN External pointing to http://yourdomain.com/svn/kits/ > config991 > /docs -> SVN External pointing to http://yourdomain.com/svn/kits/ > docs2093 > > > In this scenario, if you add http://yourdomain.com/svn/project to > CC, it will trigger any time anything in ANY of the kits is > committed. If you add just the kit''s svn repository location to CC, > it will only trigger for that particular kit. > > Does this help? > > Tag a kit and just add that kit to the repository, or > > > On Apr 21, 2008, at 1:20 PM, Eric Hanchrow wrote: > > > Naturally I want some sort of continuous integration system. But what > > makes my situation unusual is that what I want to build is structured > > in an unusual way, and it''s not clear to me if CruiseControl.rb can > > gracefully handle that structure. > > > > I want to independently build half a dozen "projects", each of which > > (in CruiseControl.NET terms) is its own "queue". Each corresponds to > > one deliverable set of files. (One project represents our web site; > > another represents a Windows installer that we build ... etc). Now, > > those projects don''t simply correspond to a single Subversion URL; > > instead, each is composed of a number of "kits", each of which has its > > own Subversion URL. We have about 50 separate kits; most are > > basically a bunch of C-sharp files and a .csproj file. Each kit > > typically depends on some others; the kits thus form a directed > > acyclic graph of dependencies. We have a tool that will build a kit, > > and all its dependencies; and will also check them out of Subversion > > if needed. I wrote this tool myself and am perfectly comfortable > > hacking it. > > > > For each project, I want CruiseControl.rb to monitor the bits of the > > subversion repository that correspond to the constituent kits, and > > trigger a build whenever anything changes in those bits. Note that > > this is not the same as monitoring the entire repository, since I > > don''t want a change in an unrelated kit to trigger a build for this > > project; nor is it the same as monitoring just one kit, since the > > project depends on many kits. Our building tool already knows how to > > determine whether a kit needs to be updated, so I''m imagining that > > CruiseControl.rb would simply invoke the build tool as a subprocess. > > > > At first I thought I would be able to teach it to do this by suitable > > fiddling of the "SourceControl" class, but I quickly confused myself > > -- partly because I haven''t yet figured out how CruiseControl.rb > > works, and partly because I barely know the fundamentals of Ruby. > > > > I also have a very vague suspicion that, fundamentally, > > CruiseControl.rb assumes that there''s a one-to-one-to-one > > correspondence between projects, Subversion URLs, and working copies; > > and that I''d need to change that assumption in order to get things > > working the way I want. > > > > Anyway: does sound doable -- for someone who''s a decent programmer but > > new to Ruby -- and if so, do you have any advice on how to start? > > _______________________________________________ > > Cruisecontrolrb-users mailing list > > Cruisecontrolrb-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users > > > > -- > > This message has been scanned for viruses and > > dangerous content by MailScanner, and is > > believed to be clean. > > > -- > This message has been scanned for viruses and > dangerous content by MailScanner, and is > believed to be clean. > > _______________________________________________ > Cruisecontrolrb-users mailing list > Cruisecontrolrb-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users >
Jeremy Stell-Smith
2008-Apr-21 23:10 UTC
[Cruisecontrolrb-users] Is CruiseControl.rb for me?
You definitely can achieve what you want, using build triggers. Each "kit" would be a project for cruise, and you can tell each project which projects trigger it - check the docs for info. However, you might run into a wall with 50 projects on the same cruise box. Each of our projects runs as a separate process, and not only that, each process has the rails runtime loaded. This means that for 50 projects, you are looking at a lot of memory. If you want to try it anyway, be sure to turn on serialized builds in your site.config. If you look around and determine that nothing does quite what you want it to, I will say that ccrb is much more hackable than most. That said you might go faster in a language that you''re comfortable with. On Mon, Apr 21, 2008 at 1:32 PM, Eric Hanchrow <eric.hanchrow at gmail.com> wrote:> Thanks for the quick response, but I don''t think externals will work, > because I don''t want to change the structure of our source tree; doing > so would force the developers to change the way they work, and one of > my goals is to avoid that. > > On Mon, Apr 21, 2008 at 1:28 PM, Billy Kimble <bkimble at inc21.com> wrote: > > You can probably use SVN externals to achieve what you want :) > > > > Structure: > > > > /project -> http://yourdomain.com/svn/project > > /lib -> SVN External pointing to > http://yourdomain.com/svn/kits/lib201 > > /config -> SVN External pointing to > http://yourdomain.com/svn/kits/ > > config991 > > /docs -> SVN External pointing to > http://yourdomain.com/svn/kits/ > > docs2093 > > > > > > In this scenario, if you add http://yourdomain.com/svn/project to > > CC, it will trigger any time anything in ANY of the kits is > > committed. If you add just the kit''s svn repository location to CC, > > it will only trigger for that particular kit. > > > > Does this help? > > > > Tag a kit and just add that kit to the repository, or > > > > > > On Apr 21, 2008, at 1:20 PM, Eric Hanchrow wrote: > > > > > Naturally I want some sort of continuous integration system. But > what > > > makes my situation unusual is that what I want to build is structured > > > in an unusual way, and it''s not clear to me if CruiseControl.rb can > > > gracefully handle that structure. > > > > > > I want to independently build half a dozen "projects", each of which > > > (in CruiseControl.NET terms) is its own "queue". Each corresponds to > > > one deliverable set of files. (One project represents our web site; > > > another represents a Windows installer that we build ... etc). Now, > > > those projects don''t simply correspond to a single Subversion URL; > > > instead, each is composed of a number of "kits", each of which has > its > > > own Subversion URL. We have about 50 separate kits; most are > > > basically a bunch of C-sharp files and a .csproj file. Each kit > > > typically depends on some others; the kits thus form a directed > > > acyclic graph of dependencies. We have a tool that will build a kit, > > > and all its dependencies; and will also check them out of Subversion > > > if needed. I wrote this tool myself and am perfectly comfortable > > > hacking it. > > > > > > For each project, I want CruiseControl.rb to monitor the bits of the > > > subversion repository that correspond to the constituent kits, and > > > trigger a build whenever anything changes in those bits. Note that > > > this is not the same as monitoring the entire repository, since I > > > don''t want a change in an unrelated kit to trigger a build for this > > > project; nor is it the same as monitoring just one kit, since the > > > project depends on many kits. Our building tool already knows how to > > > determine whether a kit needs to be updated, so I''m imagining that > > > CruiseControl.rb would simply invoke the build tool as a subprocess. > > > > > > At first I thought I would be able to teach it to do this by suitable > > > fiddling of the "SourceControl" class, but I quickly confused myself > > > -- partly because I haven''t yet figured out how CruiseControl.rb > > > works, and partly because I barely know the fundamentals of Ruby. > > > > > > I also have a very vague suspicion that, fundamentally, > > > CruiseControl.rb assumes that there''s a one-to-one-to-one > > > correspondence between projects, Subversion URLs, and working copies; > > > and that I''d need to change that assumption in order to get things > > > working the way I want. > > > > > > Anyway: does sound doable -- for someone who''s a decent programmer > but > > > new to Ruby -- and if so, do you have any advice on how to start? > > > _______________________________________________ > > > Cruisecontrolrb-users mailing list > > > Cruisecontrolrb-users at rubyforge.org > > > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users > > > > > > -- > > > This message has been scanned for viruses and > > > dangerous content by MailScanner, and is > > > believed to be clean. > > > > > > -- > > This message has been scanned for viruses and > > dangerous content by MailScanner, and is > > believed to be clean. > > > > _______________________________________________ > > Cruisecontrolrb-users mailing list > > Cruisecontrolrb-users at rubyforge.org > > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users > > > _______________________________________________ > Cruisecontrolrb-users mailing list > Cruisecontrolrb-users at rubyforge.org > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://rubyforge.org/pipermail/cruisecontrolrb-users/attachments/20080421/95014c03/attachment-0001.html
On Mon, Apr 21, 2008 at 1:32 PM, Eric Hanchrow <eric.hanchrow at gmail.com> wrote:> Thanks for the quick response, but I don''t think externals will work, > because I don''t want to change the structure of our source tree; doing > so would force the developers to change the way they work, and one of > my goals is to avoid that.As Jeremy said, you may try build triggers. If that doesn''t work, I think svn:externals could definitely work. We''ve put a lot of effort into making sure externals are detected and reported properly in change detection and changeset reporting, and I think trunk works mostly pretty well in this area :) As for having to change the source tree, how about this: You can have a separate project, which is only used to build your "kit". This project is composed of only externals, which arrange things in the proper way to achive a build (perhaps with some hacks to your build process). Your developers can use the regular repo locations the way they normally do, this separate project that ties everything together via externals will only be used by cruise.rb. Just an idea, let us know if you get it working. -- Chad