Cruisecontrolrb users - Jan 2011 - CC running tests twice with Passenger 3

Hi,

Ruby 1.8.7
Cruise Control 1.4.0
Passenger 3.0.2
git

Since upgrading to Passenger 3 we''ve noticed that our Cruise projects
try to run the test suite twice which is, in itself, causing some of our
tests to fail (on the second run).

Note that our tests pass (and only run once) on our local boxes, so
it''s
not some hidden fork call. Also, they pass if run manually on the server
hosting CC (and only run once). It''s only the automated testing that
exhibits this behavior.

Any ideas?

Regards,

Dan

Are you running two builder processes for the same project?

On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger <dberger at globe.gov>
wrote:
> Hi,
>
> Ruby 1.8.7
> Cruise Control 1.4.0
> Passenger 3.0.2
> git
>
> Since upgrading to Passenger 3 we''ve noticed that our Cruise
projects
> try to run the test suite twice which is, in itself, causing some of our
> tests to fail (on the second run).
>
> Note that our tests pass (and only run once) on our local boxes, so
it''s
> not some hidden fork call. Also, they pass if run manually on the server
> hosting CC (and only run once). It''s only the automated testing
that
> exhibits this behavior.
>
> Any ideas?
>
> Regards,
>
> Dan
>
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>


-- 
Alexey Verkhovsky
http://alex-verkhovsky.blogspot.com/
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/cruisecontrolrb-users/attachments/20110124/ca66d571/attachment.html>

Hi,

Can you help me determine if that''s the case? Where is that configured?

Dan

On Mon, 2011-01-24 at 10:14 -0700, Alexey Verkhovsky
wrote:
> Are you running two builder processes for the same project?
> 
> On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger <dberger at globe.gov>
> wrote:
>         Hi,
>         
>         Ruby 1.8.7
>         Cruise Control 1.4.0
>         Passenger 3.0.2
>         git
>         
>         Since upgrading to Passenger 3 we''ve noticed that our
Cruise
>         projects
>         try to run the test suite twice which is, in itself, causing
>         some of our
>         tests to fail (on the second run).
>         
>         Note that our tests pass (and only run once) on our local
>         boxes, so it''s
>         not some hidden fork call. Also, they pass if run manually on
>         the server
>         hosting CC (and only run once). It''s only the automated
>         testing that
>         exhibits this behavior.
>         
>         Any ideas?
>         
>         Regards,
>         
>         Dan
>         
>         _______________________________________________
>         Cruisecontrolrb-users mailing list
>         Cruisecontrolrb-users at rubyforge.org
>         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> 
> 
> 
> -- 
> Alexey Verkhovsky
> http://alex-verkhovsky.blogspot.com/
> CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> 
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users

It''s not configured anywhere, moreover is not supposed to happen. But
then,
running the same build twice is not supposed to happen, either.
ps -ef | grep build should tell you all you need.

On Mon, Jan 24, 2011 at 10:26 AM, Daniel Berger <dberger at globe.gov>
wrote:
> Hi,
>
> Can you help me determine if that''s the case? Where is that
configured?
>
> Dan
>
> On Mon, 2011-01-24 at 10:14 -0700, Alexey Verkhovsky wrote:
> > Are you running two builder processes for the same project?
> >
> > On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger <dberger at
globe.gov>
> > wrote:
> >         Hi,
> >
> >         Ruby 1.8.7
> >         Cruise Control 1.4.0
> >         Passenger 3.0.2
> >         git
> >
> >         Since upgrading to Passenger 3 we''ve noticed that our
Cruise
> >         projects
> >         try to run the test suite twice which is, in itself, causing
> >         some of our
> >         tests to fail (on the second run).
> >
> >         Note that our tests pass (and only run once) on our local
> >         boxes, so it''s
> >         not some hidden fork call. Also, they pass if run manually on
> >         the server
> >         hosting CC (and only run once). It''s only the
automated
> >         testing that
> >         exhibits this behavior.
> >
> >         Any ideas?
> >
> >         Regards,
> >
> >         Dan
> >
> >         _______________________________________________
> >         Cruisecontrolrb-users mailing list
> >         Cruisecontrolrb-users at rubyforge.org
> >         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >
> >
> >
> > --
> > Alexey Verkhovsky
> > http://alex-verkhovsky.blogspot.com/
> > CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> >
> > _______________________________________________
> > Cruisecontrolrb-users mailing list
> > Cruisecontrolrb-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>
>
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>


-- 
Alexey Verkhovsky
http://alex-verkhovsky.blogspot.com/
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/cruisecontrolrb-users/attachments/20110124/e5dd96b2/attachment.html>

Not that I can see. We do have multiple watchers setup that are pointing
at the same project but different branches.

Hm, is it possible we have projects setup pointing at the same branch by
mistake? Is there a way to tell? Would that cause it?

Regards,

Dan

On Mon, 2011-01-24 at 10:41 -0700, Alexey Verkhovsky
wrote:
> It''s not configured anywhere, moreover is not supposed to happen.
But
> then, running the same build twice is not supposed to happen, either.
> ps -ef | grep build should tell you all you need.
> 
> On Mon, Jan 24, 2011 at 10:26 AM, Daniel Berger <dberger at
globe.gov>
> wrote:
>         Hi,
>         
>         Can you help me determine if that''s the case? Where is
that
>         configured?
>         
>         Dan
>         
>         
>         On Mon, 2011-01-24 at 10:14 -0700, Alexey Verkhovsky wrote:
>         > Are you running two builder processes for the same project?
>         >
>         > On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger
>         <dberger at globe.gov>
>         > wrote:
>         >         Hi,
>         >
>         >         Ruby 1.8.7
>         >         Cruise Control 1.4.0
>         >         Passenger 3.0.2
>         >         git
>         >
>         >         Since upgrading to Passenger 3 we''ve noticed
that
>         our Cruise
>         >         projects
>         >         try to run the test suite twice which is, in itself,
>         causing
>         >         some of our
>         >         tests to fail (on the second run).
>         >
>         >         Note that our tests pass (and only run once) on our
>         local
>         >         boxes, so it''s
>         >         not some hidden fork call. Also, they pass if run
>         manually on
>         >         the server
>         >         hosting CC (and only run once). It''s only the
>         automated
>         >         testing that
>         >         exhibits this behavior.
>         >
>         >         Any ideas?
>         >
>         >         Regards,
>         >
>         >         Dan
>         >
>         >         _______________________________________________
>         >         Cruisecontrolrb-users mailing list
>         >         Cruisecontrolrb-users at rubyforge.org
>         >
>         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>         >
>         >
>         >
>         > --
>         > Alexey Verkhovsky
>         > http://alex-verkhovsky.blogspot.com/
>         > CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
>         >
>         > _______________________________________________
>         > Cruisecontrolrb-users mailing list
>         > Cruisecontrolrb-users at rubyforge.org
>         > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>         
>         
>         _______________________________________________
>         Cruisecontrolrb-users mailing list
>         Cruisecontrolrb-users at rubyforge.org
>         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>         
> 
> 
> 
> -- 
> Alexey Verkhovsky
> http://alex-verkhovsky.blogspot.com/
> CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> 
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users

BTW, here''s what I see after install a local instance of cruise and
building via the web interface:

dberger:/home/dberger >ps -ef | grep build | grep -v grep | grep village

dberger  25625 19459  0 14:45 ?        00:00:00
ruby /home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/cruise build
globe_village
dberger  28170 25625  0 15:07 ?        00:00:00 sh -c
echo /home/dberger/.cruise/projects/globe_village/work dberger$ ruby\ -e
\ \"require\ \''rubygems\''\ rescue\ nil\;\ require\
\''rake\''\;\ load\
\''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake
\''\;\ ARGV\ \<\<\ \''--nosearch\''\ \<\<\
\''cc:build\''\;\
Rake.application.run\;\ ARGV.clear\"
>> /home/dberger/.cruise/projects/globe_village/build-0dca885.2/build.log
&& ruby -e "require ''rubygems'' rescue nil;
require ''rake''; load
''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake'';
ARGV << ''--nosearch'' <<
''cc:build''; Rake.application.run; ARGV.clear" >>
/home/dberger/.cruise/projects/globe_village/build-0dca885.2/build.log
2>&1
On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
> Not that I can see. We do have multiple watchers setup that are pointing
> at the same project but different branches.
> 
> Hm, is it possible we have projects setup pointing at the same branch by
> mistake? Is there a way to tell? Would that cause it?
> 
> Regards,
> 
> Dan
> 
> On Mon, 2011-01-24 at 10:41 -0700, Alexey Verkhovsky wrote:
> > It''s not configured anywhere, moreover is not supposed to
happen. But
> > then, running the same build twice is not supposed to happen, either.
> > ps -ef | grep build should tell you all you need.
> > 
> > On Mon, Jan 24, 2011 at 10:26 AM, Daniel Berger <dberger at
globe.gov>
> > wrote:
> >         Hi,
> >         
> >         Can you help me determine if that''s the case? Where
is that
> >         configured?
> >         
> >         Dan
> >         
> >         
> >         On Mon, 2011-01-24 at 10:14 -0700, Alexey Verkhovsky wrote:
> >         > Are you running two builder processes for the same
project?
> >         >
> >         > On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger
> >         <dberger at globe.gov>
> >         > wrote:
> >         >         Hi,
> >         >
> >         >         Ruby 1.8.7
> >         >         Cruise Control 1.4.0
> >         >         Passenger 3.0.2
> >         >         git
> >         >
> >         >         Since upgrading to Passenger 3 we''ve
noticed that
> >         our Cruise
> >         >         projects
> >         >         try to run the test suite twice which is, in
itself,
> >         causing
> >         >         some of our
> >         >         tests to fail (on the second run).
> >         >
> >         >         Note that our tests pass (and only run once) on
our
> >         local
> >         >         boxes, so it''s
> >         >         not some hidden fork call. Also, they pass if run
> >         manually on
> >         >         the server
> >         >         hosting CC (and only run once). It''s
only the
> >         automated
> >         >         testing that
> >         >         exhibits this behavior.
> >         >
> >         >         Any ideas?
> >         >
> >         >         Regards,
> >         >
> >         >         Dan
> >         >
> >         >         _______________________________________________
> >         >         Cruisecontrolrb-users mailing list
> >         >         Cruisecontrolrb-users at rubyforge.org
> >         >
> >         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >         >
> >         >
> >         >
> >         > --
> >         > Alexey Verkhovsky
> >         > http://alex-verkhovsky.blogspot.com/
> >         > CruiseControl.rb
[http://cruisecontrolrb.thoughtworks.com]
> >         >
> >         > _______________________________________________
> >         > Cruisecontrolrb-users mailing list
> >         > Cruisecontrolrb-users at rubyforge.org
> >         >
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >         
> >         
> >         _______________________________________________
> >         Cruisecontrolrb-users mailing list
> >         Cruisecontrolrb-users at rubyforge.org
> >         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >         
> > 
> > 
> > 
> > -- 
> > Alexey Verkhovsky
> > http://alex-verkhovsky.blogspot.com/
> > CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> > 
> > _______________________________________________
> > Cruisecontrolrb-users mailing list
> > Cruisecontrolrb-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> 
> 
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users

BTW, I tried running this command (that showed up in ps -ef) from the
command line manually in the work directory:

ruby -e "require ''rubygems'' rescue nil; require
''rake''; load
''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake'';
ARGV << ''--nosearch'' <<
''cc:build''; Rake.application.run; ARGV.clear"

That worked fine.

Regards,

Dan

On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
> Not that I can see. We do have multiple watchers setup that are pointing
> at the same project but different branches.
> 
> Hm, is it possible we have projects setup pointing at the same branch by
> mistake? Is there a way to tell? Would that cause it?
> 
> Regards,
> 
> Dan
> 
> On Mon, 2011-01-24 at 10:41 -0700, Alexey Verkhovsky wrote:
> > It''s not configured anywhere, moreover is not supposed to
happen. But
> > then, running the same build twice is not supposed to happen, either.
> > ps -ef | grep build should tell you all you need.
> > 
> > On Mon, Jan 24, 2011 at 10:26 AM, Daniel Berger <dberger at
globe.gov>
> > wrote:
> >         Hi,
> >         
> >         Can you help me determine if that''s the case? Where
is that
> >         configured?
> >         
> >         Dan
> >         
> >         
> >         On Mon, 2011-01-24 at 10:14 -0700, Alexey Verkhovsky wrote:
> >         > Are you running two builder processes for the same
project?
> >         >
> >         > On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger
> >         <dberger at globe.gov>
> >         > wrote:
> >         >         Hi,
> >         >
> >         >         Ruby 1.8.7
> >         >         Cruise Control 1.4.0
> >         >         Passenger 3.0.2
> >         >         git
> >         >
> >         >         Since upgrading to Passenger 3 we''ve
noticed that
> >         our Cruise
> >         >         projects
> >         >         try to run the test suite twice which is, in
itself,
> >         causing
> >         >         some of our
> >         >         tests to fail (on the second run).
> >         >
> >         >         Note that our tests pass (and only run once) on
our
> >         local
> >         >         boxes, so it''s
> >         >         not some hidden fork call. Also, they pass if run
> >         manually on
> >         >         the server
> >         >         hosting CC (and only run once). It''s
only the
> >         automated
> >         >         testing that
> >         >         exhibits this behavior.
> >         >
> >         >         Any ideas?
> >         >
> >         >         Regards,
> >         >
> >         >         Dan
> >         >
> >         >         _______________________________________________
> >         >         Cruisecontrolrb-users mailing list
> >         >         Cruisecontrolrb-users at rubyforge.org
> >         >
> >         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >         >
> >         >
> >         >
> >         > --
> >         > Alexey Verkhovsky
> >         > http://alex-verkhovsky.blogspot.com/
> >         > CruiseControl.rb
[http://cruisecontrolrb.thoughtworks.com]
> >         >
> >         > _______________________________________________
> >         > Cruisecontrolrb-users mailing list
> >         > Cruisecontrolrb-users at rubyforge.org
> >         >
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >         
> >         
> >         _______________________________________________
> >         Cruisecontrolrb-users mailing list
> >         Cruisecontrolrb-users at rubyforge.org
> >         http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> >         
> > 
> > 
> > 
> > -- 
> > Alexey Verkhovsky
> > http://alex-verkhovsky.blogspot.com/
> > CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> > 
> > _______________________________________________
> > Cruisecontrolrb-users mailing list
> > Cruisecontrolrb-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> 
> 
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users

Hi again,

Invoking the build process from a local instance I see this:

dberger   9330 25625  0 16:10 ?        00:00:00 sh -c
echo /home/dberger/.cruise/projects/globe_village/work dberger$ ruby\ -e
\ \"require\ \''rubygems\''\ rescue\ nil\;\ require\
\''rake\''\;\ load\
\''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake
\''\;\ ARGV\ \<\<\ \''--nosearch\''\ \<\<\
\''cc:build\''\;\
Rake.application.run\;\ ARGV.clear\"
>> /home/dberger/.cruise/projects/globe_village/build-28091f9.8/build.log
&& ruby -e "require ''rubygems'' rescue nil;
require ''rake''; load
''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake'';
ARGV << ''--nosearch'' <<
''cc:build''; Rake.application.run; ARGV.clear" >>
/home/dberger/.cruise/projects/globe_village/build-28091f9.8/build.log
2>&1
dberger   9331  9330 16 16:10 ?        00:00:09 ruby -e require
''rubygems'' rescue nil; require ''rake''; load
''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake'';
ARGV << ''--nosearch'' <<
''cc:build''; Rake.application.run; ARGV.clear
dberger  22962     1  0 14:23 ?        00:00:01

ruby /home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/cruise build
globe_village

So, it looks like 2 processes there, 9330 and 9331. The latter has the
former as its parent pid.

I also tried switching to Passenger 2.x, but the result was the same.

In addition, I tried altering the :cruise task to just do "puts hello"
but it still seems to try to run the entire test suite. I slapped a
debug print statement in the cruise control source to make sure it was
picking up that there''s a custom cruise task. It is.

Any ideas?

Regards,

Dan

On Mon, 2011-01-24 at 15:21 -0700, Daniel Berger wrote:
> BTW, I tried running this command (that showed up in ps -ef) from the
> command line manually in the work directory:
> 
> ruby -e "require ''rubygems'' rescue nil; require
''rake''; load
>
''/home/dberger/Downloads/Ruby/cruisecontrol-1.4.0/tasks/cc_build.rake'';
> ARGV << ''--nosearch'' <<
''cc:build''; Rake.application.run; ARGV.clear"
> 
> That worked fine.
> 
> Regards,
> 
> Dan
> 
> On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
> > Not that I can see. We do have multiple watchers setup that are
pointing
> > at the same project but different branches.
> > 
> > Hm, is it possible we have projects setup pointing at the same branch
by
> > mistake? Is there a way to tell? Would that cause it?
> > 
> > Regards,
> > 
> > Dan
> > 
> > On Mon, 2011-01-24 at 10:41 -0700, Alexey Verkhovsky wrote:
> > > It''s not configured anywhere, moreover is not supposed
to happen. But
> > > then, running the same build twice is not supposed to happen,
either.
> > > ps -ef | grep build should tell you all you need.
> > > 
> > > On Mon, Jan 24, 2011 at 10:26 AM, Daniel Berger <dberger at
globe.gov>
> > > wrote:
> > >         Hi,
> > >         
> > >         Can you help me determine if that''s the case?
Where is that
> > >         configured?
> > >         
> > >         Dan
> > >         
> > >         
> > >         On Mon, 2011-01-24 at 10:14 -0700, Alexey Verkhovsky
wrote:
> > >         > Are you running two builder processes for the same
project?
> > >         >
> > >         > On Mon, Jan 24, 2011 at 9:28 AM, Daniel Berger
> > >         <dberger at globe.gov>
> > >         > wrote:
> > >         >         Hi,
> > >         >
> > >         >         Ruby 1.8.7
> > >         >         Cruise Control 1.4.0
> > >         >         Passenger 3.0.2
> > >         >         git
> > >         >
> > >         >         Since upgrading to Passenger 3
we''ve noticed that
> > >         our Cruise
> > >         >         projects
> > >         >         try to run the test suite twice which is, in
itself,
> > >         causing
> > >         >         some of our
> > >         >         tests to fail (on the second run).
> > >         >
> > >         >         Note that our tests pass (and only run once)
on our
> > >         local
> > >         >         boxes, so it''s
> > >         >         not some hidden fork call. Also, they pass
if run
> > >         manually on
> > >         >         the server
> > >         >         hosting CC (and only run once).
It''s only the
> > >         automated
> > >         >         testing that
> > >         >         exhibits this behavior.
> > >         >
> > >         >         Any ideas?
> > >         >
> > >         >         Regards,
> > >         >
> > >         >         Dan
> > >         >
> > >         >        
_______________________________________________
> > >         >         Cruisecontrolrb-users mailing list
> > >         >         Cruisecontrolrb-users at rubyforge.org
> > >         >
> > >        
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> > >         >
> > >         >
> > >         >
> > >         > --
> > >         > Alexey Verkhovsky
> > >         > http://alex-verkhovsky.blogspot.com/
> > >         > CruiseControl.rb
[http://cruisecontrolrb.thoughtworks.com]
> > >         >
> > >         > _______________________________________________
> > >         > Cruisecontrolrb-users mailing list
> > >         > Cruisecontrolrb-users at rubyforge.org
> > >         >
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> > >         
> > >         
> > >         _______________________________________________
> > >         Cruisecontrolrb-users mailing list
> > >         Cruisecontrolrb-users at rubyforge.org
> > >        
http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> > >         
> > > 
> > > 
> > > 
> > > -- 
> > > Alexey Verkhovsky
> > > http://alex-verkhovsky.blogspot.com/
> > > CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
> > > 
> > > _______________________________________________
> > > Cruisecontrolrb-users mailing list
> > > Cruisecontrolrb-users at rubyforge.org
> > > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> > 
> > 
> > _______________________________________________
> > Cruisecontrolrb-users mailing list
> > Cruisecontrolrb-users at rubyforge.org
> > http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
> 
> 
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users

On Mon, Jan 24, 2011 at 4:17 PM, Daniel Berger <dberger at globe.gov>
wrote:
> So, it looks like 2 processes there, 9330 and 9331. The latter has the
> former as its parent pid.
>
Looks exactly how it should. One is shell, the other one is ruby that it
invoked.
> On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
> > > Not that I can see. We do have multiple watchers setup that are
> pointing
> > > at the same project but different branches.
>
Accessing the same database and/or some other resource(s), I presume?
I''d
bet that''s the problem. To fix it, you want to configure CC.rb to
serialize
builds (i.e., make it so that only one build is running at any given time).
There is a commented out setting for this in the site_config.rb, you just
need to find and uncomment it.

--Alex
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/cruisecontrolrb-users/attachments/20110124/98af68da/attachment.html>

On Mon, 2011-01-24 at 18:26 -0700, Alexey Verkhovsky
wrote:
> On Mon, Jan 24, 2011 at 4:17 PM, Daniel Berger <dberger at globe.gov>
> wrote:
>         So, it looks like 2 processes there, 9330 and 9331. The latter
>         has the
>         former as its parent pid.
> 
> Looks exactly how it should. One is shell, the other one is ruby that
> it invoked.
> 
> 
>         > On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
>         > > Not that I can see. We do have multiple watchers setup
>         that are pointing
>         > > at the same project but different branches.
>         
> 
> Accessing the same database and/or some other resource(s), I presume?
> I''d bet that''s the problem. To fix it, you want to
configure CC.rb to
> serialize builds (i.e., make it so that only one build is running at
> any given time). There is a commented out setting for this in the
> site_config.rb, you just need to find and uncomment it.
I tried that but it didn''t solve my problem unfortunately. This is tied
to Passenger somehow I think, because when I run the build using WEBrick
it only runs the tests once, and they pass.

I''ll play with our :cruise task some more and see if I can narrow it
down.

Regards,

Dan

When you catch it in the act of running the same build twice, I suggest that
you look into the process tree and find out who started who. If it''s
not two
concurrent builds running of different branches clashing with one another,
then one of the build processes is probably an orphan.

When you run this stuff under Passenger, it starts and stops frontend
workers at will. These, in turn, start and stop builder processes, one per
project. There is a protection mechanism in the form of a builder lock file,
that is intended to prevent CC.rb from running multiple builders ob the same
project directory; HOWEVER, if a builder process starts a build, which then
doesn''t die when the builder dies (say, a rescue Exception clause
eating the
stop signals, which are Exceptions in Ruby), you can end up with concurrent
builds per project, I guess.

Personally, I simply don''t run CC.rb frontend under Passenger because
of
these oddball child process management scenarios. What I do instead is
follow the instructions in daemon/cruise and let init run a single frontend
process with a single set of builders.

On Tue, Jan 25, 2011 at 9:54 AM, Daniel Berger <dberger at globe.gov>
wrote:
> On Mon, 2011-01-24 at 18:26 -0700, Alexey Verkhovsky wrote:
> > On Mon, Jan 24, 2011 at 4:17 PM, Daniel Berger <dberger at
globe.gov>
> > wrote:
> >         So, it looks like 2 processes there, 9330 and 9331. The latter
> >         has the
> >         former as its parent pid.
> >
> > Looks exactly how it should. One is shell, the other one is ruby that
> > it invoked.
> >
> >
> >         > On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
> >         > > Not that I can see. We do have multiple watchers
setup
> >         that are pointing
> >         > > at the same project but different branches.
> >
> >
> > Accessing the same database and/or some other resource(s), I presume?
> > I''d bet that''s the problem. To fix it, you want to
configure CC.rb to
> > serialize builds (i.e., make it so that only one build is running at
> > any given time). There is a commented out setting for this in the
> > site_config.rb, you just need to find and uncomment it.
>
> I tried that but it didn''t solve my problem unfortunately. This is
tied
> to Passenger somehow I think, because when I run the build using WEBrick
> it only runs the tests once, and they pass.
>
> I''ll play with our :cruise task some more and see if I can narrow
it
> down.
>
> Regards,
>
> Dan
>
> _______________________________________________
> Cruisecontrolrb-users mailing list
> Cruisecontrolrb-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/cruisecontrolrb-users
>


-- 
Alexey Verkhovsky
http://alex-verkhovsky.blogspot.com/
CruiseControl.rb [http://cruisecontrolrb.thoughtworks.com]
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://rubyforge.org/pipermail/cruisecontrolrb-users/attachments/20110125/ac35613a/attachment-0001.html>

On Tue, 2011-01-25 at 09:54 -0700, Daniel Berger wrote:
> On Mon, 2011-01-24 at 18:26 -0700, Alexey Verkhovsky wrote:
> > On Mon, Jan 24, 2011 at 4:17 PM, Daniel Berger <dberger at
globe.gov>
> > wrote:
> >         So, it looks like 2 processes there, 9330 and 9331. The latter
> >         has the
> >         former as its parent pid.
> > 
> > Looks exactly how it should. One is shell, the other one is ruby that
> > it invoked.
> > 
> > 
> >         > On Mon, 2011-01-24 at 15:03 -0700, Daniel Berger wrote:
> >         > > Not that I can see. We do have multiple watchers
setup
> >         that are pointing
> >         > > at the same project but different branches.
> >         
> > 
> > Accessing the same database and/or some other resource(s), I presume?
> > I''d bet that''s the problem. To fix it, you want to
configure CC.rb to
> > serialize builds (i.e., make it so that only one build is running at
> > any given time). There is a commented out setting for this in the
> > site_config.rb, you just need to find and uncomment it.
> 
> I tried that but it didn''t solve my problem unfortunately. This is
tied
> to Passenger somehow I think, because when I run the build using WEBrick
Scratch that. It happens with WEBrick, too, but -only- if I daemonize
it. Very peculiar.

Investigating...

Regards,

Dan

On Tue, 2011-01-25 at 13:21 -0700, Alexey Verkhovsky
wrote:
> When you catch it in the act of running the same build twice, I
> suggest that you look into the process tree and find out who started
> who. If it''s not two concurrent builds running of different
branches
> clashing with one another, then one of the build processes is probably
> an orphan.
> 
> When you run this stuff under Passenger, it starts and stops frontend
> workers at will. These, in turn, start and stop builder processes, one
> per project. There is a protection mechanism in the form of a builder
> lock file, that is intended to prevent CC.rb from running multiple
> builders ob the same project directory; HOWEVER, if a builder process
> starts a build, which then doesn''t die when the builder dies (say,
a
> rescue Exception clause eating the stop signals, which are Exceptions
> in Ruby), you can end up with concurrent builds per project, I guess.
> 
> Personally, I simply don''t run CC.rb frontend under Passenger
because
> of these oddball child process management scenarios. What I do instead
> is follow the instructions in daemon/cruise and let init run a single
> frontend process with a single set of builders.
I don''t think this is related, but we''ve narrowed it down to
this bit of
code, added to a test helper library, that was added to dynamically add
routes for testing:

begin
  _routes = Our::Application.routes
  _routes.disable_clear_and_finalize = true
  _routes.clear!

  Our::Application.routes_reloader.paths.each{ |path| load(path) }

  _routes.draw do
    match "request404" => "application#request404"
    match "request500" => "application#request500"
    match "generic_request" =>
"application#generic_request"
    match "login_required_request" =>
"application#login_required_request"
    match "authorize_school_request" =>
"application#authorize_school_request"
    match "authorize_school_without_redirect_request" =>
"application#authorize_school_without_redirect_request"
    match "redirect_back_request" =>
"application#redirect_back_request"
    match "render_data_request" =>
"application#render_data_request"
    match "cancant_request" =>
"application#cancant_request"

    match "redirect" => "legacy#redirect"
  end
  ActiveSupport.on_load(:action_controller) do
    Our::Application.routes.finalize!
  end
ensure
  _routes.disable_clear_and_finalize = false
end

If we comment this out, and the tests that use the helper, the problem
goes away.

BTW, I think the code came from http://openhood.com/rails/rails%
203/2010/07/20/add-routes-at-runtime-rails-3/ originally.

Still tinkering with it...

Regards,

Dan

On Tue, Jan 25, 2011 at 1:21 PM, Alexey Verkhovsky
<alexey.verkhovsky at gmail.com> wrote:
> Personally, I simply don''t run CC.rb frontend under Passenger
because of
> these oddball child process management scenarios. What I do instead is
> follow the instructions in daemon/cruise and let init run a single frontend
> process with a single set of builders.
Also, if you use the init script ''./daemon/cruise
[start|stop]'', it is
smart enough to kill all the child processes of your builders.  Very
handy.

<snip>
> 
> I don''t think this is related, but we''ve narrowed it down
to this bit of
> code, added to a test helper library, that was added to dynamically add
> routes for testing:
> 
> begin
>   _routes = Our::Application.routes
>   _routes.disable_clear_and_finalize = true
>   _routes.clear!
> 
>   Our::Application.routes_reloader.paths.each{ |path| load(path) }
> 
>   _routes.draw do
>     match "request404" => "application#request404"
>     match "request500" => "application#request500"
>     match "generic_request" =>
"application#generic_request"
>     match "login_required_request" =>
> "application#login_required_request"
>     match "authorize_school_request" =>
> "application#authorize_school_request"
>     match "authorize_school_without_redirect_request" =>
> "application#authorize_school_without_redirect_request"
>     match "redirect_back_request" =>
"application#redirect_back_request"
>     match "render_data_request" =>
"application#render_data_request"
>     match "cancant_request" =>
"application#cancant_request"
> 
>     match "redirect" => "legacy#redirect"
>   end
>   ActiveSupport.on_load(:action_controller) do
>     Our::Application.routes.finalize!
>   end
> ensure
>   _routes.disable_clear_and_finalize = false
> end
> 
> If we comment this out, and the tests that use the helper, the problem
> goes away.
> 
> BTW, I think the code came from http://openhood.com/rails/rails%
> 203/2010/07/20/add-routes-at-runtime-rails-3/ originally.
> 
> Still tinkering with it...
I decided to remove this code and simply put the routes in the
config/routes.rb file wrapped in an if block. That wasn''t the issue,
though.

Based on my experiments, any attempt to reopen the ApplicationController
class causes the double run behavior. There were some instance variables
we wanted to get at directly, so we reopened the class and added some
accessors. If I comment that code and the related tests, CC is suddenly
happy again, even though they work fine from the command line.

Why? I haven''t a clue. Very bizarre.

For now we''re just going to comment them out and move on.

Regards,

Dan