rspec users - Aug 2009 - Sharing code and still keeping a "only commit if all is green" discipline

Hello list,

I''m working in a big project in the company I''m working on as
a "BDD
tech lead", this means I''m leading the implementation of BDD in
ongoing projects, most of them did not use BDD from the start.

I implemented the policy of "Only commit if all your tests are
passing", this in all tests abstraction levels: From Cucumber features
, scenarios, steps and rspec specs.

The point of this rule is: "Never commit partial code". If
it''s not
done/fully working (as the feature/specs say it should), don''t commit
it.

However, a co-worker of mine came to me and said that partial commits
are needed to share code. I don''t agree with that, I think we can use
patches or branches to do that (like a development branch) but at
least one branch should have only stabe/deliverable code.

For patches, however, he mentioned that we might have conflicts/code
duplication, where the guy who shares the patches will have problems
later on when pulling from the server (he already had part of the code
that was committed).

What do you think? I''m looking for some enlightenment! Any
contributions welcome.

Marcelo.

On Thu, Aug 20, 2009 at 7:00 PM, Marcelo de Moraes
Serpa<celoserpa at gmail.com> wrote:
> Hello list,
>
> I''m working in a big project in the company I''m working
on as a "BDD
> tech lead", this means I''m leading the implementation of BDD
in
> ongoing projects, most of them did not use BDD from the start.
>
> I implemented the policy of "Only commit if all your tests are
> passing", this in all tests abstraction levels: From Cucumber features
> , scenarios, steps and rspec specs.
>
> The point of this rule is: "Never commit partial code". If
it''s not
> done/fully working (as the feature/specs say it should), don''t
commit
> it.
>
> However, a co-worker of mine came to me and said that partial commits
> are needed to share code. I don''t agree with that, I think we can
use
> patches or branches to do that (like a development branch) but at
> least one branch should have only stabe/deliverable code.
>
> For patches, however, he mentioned that we might have conflicts/code
> duplication, where the guy who shares the patches will have problems
> later on when pulling from the server (he already had part of the code
> that was committed).
>
> What do you think? I''m looking for some enlightenment! Any
> contributions welcome.
My approach, which is not mine originally, nor very unique, is follows:

All developer specs (RSpec) should always be passing.
Previously passing customer specs should continue passing.
Customer specs for features in development are allowed to fail.

You can enforce those rules in a CI build in any number of ways.
Tagging features in development as @in_development, for example, and
not running those as part of the CI build, or running them separately
but allowing them to fail without considering the build a failed
build.

That all make sense?
>
> Marcelo.
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

David Chelimsky wrote:
> On Thu, Aug 20, 2009 at 7:00 PM, Marcelo de Moraes
> Serpa<celoserpa at gmail.com> wrote:
>   
>> Hello list,
>>
>> I''m working in a big project in the company I''m
working on as a "BDD
>> tech lead", this means I''m leading the implementation of
BDD in
>> ongoing projects, most of them did not use BDD from the start.
>>
>> I implemented the policy of "Only commit if all your tests are
>> passing", this in all tests abstraction levels: From Cucumber
features
>> , scenarios, steps and rspec specs.
>>
>> The point of this rule is: "Never commit partial code". If
it''s not
>> done/fully working (as the feature/specs say it should), don''t
commit
>> it.
>>
>> However, a co-worker of mine came to me and said that partial commits
>> are needed to share code. I don''t agree with that, I think we
can use
>> patches or branches to do that (like a development branch) but at
>> least one branch should have only stabe/deliverable code.
>>
>> For patches, however, he mentioned that we might have conflicts/code
>> duplication, where the guy who shares the patches will have problems
>> later on when pulling from the server (he already had part of the code
>> that was committed).
>>
>> What do you think? I''m looking for some enlightenment! Any
>> contributions welcome.
>>     
We use remote branches (we use git) to collaborate and share code that 
isn''t ready to be merged down to our mainline.  Then when that branch
is
ready to be merged down we use git rebase --interactive to squash all 
the wip/garbage commits.  This gives you the best of both worlds IMO.  
Using rebase can be dangerous though as it rewrites the repo''s history.
That means once someone rebases everyone will have to delete that branch 
and pull it again if they want to continue working on it.  That is why 
we generally wait until the work on the branch is don''t before we
rebase.
>
> My approach, which is not mine originally, nor very unique, is follows:
>
> All developer specs (RSpec) should always be passing.
> Previously passing customer specs should continue passing.
> Customer specs for features in development are allowed to fail.
>
> You can enforce those rules in a CI build in any number of ways.
> Tagging features in development as @in_development, for example, and
> not running those as part of the CI build, or running them separately
> but allowing them to fail without considering the build a failed
> build.
>
> That all make sense?
>   

Cucumber actually ships now with some additional rake tasks to help with 
this:

rake cucumber      # Alias for cucumber:ok
rake cucumber:all  # Run all features
rake cucumber:ok   # Run features that should pass
rake cucumber:wip  # Run features that are being worked on


-Ben

On Thu, Aug 20, 2009 at 8:00 PM, Marcelo de Moraes
Serpa<celoserpa at gmail.com> wrote:
>
> However, a co-worker of mine came to me and said that partial commits
> are needed to share code. I don''t agree with that, I think we can
use
> patches or branches to do that (like a development branch) but at
> least one branch should have only stabe/deliverable code.
First: you keep using that word.  I do not think it means what you
think it means.

You can''t really have a "partial commit."  Commits are an
atomic unit
of version control; you''ve changed the state of the repository or you
haven''t.  The *size* of a commit may vary, but a commit that''s
just
part of everything you''re working on is still a commit.  And a commit
to a development branch is a commit, too.

It sounds to me like you''re using "commit" not to mean
"any update
that alters the history of the repository," but an update to the
official master branch (or trunk, if you''re using Subversion). 
That''s
a bit more specialized, and I personally would advise a better word or
phrase for it.  "Push to master" works for me.  It may be that some of
your confusion with your team may just be a matter of unclear
terminology.

> For patches, however, he mentioned that we might have conflicts/code
> duplication, where the guy who shares the patches will have problems
> later on when pulling from the server (he already had part of the code
> that was committed).
Second: have you tried Git?  With all this talk of patch conflicts,
and "the server," and spotlighting development branches like
they''re a
big deal, I''m guessing you''re using Subversion or something
else where
branching and merging is a pain.  Once I learned to appreciate how
easy branches were in Git, and to use them properly and frequently, a
lot of these sorts of problems just evaporated.  (I''ve heard the same
is true in Mercurial too, FWIW.)

Get a proper tool that encourages branching and experimentation -- and
then encourage people to branch and experiment.  Your policy of having
an "all green" master branch is truly a good one, but if it
discourages people from _using_ the source control on a constant basis
then something needs to be made easier.  I personally can''t imagine
using Git but refusing to commit my changes to my personal repository
until everything worked.  I commit _constantly._  Mostly for the sake
of being able to easily undo stuff.




-- 
Have Fun,
   Steve Eley (sfeley at gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org

"Partial Commits" are not needed to share code. That is why you use
branches in Subversion: to provide a place for people working on a common
development thread to put their work when it isn''t ready for trunk.

You still stick to "only commit when green". No broken code goes into
the
repository.

Branching and merging in Subversion is fraught with peril, it''s better
in
the long run to use git. Subversion keeps track of versions of documents,
and merging involves you making decisions about which version of a document
is more recent. Git tracks diffs between documents, so merging branches is
simply a case of collecting all the changes together in one place.

It is quite possible to use git-svn to do local (ie: on the developer''s
workstation) "non-green" commits in case someone wants to try
different
ways of not solving a problem. The only catch is to remember to track a
branch on subversion using it''s own specific branch in the git
repository,
and only use rebase to bring patches in and out to the other git branches
into that svn-tracking branch.

When you share a patch you will branch from trunk, write the patch on that
branch, merge that in with trunk.

Just keep in mind that subversion deals in versioning entire documents,
while git tracks changes you make.

Thanks guys,

Well, I think that one of the approaches would be to setup the
integration server to run only "the features that should pass", and
ignore the other ones (using tagging).

@Stephen: Sorry, I used the wrong term. I did not mean partial commit
in the technical term, but a "partial feature commit", which is
committing a feature that is breaking or not fully working somehow to
the main/stable line (and with a comment like "Feature x partially
finished). **One good use-case of this, however, is when another
developer needs code/artifacts you have wrote and you need to commit,
but your commit does not represent the completion of a
feature/step/spec.** <-- Here''s the use case we are trying to come
up
with a solution for.

I''m using git, actually I love git. However, we started with svn
unfortunately and only now the management saw the benefits of git and
we migrated. However, 99% of the developers of this project does not
know how to use git, and with this I mean they don''t know what
branches are actually and how they work. The investment of time to
teach them this would ruin this project, which is already behind the
schedule.

So, for now, we are in a "transition phase", using git in a half-*****
way, with only the master branch. However, being master the only
branch, I think the code there should be stable and have only solid
code. Committing partially done or code that might break features
would put us in chaos again, and nobody would really look into the
CC.rb logs again (since they would fail all the time, as CC.rb is
using this master branch too).

My co-worder suggested running the whole test suite **only** in
staging (we only update staging when all the features are supposedly
working) but this would defeat the whole purpose of BDD/Continuous
Integration, which must be a support tool in the process and not a
specific phase.

So, for this specific situation, I thought on other ways we could
share unstable code (code that might break tests) without committing
to the master branch -- which include patches or even just copy and
paste.

Or maybe we should just use branches, but won''t be easy to convince
the management to do this now.

Does that make sense?

Thanks again,

Marcelo.

On Thu, Aug 20, 2009 at 9:28 PM, Stephen Eley<sfeley at gmail.com>
wrote:
> On Thu, Aug 20, 2009 at 8:00 PM, Marcelo de Moraes
> Serpa<celoserpa at gmail.com> wrote:
>>
>> However, a co-worker of mine came to me and said that partial commits
>> are needed to share code. I don''t agree with that, I think we
can use
>> patches or branches to do that (like a development branch) but at
>> least one branch should have only stabe/deliverable code.
>
> First: you keep using that word. ?I do not think it means what you
> think it means.
>
> You can''t really have a "partial commit." ?Commits are
an atomic unit
> of version control; you''ve changed the state of the repository or
you
> haven''t. ?The *size* of a commit may vary, but a commit
that''s just
> part of everything you''re working on is still a commit. ?And a
commit
> to a development branch is a commit, too.
>
> It sounds to me like you''re using "commit" not to mean
"any update
> that alters the history of the repository," but an update to the
> official master branch (or trunk, if you''re using Subversion).
?That''s
> a bit more specialized, and I personally would advise a better word or
> phrase for it. ?"Push to master" works for me. ?It may be that
some of
> your confusion with your team may just be a matter of unclear
> terminology.
>
>
>> For patches, however, he mentioned that we might have conflicts/code
>> duplication, where the guy who shares the patches will have problems
>> later on when pulling from the server (he already had part of the code
>> that was committed).
>
> Second: have you tried Git? ?With all this talk of patch conflicts,
> and "the server," and spotlighting development branches like
they''re a
> big deal, I''m guessing you''re using Subversion or
something else where
> branching and merging is a pain. ?Once I learned to appreciate how
> easy branches were in Git, and to use them properly and frequently, a
> lot of these sorts of problems just evaporated. ?(I''ve heard the
same
> is true in Mercurial too, FWIW.)
>
> Get a proper tool that encourages branching and experimentation -- and
> then encourage people to branch and experiment. ?Your policy of having
> an "all green" master branch is truly a good one, but if it
> discourages people from _using_ the source control on a constant basis
> then something needs to be made easier. ?I personally can''t
imagine
> using Git but refusing to commit my changes to my personal repository
> until everything worked. ?I commit _constantly._ ?Mostly for the sake
> of being able to easily undo stuff.
>
>
>
>
> --
> Have Fun,
> ? Steve Eley (sfeley at gmail.com)
> ? ESCAPE POD - The Science Fiction Podcast Magazine
> ? http://www.escapepod.org
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users
>

On Fri, Aug 21, 2009 at 11:14 AM, Marcelo de Moraes
Serpa<celoserpa at gmail.com> wrote:
> we started with svn
> unfortunately and only now the management saw the benefits of git and
> we migrated. However, 99% of the developers of this project does not
> know how to use git, and with this I mean they don''t know what
> branches are actually and how they work. The investment of time to
> teach them this would ruin this project, which is already behind the
> schedule.
Ahh, the old "Our problems are so bad that we can''t afford to
address them."

Perhaps the reason you are behind schedule is that your tools/process
are holding you back.

-- 
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Twitter: http://twitter.com/RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

On Fri, Aug 21, 2009 at 11:14 AM, Marcelo de Moraes
Serpa<celoserpa at gmail.com> wrote:
>
> I''m using git, actually I love git. However, we started with svn
> unfortunately and only now the management saw the benefits of git and
> we migrated. However, 99% of the developers of this project does not
> know how to use git, and with this I mean they don''t know what
> branches are actually and how they work. The investment of time to
> teach them this would ruin this project, which is already behind the
> schedule.
Sounds like you have a risk assessment decision to make.  You are
*already* introducing changes to a late project: you moved to Git,
you''re trying to implement CI, you''re talking about version
control
policy.  Given that -- which is more likely to pose a greater threat
to your delivery schedule?

1.) Taking an hour for a mandatory lunch-and-learn, buying everyone
pizza, and saying "Listen up folks.  We''re about to introduce our
friends ''git checkout -b'' and ''git merge'',
and then I''m going to tell
you the version control policies, and you _will_ follow them or you
get to be the one who babysits the continuous integration server."
(Really, you _can_ explain Git branching in a few minutes.  Hell, just
show everyone the Peepcode screencast.)

2.) Stopping short of introducing changes that are responsive to the
changes you''ve already made, and allowing people to continue the
development practices that made your project late -- only now with
tools they''re unfamiliar with and using badly.

I really don''t think the answer to this is automatic.  Sometimes #2
_is_ the right answer.  But if that''s the case, you should probably
also abandon any attempt to maintain continuous integration policies.
What you''ve just said then is that this is the wrong time to make
things better.  So stop.  Don''t make any decisions, don''t
change
anything.  And make sure you''re out of the blame path for the
project''s schedule.

> Or maybe we should just use branches, but won''t be easy to
convince
> the management to do this now.
Question for you: is "management" committing code?

If not, why should you need to convince them of anything?  Don''t ask
permission.  Branching doesn''t need any extra budget.  Just start
doing it.  Follow good practices, and let your teammates know what
you''re doing and why.

You guys are there to do your jobs.  Management''s job is to empower
you to do so.  If this is the sort of company where you have to raise
your hand every time you need to go to the bathroom, then you have
issues a continuous integration server won''t solve.



-- 
Have Fun,
   Steve Eley (sfeley at gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org

On Fri, Aug 21, 2009 at 1:17 PM, Stephen Eley<sfeley at gmail.com>
wrote:
> Question for you: is "management" committing code?
>
> If not, why should you need to convince them of anything? ?Don''t
ask
> permission. ?Branching doesn''t need any extra budget. ?Just start
> doing it. ?Follow good practices, and let your teammates know what
> you''re doing and why.
Actually, the way I read it was that management was already on board
with git and approved converting over to it which they have, but it''s
the cow orkers and their lack of training which is "preventing" them
from actually benefitting from it.

-- 
Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/
Twitter: http://twitter.com/RickDeNatale
WWR: http://www.workingwithrails.com/person/9021-rick-denatale
LinkedIn: http://www.linkedin.com/in/rickdenatale

On Fri, Aug 21, 2009 at 4:44 PM, Rick DeNatale<rick.denatale at gmail.com>
wrote:
>
> Actually, the way I read it was that management was already on board
> with git and approved converting over to it which they have, but
it''s
> the cow orkers and their lack of training which is "preventing"
them
> from actually benefitting from it.
He specifically said he didn''t think he could "convince the
management" to allow using branches in git.

A couple months ago I read _Joel on Software._  I''d resisted it for a
while because I''d listened to Joel Spolsky''s podcast for a
while and
thought he was too much of a blowhard.  But the book really was good.
This thread reminded me of one of the essays in it, but I couldn''t
remember what.  I just dug around and found it:

"Getting Things Done When You''re Only a Grunt"
http://www.joelonsoftware.com/articles/fog0000000332.html

-- 
Have Fun,
   Steve Eley (sfeley at gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org

On 21 Aug 2009, at 16:14, Marcelo de Moraes Serpa wrote:
> Thanks guys,
>
> Well, I think that one of the approaches would be to setup the
> integration server to run only "the features that should pass",
and
> ignore the other ones (using tagging).
This is good practice anyway - the @wip thing is a really useful idea  
so you can separate regression tests from development feedback tests.  
This really means you need to run two builds though - one on the  
development branch that might still have @wip hiding in it, and  
another one on the ''stable'' branch that should not have any
@wip left
in it.
> @Stephen: Sorry, I used the wrong term. I did not mean partial commit
> in the technical term, but a "partial feature commit", which is
> committing a feature that is breaking or not fully working somehow to
> the main/stable line (and with a comment like "Feature x partially
> finished). **One good use-case of this, however, is when another
> developer needs code/artifacts you have wrote and you need to commit,
> but your commit does not represent the completion of a
> feature/step/spec.** <-- Here''s the use case we are trying to
come up
> with a solution for.
>
> I''m using git, actually I love git. However, we started with svn
> unfortunately and only now the management saw the benefits of git and
> we migrated. However, 99% of the developers of this project does not
> know how to use git, and with this I mean they don''t know what
> branches are actually and how they work. The investment of time to
> teach them this would ruin this project, which is already behind the
> schedule.
This will not ruin the project - it takes a few minutes to learn,  
honestly. Get everyone on the team to read this:
http://blog.hasmanythrough.com/2008/12/18/agile-git-and-the-story-branch-pattern

Then, get them all to install the git_remote_branch gem
> So, for now, we are in a "transition phase", using git in a
half-*****
> way, with only the master branch. However, being master the only
> branch, I think the code there should be stable and have only solid
> code. Committing partially done or code that might break features
> would put us in chaos again, and nobody would really look into the
> CC.rb logs again (since they would fail all the time, as CC.rb is
> using this master branch too).
You''re right here. I think you have two choices: use @wip tags to hide
partially complete scenarios from the build, or use branches. I think  
@wip might be better for you as it means you won''t have to teach  
people how to use branches. However it does leave you with the problem  
that people might forget to remove the @wip tags... More on that in a  
moment.
> My co-worder suggested running the whole test suite **only** in
> staging (we only update staging when all the features are supposedly
> working) but this would defeat the whole purpose of BDD/Continuous
> Integration, which must be a support tool in the process and not a
> specific phase.
What we do at songkick, is maintain a ''staging'' branch which
we merge
into when we have a green build of the master branch. Our cap tasks  
for staging deployment use this branch (I think you have to call it  
tag or somesuch in capistrano-speak). You don''t have to touch this  
branch all that often, and when you do, the merges should be easy  
since nobody should really be making changes on the staging branch.  
This is the branch which you can run your ''no @wip'' build on
that will
fail if there are any @wip tags left.
> So, for this specific situation, I thought on other ways we could
> share unstable code (code that might break tests) without committing
> to the master branch -- which include patches or even just copy and
> paste.
>
> Or maybe we should just use branches, but won''t be easy to
convince
> the management to do this now.
>
> Does that make sense?
>
> Thanks again,
>
> Marcelo.
>
> On Thu, Aug 20, 2009 at 9:28 PM, Stephen Eley<sfeley at gmail.com>
wrote:
>> On Thu, Aug 20, 2009 at 8:00 PM, Marcelo de Moraes
>> Serpa<celoserpa at gmail.com> wrote:
>>>
>>> However, a co-worker of mine came to me and said that partial  
>>> commits
>>> are needed to share code. I don''t agree with that, I think
we can
>>> use
>>> patches or branches to do that (like a development branch) but at
>>> least one branch should have only stabe/deliverable code.
>>
>> First: you keep using that word.  I do not think it means what you
>> think it means.
>>
>> You can''t really have a "partial commit."  Commits
are an atomic unit
>> of version control; you''ve changed the state of the repository
or you
>> haven''t.  The *size* of a commit may vary, but a commit
that''s just
>> part of everything you''re working on is still a commit.  And a
commit
>> to a development branch is a commit, too.
>>
>> It sounds to me like you''re using "commit" not to
mean "any update
>> that alters the history of the repository," but an update to the
>> official master branch (or trunk, if you''re using Subversion).
>> That''s
>> a bit more specialized, and I personally would advise a better word  
>> or
>> phrase for it.  "Push to master" works for me.  It may be
that some
>> of
>> your confusion with your team may just be a matter of unclear
>> terminology.
>>
>>
>>> For patches, however, he mentioned that we might have
conflicts/code
>>> duplication, where the guy who shares the patches will have
problems
>>> later on when pulling from the server (he already had part of the  
>>> code
>>> that was committed).
>>
>> Second: have you tried Git?  With all this talk of patch conflicts,
>> and "the server," and spotlighting development branches like
>> they''re a
>> big deal, I''m guessing you''re using Subversion or
something else
>> where
>> branching and merging is a pain.  Once I learned to appreciate how
>> easy branches were in Git, and to use them properly and frequently, a
>> lot of these sorts of problems just evaporated.  (I''ve heard
the same
>> is true in Mercurial too, FWIW.)
>>
>> Get a proper tool that encourages branching and experimentation --  
>> and
>> then encourage people to branch and experiment.  Your policy of  
>> having
>> an "all green" master branch is truly a good one, but if it
>> discourages people from _using_ the source control on a constant  
>> basis
>> then something needs to be made easier.  I personally can''t
imagine
>> using Git but refusing to commit my changes to my personal repository
>> until everything worked.  I commit _constantly._  Mostly for the sake
>> of being able to easily undo stuff.
>>
>>
>>
>>
>> --
>> Have Fun,
>>   Steve Eley (sfeley at gmail.com)
>>   ESCAPE POD - The Science Fiction Podcast Magazine
>>   http://www.escapepod.org
>> _______________________________________________
>> rspec-users mailing list
>> rspec-users at rubyforge.org
>> http://rubyforge.org/mailman/listinfo/rspec-users
>>
> _______________________________________________
> rspec-users mailing list
> rspec-users at rubyforge.org
> http://rubyforge.org/mailman/listinfo/rspec-users