Ocfs2 devel - Nov 2008 - [RFC] Splitting cluster.git into separate projects/trees

Hi everybody,

as discussed and agreed at the Cluster Summit we need to split our tree
to make life easier in the long run (etc. etc.).

We need to decide how we want to do it and there are different
approaches to that. I was able to think of 3. There might be more and I
might not have taken everything into consideration so comments and ideas
are welcome.

At this point we haven't really settled how many (sub) project will be
created out of this split. This will come once we agree how to split.

= first approach 
We maintain cluster.git as single entity with all source code in one
place. We change the build system in such a way each single component
can be released standalone (similar to how it was done in the RHEL*
branches).

Pro:
 - preserve current development model.
 - allow release of separate tarball for each (sub) project.
 - external users don't need to build the whole tree for one (sub)
project.

Cons:
 - move all the burden to the build system (by duplicating tons of
stuff, maybe solvable but needs investigation) and release manager.
 - tagging for releases will require changes as it's not possible to tag
only one (sub) project.

= second approach 
We maintain cluster.git as single entity. Each (sub) project would
become a separate branch.

So for example all the gnbd code will be branched into master-gnbd (and
so on for all the others).

Checking out one specific HEAD will only show the code for that project.

Pro:
 - cleaner look at the tree.
 - partially preserve current development model (still easy to cherry
pick changes between branches)
 - external users don't need to build the whole tree.

Cons:
 - more expensive branch management.
 - tagging for releases will require small changes.

= third approach 
We copy cluster.git N times for each (sub) project, clean the master
branch to match only that (sub)project.

Pro:
 - very clean tree from checkout
 - each (sub) project is really separated and will have its own
identity.
 - external users don't need to build the whole tree.
 - easier to fine tune access to each single component (for example we
can allow user foo to access dlm but not gfs... or whatever combination)

Cons:
 - more complex process to perform cherry-pick between branches.
 - higher risk to commit fixes in one branch and forget in another.
 - requires a lot more developer attention.

Fabio

Hi,

I'm not so keen on option #2, but aside from that I have no strong
opinions,

Steve.

On Fri, 2008-11-14 at 10:18 +0100, Fabio M. Di Nitto
wrote:
> Hi everybody,
> 
> as discussed and agreed at the Cluster Summit we need to split our tree
> to make life easier in the long run (etc. etc.).
> 
> We need to decide how we want to do it and there are different
> approaches to that. I was able to think of 3. There might be more and I
> might not have taken everything into consideration so comments and ideas
> are welcome.
> 
> At this point we haven't really settled how many (sub) project will be
> created out of this split. This will come once we agree how to split.
> 
> = first approach > 
> We maintain cluster.git as single entity with all source code in one
> place. We change the build system in such a way each single component
> can be released standalone (similar to how it was done in the RHEL*
> branches).
> 
> Pro:
>  - preserve current development model.
>  - allow release of separate tarball for each (sub) project.
>  - external users don't need to build the whole tree for one (sub)
> project.
> 
> Cons:
>  - move all the burden to the build system (by duplicating tons of
> stuff, maybe solvable but needs investigation) and release manager.
>  - tagging for releases will require changes as it's not possible to
tag
> only one (sub) project.
> 
> = second approach > 
> We maintain cluster.git as single entity. Each (sub) project would
> become a separate branch.
> 
> So for example all the gnbd code will be branched into master-gnbd (and
> so on for all the others).
> 
> Checking out one specific HEAD will only show the code for that project.
> 
> Pro:
>  - cleaner look at the tree.
>  - partially preserve current development model (still easy to cherry
> pick changes between branches)
>  - external users don't need to build the whole tree.
> 
> Cons:
>  - more expensive branch management.
>  - tagging for releases will require small changes.
> 
> = third approach > 
> We copy cluster.git N times for each (sub) project, clean the master
> branch to match only that (sub)project.
> 
> Pro:
>  - very clean tree from checkout
>  - each (sub) project is really separated and will have its own
> identity.
>  - external users don't need to build the whole tree.
>  - easier to fine tune access to each single component (for example we
> can allow user foo to access dlm but not gfs... or whatever combination)
> 
> Cons:
>  - more complex process to perform cherry-pick between branches.
>  - higher risk to commit fixes in one branch and forget in another.
>  - requires a lot more developer attention.
> 
> Fabio
>

Fabio M. Di Nitto wrote:
> Hi everybody,
> 
> as discussed and agreed at the Cluster Summit we need to split our tree
> to make life easier in the long run (etc. etc.).
> 
> We need to decide how we want to do it and there are different
> approaches to that. I was able to think of 3. There might be more and I
> might not have taken everything into consideration so comments and ideas
> are welcome.
> 
> At this point we haven't really settled how many (sub) project will be
> created out of this split. This will come once we agree how to split.
> 
> = first approach > 
> We maintain cluster.git as single entity with all source code in one
> place. We change the build system in such a way each single component
> can be released standalone (similar to how it was done in the RHEL*
> branches).
> 
> Pro:
>  - preserve current development model.
>  - allow release of separate tarball for each (sub) project.
>  - external users don't need to build the whole tree for one (sub)
> project.
> 
> Cons:
>  - move all the burden to the build system (by duplicating tons of
> stuff, maybe solvable but needs investigation) and release manager.
>  - tagging for releases will require changes as it's not possible to
tag
> only one (sub) project.
> 
> = second approach > 
> We maintain cluster.git as single entity. Each (sub) project would
> become a separate branch.
> 
> So for example all the gnbd code will be branched into master-gnbd (and
> so on for all the others).
> 
> Checking out one specific HEAD will only show the code for that project.
> 
> Pro:
>  - cleaner look at the tree.
>  - partially preserve current development model (still easy to cherry
> pick changes between branches)
>  - external users don't need to build the whole tree.
> 
> Cons:
>  - more expensive branch management.
>  - tagging for releases will require small changes.
> 
> = third approach > 
> We copy cluster.git N times for each (sub) project, clean the master
> branch to match only that (sub)project.
> 
> Pro:
>  - very clean tree from checkout
>  - each (sub) project is really separated and will have its own
> identity.
>  - external users don't need to build the whole tree.
>  - easier to fine tune access to each single component (for example we
> can allow user foo to access dlm but not gfs... or whatever combination)
> 
> Cons:
>  - more complex process to perform cherry-pick between branches.
>  - higher risk to commit fixes in one branch and forget in another.
>  - requires a lot more developer attention.

I think I would votes for 3, 1, 2 in that order.

3 is definitely the best option IMHO. The cons don't make much
difference really - as I understand it, we're not splitting branches but
projects so there will be no, or very little, need to copy fixes across
git trees. Even for the few occasions when it might be necessary, git is
quite capable of generating usable patches.

2 is an inholy mess and an abuse of git's branching system I think.
Please don't do this!

1 is more trouble than it sounds - speaking from experience with the old
RHEL4 tree. You have a tree that looks like it's self-contained, but you
still have to install bits of it to make things build properly. That's
just confusing and annoying.

Chrissie

Top-posting...

#1 is what the current "situation" evolved from so I think its pretty
clear that despite the best of intentions, it doesn't work out (far
too easy for cross-dependancies to develop).

#2 - I share chrissie's feelings on this one ;-)

#3 - Thats got my vote - basically what we did with Pacemaker.

The biggest question here is do you want old versions of the split out
code to be buildable?

Obviously you'd keep the combined repo somewhere, but you may also
want to be able to build old versions from within the split-out repo.
If people don't want this, the split out repos _could_ be filtered
copies (smaller and only contains relevant history).

Just an idea - I don't have a strong opinion either way (For Pacemaker
I didn't filter, for the Pacemaker GUI I did).

Oh, and I'm assuming git can do this since Mercurial can (and it also
knows how to match the old and new commit IDs - since they'll
obviously change in the new repo)


On Fri, Nov 14, 2008 at 10:18, Fabio M. Di Nitto <fdinitto at redhat.com>
wrote:
> Hi everybody,
>
> as discussed and agreed at the Cluster Summit we need to split our tree
> to make life easier in the long run (etc. etc.).
>
> We need to decide how we want to do it and there are different
> approaches to that. I was able to think of 3. There might be more and I
> might not have taken everything into consideration so comments and ideas
> are welcome.
>
> At this point we haven't really settled how many (sub) project will be
> created out of this split. This will come once we agree how to split.
>
> = first approach >
> We maintain cluster.git as single entity with all source code in one
> place. We change the build system in such a way each single component
> can be released standalone (similar to how it was done in the RHEL*
> branches).
>
> Pro:
>  - preserve current development model.
>  - allow release of separate tarball for each (sub) project.
>  - external users don't need to build the whole tree for one (sub)
> project.
>
> Cons:
>  - move all the burden to the build system (by duplicating tons of
> stuff, maybe solvable but needs investigation) and release manager.
>  - tagging for releases will require changes as it's not possible to
tag
> only one (sub) project.
>
> = second approach >
> We maintain cluster.git as single entity. Each (sub) project would
> become a separate branch.
>
> So for example all the gnbd code will be branched into master-gnbd (and
> so on for all the others).
>
> Checking out one specific HEAD will only show the code for that project.
>
> Pro:
>  - cleaner look at the tree.
>  - partially preserve current development model (still easy to cherry
> pick changes between branches)
>  - external users don't need to build the whole tree.
>
> Cons:
>  - more expensive branch management.
>  - tagging for releases will require small changes.
>
> = third approach >
> We copy cluster.git N times for each (sub) project, clean the master
> branch to match only that (sub)project.
>
> Pro:
>  - very clean tree from checkout
>  - each (sub) project is really separated and will have its own
> identity.
>  - external users don't need to build the whole tree.
>  - easier to fine tune access to each single component (for example we
> can allow user foo to access dlm but not gfs... or whatever combination)
>
> Cons:
>  - more complex process to perform cherry-pick between branches.
>  - higher risk to commit fixes in one branch and forget in another.
>  - requires a lot more developer attention.
>
> Fabio
>
>
> _______________________________________________
> Pacemaker mailing list
> Pacemaker at clusterlabs.org
> http://list.clusterlabs.org/mailman/listinfo/pacemaker
>

On Fri, Nov 14, 2008 at 10:18:13AM +0100, Fabio M. Di Nitto
wrote:
> At this point we haven't really settled how many (sub) project will be
> created out of this split. This will come once we agree how to split.
I like the third option as long as the number of new git trees doesn't
explode (obviously no one wants 10 new git trees.)  Not to get ahead of
you, but for my own curiosity I looked at what minimum number of git trees
I'd have to start juggling... it's not too bad, but more than this might
get out of hand.

dlm.git:
  libdlm, dlm_controld, libdlmcontrol, dlm_tool

fence.git:
  libfence, fenced, libfenced, fence_tool, fence_node

fence-agents.git:
  <lots>

cman.git:
  libcman, cman_tool, cmannotifyd, qdiskd, mkqdisk
  cluster/config/*
  move plugins into corosync tree?
  group_tool (groupd/libgroup won't exist, group_tool will just be a
  wrapper/shortcut for fence_tool/dlm_tool/gfs_control queries;
  maybe include queries of other related daemons, like ocfs2_controld?)

gfs2-utils.git:
  gfs_controld, libgfscontrol, gfs_control
  mount.gfs2, mount.gfs, libgfs, libgfs2
  gfs_debug, gfs_fsck, gfs_grow, gfs_jadd, gfs_mkfs, gfs_quota, gfs_tool
  gfs2_convert, gfs2_edit, gfs2_fsck, gfs2_mkfs, gfs2_quota, gfs2_tool

gfs-kernel.git:
  gfs.ko

rgmanager.git:

gnbd goes away
cmirror moves away

On Fri, Nov 14, 2008 at 09:33:48AM +0000, Christine Caulfield
wrote:
> Fabio M. Di Nitto wrote:
> > Hi everybody,
> > 
> > as discussed and agreed at the Cluster Summit we need to split our
tree
> > to make life easier in the long run (etc. etc.).
Thanks for taking this on Fabio.

> > = third approach > > 
> > We copy cluster.git N times for each (sub) project, clean the master
> > branch to match only that (sub)project.
> > 
> > Pro:
> >  - very clean tree from checkout
> >  - each (sub) project is really separated and will have its own
> > identity.
> >  - external users don't need to build the whole tree.
> >  - easier to fine tune access to each single component (for example we
> > can allow user foo to access dlm but not gfs... or whatever
combination)
> > 
> > Cons:
> >  - more complex process to perform cherry-pick between branches.
> >  - higher risk to commit fixes in one branch and forget in another.
> >  - requires a lot more developer attention.
> 
> 
> I think I would votes for 3, 1, 2 in that order.
> 
> 3 is definitely the best option IMHO. The cons don't make much
> difference really - as I understand it, we're not splitting branches
but
> projects so there will be no, or very little, need to copy fixes across
> git trees. Even for the few occasions when it might be necessary, git is
> quite capable of generating usable patches.
I completely agree. Also, yeah the cons don't seem like real cons.

Split out repos for each component is a rather common development model for
free software projects, so I think we're in good company there.

1 and 2 don't really seem like viable long term options to me.
	--Mark

--
Mark Fasheh