Lustre discuss - Feb 2007 - Anonymous CVS access?

I was wondering if there''s any anonymous cvs access out there?
The wiki says a few things but they are very out of date or don''t work
at all.

Thanks
- David Brown

On Thu, Feb 15, 2007 at 09:46:16AM -0800, David Brown
wrote:
> I was wondering if there''s any anonymous cvs access out there?
> The wiki says a few things but they are very out of date or don''t
work at
> all.
> 
I was wondering pretty much the same thing.. Getting Lustre out into the
open source community would be something that a lot of Clusterfs''s
paying customers would like. I know DOE for one has mandates to support
open source projects.

> I was wondering pretty much the same thing.. Getting Lustre out into the
> open source community would be something that a lot of Clusterfs''s
> paying customers would like. I know DOE for one has mandates to support
> open source projects.
Agreed, I work for a DOE funded lab that uses lustre and I know others
have asked for anonymous cvs access in the past and have just been
ignores by the cfs guys.

This is one thing I don''t understand, I have experience with open
source projects and know full well the benefits of providing anonymous
scm access to the latest development trees that the developers are
working on. Latest development trees include side projects and
exploring potential forks in the code.

Providing the development trees to the customer means the support
teams for the clusters that are both in a production role and for
experimental use can more easily maintain their clusters.  We wouldn''t
have to bounce back to the lustre developers or attempt to search
bugzilla for solutions to our problems, those avenues are available to
us but providing anonymous scm access is yet another way we can find
solutions without having some form of human initiated response with
the developers. I''m sure most maintainers of production systems know
what it means to have that production system go down and sitting on
your thumbs waiting from support for third party vendor to respond.

Also I''m sure the cfs guys have seen many more bugs and know what the
outputs look like than we do, and I know they''ve gotten repeat bugs
for versions of their software both in bugzilla and on the mailing
lists. Instead of answering the questions with the same patch over and
over you can provide an scm commit id (for whatever scm you are using)
and keep the emails short. Providing the commit id also means
consistency in the patch you provide that will fix our problems.

Also providing development trees freely to the customer provides the
customer with insight as to where you are going with your software.
Most of us are fairly well adept in writing C code and understanding
the bigger pictures. We also can provide feed back and inevitably code
for free to the head of the scm.  Which gives the cfs guys a better
picture of what their customers are looking for in the future, and a
little bit of free development.

Tracking also works a lot better when you provide anonymous scm
access.  Maintainers of the clusters have one centralized place to
look for code solutions to the clusters they manage. So providing an
scm commit id instead of patches saves us time when searching for
kernel panics or bugs. If we find the problem we are experiencing and
are given an scm commit id to look at we can then push the patches
around locally and then resubmit to the devel mailing list when we
make it work.
>From my experience you either go open source all the way, or close
everything off, only going half way in between just frustrates people
more...

- David Brown

Hi David and Troy,

On 2007-02-15, at 23:30 , Troy Benjegerdes wrote:
> On Thu, Feb 15, 2007 at 09:46:16AM -0800, David Brown wrote:
>> I was wondering if there''s any anonymous cvs access out there?
>> The wiki says a few things but they are very out of date or
don''t
>> work at
>> all.
>
> I was wondering pretty much the same thing.. Getting Lustre out  
> into the
> open source community would be something that a lot of Clusterfs''s
> paying customers would like. I know DOE for one has mandates to  
> support
> open source projects.
Thanks for this suggestion. First, I should point out that full  
sources are available for every one of our production and beta Lustre  
releases through our download site.

I understand that external engineers can benefit from having access  
to the scm as well, and external read-only access is available to  
support customers upon request. If you are a support customer, and  
you are interested in this service, please send an email to support@.

Further, as I explained at the Lustre.org meeting at SC''06, we have  
been contemplating a move to subversion and moving our entire source  
code repository to the outside. But, since we have some non-GPL  
libraries and sources in our tree related to other projects, this is  
not a trivial transition for us. I can''t make any firm commitments on  
a date at this time.

Cheers,
Bojanic

> Thanks for this suggestion. First, I should point out that full
> sources are available for every one of our production and beta Lustre
> releases through our download site.
And we appreciate these releases a lot :)
Another method might also be nightly snapshots of the source tree, at
least for the interim.
> I understand that external engineers can benefit from having access
> to the scm as well, and external read-only access is available to
> support customers upon request. If you are a support customer, and
> you are interested in this service, please send an email to support@.
I''ll be sure to send an email. However, its been mentioned several
times to me that these emails have been ignored/lost somewhere along
the way. I''ll send another one.
> Further, as I explained at the Lustre.org meeting at SC''06, we
have
> been contemplating a move to subversion and moving our entire source
> code repository to the outside. But, since we have some non-GPL
> libraries and sources in our tree related to other projects, this is
> not a trivial transition for us. I can''t make any firm commitments
on
> a date at this time.
Sorry, wasn''t at SC''06 :( ... and I''m glad you are
trying to move to a
better scm than cvs. Also I wish you luck on getting the non-GPL''ed
code out of the repository.

Can we get an update on which libraries in the repository aren''t GPL
and what their status towards removal are?

- David Brown

I have sent an email directly to David about this, but I would like to 
say on the public fourm that I am concerned to hear this and I would 
like to encourage anyone who has had similar experiences to drop me a 
line. If things have been dropping through the cracks somewhere I  want 
to make sure that we put an end to it. Thanks.
>
> I''ll be sure to send an email. However, its been mentioned several
> times to me that these emails have been ignored/lost somewhere along
> the way. I''ll send another one.
>

On Fri, Feb 16, 2007 at 03:05:43PM -0400, Peter Bojanic
wrote:
> Hi David and Troy,
> 
> On 2007-02-15, at 23:30 , Troy Benjegerdes wrote:
> 
> >On Thu, Feb 15, 2007 at 09:46:16AM -0800, David Brown wrote:
> >>I was wondering if there''s any anonymous cvs access out
there?
> >>The wiki says a few things but they are very out of date or
don''t
> >>work at
> >>all.
> >
> >I was wondering pretty much the same thing.. Getting Lustre out  
> >into the
> >open source community would be something that a lot of
Clusterfs''s
> >paying customers would like. I know DOE for one has mandates to  
> >support
> >open source projects.
> 
> Thanks for this suggestion. First, I should point out that full  
> sources are available for every one of our production and beta Lustre  
> releases through our download site.
> 
> I understand that external engineers can benefit from having access  
> to the scm as well, and external read-only access is available to  
> support customers upon request. If you are a support customer, and  
> you are interested in this service, please send an email to support@.
> 
> Further, as I explained at the Lustre.org meeting at SC''06, we
have
> been contemplating a move to subversion and moving our entire source  
> code repository to the outside. But, since we have some non-GPL  
> libraries and sources in our tree related to other projects, this is  
> not a trivial transition for us. I can''t make any firm commitments
on
> a date at this time.
I would like to put in a suggestion for Mercurial instead of SVN. This
also might make for an easier transition since it looks like the FreeBSD
guys made a CVS to Mercurial mirror program they are using to mirror the
FreeBSD CVS.  

http://hg.fr.freebsd.org/

Maybe one of the paying customers could look at setting this up...

Hi David,

On 2007-02-16, at 15:47 , David Brown wrote:
>> Thanks for this suggestion. First, I should point out that full
>> sources are available for every one of our production and beta Lustre
>> releases through our download site.
>
> And we appreciate these releases a lot :)
> Another method might also be nightly snapshots of the source tree, at
> least for the interim.
We''ve discussed this as well -- nightly builds will be part of our  
new Lustre Test System that we''re developing. I don''t know the
timeframe for this, off the top of my head.
>> I understand that external engineers can benefit from having access
>> to the scm as well, and external read-only access is available to
>> support customers upon request. If you are a support customer, and
>> you are interested in this service, please send an email to support@.
>
> I''ll be sure to send an email. However, its been mentioned several
> times to me that these emails have been ignored/lost somewhere along
> the way. I''ll send another one.
Peter Jones has offered publicly to take care of you ;)
>> Further, as I explained at the Lustre.org meeting at SC''06, we
have
>> been contemplating a move to subversion and moving our entire source
>> code repository to the outside. But, since we have some non-GPL
>> libraries and sources in our tree related to other projects, this is
>> not a trivial transition for us. I can''t make any firm
commitments on
>> a date at this time.
>
> Sorry, wasn''t at SC''06 :( ... and I''m glad you
are trying to move to a
> better scm than cvs. Also I wish you luck on getting the
non-GPL''ed
> code out of the repository.
>
> Can we get an update on which libraries in the repository aren''t
GPL
> and what their status towards removal are?
This is in our architecture team''s capable hands. I''d rather
not get
into a deep discussion about what else we''ve got in our scm.

Cheers,
Bojanic

Hi Troy,

On 2007-02-16, at 23:47 , Troy Benjegerdes wrote:
>> Further, as I explained at the Lustre.org meeting at SC''06, we
have
>> been contemplating a move to subversion and moving our entire source
>> code repository to the outside. But, since we have some non-GPL
>> libraries and sources in our tree related to other projects, this is
>> not a trivial transition for us. I can''t make any firm
commitments on
>> a date at this time.
>
> I would like to put in a suggestion for Mercurial instead of SVN. This
> also might make for an easier transition since it looks like the  
> FreeBSD
> guys made a CVS to Mercurial mirror program they are using to  
> mirror the
> FreeBSD CVS.
>
> http://hg.fr.freebsd.org/
>
> Maybe one of the paying customers could look at setting this up...
Thanks for the suggestion. I''ve passed this onto our architecture  
team to ensure that they take it under consideration.

Cheers,
Bojanic

> > I would like to put in a suggestion for Mercurial instead of SVN. This
 > > also might make for an easier transition since it looks like the  
 > > FreeBSD
 > > guys made a CVS to Mercurial mirror program they are using to  
 > > mirror the
 > > FreeBSD CVS.
 > >
 > > http://hg.fr.freebsd.org/

Thanks for the reminder, I''d like to add that this work was very well
documented in:
http://ns2.freenix.org/~roberto/paper.pdf
http://ns2.freenix.org/~roberto/slides.pdf
nice to see that it''s in place.

Another enlightening discussion is the xorg move from CVS to git:
http://keithp.com/blog/Repository_Formats_Matter.html

Of course svn is the most "natural" (smooth) migration path from CVS:
it has been made for this.
  
 > Thanks for the suggestion. I''ve passed this onto our architecture
 > team to ensure that they take it under consideration.

Note that the internal choice of SCM tool and the external one are two
different problems. External tool choice problem is (probably much)
easier, basically anything (even CVS/cvsup or old fashioned snapshots)
should fill the requirements.

Maybe what would help, would be to have identifiers for the snapshots
inside the code to be able to determine which release/snapshot has a
specific issue and which have not.

Last point, as already discussed here, this has been done with svn for
Debian (BTW, packages just entered unstable, congrats!), but I don''t
know what additional amount of work would be needed to integrate the
snapshots (some packages already do this, e.g. emacs-snapshot).

-- 
solofo

Thanks, Solofo. I''m forward this also to our architecture group, who  
I''m certain will be thankful for the additional input.

Cheers,
Bojanic

On 2007-02-20, at 6:04 , Solofo.Ramangalahy@bull.net wrote:
>
>>> I would like to put in a suggestion for Mercurial instead of SVN.  
>>> This
>>> also might make for an easier transition since it looks like the
>>> FreeBSD
>>> guys made a CVS to Mercurial mirror program they are using to
>>> mirror the
>>> FreeBSD CVS.
>>>
>>> http://hg.fr.freebsd.org/
>
> Thanks for the reminder, I''d like to add that this work was very
well
> documented in:
> http://ns2.freenix.org/~roberto/paper.pdf
> http://ns2.freenix.org/~roberto/slides.pdf
> nice to see that it''s in place.
>
> Another enlightening discussion is the xorg move from CVS to git:
> http://keithp.com/blog/Repository_Formats_Matter.html
>
> Of course svn is the most "natural" (smooth) migration path from
CVS:
> it has been made for this.
>
>> Thanks for the suggestion. I''ve passed this onto our
architecture
>> team to ensure that they take it under consideration.
>
> Note that the internal choice of SCM tool and the external one are two
> different problems. External tool choice problem is (probably much)
> easier, basically anything (even CVS/cvsup or old fashioned snapshots)
> should fill the requirements.
>
> Maybe what would help, would be to have identifiers for the snapshots
> inside the code to be able to determine which release/snapshot has a
> specific issue and which have not.
>
> Last point, as already discussed here, this has been done with svn for
> Debian (BTW, packages just entered unstable, congrats!), but I
don''t
> know what additional amount of work would be needed to integrate the
> snapshots (some packages already do this, e.g. emacs-snapshot).
>
> -- 
> solofo

On Fri, Feb 16, 2007 at 09:47:02PM -0600, Troy Benjegerdes wrote:
> > 
> > Further, as I explained at the Lustre.org meeting at SC''06,
we have
> > been contemplating a move to subversion and moving our entire source  
> > code repository to the outside. But, since we have some non-GPL  
> > libraries and sources in our tree related to other projects, this is  
> > not a trivial transition for us. I can''t make any firm
commitments on
> > a date at this time.
> 
> I would like to put in a suggestion for Mercurial instead of SVN. This
> also might make for an easier transition since it looks like the FreeBSD
> guys made a CVS to Mercurial mirror program they are using to mirror the
> FreeBSD CVS.  
> 
> http://hg.fr.freebsd.org/
> 
> Maybe one of the paying customers could look at setting this up...
> 
perhaps the transition to `git'' might be more suited to lustre, since
the linux kernel devs are using it, and there are various patches to the
kernel anyway and most who have played around with the newer
developmental kernels and other patches for things like openib are
probably using git as well, it seems like a natural choice.



Jimmy.

-- 
Jimmy Tang
Trinity Centre for High Performance Computing,
Lloyd Building, Trinity College Dublin, Dublin 2, Ireland.
http://www.tchpc.tcd.ie/ | http://www.tchpc.tcd.ie/~jtang

> perhaps the transition to `git'' might be more suited to lustre,
since
> the linux kernel devs are using it, and there are various patches to the
> kernel anyway and most who have played around with the newer
> developmental kernels and other patches for things like openib are
> probably using git as well, it seems like a natural choice.
I would also suggest git. I''ve used it in many different situations
and with many different development styles. There is a cvs daemon to
make a more cvs-like interface to a git repository (helps with
transition).  My experience primarily comes from working with the
Source Mage GNU/Linux project (www.sourcemage.org) who recently moved
all their development to git. We ended up going through a half dozen
SCM''s before going to git.

IRC help is also very available and on irc.freenode.net #git

- David Brown

> I would also suggest git. I''ve used it in many different
situations
> and with many different development styles. There is a cvs daemon to
> make a more cvs-like interface to a git repository (helps with
> transition).  My experience primarily comes from working with the
> Source Mage GNU/Linux project (www.sourcemage.org) who recently moved
> all their development to git. We ended up going through a half dozen
> SCM''s before going to git.
>
> IRC help is also very available and on irc.freenode.net #git
Here''s the start of one of our threads on the mailing list for scm
discussion.

http://lists.ibiblio.org/pipermail/sm-discuss/2006-May/014508.html

- David Brown