llvm dev - May 2004 - [LLVMdev] Open Source Contributions (was Re: Benchmarks)

On Fri, 2004-04-30 at 10:57, Chris Lattner wrote:
> There is a chance, but as Vikram mentioned, it's not extremely likely
to
> happen in the immediate future.  However, if you or someone else wrote the
> makefiles neccesary to the suite to the llvm/test/Programs hierarchy, we
> would be happy to add them and have our automated testers run them.  :)
> 
> In particular, I'd really like to try to foster some more community
> involvement in all aspects of LLVM, including coding, porting, testing,
> documentation, performance tweaking, ...  Our group obviously has a lot
> invested in LLVM and we will continue doing lots of interesting things,
> but unfortunately we do have limited people-bandwidth.  If you *really*
> want something done (like any open source project), the best way is to do
> it yourself of convince someone to do it for you.
> 
> I'm not try to complain here: we have had a number of high-quality
> contributions from several different people, I just wanted to point out
> that asking us to do something isn't the only way to get it done.  :-)
> 
> -Chris
I'd be happy to contribute more and in fact, there are lots of things I
could and would contribute. Pretty much the only thing that stops me  is
the project's CVS policy. Submitting patches is fine for smaller tasks
(single file bug fixes etc.) but larger tasks (like adding a whole test
suite) really need CVS write access to be done efficiently and
correctly.

Perhaps its just me, but I don't have time to dicker with patch
creation, wait for individual files to be added, processing emails that
let me know when things have been added, wait for further responses on
modifications, etc.  I have a full-time job, a full-time family, and a
full-time second career (erm, perhaps I'm just over-booked? :)  I need
some efficiency if I'm to do anything. From my perspective, the process
is _way_ more efficient if I can just create a branch, do what I need to
do and then tell you "look at branch xyz and merge if you like it".
Its
possible with CVS to restrict access to main line commits to certain
users so you can still maintain control of the main development trend.
Furthermore, providing CVS access should reduce your administrative
burden. Instead of patching a pile of individual files, you can simply
look at the changes introduced by a branch and decide if its something
you want to keep or inform the branch author of the things that need to
be modified. You can also set the "gold standard" for contributions to
make sure that (a) branches are maintained with the mainline by the
author (thereby reducing mainline conflicts on merge), (b) the software
on the branch must build correctly, (c) the branch software must be well
integrated into the build system, (d) the branch software must not break
any existing tests, (e) etc., etc.  

I understand the University has certain legal restrictions about
granting access to non-student and non-faculty users. That may be the
trump card that prevents wider use of CVS by contributors to LLVM. If
that is the case, I would suggest that (a) the project simply accept
that contributions from others will be minimal or (b) move the CVS
repository somewhere that doesn't have the University's restrictions.
That last option, however, may have additional intellectual property
issues.
 
While it would be unwise to freely grant write access to the CVS
repository to anyone that asked for it, you might want to think about
some qualifications necessary to allow that to happen in a controlled
fashion.  I for one don't have any problems being asked to qualify for
CVS write access. If such access were available to serious and capable
contributors, I believe you'd get a lot more contributions (as I've seen
on other projects).  Furthermore, the contributions are generally of a
higher quality because the technical requirements go up. Its something
of a self-sorting process.

Ultimately the decision is yours. LLVM is still great either way. :)

Reid.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20040501/a5ff2573/attachment.sig>

On Sat, 1 May 2004, Reid Spencer wrote:
> I'd be happy to contribute more and in fact, there are lots of things I
> could and would contribute. Pretty much the only thing that stops me  is
> the project's CVS policy.
Wow, I really had no idea that this was such a problem!
> (erm, perhaps I'm just over-booked? :)
Heh, I know a little bit of that feeling ;)
> I need some efficiency if I'm to do anything. From my perspective, the
> process is _way_ more efficient if I can just create a branch, do what I
> need to do and then tell you "look at branch xyz and merge if you like
> it".
Okay, that makes sense.
> Its possible with CVS to restrict access to main line commits to
> certain users so you can still maintain control of the main development
> trend.
The problem is that CVS was not at all designed for this.  It certainly is
possible to hack it enough with the right set of scripts though, and
several projects have them.
> Furthermore, providing CVS access should reduce your administrative
> burden. Instead of patching a pile of individual files, you can simply
> look at the changes introduced by a branch and decide if its something
> you want to keep or inform the branch author of the things that need to
> be modified.
That would certainly be nice.  :)
> I understand the University has certain legal restrictions about
> granting access to non-student and non-faculty users. That may be the
> trump card that prevents wider use of CVS by contributors to LLVM.
Unfortunately this is a big problem.
> If that is the case, I would suggest that (a) the project simply accept
> that contributions from others will be minimal or
Hrm, that's not very attractive :)
> (b) move the CVS repository somewhere that doesn't have the
University's
> restrictions. That last option, however, may have additional
> intellectual property issues.
I don't think that there would be IP issues: LLVM is (effectively) BSD
licensed, so it could be forked at any time without a problem.  This would
obviously be very bad for LLVM, but it's possible.

The more I've thought about this, the more that I'm beginning to realize
that CVS is the root of the problem.  Perhaps it is time for LLVM to
seriously start looking at switching over to a decentralized version
control system?  I really am not "up" on the various options, but
I've
heard rumars that there are now several good options.

Take 'arch' for example: its approach seems like it would solve almost
all
of the version control issues that we are facing, and supports
decentralized development in particular.  From what I understand, you
would be able to do all of your development on your own "local"
branch,
others could have access to it, and when it's ready, we could pull it in
as one big patch or set of changes.

On Sat, 2004-05-01 at 20:57, Chris Lattner wrote:
> > (b) move the CVS repository somewhere that doesn't have the
University's
> > restrictions. That last option, however, may have additional
> > intellectual property issues.
> 
> I don't think that there would be IP issues: LLVM is (effectively) BSD
> licensed, so it could be forked at any time without a problem.  This would
> obviously be very bad for LLVM, but it's possible.
I don't see moving the repository to another system and forking the code
base as equivalent. I agree that its definitely not time to fork the
code base. But we can change the source code control repository without
forking.
> 
> The more I've thought about this, the more that I'm beginning to
realize
> that CVS is the root of the problem.  Perhaps it is time for LLVM to
> seriously start looking at switching over to a decentralized version
> control system?  I really am not "up" on the various options, but
I've
> heard rumars that there are now several good options.
> 
> Take 'arch' for example: its approach seems like it would solve
almost all
> of the version control issues that we are facing, and supports
> decentralized development in particular.  From what I understand, you
> would be able to do all of your development on your own "local"
branch,
> others could have access to it, and when it's ready, we could pull it
in
> as one big patch or set of changes.
I've looked at subversion recently. Setting it up wasn't a big deal but
its much more complicated than CVS (e.g. you have to get a specific
version of Berkley DB). Also, there are enough problems with it that I
deem it unstable. The last thing we need is a buggy SCC system. I think
Subversion is a good choice (functionality wise), it just isn't quite
ready yet. Perhaps by release 1.2 or so the main issues will have
settled down. I haven't looked at arch but I will.
> 
> >From your perspective, would this solve the problem?  I used arch just
as
> an example, I'm sure there are others (bitkeeper at least supports
these
> features, but has unattractive licensing issues).
The main problem isn't the SCC tool that we use. They all have their
good and bad points. The main problem is not being able to check things
in to a branch. As long as the SCC tool supports distributed and
parallel development (i.e. is WAN network based and supports branches),
it will be fine. So, I'm rejecting SCCS and RCS but not CVS. 
> 
> If it is really time to switch version control systems we probably should
> have someone do some research and find out which one is the most
> appropriate.  Assuming that we can come up with a reasonable transition
> phase, I think that this could be done.
Sounds right. Its not a decision to be made lightly.

Reid.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20040502/dfc68e91/attachment.sig>

Reid,

There are no IP issues or restrictions I know of that prevent us from 
accepting contributions or providing direct CVS write access to 
non-UIUC people.  If we can solve the technical issues, Chris and I 
would both be in favor of making write access available, in some 
controlled way.  (As Chris said, I think it would be really unfortunate 
if we had to fork off the CVS repository but again, inside or outside 
the UIUC domain is a non-issue.)

Replacing CVS with something else may be our only option, but I would 
want to make sure that it is as widely available as CVS, so that 
prospective users don't have to download and install version management 
sofware to use llvm.

--Vikram
http://www.cs.uiuc.edu/~vadve
http://llvm.cs.uiuc.edu/


On May 1, 2004, at 8:03 PM, Reid Spencer wrote:
> On Fri, 2004-04-30 at 10:57, Chris Lattner wrote:
>> There is a chance, but as Vikram mentioned, it's not extremely
likely
>> to
>> happen in the immediate future.  However, if you or someone else 
>> wrote the
>> makefiles neccesary to the suite to the llvm/test/Programs hierarchy, 
>> we
>> would be happy to add them and have our automated testers run them.  
>> :)
>>
>> In particular, I'd really like to try to foster some more community
>> involvement in all aspects of LLVM, including coding, porting, 
>> testing,
>> documentation, performance tweaking, ...  Our group obviously has a 
>> lot
>> invested in LLVM and we will continue doing lots of interesting 
>> things,
>> but unfortunately we do have limited people-bandwidth.  If you 
>> *really*
>> want something done (like any open source project), the best way is 
>> to do
>> it yourself of convince someone to do it for you.
>>
>> I'm not try to complain here: we have had a number of high-quality
>> contributions from several different people, I just wanted to point 
>> out
>> that asking us to do something isn't the only way to get it done. 
:-)
>>
>> -Chris
>
> I'd be happy to contribute more and in fact, there are lots of things I
> could and would contribute. Pretty much the only thing that stops me  
> is
> the project's CVS policy. Submitting patches is fine for smaller tasks
> (single file bug fixes etc.) but larger tasks (like adding a whole test
> suite) really need CVS write access to be done efficiently and
> correctly.
>
> Perhaps its just me, but I don't have time to dicker with patch
> creation, wait for individual files to be added, processing emails that
> let me know when things have been added, wait for further responses on
> modifications, etc.  I have a full-time job, a full-time family, and a
> full-time second career (erm, perhaps I'm just over-booked? :)  I need
> some efficiency if I'm to do anything. From my perspective, the process
> is _way_ more efficient if I can just create a branch, do what I need 
> to
> do and then tell you "look at branch xyz and merge if you like
it". Its
> possible with CVS to restrict access to main line commits to certain
> users so you can still maintain control of the main development trend.
> Furthermore, providing CVS access should reduce your administrative
> burden. Instead of patching a pile of individual files, you can simply
> look at the changes introduced by a branch and decide if its something
> you want to keep or inform the branch author of the things that need to
> be modified. You can also set the "gold standard" for
contributions to
> make sure that (a) branches are maintained with the mainline by the
> author (thereby reducing mainline conflicts on merge), (b) the software
> on the branch must build correctly, (c) the branch software must be 
> well
> integrated into the build system, (d) the branch software must not 
> break
> any existing tests, (e) etc., etc.
>
> I understand the University has certain legal restrictions about
> granting access to non-student and non-faculty users. That may be the
> trump card that prevents wider use of CVS by contributors to LLVM. If
> that is the case, I would suggest that (a) the project simply accept
> that contributions from others will be minimal or (b) move the CVS
> repository somewhere that doesn't have the University's
restrictions.
> That last option, however, may have additional intellectual property
> issues.
>
> While it would be unwise to freely grant write access to the CVS
> repository to anyone that asked for it, you might want to think about
> some qualifications necessary to allow that to happen in a controlled
> fashion.  I for one don't have any problems being asked to qualify for
> CVS write access. If such access were available to serious and capable
> contributors, I believe you'd get a lot more contributions (as I've
> seen
> on other projects).  Furthermore, the contributions are generally of a
> higher quality because the technical requirements go up. Its something
> of a self-sorting process.
>
> Ultimately the decision is yours. LLVM is still great either way. :)
>
> Reid.

On Sun, 2004-05-02 at 08:26, Vikram S. Adve wrote:
> Reid,
> 
> There are no IP issues or restrictions I know of that prevent us from 
> accepting contributions or providing direct CVS write access to 
> non-UIUC people. 
That's good to know. When I originally asked for CVS write access (last
fall), the issue was raised that an account would have to be created on
a UIUC machine and that current policy does not permit that for non-UIUC
people. 
>  If we can solve the technical issues, Chris and I 
> would both be in favor of making write access available, in some 
> controlled way. 
Please note that it is possible with recent versions of CVS to provide
password protected access without creating a system account.  This
facility, pserver, is already set up, and is how anonymous CVS access
works now. All that needs to be done is to add some users to the
$CVSROOT/CVSROOT/passwd file. You may additionally need to set up a
separate real (Unix) user to control access. For example "anoncvs" and
"pubcvs" are common to provide read-only and read-write access to the
CVS repository, respectively.  
>  (As Chris said, I think it would be really unfortunate 
> if we had to fork off the CVS repository but again, inside or outside 
> the UIUC domain is a non-issue.)
The issues of forking LLVM code base and providing write access to CVS
are orthogonal. I fully agree that forking LLVM at this time would be
bad. Providing write access to the repository would be good. :) 
> 
> Replacing CVS with something else may be our only option, but I would 
> want to make sure that it is as widely available as CVS, so that 
> prospective users don't have to download and install version management
> sofware to use llvm.
I wouldn't vote for replacing CVS at this time. Yes, its old and clunky
but it gets the job done and I'm sure you could live without the project
interruption right now. 

In my opinion, Subversion is well suited to open source development and
is "familiar" for CVS users. However, moving to Subversion at this
time
would be premature. It is just recently at version 1.0 and needs a
little time (few months perhaps) to settle down and get the kinks out.

Perhaps it would be prudent to wait for summer before thinking about
switching version control systems. The impact will be less and
Subversion should be in better shape by then.  In the mean time,
providing write access to existing CVS server sounds doable both policy
wise and technically.

Reid.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20040502/a8bbbb37/attachment.sig>

Chris Lattner wrote:
> The more I've thought about this, the more that I'm beginning to
realize
> that CVS is the root of the problem.  Perhaps it is time for LLVM to
> seriously start looking at switching over to a decentralized version
> control system?  I really am not "up" on the various options, but
I've
> heard rumars that there are now several good options.
>
> Take 'arch' for example: its approach seems like it would solve
almost all
> of the version control issues that we are facing, and supports
> decentralized development in particular.  From what I understand, you
> would be able to do all of your development on your own "local"
branch,
> others could have access to it, and when it's ready, we could pull it
in
> as one big patch or set of changes.
There are a couple of problems. First, arch is not portable to Windows. Are 
you really sure nobody will port ALVA (or parts of it) to that platform?

Second, local repository is fine, but what if two persons ever decide to work 
on the same branch?

- Volodya