llvm dev - Feb 2016 - RFC: Move the test-suite LLVM project to GitHub?

My biggest concerns and care-abouts are largely what Matthias expressed 
below.
Most of them have been addressed already further down the thread.

I hope that the move to github in practice would mean that the only 
difference is that I 'git clone https://github...' instead of 'git
clone
http://llvm.org/git/test-suite.git' ?

Thanks,

Kristof

On 24/02/2016 22:25, Matthias Braun via llvm-dev wrote:
> I don't really care where the repository is located, but I do have 
> some comments on the future test-suite directions:
>
>> On Feb 24, 2016, at 12:57 PM, Chandler Carruth via llvm-dev 
>> <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>
>> Subject kinda says it all. Here is my rationale:
>>
>> The test-suite is really weird relative to the rest of the LLVM
project:
>> 1) It contains all manner of crazily licensed code.
> That's indeed a good reason to move the repository away.
>
>> 2) We don't really care about the history at all. Any concerns
around
>> linear history or bisection are pretty much irrelevant.
> We do care about the history. Sometimes benchmarks get fixed or 
> tweaked which may change the results, we should be able to dig into 
> the history to see what happened when. In any way retaining the 
> history wouldn't be a problem, would it?
>
>> 3) We don't ever plan to have LLVM code move into or out from the 
>> test-suite
> I could actually see moving llvm code into the test-suite (we already 
> use lit code from llvm) but indeed move code out of the testsuite into 
> llvm I don't foresee happening.
>
>> 4) Its already big, and really should be much bigger. We shouldn't 
>> have incentives to keep stuff out of the test suite because of size, 
>> hosting cost, or anything else.
> I agree with the goal of having a big test-suite. However I think 
> there is a point where we should rather strive to have a stable base 
> system for building and running tests, etc. and then have the actual 
> benchmarks/tests being modules on top of that. We already have that 
> situation today with External/SPEC* and I think it would be a good 
> idea to have a mode where you just checkout more benchmarks into a 
> test-suite subdirectory and they are automatically recognized and used 
> (in fact that is something on my TODO list though at a very low position).
>
> - Matthias
>
>>
>> For all of these reasons, and also because I'd like to see how well
>> (or rather, how poorly) a service like GitHub actually works for the 
>> project, it seems like splitting the test-suite out of the current 
>> subversion repository and moving it there is the right call.
>>
>> When I chatted with folks on the board, this made sense to them as 
>> well, and I've made sure we have a reasonable LLVM organization set
>> up on GitHub and all the board members are on it: 
>> https://github.com/llvm (I think only my membership is public at the 
>> moment).
>>
>> There is still plenty to figure out about how to manage this on 
>> github, but before doing anything else I just wanted to shoot an 
>> email and see if folks like this idea.
>>
>> Thanks!
>> -Chandler
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>
>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160225/38d2e450/attachment.html>

Yes, I would hope that for most this boils down to cloning a different URL.
Except for folks pushing patches to the test suite, I'm moderately
confident there would be no other difference.

It's the pushing patches side that would need to be sorted out in more
detail.

On Thu, Feb 25, 2016 at 12:48 AM Kristof Beyls <kristof.beyls at arm.com>
wrote:
> My biggest concerns and care-abouts are largely what Matthias expressed
> below.
> Most of them have been addressed already further down the thread.
>
> I hope that the move to github in practice would mean that the only
> difference is that I 'git clone https://github...' instead of
'git clone
> http://llvm.org/git/test-suite.git' ?
>
> Thanks,
>
> Kristof
>
>
> On 24/02/2016 22:25, Matthias Braun via llvm-dev wrote:
>
> I don't really care where the repository is located, but I do have some
> comments on the future test-suite directions:
>
> On Feb 24, 2016, at 12:57 PM, Chandler Carruth via llvm-dev <
> llvm-dev at lists.llvm.org> wrote:
>
> Subject kinda says it all. Here is my rationale:
>
> The test-suite is really weird relative to the rest of the LLVM project:
> 1) It contains all manner of crazily licensed code.
>
> That's indeed a good reason to move the repository away.
>
> 2) We don't really care about the history at all. Any concerns around
> linear history or bisection are pretty much irrelevant.
>
> We do care about the history. Sometimes benchmarks get fixed or tweaked
> which may change the results, we should be able to dig into the history to
> see what happened when. In any way retaining the history wouldn't be a
> problem, would it?
>
> 3) We don't ever plan to have LLVM code move into or out from the
> test-suite
>
> I could actually see moving llvm code into the test-suite (we already use
> lit code from llvm) but indeed move code out of the testsuite into llvm I
> don't foresee happening.
>
> 4) Its already big, and really should be much bigger. We shouldn't have
> incentives to keep stuff out of the test suite because of size, hosting
> cost, or anything else.
>
> I agree with the goal of having a big test-suite. However I think there is
> a point where we should rather strive to have a stable base system for
> building and running tests, etc. and then have the actual benchmarks/tests
> being modules on top of that. We already have that situation today with
> External/SPEC* and I think it would be a good idea to have a mode where you
> just checkout more benchmarks into a test-suite subdirectory and they are
> automatically recognized and used (in fact that is something on my TODO
> list though at a very low position).
>
> - Matthias
>
>
> For all of these reasons, and also because I'd like to see how well (or
> rather, how poorly) a service like GitHub actually works for the project,
> it seems like splitting the test-suite out of the current subversion
> repository and moving it there is the right call.
>
> When I chatted with folks on the board, this made sense to them as well,
> and I've made sure we have a reasonable LLVM organization set up on
GitHub
> and all the board members are on it: https://github.com/llvm (I think
> only my membership is public at the moment).
>
> There is still plenty to figure out about how to manage this on github,
> but before doing anything else I just wanted to shoot an email and see if
> folks like this idea.
>
> Thanks!
> -Chandler
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
>
> _______________________________________________
> LLVM Developers mailing listllvm-dev at
lists.llvm.orghttp://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160225/b97168e7/attachment.html>

Ah - I do commit to the test-suite from time to time.
So, could the github-based project be set up so that we can just do 'git 
push'? Or would we have to go through the pull requests route on every 
commit?
I'm afraid I've never committed to any github project before, so I am a 
bit confused on how committing would work in practice.
Obviously, I hope for the mechanism to commit to be simple - e.g. just 
do a git push from the command line.

Before making the decision to move the repo, I think this part also 
should be sorted out.

On 25/02/2016 09:50, Chandler Carruth wrote:
> Yes, I would hope that for most this boils down to cloning a different 
> URL. Except for folks pushing patches to the test suite, I'm 
> moderately confident there would be no other difference.
>
> It's the pushing patches side that would need to be sorted out in more 
> detail.
>
> On Thu, Feb 25, 2016 at 12:48 AM Kristof Beyls <kristof.beyls at arm.com
> <mailto:kristof.beyls at arm.com>> wrote:
>
>     My biggest concerns and care-abouts are largely what Matthias
>     expressed below.
>     Most of them have been addressed already further down the thread.
>
>     I hope that the move to github in practice would mean that the
>     only difference is that I 'git clone https://github...' instead
of
>     'git clone http://llvm.org/git/test-suite.git' ?
>
>     Thanks,
>
>     Kristof
>
>
>     On 24/02/2016 22:25, Matthias Braun via llvm-dev wrote:
>>     I don't really care where the repository is located, but I do
>>     have some comments on the future test-suite directions:
>>
>>>     On Feb 24, 2016, at 12:57 PM, Chandler Carruth via llvm-dev
>>>     <llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>> wrote:
>>>
>>>     Subject kinda says it all. Here is my rationale:
>>>
>>>     The test-suite is really weird relative to the rest of the LLVM
>>>     project:
>>>     1) It contains all manner of crazily licensed code.
>>     That's indeed a good reason to move the repository away.
>>
>>>     2) We don't really care about the history at all. Any
concerns
>>>     around linear history or bisection are pretty much irrelevant.
>>     We do care about the history. Sometimes benchmarks get fixed or
>>     tweaked which may change the results, we should be able to dig
>>     into the history to see what happened when. In any way retaining
>>     the history wouldn't be a problem, would it?
>>
>>>     3) We don't ever plan to have LLVM code move into or out
from
>>>     the test-suite
>>     I could actually see moving llvm code into the test-suite (we
>>     already use lit code from llvm) but indeed move code out of the
>>     testsuite into llvm I don't foresee happening.
>>
>>>     4) Its already big, and really should be much bigger. We
>>>     shouldn't have incentives to keep stuff out of the test
suite
>>>     because of size, hosting cost, or anything else.
>>     I agree with the goal of having a big test-suite. However I think
>>     there is a point where we should rather strive to have a stable
>>     base system for building and running tests, etc. and then have
>>     the actual benchmarks/tests being modules on top of that. We
>>     already have that situation today with External/SPEC* and I think
>>     it would be a good idea to have a mode where you just checkout
>>     more benchmarks into a test-suite subdirectory and they are
>>     automatically recognized and used (in fact that is something on
>>     my TODO list though at a very low position).
>>
>>     - Matthias
>>
>>>
>>>     For all of these reasons, and also because I'd like to see
how
>>>     well (or rather, how poorly) a service like GitHub actually
>>>     works for the project, it seems like splitting the test-suite
>>>     out of the current subversion repository and moving it there is
>>>     the right call.
>>>
>>>     When I chatted with folks on the board, this made sense to them
>>>     as well, and I've made sure we have a reasonable LLVM
>>>     organization set up on GitHub and all the board members are on
>>>     it: https://github.com/llvm (I think only my membership is
>>>     public at the moment).
>>>
>>>     There is still plenty to figure out about how to manage this on
>>>     github, but before doing anything else I just wanted to shoot
an
>>>     email and see if folks like this idea.
>>>
>>>     Thanks!
>>>     -Chandler
>>>     _______________________________________________
>>>     LLVM Developers mailing list
>>>     llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>>>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
>>
>>
>>     _______________________________________________
>>     LLVM Developers mailing list
>>     llvm-dev at lists.llvm.org <mailto:llvm-dev at
lists.llvm.org>
>>     http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160225/4020d3a0/attachment.html>