llvm dev - Jul 2016 - [RFC] One or many git repositories?

On 22 Jul 2016, at 07:14, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> Just tried to set it up there: https://github.com/joker-eph/llvm-unified
> (git log —follow is working fine with this setup).
> 
> While it preserves the history fine (I.e. the hashes are identical to the
current git), it has a drawback: there isn’t anymore a common ancestor for the
parents of the merge (this may or may not be an issue in practice, not sure yet,
but it is uncommon for git).
Perhaps I’m missing something, but on GitHub I can’t see any of the history. 
Assuming that this is just a GitHub UI issue, can you explain the steps for a
downstream user who has clones of the llvm and clang repos that were forked at
some point in the past, have been periodically merged, and contain a load of
commits on top of upstream would migrate to using this?

David

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 3719 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160722/a19c4213/attachment.bin>

> On Jul 22, 2016, at 12:31 AM, David Chisnall <David.Chisnall at
cl.cam.ac.uk> wrote:
> 
> On 22 Jul 2016, at 07:14, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>> 
>> Just tried to set it up there:
https://github.com/joker-eph/llvm-unified
>> (git log —follow is working fine with this setup).
>> 
>> While it preserves the history fine (I.e. the hashes are identical to
the current git), it has a drawback: there isn’t anymore a common ancestor for
the parents of the merge (this may or may not be an issue in practice, not sure
yet, but it is uncommon for git).
> 
> Perhaps I’m missing something, but on GitHub I can’t see any of the
history.  Assuming that this is just a GitHub UI issue,
Probably, this should be populated:
https://github.com/joker-eph/llvm-unified/commits/master
History for one file isn’t great on github because it does not follow move:
https://github.com/joker-eph/llvm-unified/commits/master/llvm/CMakeLists.txt

On the command line, `git log —follow llvm/CMakeLists.txt` works fine.
> can you explain the steps for a downstream user who has clones of the llvm
and clang repos that were forked at some point in the past, have been
periodically merged, and contain a load of commits on top of upstream would
migrate to using this?
Taking cheri as an example, it seems you have a repo for clang and one for llvm
(https://github.com/CTSRD-CHERI/clang and https://github.com/CTSRD-CHERI/llvm)

You can create the unified repo for cheri quite easily, it is roughly doing
this:

1) checkout llvm, move everything to a llvm subdir, commit
2) checkout clang, move everything to a clang subdir, commit
3) merge clang/master into llvm/master.
4) done.

From now you should be able to merge the unified repo (the first merge may
require extra git-fu)

I just tried the attached script to unify cheri, it takes a few minutes to fetch
from github, and a few seconds to merge. To be used as `./unify-cheri.sh
cheri-unified` (it’ll create a directory cheri-unified, fetch and merge).

-------------- next part --------------
A non-text attachment was scrubbed...
Name: unify-cheri.sh
Type: application/octet-stream
Size: 1104 bytes
Desc: not available
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20160722/be68a610/attachment.obj>
-------------- next part --------------



The only advantages of this approach over Justin’s script is to preserve the
history of the existing official git repositories and yours, including not
changing the hashes (t may or may not matter that much).


— 
Mehdi

> On Jul 22, 2016, at 7:28 PM, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
> 
> 
>> On Jul 22, 2016, at 12:31 AM, David Chisnall <David.Chisnall at
cl.cam.ac.uk> wrote:
>> 
>> On 22 Jul 2016, at 07:14, Mehdi Amini via llvm-dev <llvm-dev at
lists.llvm.org> wrote:
>>> 
>>> Just tried to set it up there:
https://github.com/joker-eph/llvm-unified
>>> (git log —follow is working fine with this setup).
>>> 
>>> While it preserves the history fine (I.e. the hashes are identical
to the current git), it has a drawback: there isn’t anymore a common ancestor
for the parents of the merge (this may or may not be an issue in practice, not
sure yet, but it is uncommon for git).
>> 
>> Perhaps I’m missing something, but on GitHub I can’t see any of the
history.  Assuming that this is just a GitHub UI issue,
> 
> Probably, this should be populated:
https://github.com/joker-eph/llvm-unified/commits/master
> History for one file isn’t great on github because it does not follow move:
https://github.com/joker-eph/llvm-unified/commits/master/llvm/CMakeLists.txt
> 
> On the command line, `git log —follow llvm/CMakeLists.txt` works fine.
> 
>> can you explain the steps for a downstream user who has clones of the
llvm and clang repos that were forked at some point in the past, have been
periodically merged, and contain a load of commits on top of upstream would
migrate to using this?
> 
> Taking cheri as an example, it seems you have a repo for clang and one for
llvm (https://github.com/CTSRD-CHERI/clang and
https://github.com/CTSRD-CHERI/llvm)
> 
> You can create the unified repo for cheri quite easily, it is roughly doing
this:
> 
> 1) checkout llvm, move everything to a llvm subdir, commit
> 2) checkout clang, move everything to a clang subdir, commit
> 3) merge clang/master into llvm/master.
> 4) done.
> 
> From now you should be able to merge the unified repo (the first merge may
require extra git-fu)
> 
> I just tried the attached script to unify cheri
I forgot to mention that I pushed the result there:
https://github.com/joker-eph/llvm-unified/tree/cheri-unified
(History: https://github.com/joker-eph/llvm-unified/commits/cheri-unified )

— 
Mehdi




> , it takes a few minutes to fetch from github, and a few seconds to merge.
To be used as `./unify-cheri.sh cheri-unified` (it’ll create a directory
cheri-unified, fetch and merge).
> 
> <unify-cheri.sh>
> 
> 
> The only advantages of this approach over Justin’s script is to preserve
the history of the existing official git repositories and yours, including not
changing the hashes (t may or may not matter that much).
> 
> 
> — 
> Mehdi
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev