llvm dev - Sep 2011 - [LLVMdev] git Status

> On Aug 18, 2011, at 10:57 AM, David Greene wrote:
>>
>> Did the project ever come to a decision about making a transition to
>> git?  I'm trying to do some longer-term planning and it would be
helpful
>> to know what the roadmap is.
It's stuck on:

1) A misunderstanding that global revision numbers are necessary and 
that 'git describe' along with frequent tagging (commit hooks) isn't
good enough.

2) Nobody writing up how git should be used with the current llvm 
workflow (which is not going to adapt to an SCM, but the other way 
around, which is understandable.)

On Aug 19, 2011, at 2:57 AM, FlyLanguage wrote:
>> On Aug 18, 2011, at 10:57 AM, David Greene wrote:
>>> 
>>> Did the project ever come to a decision about making a transition
to
>>> git?  I'm trying to do some longer-term planning and it would
be helpful
>>> to know what the roadmap is.
> 
> It's stuck on:
> 
> 1) A misunderstanding that global revision numbers are necessary and 
> that 'git describe' along with frequent tagging (commit hooks)
isn't
> good enough.
> 
> 2) Nobody writing up how git should be used with the current llvm 
> workflow (which is not going to adapt to an SCM, but the other way 
> around, which is understandable.)
>From chats with some llvm/clang developers at Apple, we're also stuck
on:
3) Somebody doing all of the thankless infrastructure work to ensure that we
don't regress on basic things like the web interface, post-commit email
hooks, etc.

4) Having a few stakeholders audit the conversion process to ensure that
"we do it right".

5) There are organizations that use the current git-svn bridge extensively to
maintain long term topic branches before proposing/integrating them into the
public repository. If the conversion changes the SHA1 of commits (and it likely
will due to goal #4), then these organizations will need time to prepare for the
conversion (to avoid unscheduled downtime and/or any loss of history).

davez

FlyLanguage <flylanguage at gmail.com> writes:
> 1) A misunderstanding that global revision numbers are necessary and 
> that 'git describe' along with frequent tagging (commit hooks)
isn't
> good enough.
I'm not sure what's needed here.  I don't care about global revision
numbers at all so I can't really comment on what should be used to
replace them with git.  Hopefully someone else who cares can chime in.
> 2) Nobody writing up how git should be used with the current llvm 
> workflow (which is not going to adapt to an SCM, but the other way 
> around, which is understandable.)
I'm willing to take a crack at this as I have to do something like this
already anyway.  I won't be able to get to it until next week, though.

                                -Dave

Dave Zarzycki <zarzycki at apple.com> writes:
>>From chats with some llvm/clang developers at Apple, we're also
stuck on:
>
> 3) Somebody doing all of the thankless infrastructure work to ensure
> that we don't regress on basic things like the web interface,
> post-commit email hooks, etc.
It's certainly a concern.  Is Apple the only entity that has access to
this stuff or can others help?
> 4) Having a few stakeholders audit the conversion process to ensure
> that "we do it right".
Has anyone been identified to play this role?  I haven't seen any
requests go out on the list for volunteers.
> 5) There are organizations that use the current git-svn bridge
> extensively to maintain long term topic branches before
> proposing/integrating them into the public repository. If the
> conversion changes the SHA1 of commits (and it likely will due to goal
> #4), then these organizations will need time to prepare for the
> conversion (to avoid unscheduled downtime and/or any loss of history).
I am probably not one of the organizations you're thinking of, but I
currently do a lot of this and am willing to throw away everything I
have to move to git proper.  It's that much better.

                             -Dave

FlyLanguage <flylanguage at gmail.com> writes:
> 2) Nobody writing up how git should be used with the current llvm 
> workflow (which is not going to adapt to an SCM, but the other way 
> around, which is understandable.)
Here is a first cut at that.  Other git users, please chime in with
suggestions, edits, etc.  Non-git users, please ask for clarification
where needed.  This is based on my notes on working with LLVM via
git-svn, modified to assume native git.  There are hundreds of ways to
design a workflow that works with the current review process.  This is
but one.

We should add this to the web page once it's polished if we make the
transition to git.

                         -Dave

-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: LLVMgit.txt
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20110822/73f364d4/attachment.txt>

David,

A few comments.

> Naming Upstream
> ---------------
>
> The intial clone from upstream results in a git remote reference with
> the rather unhelpful name of "origin."  As more remote sources
get
> added, it is easy to forget what "origin" is.  Therefore, add a
remote
> with a more descriptive name.
>
> git remote add llvm-upstream http://llvm.org/git/llvm.git master
>
>
If the intent is to rename origin, this can be done directly:
  git remote rename origin llvm-upstream


> Updating LLVM - no local changes
> --------------------------------
>
>
Splitting this into "no local changes" / "with local
changes" seems
unnecessarily complicated. Why not just recommend doing 'git pull
--rebase'
all of the time?

I have to ask as well - is a linear history really desired? That seems to be
the intent of your instructions.

I ask because for something like a reasonably sized feature that might have
multiple commits, having the merge history in place can be useful, if only
to separate the set of related changes from sets of unrelated changes. It
also aids in reverting an entire feature composed of multiple commits.

One other comment - the way I use git (and from what I've read I am not
alone), I end up committing multiple times per hour / many times per day,
often into different branches. I then go back and use rebase to reorder and
squash commits into logically related bite-sized chunks (e.g. combine some
commits related to feature A into one or more commits, some related to
feature B into one or more commits, and then there might be three bug fix
commits for bugs C, D, and E.). It might be helpful to add some guidelines
related to this that are in line with current LLVM review process - somewhat
the reverse scenario of someone asking for commits to be split. In this
case, they probably don't want to see 15 different commits that in total add
75 lines of code to feature A.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20110822/f6c37ff0/attachment.html>

On 08/22/2011 09:37 PM, David A. Greene wrote:
> FlyLanguage<flylanguage at gmail.com>  writes:
>
>> >  2) Nobody writing up how git should be used with the current llvm
>> >  workflow (which is not going to adapt to an SCM, but the other
way
>> >  around, which is understandable.)
> Here is a first cut at that.  Other git users, please chime in with
> suggestions, edits, etc.  Non-git users, please ask for clarification
> where needed.  This is based on my notes on working with LLVM via
> git-svn, modified to assume native git.  There are hundreds of ways to
> design a workflow that works with the current review process.  This is
> but one.
>
> We should add this to the web page once it's polished if we make the
> transition to git.
>
>                           -Dave
Hi Dave,

thanks a lot. This already reads very nice. Two smaller
comments:
> Sending Patches for Review
> --------------------------
>
> git includes a whole set of tools for managing the patch review
> process.  We kick things off with git format-patch:
>
> git format-patch -o $HOME/patches/ifconvert --thread --src-prefix=old/ \
>                   --dst-prefix=new/ --cover-letter HEAD~1..HEAD
I personally dislike typing all the time such a long command line. Maybe 
you can also point out, how to configure this in .git/config.
> This places three text files in $HOME/patches/ifconvert, one for each
> commit, plus a cover letter to send before each patch.  These will get
> sent to the e-mail list with the subject "[PATCH n/2]<commit
> subject>" where "n" is the patch number (0 for the cover
letter) and
> <commit subject>  is the first line of the commit message.
>
> Edit these files to add any commentary you desire.
>
> It is helpful for format-patch and send-email to have various bits of
> information pre-selected for e-mail interaction.  For example, I put
> this in my .git/config file in the local repository:
>
> [format]
>          numbered = auto
>          to =llvm-commits at cs.uiuc.edu
>          inline = "---------"
I believe the current policy is not to inline patches, but to attach 
them. I believe we should keep following this policy to reduce the 
changes of this transition.
> Updating Patches
> ----------------
>
> Your patches will probably require some editing.  git rebase -i and
> git add -i are your friends.
>
> For the typical case of editing your patches a bit, use git rebase -i:
>
> git rebase -i HEAD~2
>
> This brings up an editor with a document that looks something like
> this:
>
>    pick ef723 Start if conversion work
>    pick 443de Middle of if conversion work, something interesting to commit
>
> This is a control file you edit to state how git-rebase should work.
> There are essentially three commands pick, edit and squash.
You missed the one I use most: 'fixup'. I use a different approach to 
edit patches. Here the text, feel free to add parts of it or to ignore 
it for the sake of simplicity.

-------------------
For the typical case of editing your patches a bit, use git rebase -i:

Let's assume you have the following two patches in your queue:

$ git log --oneline
ef723 Start if conversion work
443de Middle of if conversion work, something interesting to commit

To fix some problems with commit 'Start if conversion work', add your 
fixes directly to the working copy and commit them as new changeset 
'Fixes for: Start if conversion work'.

Now we call

$ git rebase -i HEAD~2

This gives us the following text file:

pick ef723 Start if conversion work
pick 443de Middle of if conversion work, something interesting to commit
pick 33ed3 Fixes for: Start if conversion work

Here, we move the fixup right after the commit to fix and replace the 
default action 'pick' by the comment 'fixup'.

pick ef723 Start if conversion work
fixup 33ed3 Fixes for: Start if conversion work
pick 443de Middle of if conversion work, something interesting to commit

By saving and closing this file, the patches will be merged and
there will be a new history.

sdd24 Start if conversion work
eba32 Middle of if conversion work, something interesting to commit

Commit sdd24 now contains the changes of both ef723 and 443de. In case 
you want to edit the commit message of the commit to fix, you can also 
use 'squash' instead of 'fixup'.
---------------------------

Cheers
Tobi

greened at obbligato.org (David A. Greene) writes:
> Updating LLVM - no local changes
> --------------------------------
>
> To update your clone to the latest LLVM sources, use git pull:
>
> git pull llvm-upstream
That should probably be either

  git pull

or

  git pull llvm-upstream master

since the first is shorter and works if llvm-upstream is the upstream
for the current branch, and "git pull llvm-upstream" will ask for a
branch name if it isn't.
> git cherry-pick 44ef3
> git cherry-pick afe3d
>
> The dag looks like this:
>
> ...-A---E-F{master}[HEAD]
>      \ / /
>       B-C-D{ifconvert}
Err, no. cherry-pick won't record B and C as parents.

A common convention would be to call E and F as B' and C' to reflect the
fact that they are different, but similar to B and C.

Also, if you are to draw DAGs in your explanations (which is good), you
should mention gitk and/or git log --oneline --graph so that users can
experiment by themselves.
> git rebase llvm-upstream/master
>
> There may be conflicts from this operation.  If so, resolve them in
> the usual way.
... and use "git rebase --continue" as needed.

Perhaps point to git-rebase(1) for more details.
> Now our history looks like this:
>
> ...-A-----G-H-I-E'-F'{master}[HEAD]
>      \   /
>       B-C-D{ifconvert}
I don't get it. B and C were local commits, and you didn't explain the
user how to push them, so they're still local. As I Understand It, G and
H are the remote commits you've just fetched, so G cannot have C as
parent.

You probably wanted to show this history after fetch

      G-H{llvm-uptream}
     /
...-A--E-F{master}[HEAD]

and this one after rebase

...-A-G-H-E'-F'{master}[HEAD]
    
> git format-patch -o $HOME/patches/ifconvert --thread --src-prefix=old/ \
>                  --dst-prefix=new/ --cover-letter HEAD~1..HEAD
I usually use git send-email right away, it can call format-patch in
modern Gits.
> Edit these files to add any commentary you desire.
... comments that are not meant to appear in the commit message should
be added after the --- and before the diffstat in the patches.

?
> [format]
>         numbered = auto
>         thread = shallow
these are the default, so better not annoy new users with them.
> git rebase -i HEAD~2
I usually do

git rebase -i origin/master

or in this case

git rebase -i llvm-upstream/master

which has the advantage of offering me to edit local commits, and only
them. I cannot rewrite upstream history by mistake (which would lead to
weird things afterwards).
> In this case, let's say that the second commit needs some work.  We
> edit the control file to do that:
>
>   pick ef723 Start if conversion work
>   edit 443de Middle of if conversion work, something interesting to commit
It's not the best example: rebase is not needed at all to edit the last
commit. You should edit the first to make the example more relevant.
> git reset --soft HEAD^
> git commit -a -c ORIG_HEAD
That seems to be a rather complex and potentially dangerous way of
saying

git commit --amend
> git reset --mixed HEAD^
>
> Your files are left in the "untracked files" 
... or "changed, but not staged for commit"
> state
> Now commit:
>
> git push llvm-upstream master
"Now commit" is confusing. If you talk to former SVN users, you can
say
"now, do the equivalent of svn commit", and otherwise, "Now, send
your
changes to the upstream repository".

My 2 (perhaps 3?) cents,

-- 
Matthieu Moy
http://www-verimag.imag.fr/~moy/

>> 2) Nobody writing up how git should be used with the current llvm
>> workflow (which is not going to adapt to an SCM, but the other way
>> around, which is understandable.)
>
> Here is a first cut at that.  Other git users, please chime in with
> suggestions, edits, etc.  Non-git users, please ask for clarification
> where needed.  This is based on my notes on working with LLVM via
> git-svn, modified to assume native git.  There are hundreds of ways to
> design a workflow that works with the current review process.  This is
> but one.
>
> We should add this to the web page once it's polished if we make the
> transition to git.
Great that you work on this, but I don't quite get the approach, nor 
some of your examples (which seems buggy).

I feel that:

a) We don't need another git tutorial - there's a lot of really good 
ones out there, even ones explaining git from the perspective of a 
current SVN user.

b) We need a document explaining how patches are reviewed and referred 
to, what commit rules (and hooks) are in place and how this would work 
in a git world. It would be short and sweet.

Now I can convert llvm and clang to mercurial with full history,
especially for clang, the current git mirror for clang is not tracking
the whole history(for those revisions under cfe/cfe/trunk), but the
one I converted is good on this, I was using an own modified
hgsubversion to do this, but I got problems on converting hg to git.


2011/8/19, FlyLanguage <flylanguage at gmail.com>:
>> On Aug 18, 2011, at 10:57 AM, David Greene wrote:
>>>
>>> Did the project ever come to a decision about making a transition
to
>>> git?  I'm trying to do some longer-term planning and it would
be helpful
>>> to know what the roadmap is.
>
> It's stuck on:
>
> 1) A misunderstanding that global revision numbers are necessary and
> that 'git describe' along with frequent tagging (commit hooks)
isn't
> good enough.
>
> 2) Nobody writing up how git should be used with the current llvm
> workflow (which is not going to adapt to an SCM, but the other way
> around, which is understandable.)
> _______________________________________________
> LLVM Developers mailing list
> LLVMdev at cs.uiuc.edu         http://llvm.cs.uiuc.edu
> http://lists.cs.uiuc.edu/mailman/listinfo/llvmdev
>
-- 
从我的移动设备发送

         此致
礼
罗勇刚
Yours
    sincerely,
Yonggang Luo

2011/9/4 罗勇刚(Yonggang Luo) <luoyonggang at
gmail.com>:
> Now I can convert llvm and clang to mercurial with full history,
> especially for clang, the current git mirror for clang is not tracking
> the whole history(for those revisions under cfe/cfe/trunk), but the
> one I converted is good on this, I was using an own modified
> hgsubversion to do this, but I got problems on converting hg to git.
FYI, we have such git mirror at https://github.com/spurious/clang-mirror
and I think this one is compatible with hg using hg-git.
But i'd recommend hg users to use direct hgsubversion mirror.

-- oku

Hello
> Now I can convert llvm and clang to mercurial with full history,
> especially for clang, the current git mirror for clang is not tracking
> the whole history(for those revisions under cfe/cfe/trunk),
For example? Which revisions are not tracked?

-- 
With best regards, Anton Korobeynikov
Faculty of Mathematics and Mechanics, Saint Petersburg State University