llvm dev - Apr 2019 - How to deal with accidental directory tree deletes, downstream?

Hello fellow downstream residents,

I see that r358546 accidentally deleted an entire subtree, which was
reverted in r358552.  This of course caused a big merge conflict in 
our local repo, and internally we've been debating tactics for dealing 
with it, hopefully without losing our original history.

Has anyone else handled this in a way that they are happy with?  We
found a StackOverflow post that is potentially helpful:
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
Doing this on our local copy of the upstream repo means we would not
have an exact copy anymore, which seems like a Bad Idea.

Is there a smooth way to resolve the merge conflict that does *not*
delete our local tree? I suppose we can somehow not accept the
accidental deletes, and then when we run forward to r358552 it will
decide we already have those files and it will Just Work?

Tips and hints welcome.
Thanks,
--paulr

Hey Paul,
Maybe use a pull request type model for merging in upstream commits to your
internal repository?  I know it is not the cleanest method of merging
upstream commits, but it would allow you to gate your upstream merges on a
quick and dirty merge test and then you could alert an engineer to the
merge problem before it hits your mainline branch?  In the scenario you
described below, r358546 would have come in on a separate branch and
attempted to merge with master (assuming your main dev branch is called
master).  When the merge failed an engineer could have been notified and
begun to work through the issue to see what the best course of action would
be.  I'm really over simplifying here, but the point is, when possible,
test each commit coming from upstream in as much isolation as possible,
prior to merging.  I'm a big fan of pre-commit testing and of pull
requests.  If there is a way, which works for you to integrate automated
pull requests into the upstream merging workflow, it may be of benefit.
Hope this helps.

-Mike

On Wed, Apr 17, 2019 at 5:36 AM via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
> Hello fellow downstream residents,
>
> I see that r358546 accidentally deleted an entire subtree, which was
> reverted in r358552.  This of course caused a big merge conflict in
> our local repo, and internally we've been debating tactics for dealing
> with it, hopefully without losing our original history.
>
> Has anyone else handled this in a way that they are happy with?  We
> found a StackOverflow post that is potentially helpful:
>
>
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
> Doing this on our local copy of the upstream repo means we would not
> have an exact copy anymore, which seems like a Bad Idea.
>
> Is there a smooth way to resolve the merge conflict that does *not*
> delete our local tree? I suppose we can somehow not accept the
> accidental deletes, and then when we run forward to r358552 it will
> decide we already have those files and it will Just Work?
>
> Tips and hints welcome.
> Thanks,
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190417/43da5bcd/attachment.html>

Hi Paul,

This caused merge conflicts for us due to local changes in
llvm/test/Transforms so was blocked from being merged. As we merge commit
by commit, I manually resolved the conflict merging r358546 by keeping
"ours" then restored the rest of test/Transforms ("git reset HEAD
llvm/test/Transforms", "git checkout llvm/test/Transforms") apart
from
test/Transforms/LoopFusion which was supposed to be part of the commit
("git rm -r llvm/test/Transforms/LoopFusion"). When r358552 came to be
merged (also a conflict as we still had local files which were different
from the opensource files being re-added) I resolved in favour of
"ours"
again. This effectively "edits out" the accidental delete between
r358546
and r358552. This means that the files still exist in our tree through this
period even if they don't in opensource so git blame etc. still work (for
us locally). A slight discrepancy but hopefully not a problematic one.

Regards
Russ

On Wed, 17 Apr 2019 at 13:35, via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
> Hello fellow downstream residents,
>
> I see that r358546 accidentally deleted an entire subtree, which was
> reverted in r358552.  This of course caused a big merge conflict in
> our local repo, and internally we've been debating tactics for dealing
> with it, hopefully without losing our original history.
>
> Has anyone else handled this in a way that they are happy with?  We
> found a StackOverflow post that is potentially helpful:
>
>
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
> Doing this on our local copy of the upstream repo means we would not
> have an exact copy anymore, which seems like a Bad Idea.
>
> Is there a smooth way to resolve the merge conflict that does *not*
> delete our local tree? I suppose we can somehow not accept the
> accidental deletes, and then when we run forward to r358552 it will
> decide we already have those files and it will Just Work?
>
> Tips and hints welcome.
> Thanks,
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190417/30d4054f/attachment.html>

I'm pretty sure you could use 'git rebase' then 'git
replace' to graft an
alternate history over that patch of commits. Perform your merges, then
optionally remove the replacements?

On Wed, 17 Apr 2019 at 22:05, via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
> Hello fellow downstream residents,
>
> I see that r358546 accidentally deleted an entire subtree, which was
> reverted in r358552.  This of course caused a big merge conflict in
> our local repo, and internally we've been debating tactics for dealing
> with it, hopefully without losing our original history.
>
> Has anyone else handled this in a way that they are happy with?  We
> found a StackOverflow post that is potentially helpful:
>
>
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
> Doing this on our local copy of the upstream repo means we would not
> have an exact copy anymore, which seems like a Bad Idea.
>
> Is there a smooth way to resolve the merge conflict that does *not*
> delete our local tree? I suppose we can somehow not accept the
> accidental deletes, and then when we run forward to r358552 it will
> decide we already have those files and it will Just Work?
>
> Tips and hints welcome.
> Thanks,
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190418/0ff4d796/attachment.html>

FWIW I do apologize... I'm not entirely certain how that happened
since neither the commit nor the email for the commit had 5k files
being deleted, but I can imagine it was quite inconvenient.

-eric

On Wed, Apr 17, 2019 at 5:35 AM via llvm-dev <llvm-dev at lists.llvm.org>
wrote:
>
> Hello fellow downstream residents,
>
> I see that r358546 accidentally deleted an entire subtree, which was
> reverted in r358552.  This of course caused a big merge conflict in
> our local repo, and internally we've been debating tactics for dealing
> with it, hopefully without losing our original history.
>
> Has anyone else handled this in a way that they are happy with?  We
> found a StackOverflow post that is potentially helpful:
>
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
> Doing this on our local copy of the upstream repo means we would not
> have an exact copy anymore, which seems like a Bad Idea.
>
> Is there a smooth way to resolve the merge conflict that does *not*
> delete our local tree? I suppose we can somehow not accept the
> accidental deletes, and then when we run forward to r358552 it will
> decide we already have those files and it will Just Work?
>
> Tips and hints welcome.
> Thanks,
> --paulr
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

Hi,

Isn't this a bit of a nuisance also directly on trunk and not only for 
downstream repos?

If I do e.g.

  git blame test/Transforms/LoopUnroll/basic.ll

then every line in the file is attributed to the commit where the 
testcases were added back again.

With

  gitk test/Transforms/LoopUnroll/basic.ll

older commits are shown as well so it's not like all history is gone, 
but at least blame is messed up.

For our downstream clone, we didn't merge from trunk to our 
development-branch while the testcases were gone, so we never got any 
conflicts at all, and since the tests were later put back I think the 
situation for us now is the same as being directly on trunk.

/Mikael


On 4/17/19 2:35 PM, via llvm-dev wrote:
> Hello fellow downstream residents,
> 
> I see that r358546 accidentally deleted an entire subtree, which was
> reverted in r358552.  This of course caused a big merge conflict in
> our local repo, and internally we've been debating tactics for dealing
> with it, hopefully without losing our original history.
> 
> Has anyone else handled this in a way that they are happy with?  We
> found a StackOverflow post that is potentially helpful:
>
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
> Doing this on our local copy of the upstream repo means we would not
> have an exact copy anymore, which seems like a Bad Idea.
> 
> Is there a smooth way to resolve the merge conflict that does *not*
> delete our local tree? I suppose we can somehow not accept the
> accidental deletes, and then when we run forward to r358552 it will
> decide we already have those files and it will Just Work?
> 
> Tips and hints welcome.
> Thanks,
> --paulr
> 
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>

test/Transforms svn log has been lost in the same way as git blame - if 
there's any way that we can get both back it'd be very useful!

On 18/04/2019 08:11, Mikael Holmén via llvm-dev wrote:
> Hi,
>
> Isn't this a bit of a nuisance also directly on trunk and not only for
> downstream repos?
>
> If I do e.g.
>
>    git blame test/Transforms/LoopUnroll/basic.ll
>
> then every line in the file is attributed to the commit where the
> testcases were added back again.
>
> With
>
>    gitk test/Transforms/LoopUnroll/basic.ll
>
> older commits are shown as well so it's not like all history is gone,
> but at least blame is messed up.
>
> For our downstream clone, we didn't merge from trunk to our
> development-branch while the testcases were gone, so we never got any
> conflicts at all, and since the tests were later put back I think the
> situation for us now is the same as being directly on trunk.
>
> /Mikael
>
>
> On 4/17/19 2:35 PM, via llvm-dev wrote:
>> Hello fellow downstream residents,
>>
>> I see that r358546 accidentally deleted an entire subtree, which was
>> reverted in r358552.  This of course caused a big merge conflict in
>> our local repo, and internally we've been debating tactics for
dealing
>> with it, hopefully without losing our original history.
>>
>> Has anyone else handled this in a way that they are happy with?  We
>> found a StackOverflow post that is potentially helpful:
>>
https://stackoverflow.com/questions/44990103/how-to-undelete-file-in-git-and-keep-his-original-blame-history
>> Doing this on our local copy of the upstream repo means we would not
>> have an exact copy anymore, which seems like a Bad Idea.
>>
>> Is there a smooth way to resolve the merge conflict that does *not*
>> delete our local tree? I suppose we can somehow not accept the
>> accidental deletes, and then when we run forward to r358552 it will
>> decide we already have those files and it will Just Work?
>>
>> Tips and hints welcome.
>> Thanks,
>> --paulr
>>
>> _______________________________________________
>> LLVM Developers mailing list
>> llvm-dev at lists.llvm.org
>> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev
>>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev