llvm dev - Feb 2019 - [monorepo] Exception in migrate-downstream-fork during monorepo transition

Our team decided to try out the conversion process utilizing the tool from
https://llvm.org/docs/Proposals/GitHubMove.html#basic-migration

However, we're currently blocked by an issue that I'd hope at least one
other group who had used this script would have run into. I'm wondering if
we've missed something fundamental, or if there's a new, more robust
process in place and the documentation is out of date.

When running the script, I get the exception:

...
  File "../llvm-git-migration/migrate-downstream-fork.py", line 151,
in commit_filter
    candidate_upstreams = [self.svn_to_newrev[p[0]] for p in parent_svn_info]
KeyError: 181274

svn_to_newrev is the mapping which help bridge the gap between old and new repos
using the common svn revision.

In the monorepo, there's commit 2f2aeb9b. It seems to be an amalgamation of
3 svn revisions.
The log for this commit contains:

llvm-svn: 181277
llvm-svn: 181274
llvm-svn: 181273

However, when dumping out the svn_to_newrev map, I only get one reference to
this commit:

181273: '2f2aeb9b6912046e9ea21ae259932c103333c444',

181274 is not present at all. This is a problem for commits whose parents
include clang mirror commit f9abf237
https://github.com/llvm-mirror/clang/commit/f9abf23757fd6d417e20f1862dcf9a2794c76f16

The apparent assumption is that 1 monorepo commit is 1 svn revision, but this
commit seems to break that assumption.
Any information or help is greatly appreciated! I loathe to attempt applying all
of our changes manually, and would be happy to patch the script if there is in
fact an error :)

I haven't tested it, but I'd bet the way to reproduce this would be:


1)      Checkout the llvm-mirror for clang

2)      Make a branch off of f9abf2375 and make a commit or two

3)      Attempt to use the migration script to move that branch onto the
monorepo

Because that seems to be precisely what we did to cause the error.

Regards,
J.B. Nagurne
Code Generation
Texas Instruments
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190222/54be114b/attachment.html>

It's probably worth filing a GitHub issue here:

https://github.com/jyknight/llvm-git-migration/issues

                        -David

"Nagurne, James via llvm-dev" <llvm-dev at lists.llvm.org>
writes:
> Our team decided to try out the conversion process utilizing the tool
> from https://llvm.org/docs/Proposals/GitHubMove.html#basic-migration
>
> However, we’re currently blocked by an issue that I’d hope at least
> one other group who had used this script would have run into. I’m
> wondering if we’ve missed something fundamental, or if there’s a new,
> more robust process in place and the documentation is out of date.
>
> When running the script, I get the exception:
>
> …
>
> File "../llvm-git-migration/migrate-downstream-fork.py", line
151, in
> commit_filter
>
> candidate_upstreams = [self.svn_to_newrev[p[0]] for p in
> parent_svn_info]
>
> KeyError: 181274
>
> svn_to_newrev is the mapping which help bridge the gap between old and
> new repos using the common svn revision.
>
> In the monorepo, there’s commit 2f2aeb9b. It seems to be an
> amalgamation of 3 svn revisions.
>
> The log for this commit contains:
>
> llvm-svn: 181277
>
> llvm-svn: 181274
>
> llvm-svn: 181273
>
> However, when dumping out the svn_to_newrev map, I only get one
> reference to this commit:
>
> 181273: '2f2aeb9b6912046e9ea21ae259932c103333c444',
>
> 181274 is not present at all. This is a problem for commits whose
> parents include clang mirror commit f9abf237
>
>
https://github.com/llvm-mirror/clang/commit/f9abf23757fd6d417e20f1862dcf9a2794c76f16
>
> The apparent assumption is that 1 monorepo commit is 1 svn revision,
> but this commit seems to break that assumption.
>
> Any information or help is greatly appreciated! I loathe to attempt
> applying all of our changes manually, and would be happy to patch the
> script if there is in fact an error J
>
> I haven’t tested it, but I’d bet the way to reproduce this would be:
>
> 1) Checkout the llvm-mirror for clang
>
> 2) Make a branch off of f9abf2375 and make a commit or two
>
> 3) Attempt to use the migration script to move that branch onto the
> monorepo
>
> Because that seems to be precisely what we did to cause the error.
>
> Regards,
>
> J.B. Nagurne
>
> Code Generation
>
> Texas Instruments
>
>
> _______________________________________________
> LLVM Developers mailing list
> llvm-dev at lists.llvm.org
> https://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev

I think it'd probably work to simply add all the svn revision numbers from
such commits into the mapping.

On Thu, Feb 21, 2019 at 8:04 PM Nagurne, James <j-nagurne at ti.com>
wrote:
> Our team decided to try out the conversion process utilizing the tool from
> https://llvm.org/docs/Proposals/GitHubMove.html#basic-migration
>
>
>
> However, we’re currently blocked by an issue that I’d hope at least one
> other group who had used this script would have run into. I’m wondering if
> we’ve missed something fundamental, or if there’s a new, more robust
> process in place and the documentation is out of date.
>
>
>
> When running the script, I get the exception:
>
>
>
> …
>
>   File "../llvm-git-migration/migrate-downstream-fork.py", line
151, in
> commit_filter
>
>     candidate_upstreams = [self.svn_to_newrev[p[0]] for p in
> parent_svn_info]
>
> KeyError: 181274
>
>
>
> svn_to_newrev is the mapping which help bridge the gap between old and new
> repos using the common svn revision.
>
>
>
> In the monorepo, there’s commit 2f2aeb9b. It seems to be an amalgamation
> of 3 svn revisions.
>
> The log for this commit contains:
>
>
>
> llvm-svn: 181277
>
> llvm-svn: 181274
>
> llvm-svn: 181273
>
>
>
> However, when dumping out the svn_to_newrev map, I only get one reference
> to this commit:
>
>
>
> 181273: '2f2aeb9b6912046e9ea21ae259932c103333c444',
>
>
>
> 181274 is not present at all. This is a problem for commits whose parents
> include clang mirror commit f9abf237
>
>
>
https://github.com/llvm-mirror/clang/commit/f9abf23757fd6d417e20f1862dcf9a2794c76f16
>
>
>
> The apparent assumption is that 1 monorepo commit is 1 svn revision, but
> this commit seems to break that assumption.
>
> Any information or help is greatly appreciated! I loathe to attempt
> applying all of our changes manually, and would be happy to patch the
> script if there is in fact an error J
>
>
>
> I haven’t tested it, but I’d bet the way to reproduce this would be:
>
>
>
> 1)      Checkout the llvm-mirror for clang
>
> 2)      Make a branch off of f9abf2375 and make a commit or two
>
> 3)      Attempt to use the migration script to move that branch onto the
> monorepo
>
>
>
> Because that seems to be precisely what we did to cause the error.
>
>
>
> Regards,
>
> J.B. Nagurne
>
> Code Generation
>
> Texas Instruments
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.llvm.org/pipermail/llvm-dev/attachments/20190222/b87ebde8/attachment.html>