So the Github-Importer screws up the old dark places of the SVN-History and the scripts try to handle this? 2017-01-13 22:57 GMT+01:00 James Y Knight via llvm-dev < llvm-dev at lists.llvm.org>:> I've been working on some scripting for re-converting the svn repository > into git. > > The existing git conversions are entirely sufficient for day-to-day > development purposes, but not up to the standard (at least, *my* standard) > for replacing svn as the authoritative source repository. For one example, > clang.git doesn't actually go back to the first commit of clang, because > weird stuff happened early in the svn history that threw off git-svn. > > Here's my work in progress, but there's still more work to be done: > https://github.com/jyknight/llvm-git-migration > <https://github.com/jyknight/llvm-monorepo> (conversion scripts) > https://github.com/jyknight/llvm-monorepo (test repository) > > > > On Fri, Jan 13, 2017 at 4:07 PM, Keane, Erich via llvm-dev < > llvm-dev at lists.llvm.org> wrote: > >> Hi all- >> I was wondering if anyone knew what the status/schedule of the SVN to >> git/github transition was? I thought I saw that at the November meeting it >> was agreed upon, but I'm not sure I saw any progress since? >> >> Thanks, >> Erich >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/067a9ab3/attachment-0001.html>
Something like that. FWIW: This is not surprising. When i moved GCC from CVS to SVN, the older versions of CVS had bugs that the SVN importer couldn't handle, and i essentially had to rewrite subversion's cvs2svn to make it work (plus, it was originally so slow it would have taken 2 real-time months to do the import). On the plus side, once we fixed all the bugs, we also were able to get RCS history into the SVN history :) On Mon, Jan 16, 2017 at 12:29 AM, Alexander Benikowski via llvm-dev < llvm-dev at lists.llvm.org> wrote:> So the Github-Importer screws up the old dark places of the SVN-History > and the scripts try to handle this? > > 2017-01-13 22:57 GMT+01:00 James Y Knight via llvm-dev < > llvm-dev at lists.llvm.org>: > >> I've been working on some scripting for re-converting the svn repository >> into git. >> >> The existing git conversions are entirely sufficient for day-to-day >> development purposes, but not up to the standard (at least, *my* standard) >> for replacing svn as the authoritative source repository. For one example, >> clang.git doesn't actually go back to the first commit of clang, because >> weird stuff happened early in the svn history that threw off git-svn. >> >> Here's my work in progress, but there's still more work to be done: >> https://github.com/jyknight/llvm-git-migration >> <https://github.com/jyknight/llvm-monorepo> (conversion scripts) >> https://github.com/jyknight/llvm-monorepo (test repository) >> >> >> >> On Fri, Jan 13, 2017 at 4:07 PM, Keane, Erich via llvm-dev < >> llvm-dev at lists.llvm.org> wrote: >> >>> Hi all- >>> I was wondering if anyone knew what the status/schedule of the SVN to >>> git/github transition was? I thought I saw that at the November meeting it >>> was agreed upon, but I'm not sure I saw any progress since? >>> >>> Thanks, >>> Erich >>> _______________________________________________ >>> LLVM Developers mailing list >>> llvm-dev at lists.llvm.org >>> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >>> >> >> >> _______________________________________________ >> LLVM Developers mailing list >> llvm-dev at lists.llvm.org >> http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev >> >> > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/f05e5488/attachment.html>
I had a similar experience when I did the same migration for PHP's repos in 2009. It took several months of prep and a multi-thousand-line script around cvs2svn, and they still lost some history in the process. I don't know of any plus sides, aside from they weren't on CVS anymore ^^; It's safe to say there will always be some rough edges when dealing with so much history. -- Gwynne Raskind> On Jan 16, 2017, at 02:35, Daniel Berlin via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > Something like that. > FWIW: This is not surprising. > > When i moved GCC from CVS to SVN, the older versions of CVS had bugs that the SVN importer couldn't handle, and i essentially had to rewrite subversion's cvs2svn to make it work (plus, it was originally so slow it would have taken 2 real-time months to do the import). > > On the plus side, once we fixed all the bugs, we also were able to get RCS history into the SVN history :) > > > > > On Mon, Jan 16, 2017 at 12:29 AM, Alexander Benikowski via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > So the Github-Importer screws up the old dark places of the SVN-History and the scripts try to handle this? > > 2017-01-13 22:57 GMT+01:00 James Y Knight via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>: > I've been working on some scripting for re-converting the svn repository into git. > > The existing git conversions are entirely sufficient for day-to-day development purposes, but not up to the standard (at least, *my* standard) for replacing svn as the authoritative source repository. For one example, clang.git doesn't actually go back to the first commit of clang, because weird stuff happened early in the svn history that threw off git-svn. > > Here's my work in progress, but there's still more work to be done: > https://github.com/jyknight/llvm-git-migration <https://github.com/jyknight/llvm-monorepo> (conversion scripts) > https://github.com/jyknight/llvm-monorepo <https://github.com/jyknight/llvm-monorepo> (test repository) > > > > On Fri, Jan 13, 2017 at 4:07 PM, Keane, Erich via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > Hi all- > I was wondering if anyone knew what the status/schedule of the SVN to git/github transition was? I thought I saw that at the November meeting it was agreed upon, but I'm not sure I saw any progress since? > > Thanks, > Erich > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/8b1a87a0/attachment.html>
Hi,> On Jan 16, 2017, at 12:29 AM, Alexander Benikowski via llvm-dev <llvm-dev at lists.llvm.org> wrote: > > So the Github-Importer screws up the old dark places of the SVN-History and the scripts try to handle this?I don’t think we tried the Github-Importer, but keep in mind that the project structure is a bit special. In SVN we have: llvm/trunk llvm/branch llvm/tags clang/trunk clang/branch clang/tags … And we want to map in the destination repo llvm/trunk to llvm/, clang/trunk to clang, .. We also want to map nicely the author names/emails to commits, possible catching “Patch by:” in the commit message to attribute correctly authorship (while keeping the “committer” field in git. Also there are some specific cases in the history we want to filter: for instance all of LLVM was committed into LLDB as a zip file at some point. This is already filtered out of the git repo on llvm.org <http://llvm.org/>, and we want to filter this out in the monorepo. Other craziness can be found by looking at the script James wrote: https://github.com/jyknight/llvm-git-migration/blob/master/llvm-svn2git.rules <https://github.com/jyknight/llvm-git-migration/blob/master/llvm-svn2git.rules> : - "Skip the revisions that deleted cfe/cfe/ and moved it to cfe/“ - "Handle compiler-rt's initial revision, which was out of trunk/" - “Ignore move from gcc-plugin/ to dragonegg/, and re-addition of gcc-plugin” - "Some branches are at a different level.” I’m very impressed by the amount of dedication James is putting into this, and he deserves a big thanks for this archeology work :) Best, Mehdi> 2017-01-13 22:57 GMT+01:00 James Y Knight via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>>: > I've been working on some scripting for re-converting the svn repository into git. > > The existing git conversions are entirely sufficient for day-to-day development purposes, but not up to the standard (at least, *my* standard) for replacing svn as the authoritative source repository. For one example, clang.git doesn't actually go back to the first commit of clang, because weird stuff happened early in the svn history that threw off git-svn. > > Here's my work in progress, but there's still more work to be done: > https://github.com/jyknight/llvm-git-migration <https://github.com/jyknight/llvm-monorepo> (conversion scripts) > https://github.com/jyknight/llvm-monorepo <https://github.com/jyknight/llvm-monorepo> (test repository) > > > > On Fri, Jan 13, 2017 at 4:07 PM, Keane, Erich via llvm-dev <llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org>> wrote: > Hi all- > I was wondering if anyone knew what the status/schedule of the SVN to git/github transition was? I thought I saw that at the November meeting it was agreed upon, but I'm not sure I saw any progress since? > > Thanks, > Erich > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org <mailto:llvm-dev at lists.llvm.org> > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev <http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev> > > > _______________________________________________ > LLVM Developers mailing list > llvm-dev at lists.llvm.org > http://lists.llvm.org/cgi-bin/mailman/listinfo/llvm-dev-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.llvm.org/pipermail/llvm-dev/attachments/20170116/5a2e1399/attachment.html>