Hans van Kranenburg writes ("Re: git workflow,
redux"):> On 08/23/2018 08:07 PM, Ian Jackson wrote:
> > I think git-debrebase is going to be easier for all these things than
> > the current approach.
>
> Ok, let's try it! Thanks a lot for doing the above writeup.
Great, thanks. (I hope it's OK that I have snipped most of your
responses to the discussion, that didn't seem to want a reply.)
> How to proceed:
>
> Can you provide me with some rtfm pointers and instructions about how
> you would like to see things being transformed? Or would you like to do
> it together? Do we want to do this before or after Sep 10?
I would like to do this as soon as possible.
There is no pre-cooked recipe or tool for converting a packaging-only
style branch to a git-debrebase branch. And this kind of surgery
involves more a thorough understanding of the gdr branch format than
merely using the tool on an existing branch does.
If you would like to try it then you're welcome to, but maybe it would
be better for me to do it. How about I do this today, and push it to
a wip branch on salsa for you and Wolodja to take a look at.
(FTR: I think that git-merge to bring in upstream, plus git-debrebase
convert-from-gbp, might do the right thing, so I will try that first.)
> I actually already have some questions about the git-debrebase behavior
> to start with:
Of course. This is quite new tool and in particular the user
documentation hasn't had much contact with users who aren't intimately
familiar with how it works. I think this is particularly true for
people who are more familiar with older Debian packaging workflows.
So please do ask me questions here, or on irc, or whatever.
> 1: meta information
>
> When writing software and git commits, my main incentive is to have a
> future reader understand what I'm doing and why. I tend to put quite
> some effort in this when I do things, not only because...
Absolutely.
> The quilt workflow allows to add a patch and add (meta-) comments to the
> git commit adding that patch, why it needed to be added etc. When
> directly adding patches on git level, this is not possible.
I'm not sure what you mean. The metainformation which would
previously go into the patch description, now goes into the commit
message.
> How is this meta information stored for future readers? Does the
> reader have to search in the debian/changelog and hope something is
> mentioned about it?
Certainly not - quite the opposite. The commit will show up in gitk
and git log and so on. You can look at its commit message.
> 2: usage of git
>
> One thing that is quite disturbing for me personally, is the result that
> is left behind in a git repository when using git-debrebase. Instead of
> having a meaningful history (which is my favourite thing to end up with,
> something you can "browse" and read meaningful things, see above)
git
> seems to be used as some sort of ftp server, where a rm *; cp -a * is
> done every time.
Again, I'm not sure what you mean. Perhaps you're referring to what
you see in gitk without --date-order and without --first-parent ?
It's true that that the full history view is rather complex.
Or perhaps you are running git diff, git log, etc. on debian/patches.
When using git-debrebase, don't do that. In a git-debrebase branch,
the files in debian/patches are an output which is best ignored.
Instead, look at the history of the actual upstream files.
> The relationship between patches and changes in them is not stored in
> git history because every time things end up in new unrelated commits.
It is true that you can't interdiff patches so easily. IME one
probably wants to do that rarely anyway.
> The history of the current master branch, doing the packaging from 4.8
> via 4.9 and 4.10 to 4.11 and all the changes still does fit on your
> screen. I like this. I can just read back what I did and why.
>
> So my question is: how does debrebase care about future readers? How do
> I quickly find out what *actual* changes have happened to the packaging
> from the large commit blurb that I'm looking at? Does this need a
README
> with git commands to filter commits in debian/ or other things, or does
> the debrebase tooling need to get features to analyze the git history
> again to produce a useful overview of this information?
The normal git commands for viewing subsets of the history can be
used. Let me make some suggestions:
History of package in Debian (disregards upstream history)
gitk --first-parent
In a laundered branch, the delta queue is at the top.
History of the packaging (excluding the delta queue)
gitk :/debian :!/debian/patches
Just the delta queue (ie, Debian's changes to upstream):
gitk --first-parent -- :/ :!/debian
Full history including old versions of the delta queue:
gitk --date-order
The "Declare fast forward" commits you see have an older
history (usually, an older delta queue) as one parent,
and a newer history as the other. --date-order makes
gitk show the delta queues in order.
Show complete diff since the last upload:
git diff dgit/dgit/sid..HEAD -- :/ :!/debian/patches
(Includes changes to upstream files.)
Show interdiff of patches, if you really want that:
git debrebase make-patches
git diff dgit/dgit/sid..HEAD -- debian/patches
Also of course there is
git debrebase status
I think some of this information should be in the manpages.
I'll file a bug about that.
> > Update d/control edit control.in # edit control.in
> > run rules to debian/rules debian/control
> > see output, git commit debian/control
> > debdifff? git diff
> >
> > git blame on not possible git blame
> > upstream source
>
> Not possible? Well, not in the same repo no, but I can't imagine you
> don't know how to do a git blame on the upstream source. Sorry, but
this
> does not remotely make sense at all.
This part I wanted to comment on, since it's a useful feature of
git-debrebase branches. What I mean is that, if I look at some file
in the dgit/stretch branch, which was done by git-debrebase, and do
`git-blame' or `git-log' on that file, I see the upstream history
*and* the patches, all combined. That is, in the blame output: for
lines which were edited in a patch, it shows the patch, and for lines
which are from upstream, it shows the relevant upstream commit.
Thanks,
Ian.
On 08/24/2018 01:27 PM, Ian Jackson wrote:> Hans van Kranenburg writes ("Re: git workflow, redux"): >> On 08/23/2018 08:07 PM, Ian Jackson wrote: >>> I think git-debrebase is going to be easier for all these things than >>> the current approach. >> >> Ok, let's try it! Thanks a lot for doing the above writeup. > > Great, thanks. (I hope it's OK that I have snipped most of your > responses to the discussion, that didn't seem to want a reply.)I see that the tone of my text was also not correct, and you don't deserve that. We just talked off-list about this. I am not angry, and am actually very happy with this mail thread, so I will do my best to type things in a positive and unambiguous style.>> How to proceed: >> >> Can you provide me with some rtfm pointers and instructions about how >> you would like to see things being transformed? Or would you like to do >> it together? Do we want to do this before or after Sep 10? > > I would like to do this as soon as possible.Ok. I suspect it makes sense to do the 00cvs dfsg problem afterwards instead in that case?> There is no pre-cooked recipe or tool for converting a packaging-only > style branch to a git-debrebase branch. And this kind of surgery > involves more a thorough understanding of the gdr branch format than > merely using the tool on an existing branch does. > > If you would like to try it then you're welcome to, but maybe it would > be better for me to do it. How about I do this today, and push it to > a wip branch on salsa for you and Wolodja to take a look at.Sure.> (FTR: I think that git-merge to bring in upstream, plus git-debrebase > convert-from-gbp, might do the right thing, so I will try that first.) > > >> I actually already have some questions about the git-debrebase behavior >> to start with: > > Of course. This is quite new tool and in particular the user > documentation hasn't had much contact with users who aren't intimately > familiar with how it works. I think this is particularly true for > people who are more familiar with older Debian packaging workflows. > > So please do ask me questions here, or on irc, or whatever. > > >> 1: meta information >> >> When writing software and git commits, my main incentive is to have a >> future reader understand what I'm doing and why. I tend to put quite >> some effort in this when I do things, not only because... > > Absolutely. > >> The quilt workflow allows to add a patch and add (meta-) comments to the >> git commit adding that patch, why it needed to be added etc. When >> directly adding patches on git level, this is not possible. > > I'm not sure what you mean. The metainformation which would > previously go into the patch description, now goes into the commit > message.Then I meant the meta-meta information about what the reason is to include some extra patch. Who reported it, what problem it solves, which bts number it is related to etc. Like this piece of information as an example: * Ship xen-diag (by cherry-picking the appropriate commits from upstream). This can help with diagnosis of #880554.>> How is this meta information stored for future readers? Does the >> reader have to search in the debian/changelog and hope something is >> mentioned about it? > > Certainly not - quite the opposite. The commit will show up in gitk > and git log and so on. You can look at its commit message.But I think I answered my own question already above, with changelog being a good place to put this information instead.>> 2: usage of git >> >> One thing that is quite disturbing for me personally, is the result that >> is left behind in a git repository when using git-debrebase. Instead of >> having a meaningful history (which is my favourite thing to end up with, >> something you can "browse" and read meaningful things, see above) git >> seems to be used as some sort of ftp server, where a rm *; cp -a * is >> done every time. > > Again, I'm not sure what you mean. Perhaps you're referring to what > you see in gitk without --date-order and without --first-parent ? > It's true that that the full history view is rather complex. > > Or perhaps you are running git diff, git log, etc. on debian/patches. > When using git-debrebase, don't do that. In a git-debrebase branch, > the files in debian/patches are an output which is best ignored. > > Instead, look at the history of the actual upstream files. > > >> The relationship between patches and changes in them is not stored in >> git history because every time things end up in new unrelated commits. > > It is true that you can't interdiff patches so easily. IME one > probably wants to do that rarely anyway. > > >> The history of the current master branch, doing the packaging from 4.8 >> via 4.9 and 4.10 to 4.11 and all the changes still does fit on your >> screen. I like this. I can just read back what I did and why. >> >> So my question is: how does debrebase care about future readers? How do >> I quickly find out what *actual* changes have happened to the packaging >> from the large commit blurb that I'm looking at? Does this need a README >> with git commands to filter commits in debian/ or other things, or does >> the debrebase tooling need to get features to analyze the git history >> again to produce a useful overview of this information? > > The normal git commands for viewing subsets of the history can be > used. Let me make some suggestions: > > History of package in Debian (disregards upstream history) > gitk --first-parent > In a laundered branch, the delta queue is at the top. > > History of the packaging (excluding the delta queue) > gitk :/debian :!/debian/patches > > Just the delta queue (ie, Debian's changes to upstream): > gitk --first-parent -- :/ :!/debian > > Full history including old versions of the delta queue: > gitk --date-order > The "Declare fast forward" commits you see have an older > history (usually, an older delta queue) as one parent, > and a newer history as the other. --date-order makes > gitk show the delta queues in order. > > Show complete diff since the last upload: > git diff dgit/dgit/sid..HEAD -- :/ :!/debian/patches > (Includes changes to upstream files.) > > Show interdiff of patches, if you really want that: > git debrebase make-patches > git diff dgit/dgit/sid..HEAD -- debian/patches > > Also of course there is > git debrebase status > > I think some of this information should be in the manpages. > I'll file a bug about that.This information is really helpful yes.>>> Update d/control edit control.in # edit control.in >>> run rules to debian/rules debian/control >>> see output, git commit debian/control >>> debdifff? git diff >>> >>> git blame on not possible git blame >>> upstream source >> >> Not possible? Well, not in the same repo no, but I can't imagine you >> don't know how to do a git blame on the upstream source. Sorry, but this >> does not remotely make sense at all. > > This part I wanted to comment on, since it's a useful feature of > git-debrebase branches. What I mean is that, if I look at some file > in the dgit/stretch branch, which was done by git-debrebase, and do > `git-blame' or `git-log' on that file, I see the upstream history > *and* the patches, all combined. That is, in the blame output: for > lines which were edited in a patch, it shows the patch, and for lines > which are from upstream, it shows the relevant upstream commit.Aha. Now I see. It's about the combined upstream source + changes. Yes, this is very useful indeed and yes, a 'not possible' in the left column. Hans
Hans van Kranenburg writes ("Re: git workflow,
redux"):> On 08/24/2018 01:27 PM, Ian Jackson wrote:
> > There is no pre-cooked recipe or tool for converting a packaging-only
> > style branch to a git-debrebase branch. And this kind of surgery
> > involves more a thorough understanding of the gdr branch format than
> > merely using the tool on an existing branch does.
> >
> > If you would like to try it then you're welcome to, but maybe it
would
> > be better for me to do it. How about I do this today, and push it to
> > a wip branch on salsa for you and Wolodja to take a look at.
>
> Sure.
I have now done this. It is in:
salsa/diziet/dgit/experimental.draft
Some notes:
1. Bugs in git-debrebase
This is the first time I have used git-debrebase on a package whose
patches were in subdirectories. Evidently it is the first time anyone
else has, either: git-debrebase had some trouble with that. I think
you can use the git-debrebase in sid/buster to examine my branch, and
you can build it with dpkg-buildpackage, but `make-patches' fails.
I have filed bugs, and fixed them locally and will upload a fixed
git-debrebase over the weekend. Apologies for selling you on my new
tool and then having it crap out...
2. Autogenerated files
gbp pq, which git-debrebase uses, requires debian/control. There are
some other things in devscripts etc. which also require
debian/control. I decided that debian/control should be in git.
There's a commit message explaning the rationale and some future ideas
I have for making this less annoying and less baroque, in the commit
where I edited .gitignore.
Regards,
Ian.
--
Ian Jackson <ijackson at chiark.greenend.org.uk> These opinions are my
own.
If I emailed you from an address @fyvzl.net or @evade.org.uk, that is
a private address which bypasses my fierce spamfilter.