Xen devel - Feb 2013 - Changeset / commit id not incorporated in build after switch to git

Hi All,

After the switching from mercurial to git, the changeset isn''t
incorporated anymore in the build.
This makes error reports possibly a bit less verbose (xl dmesg, serial logs and
xl info now omit the changeset (or commit) info)

Git doesn''t have the concept of changesets afaik and mercurial is,
while deprecated, still used as mirror.

So what would be wise:
   - just replace the changeset with the git commit sha-1 hash (always)
   - use changeset when a mercurial tree is detected or the last git commit
sha-1 (and date ?) when a git tree is detected
   - make a separate "commit" entry besides the changeset and leave
one undefined

xen/Makefile currently has:

# compile.h contains dynamic build info. Rebuilt on every
''make'' invocation.
include/xen/compile.h: include/xen/compile.h.in .banner
        @sed -e ''s/@@date@@/$(shell LC_ALL=C date)/g'' \
            -e ''s/@@time@@/$(shell LC_ALL=C date +%T)/g'' \
            -e ''s/@@whoami@@/$(XEN_WHOAMI)/g'' \
            -e ''s/@@domain@@/$(XEN_DOMAIN)/g'' \
            -e ''s/@@hostname@@/$(shell hostname)/g'' \
            -e ''s!@@compiler@@!$(shell $(CC) $(CFLAGS) --version
2>&1 | head -1)!g'' \
            -e ''s/@@version@@/$(XEN_VERSION)/g'' \
            -e ''s/@@subversion@@/$(XEN_SUBVERSION)/g'' \
            -e ''s/@@extraversion@@/$(XEN_EXTRAVERSION)/g'' \
            -e ''s!@@changeset@@!$(shell ((hg parents --template
"{date|date} {rev}:{node|short}" >/dev/null && hg parents
--template "{date|date} {rev}:{node|short}") || echo
"unavailable") 2>/dev/null)!g'' \
            < include/xen/compile.h.in > $@.new
        @grep \" .banner >> $@.new
        @grep -v \" .banner
        @mv -f $@.new $@

--
Sander

Hello Sander,

Monday, February 25, 2013, 11:28:53 PM, you wrote:
> Date: Mon, 25 Feb 2013 23:28:53 +0100
> From: Sander Eikelenboom <linux@eikelenboom.it>
> Organization: Eikelenboom IT services
> X-Priority: 3 (Normal)
> Message-ID: <1208255021.20130225232853@eikelenboom.it>
> To: xen-devel <xen-devel@lists.xen.org>
> Subject: Changeset / commit id not incorporated in build after switch to
git
> MIME-Version: 1.0
> Content-Type: text/plain; charset=us-ascii
> Content-Transfer-Encoding: 7bit
> Hi All,
> After the switching from mercurial to git, the changeset isn''t
incorporated anymore in the build.
> This makes error reports possibly a bit less verbose (xl dmesg, serial logs
and xl info now omit the changeset (or commit) info)
> Git doesn''t have the concept of changesets afaik and mercurial is,
while deprecated, still used as mirror.
> So what would be wise:
>    - just replace the changeset with the git commit sha-1 hash (always)
>    - use changeset when a mercurial tree is detected or the last git commit
sha-1 (and date ?) when a git tree is detected
>    - make a separate "commit" entry besides the changeset and
leave one undefined
> xen/Makefile currently has:
> # compile.h contains dynamic build info. Rebuilt on every
''make'' invocation.
> include/xen/compile.h: include/xen/compile.h.in .banner
>         @sed -e ''s/@@date@@/$(shell LC_ALL=C date)/g'' \
>             -e ''s/@@time@@/$(shell LC_ALL=C date +%T)/g''
\
>             -e ''s/@@whoami@@/$(XEN_WHOAMI)/g'' \
>             -e ''s/@@domain@@/$(XEN_DOMAIN)/g'' \
>             -e ''s/@@hostname@@/$(shell hostname)/g'' \
>             -e ''s!@@compiler@@!$(shell $(CC) $(CFLAGS) --version
2>&1 | head -1)!g'' \
>             -e ''s/@@version@@/$(XEN_VERSION)/g'' \
>             -e ''s/@@subversion@@/$(XEN_SUBVERSION)/g'' \
>             -e ''s/@@extraversion@@/$(XEN_EXTRAVERSION)/g''
\
>             -e ''s!@@changeset@@!$(shell ((hg parents --template
"{date|date} {rev}:{node|short}" >/dev/null && hg parents
--template "{date|date} {rev}:{node|short}") || echo
"unavailable") 2>/dev/null)!g'' \
>             < include/xen/compile.h.in > $@.new
>         @grep \" .banner >> $@.new
>         @grep -v \" .banner
>         @mv -f $@.new $@
> --
> Sander


Perhaps use about the same as linux has in scripts/setlocalversion ?

scm_version()
{
        local short
        short=false

        cd "$srctree"
        if test -e .scmversion; then
                cat .scmversion
                return
        fi
        if test "$1" = "--short"; then
                short=true
        fi

        # Check for git and a git repo.
        if test -d .git && head=`git rev-parse --verify --short HEAD
2>/dev/null`; then

                # If we are at a tagged commit (like "v2.6.30-rc6"),
we ignore
                # it, because this version is defined in the top level Makefile.
                if [ -z "`git describe --exact-match 2>/dev/null`"
]; then

                        # If only the short version is requested, don''t
bother
                        # running further git commands
                        if $short; then
                                echo "+"
                                return
                        fi
                        # If we are past a tagged commit (like
                        # "v2.6.30-rc5-302-g72357d5"), we pretty print
it.
                        if atag="`git describe 2>/dev/null`"; then
                                echo "$atag" | awk -F-
''{printf("-%05d-%s", $(NF-1),$(NF))}''

                        # If we don''t have a tag at all we print
-g{commitish}.
                        else
                                printf ''%s%s'' -g $head
                        fi
                fi

                # Is this git on svn?
                if git config --get svn-remote.svn.url >/dev/null; then
                        printf -- ''-svn%s'' "`git svn
find-rev $head`"
                fi

                # Update index only on r/w media
                [ -w . ] && git update-index --refresh --unmerged >
/dev/null

                # Check for uncommitted changes
                if git diff-index --name-only HEAD | grep -qv
"^scripts/package"; then
                        printf ''%s'' -dirty
                fi

                # All done with git
                return
        fi

        # Check for mercurial and a mercurial repo.
        if test -d .hg && hgid=`hg id 2>/dev/null`; then
                # Do we have an tagged version?  If so, latesttagdistance == 1
                if [ "`hg log -r . --template
''{latesttagdistance}''`" == "1" ]; then
                        id=`hg log -r . --template
''{latesttag}''`
                        printf ''%s%s'' -hg "$id"
                else
                        tag=`printf ''%s'' "$hgid" |
cut -d'' '' -f2`
                        if [ -z "$tag" -o "$tag" = tip ];
then
                                id=`printf ''%s''
"$hgid" | sed ''s/[+ ].*//''`
                                printf ''%s%s'' -hg
"$id"
                        fi
                fi

                # Are there uncommitted changes?
                # These are represented by + after the changeset id.
                case "$hgid" in
                        *+|*+\ *) printf ''%s'' -dirty ;;
                esac

                # All done with mercurial
                return
        fi

On Mon, 2013-02-25 at 23:00 +0000, Sander Eikelenboom
wrote:
> Hello Sander,
Talking to yourself is a sign of something or other I think ;-)
> Monday, February 25, 2013, 11:28:53 PM, you wrote:
> 
> > Date: Mon, 25 Feb 2013 23:28:53 +0100
> > From: Sander Eikelenboom <linux@eikelenboom.it>
> > Organization: Eikelenboom IT services
> > X-Priority: 3 (Normal)
> > Message-ID: <1208255021.20130225232853@eikelenboom.it>
> > To: xen-devel <xen-devel@lists.xen.org>
> > Subject: Changeset / commit id not incorporated in build after switch
to git
> > MIME-Version: 1.0
> > Content-Type: text/plain; charset=us-ascii
> > Content-Transfer-Encoding: 7bit
> 
> > Hi All,
> 
> > After the switching from mercurial to git, the changeset
isn''t incorporated anymore in the build.
> > This makes error reports possibly a bit less verbose (xl dmesg, serial
logs and xl info now omit the changeset (or commit) info)
> 
> > Git doesn''t have the concept of changesets afaik and
mercurial is, while deprecated, still used as mirror.
> 
> > So what would be wise:
> >    - just replace the changeset with the git commit sha-1 hash
(always)
> >    - use changeset when a mercurial tree is detected or the last git
commit sha-1 (and date ?) when a git tree is detected
> >    - make a separate "commit" entry besides the changeset
and leave one undefined
> 
> > xen/Makefile currently has:
> 
> > # compile.h contains dynamic build info. Rebuilt on every
''make'' invocation.
> > include/xen/compile.h: include/xen/compile.h.in .banner
> >         @sed -e ''s/@@date@@/$(shell LC_ALL=C
date)/g'' \
> >             -e ''s/@@time@@/$(shell LC_ALL=C date
+%T)/g'' \
> >             -e ''s/@@whoami@@/$(XEN_WHOAMI)/g'' \
> >             -e ''s/@@domain@@/$(XEN_DOMAIN)/g'' \
> >             -e ''s/@@hostname@@/$(shell hostname)/g''
\
> >             -e ''s!@@compiler@@!$(shell $(CC) $(CFLAGS)
--version 2>&1 | head -1)!g'' \
> >             -e ''s/@@version@@/$(XEN_VERSION)/g'' \
> >             -e
''s/@@subversion@@/$(XEN_SUBVERSION)/g'' \
> >             -e
''s/@@extraversion@@/$(XEN_EXTRAVERSION)/g'' \
> >             -e ''s!@@changeset@@!$(shell ((hg parents
--template "{date|date} {rev}:{node|short}" >/dev/null &&
hg parents --template "{date|date} {rev}:{node|short}") || echo
"unavailable") 2>/dev/null)!g'' \
> >             < include/xen/compile.h.in > $@.new
> >         @grep \" .banner >> $@.new
> >         @grep -v \" .banner
> >         @mv -f $@.new $@
> 
> > --
> > Sander
> 
> 
> 
> Perhaps use about the same as linux has in scripts/setlocalversion ?
This is what I was going to suggest. Actually, I had it in mind we
already did something like!

Anyway, I think including whatever information the VCS we are building
from supplied is the right answer and the way Linux does it seems useful
and appropriate (e.g. using the tag name, appending -dirty etc).

If we can also extend that to do something sensible in the tarball case
then even better!

Ian.

Tuesday, February 26, 2013, 11:20:07 AM, you wrote:
> On Mon, 2013-02-25 at 23:00 +0000, Sander Eikelenboom wrote:
>> Hello Sander,
> Talking to yourself is a sign of something or other I think ;-)
primarily hitting the send button a bit before my thoughts actually ended :-p
(at least i hope)

>> Monday, February 25, 2013, 11:28:53 PM, you wrote:
>> 
>> > Date: Mon, 25 Feb 2013 23:28:53 +0100
>> > From: Sander Eikelenboom <linux@eikelenboom.it>
>> > Organization: Eikelenboom IT services
>> > X-Priority: 3 (Normal)
>> > Message-ID: <1208255021.20130225232853@eikelenboom.it>
>> > To: xen-devel <xen-devel@lists.xen.org>
>> > Subject: Changeset / commit id not incorporated in build after
switch to git
>> > MIME-Version: 1.0
>> > Content-Type: text/plain; charset=us-ascii
>> > Content-Transfer-Encoding: 7bit
>> 
>> > Hi All,
>> 
>> > After the switching from mercurial to git, the changeset
isn''t incorporated anymore in the build.
>> > This makes error reports possibly a bit less verbose (xl dmesg,
serial logs and xl info now omit the changeset (or commit) info)
>> 
>> > Git doesn''t have the concept of changesets afaik and
mercurial is, while deprecated, still used as mirror.
>> 
>> > So what would be wise:
>> >    - just replace the changeset with the git commit sha-1 hash
(always)
>> >    - use changeset when a mercurial tree is detected or the last
git commit sha-1 (and date ?) when a git tree is detected
>> >    - make a separate "commit" entry besides the
changeset and leave one undefined
>> 
>> > xen/Makefile currently has:
>> 
>> > # compile.h contains dynamic build info. Rebuilt on every
''make'' invocation.
>> > include/xen/compile.h: include/xen/compile.h.in .banner
>> >         @sed -e ''s/@@date@@/$(shell LC_ALL=C
date)/g'' \
>> >             -e ''s/@@time@@/$(shell LC_ALL=C date
+%T)/g'' \
>> >             -e ''s/@@whoami@@/$(XEN_WHOAMI)/g'' \
>> >             -e ''s/@@domain@@/$(XEN_DOMAIN)/g'' \
>> >             -e ''s/@@hostname@@/$(shell
hostname)/g'' \
>> >             -e ''s!@@compiler@@!$(shell $(CC) $(CFLAGS)
--version 2>&1 | head -1)!g'' \
>> >             -e ''s/@@version@@/$(XEN_VERSION)/g''
\
>> >             -e
''s/@@subversion@@/$(XEN_SUBVERSION)/g'' \
>> >             -e
''s/@@extraversion@@/$(XEN_EXTRAVERSION)/g'' \
>> >             -e ''s!@@changeset@@!$(shell ((hg parents
--template "{date|date} {rev}:{node|short}" >/dev/null &&
hg parents --template "{date|date} {rev}:{node|short}") || echo
"unavailable") 2>/dev/null)!g'' \
>> >             < include/xen/compile.h.in > $@.new
>> >         @grep \" .banner >> $@.new
>> >         @grep -v \" .banner
>> >         @mv -f $@.new $@
>> 
>> > --
>> > Sander
>> 
>> 
>> 
>> Perhaps use about the same as linux has in scripts/setlocalversion ?
> This is what I was going to suggest. Actually, I had it in mind we
> already did something like!
> Anyway, I think including whatever information the VCS we are building
> from supplied is the right answer and the way Linux does it seems useful
> and appropriate (e.g. using the tag name, appending -dirty etc).
> If we can also extend that to do something sensible in the tarball case
> then even better!
Not much info to get form a extracted tarball i''m afraid ?
Or one should include some extra versioning in the tarball itself and in that
case you don''t know if its "dirty" or not, just a "based
on tarball with changeset/commit"

Since in that case it will describe something more/else than a mecurial
changeset,
question is should the name "changeset" also be replaced by some more
general naming ?


> Ian.

On Tue, 2013-02-26 at 10:31 +0000, Sander Eikelenboom
wrote:
> Tuesday, February 26, 2013, 11:20:07 AM, you wrote:
> 
> > On Mon, 2013-02-25 at 23:00 +0000, Sander Eikelenboom wrote:
> >> Hello Sander,
> 
> > Talking to yourself is a sign of something or other I think ;-)
> 
> primarily hitting the send button a bit before my thoughts actually ended
:-p (at least i hope)
;-)
> 
> 
> >> Monday, February 25, 2013, 11:28:53 PM, you wrote:
> >> 
> >> > Date: Mon, 25 Feb 2013 23:28:53 +0100
> >> > From: Sander Eikelenboom <linux@eikelenboom.it>
> >> > Organization: Eikelenboom IT services
> >> > X-Priority: 3 (Normal)
> >> > Message-ID: <1208255021.20130225232853@eikelenboom.it>
> >> > To: xen-devel <xen-devel@lists.xen.org>
> >> > Subject: Changeset / commit id not incorporated in build
after switch to git
> >> > MIME-Version: 1.0
> >> > Content-Type: text/plain; charset=us-ascii
> >> > Content-Transfer-Encoding: 7bit
> >> 
> >> > Hi All,
> >> 
> >> > After the switching from mercurial to git, the changeset
isn''t incorporated anymore in the build.
> >> > This makes error reports possibly a bit less verbose (xl
dmesg, serial logs and xl info now omit the changeset (or commit) info)
> >> 
> >> > Git doesn''t have the concept of changesets afaik and
mercurial is, while deprecated, still used as mirror.
> >> 
> >> > So what would be wise:
> >> >    - just replace the changeset with the git commit sha-1
hash (always)
> >> >    - use changeset when a mercurial tree is detected or the
last git commit sha-1 (and date ?) when a git tree is detected
> >> >    - make a separate "commit" entry besides the
changeset and leave one undefined
> >> 
> >> > xen/Makefile currently has:
> >> 
> >> > # compile.h contains dynamic build info. Rebuilt on every
''make'' invocation.
> >> > include/xen/compile.h: include/xen/compile.h.in .banner
> >> >         @sed -e ''s/@@date@@/$(shell LC_ALL=C
date)/g'' \
> >> >             -e ''s/@@time@@/$(shell LC_ALL=C date
+%T)/g'' \
> >> >             -e
''s/@@whoami@@/$(XEN_WHOAMI)/g'' \
> >> >             -e
''s/@@domain@@/$(XEN_DOMAIN)/g'' \
> >> >             -e ''s/@@hostname@@/$(shell
hostname)/g'' \
> >> >             -e ''s!@@compiler@@!$(shell $(CC)
$(CFLAGS) --version 2>&1 | head -1)!g'' \
> >> >             -e
''s/@@version@@/$(XEN_VERSION)/g'' \
> >> >             -e
''s/@@subversion@@/$(XEN_SUBVERSION)/g'' \
> >> >             -e
''s/@@extraversion@@/$(XEN_EXTRAVERSION)/g'' \
> >> >             -e ''s!@@changeset@@!$(shell ((hg parents
--template "{date|date} {rev}:{node|short}" >/dev/null &&
hg parents --template "{date|date} {rev}:{node|short}") || echo
"unavailable") 2>/dev/null)!g'' \
> >> >             < include/xen/compile.h.in > $@.new
> >> >         @grep \" .banner >> $@.new
> >> >         @grep -v \" .banner
> >> >         @mv -f $@.new $@
> >> 
> >> > --
> >> > Sander
> >> 
> >> 
> >> 
> >> Perhaps use about the same as linux has in scripts/setlocalversion
?
> 
> > This is what I was going to suggest. Actually, I had it in mind we
> > already did something like!
> 
> > Anyway, I think including whatever information the VCS we are building
> > from supplied is the right answer and the way Linux does it seems
useful
> > and appropriate (e.g. using the tag name, appending -dirty etc).
> 
> > If we can also extend that to do something sensible in the tarball
case
> > then even better!
> 
> Not much info to get form a extracted tarball i''m afraid ?
I think we used "hg archive" in the past and expect we will use
"git
archive" in the future to generate the tarballs, which I think causes a
magic .{git,hg}foo to be dropped into the resulting tarball. I noticed
that Linux''s set_version checks for .scmversion which I suppose might
be
the same thing?
> Or one should include some extra versioning in the tarball itself and
> in that case you don''t know if its "dirty" or not, just
a "based on
> tarball with changeset/commit"
Yes, I don''t think we can track dirtyness in this case, but we can at
least note that it was "tarball based on foo".
> Since in that case it will describe something more/else than a mecurial
changeset,
> question is should the name "changeset" also be replaced by some
more general naming ?
I think that would be a fair bit of unnecessary churn for only a
moderate amount of benefit (I''m sure people can cope with the word
changeset meaning something a bit more generic).

Ian.