Pkg xen devel - Feb 2019 - Bug#923013: xen: FTBFS when built with dpkg-buildpackage -A

Package: src:xen
Version: 4.11.1+26-g87f51bf366-2
Severity: serious
Tags: ftbfs

Dear maintainer:

I tried to build this package in sid with "dpkg-buildpackage -A" but
it failed:

--------------------------------------------------------------------------------
[...]
 debian/rules build-indep
dh build-indep --with=python2
   debian/rules override_dh_update_autotools_config
make[1]: Entering directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
cp /usr/share/misc/{config.sub,config.guess} .
make[1]: Leaving directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
   dh_autoreconf -i
   debian/rules override_dh_auto_configure
make[1]: Entering directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
cp debian/xen-kconfig xen/.config
make -C xen olddefconfig  XEN_COMPILE_ARCH=x86_64 XEN_TARGET_ARCH=x86_64 
make[2]: Entering directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366/xen'
make -f
/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366/xen/tools/kconfig/Makefile.kconfig
ARCH=x86_64 SRCARCH=x86 HOSTCC="gcc" HOSTCXX="g++"
olddefconfig
make[3]: Entering directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366/xen'

[... snipped ...]

   dh_perl -i
   dh_link -i
   dh_strip_nondeterminism -i
   debian/rules override_dh_compress
make[1]: Entering directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
rdfind -makehardlinks true -makeresultsfile false \
	debian/xenstore-utils/usr/bin
Now scanning "debian/xenstore-utils/usr/bin", found 0 files.
Now have 0 files in total.
Removed 0 files due to nonunique device and inode.
Total size is 0 bytes or 0 B
Removed 0 files due to unique sizes from list.0 files left.
Now eliminating candidates based on first bytes:removed 0 files from list.0
files left.
Now eliminating candidates based on last bytes:removed 0 files from list.0 files
left.
Now eliminating candidates based on sha1 checksum:removed 0 files from list.0
files left.
It seems like you have 0 files that are not unique
Totally, 0 B can be reduced.
Now making hard links.
Making 0 links.
:
dh_compress
find debian/xen-hypervisor-*/usr/lib/debug -type f -print0 \
	| xargs -0r gzip -9vn
find: 'debian/xen-hypervisor-*/usr/lib/debug': No such file or directory
make[1]: Leaving directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
   dh_fixperms -i
   debian/rules override_dh_missing
make[1]: Entering directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
dh_missing --fail-missing
dh_missing: etc/bash_completion.d/xl.sh exists in debian/tmp but is not
installed to anywhere
dh_missing: missing files, aborting
	The following debhelper tools have reported what they installed (with files per
package)
	 * dh_install: libxen-dev (30), libxencall1 (2), libxendevicemodel1 (2),
libxenevtchn1 (2), libxenforeignmemory1 (2), libxengnttab1 (2), libxenmisc4.11
(14), libxenstore3.0 (2), libxentoolcore1 (2), libxentoollog1 (2), xen-doc (1),
xen-hypervisor-4.11-amd64 (3), xen-hypervisor-4.11-arm64 (0),
xen-hypervisor-4.11-armhf (0), xen-hypervisor-common (1), xen-system-amd64 (0),
xen-system-arm64 (0), xen-system-armhf (0), xen-utils-4.11 (4), xen-utils-common
(18), xenstore-utils (13)
	 * dh_installdocs: libxen-dev (0), libxencall1 (0), libxendevicemodel1 (0),
libxenevtchn1 (0), libxenforeignmemory1 (0), libxengnttab1 (0), libxenmisc4.11
(0), libxenstore3.0 (0), libxentoolcore1 (0), libxentoollog1 (0), xen-doc (0),
xen-hypervisor-4.11-amd64 (0), xen-hypervisor-4.11-arm64 (0),
xen-hypervisor-4.11-armhf (0), xen-hypervisor-common (0), xen-system-amd64 (0),
xen-system-arm64 (0), xen-system-armhf (0), xen-utils-4.11 (0), xen-utils-common
(0), xenstore-utils (0)
	 * dh_installexamples: libxen-dev (0), libxencall1 (0), libxendevicemodel1 (0),
libxenevtchn1 (0), libxenforeignmemory1 (0), libxengnttab1 (0), libxenmisc4.11
(0), libxenstore3.0 (0), libxentoolcore1 (0), libxentoollog1 (0), xen-doc (0),
xen-hypervisor-4.11-amd64 (0), xen-hypervisor-4.11-arm64 (0),
xen-hypervisor-4.11-armhf (0), xen-hypervisor-common (0), xen-system-amd64 (0),
xen-system-arm64 (0), xen-system-armhf (0), xen-utils-4.11 (0), xen-utils-common
(1), xenstore-utils (0)
	If the missing files are installed by another tool, please file a bug against
it.
	When filing the report, if the tool is not part of debhelper itself, please
reference the
	"Logging helpers and dh_missing" section from the
"PROGRAMMING" guide for debhelper (10.6.3+).
	  (in the debhelper package: /usr/share/doc/debhelper/PROGRAMMING.gz)
	Be sure to test with dpkg-buildpackage -A/-B as the results may vary when only
a subset is built
	For a short-term work-around: Add the files to debian/not-installed
make[1]: *** [debian/rules:316: override_dh_missing] Error 2
make[1]: Leaving directory
'/<<BUILDDIR>>/xen-4.11.1+26-g87f51bf366'
make: *** [debian/rules:147: binary-indep] Error 2
dpkg-buildpackage: error: fakeroot debian/rules binary-indep subprocess returned
exit status 2
--------------------------------------------------------------------------------

Hint: Try splitting override_dh_compress into override_dh_compress-arch
and override_dh_compress-indep.

(Also: Please try source-only uploads, i.e. dpkg-buildpackage -S, so that
this is easier to spot).

Thanks.

Hmm, I said:
> Hint: Try splitting override_dh_compress into override_dh_compress-arch
> and override_dh_compress-indep.
Not really sure that this is a good hint in this case. The first rule showing an
error is certainly this one:

find: 'debian/xen-hypervisor-*/usr/lib/debug': No such file or directory

but maybe it's override_dh_missing the one that would benefit from an
-arch/-indep split.

Thanks.

Hi,

On 2/23/19 12:59 AM, Santiago Vila wrote:
> 
>> Hint: Try splitting override_dh_compress into override_dh_compress-arch
>> and override_dh_compress-indep.
> 
> Not really sure that this is a good hint in this case. The first rule
showing an
> error is certainly this one:
> 
> find: 'debian/xen-hypervisor-*/usr/lib/debug': No such file or
directory
> 
> but maybe it's override_dh_missing the one that would benefit from an
-arch/-indep split.
Can you please help me by formulating a clear problem / question and if
possible some expected things that you would like to have as a result?

What I see flying by is: A B C and also X Y Z and you might R S T, and
also, by the way: F G H.

Hans

On 2/23/19 1:15 AM, Hans van Kranenburg wrote:
> Hi,
> 
> On 2/23/19 12:59 AM, Santiago Vila wrote:
>>
>>> Hint: Try splitting override_dh_compress into
override_dh_compress-arch
>>> and override_dh_compress-indep.
>>
>> Not really sure that this is a good hint in this case. The first rule
showing an
>> error is certainly this one:
>>
>> find: 'debian/xen-hypervisor-*/usr/lib/debug': No such file or
directory
>>
>> but maybe it's override_dh_missing the one that would benefit from
an -arch/-indep split.
> 
> Can you please help me by formulating a clear problem / question and if
> possible some expected things that you would like to have as a result?
> 
> What I see flying by is: A B C and also X Y Z and you might R S T, and
> also, by the way: F G H.
Also, your new bts results in an...

"Serious (policy violations or makes package unfit for release)"

...listed for our packages, which is not nice, in this phase of the
release freeze cycle, because...

The whole thing builds just fine in sid and also in stretch here.

Can you explain why you want to do this?

Hans

On Sat, Feb 23, 2019 at 01:15:49AM +0100, Hans van Kranenburg wrote:
> Can you please help me by formulating a clear problem / question and if
> possible some expected things that you would like to have as a result?
Sure. This is a packaging bug in debian/rules. Packages must build for the
end user with plain "dpkg-buildpackage" but also in the official
autobuilders by doing "dpkg-buildpackage -A" (to generate arch-all
packages) or "dpkg-buildpackage -B" (to generate arch-indep packages).

Either of those failing is a FTBFS bug, hence the serious severity.

To reproduce the problem, please try building the package with 
"dpkg-buildpackage -A", which is exactly what the arch:all
autobuilder would do if the package was uploaded in source-only form.

The hint of splitting dh_foo into dh_foo-arch and dh_foo-indep usually
works for packages that use dh and have this bug, i.e. for packages
which build ok when built with plain "dpkg-buildpackage" but not with
"dpkg-buildpackage -A" or "dpkg-buildpackage -B".

Thanks.

Correction. I said:
> "dpkg-buildpackage -B" (to generate arch-indep packages).
but really meant this:

"dpkg-buildpackage -B" (to generate arch-dep packages).

Sorry for the confusion.

Hi Santiago,

First of all, apologies because the tone in my previous message
yesterday may sound a bit hostile. I was tired and happy that we had a
productive day yesterday, getting ~30 bts closed and an upload with a
lot of fixes done, I just did my own rebuild for stretch-backports, and
then the report that it ftbfs came in, and I was too quick with thinking
"no way, I just built it!". :)

Thanks for your report.

On 2/23/19 7:04 AM, Santiago Vila wrote:
> On Sat, Feb 23, 2019 at 01:15:49AM +0100, Hans van Kranenburg wrote:
> 
>> Can you please help me by formulating a clear problem / question and if
>> possible some expected things that you would like to have as a result?
> 
> Sure. This is a packaging bug in debian/rules. Packages must build for the
> end user with plain "dpkg-buildpackage" but also in the official
> autobuilders by doing "dpkg-buildpackage -A" (to generate
arch-all
> packages) or "dpkg-buildpackage -B" (to generate arch-indep
packages).
I can indeed reproduce the error by doing dpkg-buildpackage -A:

---- >8 ----
dh_missing --fail-missing
dh_missing: etc/bash_completion.d/xl.sh exists in debian/tmp but is not
installed to anywhere
---- >8 ----

This file is installed into the xen-utils-common package as
/usr/share/bash-completion/completions/xl. From
debian/xen-utils-common.install:

---- >8 ----
#! /usr/bin/dh-exec --with=install

[...]

etc/bash_completion.d/xl.sh => usr/share/bash-completion/completions/xl
---- >8 ----
> Either of those failing is a FTBFS bug, hence the serious severity.
> 
> To reproduce the problem, please try building the package with 
> "dpkg-buildpackage -A", which is exactly what the arch:all
> autobuilder would do if the package was uploaded in source-only form.
> 
> The hint of splitting dh_foo into dh_foo-arch and dh_foo-indep usually
> works for packages that use dh and have this bug, i.e. for packages
> which build ok when built with plain "dpkg-buildpackage" but not
with
> "dpkg-buildpackage -A" or "dpkg-buildpackage -B".
All of this seems to be related to the dh-exec usage for putting files
in the xen-utils-common package with a different name. This is the first
time this package is using dh-exec.

If I change the following, dpkg-buildpackage -A completes:

---- >8 ----
diff --git a/debian/xen-utils-common.install
b/debian/xen-utils-common.install
index 60642c9a9c..e83dc0740b 100755
--- a/debian/xen-utils-common.install
+++ b/debian/xen-utils-common.install
@@ -2,7 +2,7 @@

 etc/xen/scripts
 etc/xen/xl*
-etc/bash_completion.d/xl.sh => usr/share/bash-completion/completions/xl
+etc/bash_completion.d/xl.sh usr/share/bash-completion/completions

 etc/default/xendomains
 etc/default/xencommons => /etc/default/xen
---- >8 ----

Interestingly, 'etc/default/xencommons => /etc/default/xen' does not
result in a similar error?

I also found this one, which seems to be related:
https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831786

Do you have an idea / suggestion how we can deal with this?

Hans

> All of this seems to be related to the dh-exec usage for putting files
> in the xen-utils-common package with a different name. This is the first
> time this package is using dh-exec.
>
> [...]
> 
> Interestingly, 'etc/default/xencommons => /etc/default/xen' does
not
> result in a similar error?
> 
> I also found this one, which seems to be related:
> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831786
> 
> Do you have an idea / suggestion how we can deal with this?
Based on your analysis, maybe applying the patch below.

After all, the "fail-missing" thing is just an extra protection which
helps avoiding mistakes when installing things, but in this case
it seems to be more troublesome than helpful.

--- a/debian/rules
+++ b/debian/rules
@@ -307,8 +307,11 @@ override_dh_compress:
 
 # By default, files in debian/tmp which are not handled by anything
 # in rules are ignored.  This makes them into errors.
-override_dh_missing:
-	dh_missing --fail-missing
+#
+# Disabled because of Bug #831786
+#
+# override_dh_missing:
+# 	dh_missing --fail-missing
 
 
 # We are dropping the config file /etc/default/xen which appeared in

Thanks.

Ok, so...

On 2/24/19 3:39 PM, Santiago Vila wrote:
>> All of this seems to be related to the dh-exec usage for putting files
>> in the xen-utils-common package with a different name. This is the
first
>> time this package is using dh-exec.
>>
>> [...]
>>
>> Interestingly, 'etc/default/xencommons => /etc/default/xen'
does not
>> result in a similar error?
>>
>> I also found this one, which seems to be related:
>> https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=831786
>>
>> Do you have an idea / suggestion how we can deal with this?
> 
> Based on your analysis, maybe applying the patch below.
> 
> After all, the "fail-missing" thing is just an extra protection
which
> helps avoiding mistakes when installing things, but in this case
> it seems to be more troublesome than helpful.
> 
> --- a/debian/rules
> +++ b/debian/rules
> @@ -307,8 +307,11 @@ override_dh_compress:
>  
>  # By default, files in debian/tmp which are not handled by anything
>  # in rules are ignored.  This makes them into errors.
> -override_dh_missing:
> -	dh_missing --fail-missing
> +#
> +# Disabled because of Bug #831786
> +#
> +# override_dh_missing:
> +# 	dh_missing --fail-missing
>  
>  
>  # We are dropping the config file /etc/default/xen which appeared in
Oh right. This accidentally quoted line helped me find out that
etc/default/xencommons is listed in debian/not-installed because it was
not installed before. However, now it is again, but it was not removed
from not-installed, and this solves the mystery of dh_missing not
complaining about it.

It seems useful to me to keep the dh_missing --fail-missing, so the
workaround for the dh-exec bug will now be to also list
etc/bash_completion.d/xl.sh in not-installed, with a short explanation
in a comment.

Now I can do dpkg-buildpackage -A without the error.

https://salsa.debian.org/xen-team/debian-xen/commit/69f65e1f7676cb387c9742a4e8ab96ee53e36d1d
(this commit hash will likely disapper again later, but today it'll be
there :) ).

Hans

Your message dated Thu, 28 Feb 2019 17:37:18 +0000
with message-id <E1gzPcU-0007w1-6M at fasolo.debian.org>
and subject line Bug#923013: fixed in xen 4.11.1+26-g87f51bf366-3
has caused the Debian Bug report #923013,
regarding xen: FTBFS when built with dpkg-buildpackage -A
to be marked as done.

This means that you claim that the problem has been dealt with.
If this is not the case it is now your responsibility to reopen the
Bug report if necessary, and/or fix the problem forthwith.

(NB: If you are a system administrator and have no idea what this
message is talking about, this may indicate a serious mail system
misconfiguration somewhere. Please contact owner at bugs.debian.org
immediately.)


-- 
923013: https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=923013
Debian Bug Tracking System
Contact owner at bugs.debian.org with problems
-------------- next part --------------
An embedded message was scrubbed...
From: Santiago Vila <sanvila at debian.org>
Subject: xen: FTBFS when built with dpkg-buildpackage -A
Date: Fri, 22 Feb 2019 23:49:19 +0000
Size: 7186
URL:
<http://alioth-lists.debian.net/pipermail/pkg-xen-devel/attachments/20190228/c3201eb1/attachment-0002.mht>
-------------- next part --------------
An embedded message was scrubbed...
From: Ian Jackson <ijackson at chiark.greenend.org.uk>
Subject: Bug#923013: fixed in xen 4.11.1+26-g87f51bf366-3
Date: Thu, 28 Feb 2019 17:37:18 +0000
Size: 16552
URL:
<http://alioth-lists.debian.net/pipermail/pkg-xen-devel/attachments/20190228/c3201eb1/attachment-0003.mht>