This is the second time I run into this linking issue. A few days ago it happend with a make -j 4 or 8 during automated rpm package build. Now it happend with a fresh tree. My local system is openSuSE 11.4 with make 3.82, the failed automated build was either openSUSE 11.4 or 12.1. For some reason the libxenstore.so symlink is created after init-xenstore-domain is about to be linked. The ln command is not visible in this output, I saw the ln invocation in the rpm build log. Hmm, and for some reason the symlink was not created anyway in my local build. A second make run worked ok, libxenstore.so was created. It looks like I can reproduce it by running make clean in tools/xenstore. Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE, and libxenstore.so is also a target. So its not clear to me how make can miss that, or how the dependencies should be listed. Any idea whats going on here? Olaf ... gcc -O1 -fno-omit-frame-pointer -m64 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -D__XEN_TOOLS__ -MMD -MF .xenstored_posix.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -Werror -I. -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/include -c -o xenstored_posix.o xenstored_posix.c gcc -O1 -fno-omit-frame-pointer -m64 -g -fno-strict-aliasing -std=gnu99 -Wall -Wstrict-prototypes -Wdeclaration-after-statement -D__XEN_TOOLS__ -MMD -MF .init-xenstore-domain.o.d -D_LARGEFILE_SOURCE -D_LARGEFILE64_SOURCE -fno-optimize-sibling-calls -Werror -I. -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/include -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc -I/work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/include -c -o init-xenstore-domain.o init-xenstore-domain.c gcc -Wl,-soname -Wl,libxenstore.so.3.0 -shared -o libxenstore.so.3.0.1 xs.opic xs_lib.opic -lpthread ar rcs libxenstore.a xs.o xs_lib.o gcc xs_tdb_dump.o utils.o tdb.o talloc.o -o xs_tdb_dump gcc xenstored_core.o xenstored_watch.o xenstored_domain.o xenstored_transaction.o xs_lib.o talloc.o utils.o tdb.o hashtable.o xenstored_linux.o xenstored_posix.o /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc/libxenctrl.so -o xenstored gcc init-xenstore-domain.o libxenstore.so /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc/libxenctrl.so /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/libxc/libxenguest.so /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/xenstore/libxenstore.so -o init-xenstore-domain gcc: libxenstore.so: Datei oder Verzeichnis nicht gefunden gcc: /work/xen/xendebug/xen-unstable.hg/tools/xenstore/../../tools/xenstore/libxenstore.so: Datei oder Verzeichnis nicht gefunden make[3]: *** [init-xenstore-domain] Fehler 1 make[3]: *** Warte auf noch nicht beendete Prozesse... make[3]: Leaving directory `/work/xen/xendebug/xen-unstable.hg/tools/xenstore'' make[2]: *** [subdir-install-xenstore] Fehler 2 make[2]: Leaving directory `/work/xen/xendebug/xen-unstable.hg/tools'' make[1]: *** [subdirs-install] Fehler 2 make[1]: Leaving directory `/work/xen/xendebug/xen-unstable.hg/tools'' make: *** [install-tools] Fehler 2 make: *** Warte auf noch nicht beendete Prozesse...
On Mon, Feb 20, Olaf Hering wrote:> Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE, > and libxenstore.so is also a target. So its not clear to me how make can > miss that, or how the dependencies should be listed.With this change, putting init-xenstore-domain first, I can not reproduce it anymore. -ALL_TARGETS = libxenstore.so libxenstore.a clients xs_tdb_dump xenstored init-xenstore-domain +ALL_TARGETS = init-xenstore-domain libxenstore.so libxenstore.a clients xs_tdb_dump xenstored Is this correct? Olaf
On Mon, 2012-02-20 at 19:26 +0000, Olaf Hering wrote:> This is the second time I run into this linking issue. A few days ago it > happend with a make -j 4 or 8 during automated rpm package build. Now it > happend with a fresh tree. My local system is openSuSE 11.4 with make > 3.82, the failed automated build was either openSUSE 11.4 or 12.1. > > For some reason the libxenstore.so symlink is created after > init-xenstore-domain is about to be linked. The ln command is not > visible in this output, I saw the ln invocation in the rpm build log. > > Hmm, and for some reason the symlink was not created anyway in my local > build. A second make run worked ok, libxenstore.so was created. It > looks like I can reproduce it by running make clean in tools/xenstore. > > Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE, > and libxenstore.so is also a target. So its not clear to me how make can > miss that, or how the dependencies should be listed. > > Any idea whats going on here?It''s pretty odd isn''t it. I tried: $ make -C tools/xenstore/ clean $ make -C tools/xenstore/ -j12 and couldn''t reproduce. I see the ln before the link lines, even with bigger and smaller -jN. "make -d" will tell you make''s thought processes, might give a hint? Could it be your filesystem? Something odd to do with timestamps on symlinks which upsets your version of make perhaps? (I''m on ext3) I''m really grasping at straws there... The chain of rules for making the symlinks is a bit convoluted. I''d be inclined to have: libxenstore.so libxenstore.so.$(MAJOR) libxenstore.so.$(MAJOR).$(MINOR): xs.opic xs_lib.opic $(CC).... ln -sf libxenstore.so.$(MAJOR).$(MINOR) libxenstore.so.$(MAJOR) ln -sf libxenstore.so.$(MAJOR) libxenstore.so But I don''t see why it would help or matter in practice. Ian.
On Tue, Feb 21, Ian Campbell wrote:> On Mon, 2012-02-20 at 19:26 +0000, Olaf Hering wrote: > > Any idea whats going on here? > > It''s pretty odd isn''t it. > > I tried: > $ make -C tools/xenstore/ clean > $ make -C tools/xenstore/ -j12 > and couldn''t reproduce. I see the ln before the link lines, even with > bigger and smaller -jN.I use this script ''time bash ../build.sh -d xen tools &> output.txt'': #!/bin/bash set -x unset LANG unset ${!LC_*} tgt="xen tools" if test -n "$1" then tgt=$@ fi if pushd tools/xenstore then make clean popd fi time XEN_DOMAIN=localhost DISTDIR=/dev/shm/install-${PWD//\//_} make -j 3 ${tgt} ; echo $? It seems to trigger with either ''make -d'' or ''make -d xen tools'', but seldom with ''make -d tools''. It does not trigger with -j 1, but with -j 2 or 3.> "make -d" will tell you make''s thought processes, might give a hint?Thanks for that hint, I will try to make sense of the (huge) output.> Could it be your filesystem? Something odd to do with timestamps on > symlinks which upsets your version of make perhaps? (I''m on ext3)I''m on ext3. libxenstore.so is not there, thats the issue. I wonder why libxenstore.so and libxenstore.a is listed at all in ALL_TARGETS? Perhaps that confuses make? But removing both does not help either. To me it looks like $(LIBXENSTORE) does not serve as a proper dependency. Olaf
On Tue, Feb 21, Ian Campbell wrote:> On Mon, 2012-02-20 at 19:26 +0000, Olaf Hering wrote: > > Any idea whats going on here? > > It''s pretty odd isn''t it.I can not reproduce the link failure with make 3.81. Olaf
Olaf Hering writes ("[Xen-devel] libxenstore.so Makefile dependency
issue"):>
> This is the second time I run into this linking issue. A few days ago it
> happend with a make -j 4 or 8 during automated rpm package build. Now it
> happend with a fresh tree. My local system is openSuSE 11.4 with make
> 3.82, the failed automated build was either openSUSE 11.4 or 12.1.
>
> For some reason the libxenstore.so symlink is created after
> init-xenstore-domain is about to be linked. The ln command is not
> visible in this output, I saw the ln invocation in the rpm build log.
>
> Hmm, and for some reason the symlink was not created anyway in my local
> build. A second make run worked ok, libxenstore.so was created. It
> looks like I can reproduce it by running make clean in tools/xenstore.
>
> Looking at the Makefile, init-xenstore-domain depends on LIBXENSTORE,
> and libxenstore.so is also a target. So its not clear to me how make can
> miss that, or how the dependencies should be listed.
This is indeed mysterious. I haven''t managed to reproduce it either.
I tried the patch below, with make -j1000, and everything worked the
way it should do.
Ian.
diff -r 6a34a42a2b5d tools/xenstore/Makefile
--- a/tools/xenstore/Makefile Tue Feb 21 18:01:04 2012 +0000
+++ b/tools/xenstore/Makefile Tue Feb 21 18:23:46 2012 +0000
@@ -82,7 +82,7 @@ libxenstore.so.$(MAJOR): libxenstore.so.
xs.opic: CFLAGS += -DUSE_PTHREAD
libxenstore.so.$(MAJOR).$(MINOR): xs.opic xs_lib.opic
- $(CC) $(LDFLAGS) -Wl,$(SONAME_LDFLAG) -Wl,libxenstore.so.$(MAJOR)
$(SHLIB_LDFLAGS) -o $@ $^ $(SOCKET_LIBS) -lpthread $(APPEND_LDFLAGS)
+ set -x; while test ! -f go$$$$; do sleep 10; done; $(CC) $(LDFLAGS)
-Wl,$(SONAME_LDFLAG) -Wl,libxenstore.so.$(MAJOR) $(SHLIB_LDFLAGS) -o $@ $^
$(SOCKET_LIBS) -lpthread $(APPEND_LDFLAGS)
libxenstore.a: xs.o xs_lib.o
$(AR) rcs $@ $^
On Tue, Feb 21, Ian Jackson wrote:> This is indeed mysterious. I haven''t managed to reproduce it either. > > I tried the patch below, with make -j1000, and everything worked the > way it should do.Is this with make 3.82? I think its time to ask on a make mailing list for advice. The NEWS file for 3.82 did not contain any (obvious related) incompatible change. Olaf
Olaf Hering writes ("Re: [Xen-devel] libxenstore.so Makefile dependency
issue"):> Is this with make 3.82?
>
> I think its time to ask on a make mailing list for advice. The NEWS file
> for 3.82 did not contain any (obvious related) incompatible change.
3.81.
You could try the patch from my last mail which I think should enable
to you repro the bug every time.
Ian.
On Tue, Feb 21, Ian Jackson wrote:> Olaf Hering writes ("Re: [Xen-devel] libxenstore.so Makefile dependency issue"): > > Is this with make 3.82? > > > > I think its time to ask on a make mailing list for advice. The NEWS file > > for 3.82 did not contain any (obvious related) incompatible change. > > 3.81. > > You could try the patch from my last mail which I think should enable > to you repro the bug every time.Not really. If I put a simple sleep 1 or 2 in there libxenstore.so is not created in time, but sleep 3 and more helps. Olaf