I''m having these problems when using "xl" in the latest c/s
of 4.2
(25912:651d965ee2c0):
- It''s not allowing multiple domains to use the same images, at least
under tapdisk2. If I start one domain using an image (a read-only CD-ROM
image, for instance), then start another, I can see in "tap-ctl list"
that only one of the domains actually receives the proper access. The
other either never gets it, or it is disconnected after it''s created --
I can''t readily tell which.
- When I use "xl destroy" on a domain that also has an associated
stubdom, "xl" tries to destroy the tapdisk2 entries for both the
domain
and its stubdomain, resulting in errors like these:
# xl destroy testvds4
libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable
to find type aio disk /servers/isos/DummyCD.iso: No such file or directory
libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable
to find type aio disk /servers/customers/testvds4.img: No such file or
directory
libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable
to find type aio disk /servers/isos/DummyCD-2.iso: No such file or directory
libxl: error: libxl.c:1466:devices_destroy_cb: libxl__devices_destroy
failed for 19
libxl: error: libxl.c:1466:devices_destroy_cb: libxl__devices_destroy
failed for 18
If I''m reading the source correctly, it seems as though the various
destruction functions aren''t passing down the domid and ultimately
libxl__device_destroy_tapdisk is calling tap_ctl_find, which just
globally matches based on the type and name of the image, causing the
errors on destroy. I was trying to find a quick fix, but it looks like
any proper one will require querying xenstore for the tapdisk entries
specific to the domain, and I''m not immediately able to make that
change, so I''ve personally just switched back to using "xm"
for the time
being. (I don''t know if this relates to the first bug re: multiple
domains using the same image files).
Another, much more minor, bug in "xl" that I''ve found is that
"xl
uptime" will not run without an argument; it is supposed to just show
all uptimes in this circumstance. This is an easy one-liner fix of
modifying line:
tools/libxl/xl_cmdimpl.c:5749: while ((opt = def_getopt(argc, argv,
"s", "uptime", 1)) != -1) {
into this:
tools/libxl/xl_cmdimpl.c:5749: while ((opt = def_getopt(argc, argv,
"s", "uptime", 0)) != -1) {
Also, a couple of small typos in "xl" --
libxl.c:556: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting
domain info list");
libxl.c:575: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting
domain info list");
libxl.c:678: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting
domain info list");
libxl.c:1393: LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "non-existant
domain %d", domid);
Also, running "xl sched-credit" always gives me an error, even though
it
seems to work. Maybe it should be silent if cpu pools aren''t defined?
Maybe it''s supposed to be creating a second cpupool for the second CPU
in this system, and isn''t?
# xl sched-credit
libxl: error: libxl.c:596:cpupool_info: failed to get info for cpupool1
: No such file or directory
Cpupool Pool-0: tslice=5ms ratelimit=500us
Name ID Weight Cap
Domain-0 0 65535 0
As an additional note about 4.2 -- this isn''t an "xl" issue,
but rather
a build problem -- even though I use "./configure
--libdir=/usr/lib64",
the build is still creating a "dist/install/usr/lib" directory, and
because of this, running ./install.sh nukes my system''s
"/usr/lib" ->
"/usr/lib64" softlink (which in turn causes all sorts of other
problems
until it''s corrected). Files that are being put in that tree:
xen-4.2-testing.hg # find dist/install/usr/lib
dist/install/usr/lib
dist/install/usr/lib/xen
dist/install/usr/lib/xen/boot
dist/install/usr/lib/xen/boot/xenstore-stubdom.gz
dist/install/usr/lib/xen/boot/ioemu-stubdom.gz
dist/install/usr/lib/xen/boot/pv-grub-x86_64.gz
dist/install/usr/lib/xen/boot/hvmloader
dist/install/usr/lib/xen/boot/pv-grub-x86_32.gz
dist/install/usr/lib/xen/bin
dist/install/usr/lib/xen/bin/stubdom-dm
dist/install/usr/lib/xen/bin/qemu-io
dist/install/usr/lib/xen/bin/qemu-nbd
dist/install/usr/lib/xen/bin/qemu-img
dist/install/usr/lib/xen/bin/xenpaging
dist/install/usr/lib/xen/bin/qemu-dm
dist/install/usr/lib/xen/bin/qemu-system-i386
dist/install/usr/lib/xen/bin/qemu-ga
dist/install/usr/lib/xen/bin/stubdompath.sh
-John
Ian Campbell
2012-Nov-09 13:29 UTC
Re: Bugs in xl that affect stubdom+tapdisk2, and others
On Thu, 2012-11-08 at 20:45 +0000, John Weekes wrote:> - It''s not allowing multiple domains to use the same images, at least > under tapdisk2. If I start one domain using an image (a read-only CD-ROM > image, for instance), then start another, I can see in "tap-ctl list" > that only one of the domains actually receives the proper access. The > other either never gets it, or it is disconnected after it''s created -- > I can''t readily tell which. > > - When I use "xl destroy" on a domain that also has an associated > stubdom, "xl" tries to destroy the tapdisk2 entries for both the domain > and its stubdomain, resulting in errors like these: > > # xl destroy testvds4 > libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable > to find type aio disk /servers/isos/DummyCD.iso: No such file or directory > libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable > to find type aio disk /servers/customers/testvds4.img: No such file or > directory > libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable > to find type aio disk /servers/isos/DummyCD-2.iso: No such file or directory > libxl: error: libxl.c:1466:devices_destroy_cb: libxl__devices_destroy > failed for 19 > libxl: error: libxl.c:1466:devices_destroy_cb: libxl__devices_destroy > failed for 18 > > If I''m reading the source correctly, it seems as though the various > destruction functions aren''t passing down the domid and ultimately > libxl__device_destroy_tapdisk is calling tap_ctl_find, which just > globally matches based on the type and name of the image, causing the > errors on destroy.Both of these sound like genuine bugs which should be fixed, thanks for reporting. Are either of them something you might be interested in tackling? I''m nore than happy to give guidance etc. For clarity it might be useful to have the config files for the various domains involved here.> Another, much more minor, bug in "xl" that I''ve found is that "xl > uptime" will not run without an argument; it is supposed to just show > all uptimes in this circumstance. This is an easy one-liner fix of > modifying line: > > tools/libxl/xl_cmdimpl.c:5749: while ((opt = def_getopt(argc, argv, > "s", "uptime", 1)) != -1) { > > into this: > > tools/libxl/xl_cmdimpl.c:5749: while ((opt = def_getopt(argc, argv, > "s", "uptime", 0)) != -1) { > > Also, a couple of small typos in "xl" -- > > libxl.c:556: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting > domain info list"); > libxl.c:575: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting > domain info list"); > libxl.c:678: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting > domain info list"); > libxl.c:1393: LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "non-existant > domain %d", domid);Would you mind submitting patches for these two issues please? See http://wiki.xen.org/wiki/Submitting_Xen_Patches for some hints/tip/guidance.> Also, running "xl sched-credit" always gives me an error, even though it > seems to work. Maybe it should be silent if cpu pools aren''t defined?George (CCd) submitted a fix for this a while back (see http://marc.info/?l=xen-devel&m=134502346714773). There is an outstanding question though.> As an additional note about 4.2 -- this isn''t an "xl" issue, but rather > a build problem -- even though I use "./configure --libdir=/usr/lib64", > the build is still creating a "dist/install/usr/lib" directory, and > because of this, running ./install.sh nukes my system''s "/usr/lib" -> > "/usr/lib64" softlink (which in turn causes all sorts of other problems > until it''s corrected). Files that are being put in that tree: > > xen-4.2-testing.hg # find dist/install/usr/lib > dist/install/usr/libThis is an interesting one. I''m not sure how we can avoid this behaviour, perhaps there is a tar option to cause it to follow rather than clobber synlinks? The only other choice I can think of is to change install.sh to use a (long) explicit list of $(FOO_DIR)/* entries instead of the single * to avoid including system directories, or to otherwise make install.sh more fragile and prone to bitrot :-( I must confess I didn''t realise anyone used install.sh -- is there some advantage to it over make foo-install? OOI which distro has this lib->lib64 link and what issues does removing it create?> dist/install/usr/lib/xenThis is $(XENFIRMWAREDIR). I have a feeling there is a reason this is not using $(LIBDIR), but I don''t recall what it was -- Roget/Matt do you remember? Thanks again for reporting all these. Ian.> dist/install/usr/lib/xen/boot > dist/install/usr/lib/xen/boot/xenstore-stubdom.gz > dist/install/usr/lib/xen/boot/ioemu-stubdom.gz > dist/install/usr/lib/xen/boot/pv-grub-x86_64.gz > dist/install/usr/lib/xen/boot/hvmloader > dist/install/usr/lib/xen/boot/pv-grub-x86_32.gz > dist/install/usr/lib/xen/bin > dist/install/usr/lib/xen/bin/stubdom-dm > dist/install/usr/lib/xen/bin/qemu-io > dist/install/usr/lib/xen/bin/qemu-nbd > dist/install/usr/lib/xen/bin/qemu-img > dist/install/usr/lib/xen/bin/xenpaging > dist/install/usr/lib/xen/bin/qemu-dm > dist/install/usr/lib/xen/bin/qemu-system-i386 > dist/install/usr/lib/xen/bin/qemu-ga > dist/install/usr/lib/xen/bin/stubdompath.sh > > -John > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Fri, Nov 09, 2012 at 01:29:04PM +0000, Ian Campbell wrote:> On Thu, 2012-11-08 at 20:45 +0000, John Weekes wrote: > > As an additional note about 4.2 -- this isn''t an "xl" issue, but rather > > a build problem -- even though I use "./configure --libdir=/usr/lib64", > > the build is still creating a "dist/install/usr/lib" directory, and > > because of this, running ./install.sh nukes my system''s "/usr/lib" -> > > "/usr/lib64" softlink (which in turn causes all sorts of other problems > > until it''s corrected). Files that are being put in that tree: > > > > xen-4.2-testing.hg # find dist/install/usr/lib > > dist/install/usr/lib > > This is an interesting one. I''m not sure how we can avoid this > behaviour, perhaps there is a tar option to cause it to follow rather > than clobber synlinks? > > The only other choice I can think of is to change install.sh to use a > (long) explicit list of $(FOO_DIR)/* entries instead of the single * to > avoid including system directories, or to otherwise make install.sh more > fragile and prone to bitrot :-(Oh dear. If we find that there''s not a good reason for it anymore, I''d think it would be better to remove install.sh instead.> I must confess I didn''t realise anyone used install.sh -- is there some > advantage to it over make foo-install? > > OOI which distro has this lib->lib64 link and what issues does removing > it create? > > > dist/install/usr/lib/xen > > This is $(XENFIRMWAREDIR). I have a feeling there is a reason this is > not using $(LIBDIR), but I don''t recall what it was -- Roget/Matt do you > remember?There''s no need for these files to be multilib compatible, as they are not runtime DSOs. They''re internal to Xen and I think that they should be moved to $LIBEXECDIR instead, like /usr/libexec/xen/. Unfortunately there''s not much agreement on the usefulness of /usr/libexec, so we''ll need to make sure that packagers can adjust the paths for their systems'' standards. For 4.2 we didn''t want to change where these files were being placed, and I believe they''ve bin in $PREFIX/lib/ for quite some time. Matt> Thanks again for reporting all these. > > Ian. > > > dist/install/usr/lib/xen/boot > > dist/install/usr/lib/xen/boot/xenstore-stubdom.gz > > dist/install/usr/lib/xen/boot/ioemu-stubdom.gz > > dist/install/usr/lib/xen/boot/pv-grub-x86_64.gz > > dist/install/usr/lib/xen/boot/hvmloader > > dist/install/usr/lib/xen/boot/pv-grub-x86_32.gz > > dist/install/usr/lib/xen/bin > > dist/install/usr/lib/xen/bin/stubdom-dm > > dist/install/usr/lib/xen/bin/qemu-io > > dist/install/usr/lib/xen/bin/qemu-nbd > > dist/install/usr/lib/xen/bin/qemu-img > > dist/install/usr/lib/xen/bin/xenpaging > > dist/install/usr/lib/xen/bin/qemu-dm > > dist/install/usr/lib/xen/bin/qemu-system-i386 > > dist/install/usr/lib/xen/bin/qemu-ga > > dist/install/usr/lib/xen/bin/stubdompath.sh
Ian Campbell
2012-Nov-12 09:49 UTC
Re: Bugs in xl that affect stubdom+tapdisk2, and others
On Fri, 2012-11-09 at 19:55 +0000, Matt Wilson wrote:> On Fri, Nov 09, 2012 at 01:29:04PM +0000, Ian Campbell wrote: > > On Thu, 2012-11-08 at 20:45 +0000, John Weekes wrote: > > > As an additional note about 4.2 -- this isn''t an "xl" issue, but rather > > > a build problem -- even though I use "./configure --libdir=/usr/lib64", > > > the build is still creating a "dist/install/usr/lib" directory, and > > > because of this, running ./install.sh nukes my system''s "/usr/lib" -> > > > "/usr/lib64" softlink (which in turn causes all sorts of other problems > > > until it''s corrected). Files that are being put in that tree: > > > > > > xen-4.2-testing.hg # find dist/install/usr/lib > > > dist/install/usr/lib > > > > This is an interesting one. I''m not sure how we can avoid this > > behaviour, perhaps there is a tar option to cause it to follow rather > > than clobber synlinks? > > > > The only other choice I can think of is to change install.sh to use a > > (long) explicit list of $(FOO_DIR)/* entries instead of the single * to > > avoid including system directories, or to otherwise make install.sh more > > fragile and prone to bitrot :-( > > Oh dear. If we find that there''s not a good reason for it anymore, I''d > think it would be better to remove install.sh instead.That''s one option. If the list could be autogenerated (e.g configure doing install.sh.in -> install.sh) then that might make it less prone to problems.> > I must confess I didn''t realise anyone used install.sh -- is there some > > advantage to it over make foo-install? > > > > OOI which distro has this lib->lib64 link and what issues does removing > > it create? > > > > > dist/install/usr/lib/xen > > > > This is $(XENFIRMWAREDIR). I have a feeling there is a reason this is > > not using $(LIBDIR), but I don''t recall what it was -- Roget/Matt do you > > remember? > > There''s no need for these files to be multilib compatible, as they are > not runtime DSOs. They''re internal to Xen and I think that they should > be moved to $LIBEXECDIR instead, like /usr/libexec/xen/. Unfortunately > there''s not much agreement on the usefulness of /usr/libexec, so we''ll > need to make sure that packagers can adjust the paths for their > systems'' standards.Our configure seem to provide --libexecdir and I think packagers on platforms which don''t use /usr/libexec (e.g. Debian & derivatives) are probably pretty used to passing that option.> For 4.2 we didn''t want to change where these files were being placed, > and I believe they''ve bin in $PREFIX/lib/ for quite some time.I remember now, yes deferring from 4.2 was the right thing to do but now that 4.3 is in swing this seems like as good a time as any to be making a change like this. Ian.
I took a look at the below concerns again in 4.3. The summarized concerns were: 1. Multiple domains not being able to use the same read-only images. 2. "xl destroy" giving error messages when tapdisk2 and stub domains are used. 3. "xl uptime" not working without an argument. 4. Minor typos in xl output. 5. A mistaken warning printed by "xl sched-credit". 6. Build issues related to /usr/lib being used even though ./configure was run with "--libdir=/usr/lib64" The current situation: 1. Worked around: It appears that tapdisk2 has been disabled for read-only images, with qdisk used instead, presumably to work around this concern. 2. Still an issue: These errors still appear. But, since tapdisk2 is no longer used for CD images, they are mostly harmless now. 3. Still an issue: I am submitting a patch to address this. 4. Still an issue: I am submitting a patch to address this. 5. Fixed: I am no longer seeing the warning. 6. Still an issue: But, not a major one. I also noticed a newly-introduced bug: 7. rtc_timeoffset is not accepting negative values. Attempting to use a negative value causes an error to be printed, with domain creation halted. I am submitting a patch separately to address #7 along with #3 and #4. I''ve tested the patch and it works properly for me. -John On 11/8/2012 12:45 PM, John Weekes wrote:> I''m having these problems when using "xl" in the latest c/s of 4.2 > (25912:651d965ee2c0): > > - It''s not allowing multiple domains to use the same images, at least > under tapdisk2. If I start one domain using an image (a read-only > CD-ROM image, for instance), then start another, I can see in "tap-ctl > list" that only one of the domains actually receives the proper > access. The other either never gets it, or it is disconnected after > it''s created -- I can''t readily tell which. > > - When I use "xl destroy" on a domain that also has an associated > stubdom, "xl" tries to destroy the tapdisk2 entries for both the > domain and its stubdomain, resulting in errors like these: > > # xl destroy testvds4 > libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable > to find type aio disk /servers/isos/DummyCD.iso: No such file or > directory > libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable > to find type aio disk /servers/customers/testvds4.img: No such file or > directory > libxl: error: libxl_blktap2.c:73:libxl__device_destroy_tapdisk: Unable > to find type aio disk /servers/isos/DummyCD-2.iso: No such file or > directory > libxl: error: libxl.c:1466:devices_destroy_cb: libxl__devices_destroy > failed for 19 > libxl: error: libxl.c:1466:devices_destroy_cb: libxl__devices_destroy > failed for 18 > > If I''m reading the source correctly, it seems as though the various > destruction functions aren''t passing down the domid and ultimately > libxl__device_destroy_tapdisk is calling tap_ctl_find, which just > globally matches based on the type and name of the image, causing the > errors on destroy. I was trying to find a quick fix, but it looks like > any proper one will require querying xenstore for the tapdisk entries > specific to the domain, and I''m not immediately able to make that > change, so I''ve personally just switched back to using "xm" for the > time being. (I don''t know if this relates to the first bug re: > multiple domains using the same image files). > > Another, much more minor, bug in "xl" that I''ve found is that "xl > uptime" will not run without an argument; it is supposed to just show > all uptimes in this circumstance. This is an easy one-liner fix of > modifying line: > > tools/libxl/xl_cmdimpl.c:5749: while ((opt = def_getopt(argc, argv, > "s", "uptime", 1)) != -1) { > > into this: > > tools/libxl/xl_cmdimpl.c:5749: while ((opt = def_getopt(argc, argv, > "s", "uptime", 0)) != -1) { > > Also, a couple of small typos in "xl" -- > > libxl.c:556: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting > domain info list"); > libxl.c:575: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting > domain info list"); > libxl.c:678: LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting > domain info list"); > libxl.c:1393: LIBXL__LOG(ctx, LIBXL__LOG_ERROR, "non-existant > domain %d", domid); > > Also, running "xl sched-credit" always gives me an error, even though > it seems to work. Maybe it should be silent if cpu pools aren''t > defined? Maybe it''s supposed to be creating a second cpupool for the > second CPU in this system, and isn''t? > > # xl sched-credit > libxl: error: libxl.c:596:cpupool_info: failed to get info for cpupool1 > : No such file or directory > Cpupool Pool-0: tslice=5ms ratelimit=500us > Name ID Weight Cap > Domain-0 0 65535 0 > > As an additional note about 4.2 -- this isn''t an "xl" issue, but > rather a build problem -- even though I use "./configure > --libdir=/usr/lib64", the build is still creating a > "dist/install/usr/lib" directory, and because of this, running > ./install.sh nukes my system''s "/usr/lib" -> "/usr/lib64" softlink > (which in turn causes all sorts of other problems until it''s > corrected). Files that are being put in that tree: > > xen-4.2-testing.hg # find dist/install/usr/lib > dist/install/usr/lib > dist/install/usr/lib/xen > dist/install/usr/lib/xen/boot > dist/install/usr/lib/xen/boot/xenstore-stubdom.gz > dist/install/usr/lib/xen/boot/ioemu-stubdom.gz > dist/install/usr/lib/xen/boot/pv-grub-x86_64.gz > dist/install/usr/lib/xen/boot/hvmloader > dist/install/usr/lib/xen/boot/pv-grub-x86_32.gz > dist/install/usr/lib/xen/bin > dist/install/usr/lib/xen/bin/stubdom-dm > dist/install/usr/lib/xen/bin/qemu-io > dist/install/usr/lib/xen/bin/qemu-nbd > dist/install/usr/lib/xen/bin/qemu-img > dist/install/usr/lib/xen/bin/xenpaging > dist/install/usr/lib/xen/bin/qemu-dm > dist/install/usr/lib/xen/bin/qemu-system-i386 > dist/install/usr/lib/xen/bin/qemu-ga > dist/install/usr/lib/xen/bin/stubdompath.sh > > -John > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel