Xen devel - Apr 2013 - blktap3: libxl complaining when a domain gets destroyed

I''m testing blktap3 and when I destroy a domain via "xl destroy
21" I get the following behaviour: "xl destroy 21" seems to pause
for a couple of seconds and then spits out the following:
libxl: error: libxl_device.c:882:device_backend_callback: unable to remove
device with path /local/domain/0/backend/vbd3/21/51712
libxl: error: libxl.c:1453:devices_destroy_cb: libxl__devices_destroy failed for
21
Apart from these messages, the domain seems to be correctly destroyed and there
aren''t any leftovers in XenStore. I''m able to successfully
start and stop the domain multiple times. What is libxl complaining about?

--
Thanos Makatos

On Wed, 2013-04-24 at 12:46 +0100, Thanos Makatos wrote:
> I''m testing blktap3 and when I destroy a domain via "xl
destroy 21" I
> get the following behaviour: "xl destroy 21" seems to pause for a
> couple of seconds and then spits out the following:
> libxl: error: libxl_device.c:882:device_backend_callback: unable to
> remove device with path /local/domain/0/backend/vbd3/21/51712
> libxl: error: libxl.c:1453:devices_destroy_cb: libxl__devices_destroy
> failed for 21
> Apart from these messages, the domain seems to be correctly destroyed
> and there aren''t any leftovers in XenStore. I''m able to
successfully
> start and stop the domain multiple times. What is libxl complaining
> about?
does xl -vvv destroy give any more hints?

The message results from a failure in the call
libxl__ev_devstate_wait(...XenbusStateClosed). Perhaps blktap3 is
cleaning up a bit too diligently and removing something which the
toolstack expects to cleanup?

Ian.

On 24/04/13 13:46, Thanos Makatos wrote:
> I''m testing blktap3 and when I destroy a domain via "xl
destroy 21" I get the following behaviour: "xl destroy 21" seems
to pause for a couple of seconds and then spits out the following:
> libxl: error: libxl_device.c:882:device_backend_callback: unable to remove
device with path /local/domain/0/backend/vbd3/21/51712
This is because libxl will wait for the backend to reach state
''6''
(closed). Even if state 6 is not reached libxl will clean up the backend
path, you should check why blktap3 doesn''t set the backend to state 6.
> libxl: error: libxl.c:1453:devices_destroy_cb: libxl__devices_destroy
failed for 21
> Apart from these messages, the domain seems to be correctly destroyed and
there aren''t any leftovers in XenStore. I''m able to
successfully start and stop the domain multiple times. What is libxl complaining
about?
> 
> --
> Thanos Makatos
> 
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

> This is because libxl will wait for the backend to reach state
''6''
> (closed). Even if state 6 is not reached libxl will clean up the
> backend path, you should check why blktap3 doesn''t set the backend
to
> state 6.
At some point during "xl destroy ..." the backend state is set to 5
(assuming by libxl?). I get a XenStore watch triggering on this event. So how
should I handle this event? Setting the backend state to 6 fails with EPERM,
which I don''t understand since I do modify the backend state when going
from initialised to connected etc.

> > This is because libxl will wait for the backend to reach state
''6''
> > (closed). Even if state 6 is not reached libxl will clean up the
> > backend path, you should check why blktap3 doesn''t set the
backend to
> > state 6.
> 
> At some point during "xl destroy ..." the backend state is set to
5
> (assuming by libxl?). I get a XenStore watch triggering on this event.
> So how should I handle this event? Setting the backend state to 6 fails
> with EPERM, which I don''t understand since I do modify the backend
> state when going from initialised to connected etc.
Hmm I''m looking at the code in libxl__initiate_device_remove
(libxl_device.c) and I see that for qemu back-ends libxl doesn''t wait
for the backend to switch to state 6, it does something different that I
don''t understand. If I just jump to the "out" label it seems
to work fine, though I don''t know the semantics of the domain
destruction process.