Hi, When does Xen exactly close the disks an instance is using after it is shut down? And how is this related to the output of `xm list`? We (the Ganeti <https://code.google.com/p/ganeti/> project) believe to experience some timing issues with disks still being used by Xen. Here''s essentially what we do: * We have a DRBD volume on top of LVM. That''s the disk for our VM. * To shutdown the VM, we do the following: - `xm shutdown <instance>` - `xm list`, and check if <instance> is still there - repeat until `xm list` no longer shows <instance> - de-configure the DRBD volume The last step fails every now and then, stating that the volume is still in use, presumably because Xen still holds the DRBD volume open. So I was wondering when `xm list` stops to display an instance. Is it removed from the list after the instance is shut down and all resources (including handles) are released by Xen? Or might it be possible that instance disks are still held open, even if `xm list` does no longer show the instance? Is there a more reliable way to find out when Xen does no longer use a disk? Please note that after a short while the DRBD volume can be de-configured manually, that''s what makes me believe that `xm list` is not 100% in sync with Xen''s open handles for disks. Thanks, Thomas -- Thomas Thrainer | Software Engineer | thomasth@google.com | Google Germany GmbH Dienerstr. 12 80331 München Registergericht und -nummer: Hamburg, HRB 86891 Sitz der Gesellschaft: Hamburg Geschäftsführer: Graham Law, Katherine Stephens _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Hi, The writing of what the host considers a file (it never mounts the guest file system so, as far as it''s concerned, the guest''s disk is just a very large file) the writing if it will probably be subject to write caching on the host. I believe the default for writing away the cache it''s every 30 seconds. It could be that there is still unwritten data in the cache. I would try either issuing a sync command before unmounting your volume to force the cache to be written. That might help. Otherwise, I''d put a 60 second sleep after the guest has disappeared from the list before dismounting its volume. Cheers, Paul. Thomas Thrainer <thomasth@google.com> wrote:>Hi, > >When does Xen exactly close the disks an instance is using after it is >shut >down? And how is this related to the output of `xm list`? > >We (the Ganeti <https://code.google.com/p/ganeti/> project) believe to >experience some timing issues with disks still being used by Xen. >Here''s >essentially what we do: > * We have a DRBD volume on top of LVM. That''s the disk for our VM. > * To shutdown the VM, we do the following: > - `xm shutdown <instance>` > - `xm list`, and check if <instance> is still there > - repeat until `xm list` no longer shows <instance> > - de-configure the DRBD volume > >The last step fails every now and then, stating that the volume is >still in >use, presumably because Xen still holds the DRBD volume open. So I was >wondering when `xm list` stops to display an instance. Is it removed >from >the list after the instance is shut down and all resources (including >handles) are released by Xen? Or might it be possible that instance >disks >are still held open, even if `xm list` does no longer show the >instance? Is >there a more reliable way to find out when Xen does no longer use a >disk? > >Please note that after a short while the DRBD volume can be >de-configured >manually, that''s what makes me believe that `xm list` is not 100% in >sync >with Xen''s open handles for disks. > >Thanks, >Thomas > > >-- >Thomas Thrainer | Software Engineer | thomasth@google.com | > > Google Germany GmbH >Dienerstr. 12 >80331 München > >Registergericht und -nummer: Hamburg, HRB 86891 >Sitz der Gesellschaft: Hamburg >Geschäftsführer: Graham Law, Katherine Stephens > > >------------------------------------------------------------------------ > >_______________________________________________ >Xen-users mailing list >Xen-users@lists.xen.org >http://lists.xen.org/xen-users-- Sent from Kaiten Mail on Android. Please excuse my brevity. _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
On Wed, 2013-05-08 at 09:24 +0100, Thomas Thrainer wrote:> Hi, > > > When does Xen exactly close the disks an instance is using after it is > shut down? And how is this related to the output of `xm list`? > > > We (the Ganeti project) believe to experience some timing issues with > disks still being used by Xen. Here''s essentially what we do: > * We have a DRBD volume on top of LVM. That''s the disk for our VM. > * To shutdown the VM, we do the following: > - `xm shutdown <instance>` > - `xm list`, and check if <instance> is still there > - repeat until `xm list` no longer shows <instance> > - de-configure the DRBD volume > > > The last step fails every now and then, stating that the volume is > still in use, presumably because Xen still holds the DRBD volume open. > So I was wondering when `xm list` stops to display an instance. Is it > removed from the list after the instance is shut down and all > resources (including handles) are released by Xen?I can''t say for sure with xm/xend but I think when the domain is destroyed the *request* for all resources to be freed have been made (i.e. blkback has been asked to shutdown) but I don''t know if xend will actually wait for them to finish. AFAIK libxl/xl does wait for the backend to finish clearing up.> Or might it be possible that instance disks are still held open, even > if `xm list` does no longer show the instance? Is there a more > reliable way to find out when Xen does no longer use a disk?You might be able to poll xenstore for the backend going away or perhaps integrate with the block hotplug scripts. However I can''t say for sure if either of those will actually work with xend... Ian.