14.09.2018 15:43, Jiri Denemark пишет:> On Thu, Sep 13, 2018 at 19:37:00 +0400, Dmitry Melekhov wrote: >> >> 13.09.2018 18:57, Jiri Denemark пишет: >>> On Thu, Sep 13, 2018 at 18:38:57 +0400, Dmitry Melekhov wrote: >>>> 13.09.2018 17:47, Jiri Denemark пишет: >>>>> On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: >>>>>> After some mistakes yesterday we ( me and my colleague ) think that it >>>>>> will be wise for libvirt to check config file existence on remote side >>>>> Which config file? >>>>> >>>> VM config file, namely qemu. >>> Do you mean the domain XML definition found in /etc/libvirt/qemu/*.xml ? >> Yes and no, xml, but at another location. >> >>>> We forgot to mount shared storage (namely gluster volume), on which we >>>> have config files, twice... >>> If you're talking about the domain XML file, then you should never store >>> them on a shared storage. All XML files in /etc/libvirt are supposed to >>> be local to each host. >> They are not stored in /etc/libvirt , but in other location, which is >> suitable for sharing. > I'm even more confused now. How could libvirt even know about such > files? Unless you use transient domains, i.e., point libvirt to the > right XML when starting the domain.Yes, it transient, really, we use pacemaker's VirtualDomain http://www.linux-ha.org/wiki/VirtualDomain_(resource_agent) pacemaker creates domain and destroys it. As I can see from sources migration looks like: virsh ${VIRSH_OPTIONS} migrate --live $migrate_opts $DOMAIN_NAME $remoteuri $migrateuri & Don't know which exact options it uses though. Do we need xml on another host if domain is transient for live migration?> >> Unfortunately, my colleague left today for 2 weeks vacations and I have >> no access to these hosts, >> so I will be able to look in logs or do some tests only in October. >> So, I'll write there again after 2 weeks or so. > Yeah, it would be nice if you could share the actual domain XML and the > command you used for migrating it to the other host, too. >Don't sure I can do this, I know English a little bit ( my colleague can't write ;-) ), but this is his cluster, not my, so I prefer to wait 2 weeks, not try to request access.... Thank you!
On Fri, Sep 14, 2018 at 16:00:43 +0400, Dmitry Melekhov wrote:> 14.09.2018 15:43, Jiri Denemark пишет: > > On Thu, Sep 13, 2018 at 19:37:00 +0400, Dmitry Melekhov wrote: > >> > >> 13.09.2018 18:57, Jiri Denemark пишет: > >>> On Thu, Sep 13, 2018 at 18:38:57 +0400, Dmitry Melekhov wrote: > >>>> 13.09.2018 17:47, Jiri Denemark пишет: > >>>>> On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: > >>>>>> After some mistakes yesterday we ( me and my colleague ) think that it > >>>>>> will be wise for libvirt to check config file existence on remote side > >>>>> Which config file? > >>>>> > >>>> VM config file, namely qemu. > >>> Do you mean the domain XML definition found in /etc/libvirt/qemu/*.xml ? > >> Yes and no, xml, but at another location. > >> > >>>> We forgot to mount shared storage (namely gluster volume), on which we > >>>> have config files, twice... > >>> If you're talking about the domain XML file, then you should never store > >>> them on a shared storage. All XML files in /etc/libvirt are supposed to > >>> be local to each host. > >> They are not stored in /etc/libvirt , but in other location, which is > >> suitable for sharing. > > I'm even more confused now. How could libvirt even know about such > > files? Unless you use transient domains, i.e., point libvirt to the > > right XML when starting the domain. > > Yes, it transient, really, we use pacemaker's VirtualDomain > http://www.linux-ha.org/wiki/VirtualDomain_(resource_agent) > > pacemaker creates domain and destroys it. > > As I can see from sources migration looks like: > virsh ${VIRSH_OPTIONS} migrate --live $migrate_opts $DOMAIN_NAME > $remoteuri $migrateuri & > > Don't know which exact options it uses though. > > Do we need xml on another host if domain is transient for live migration?No, the XML is only needed when starting the domain. At the beginning of migration libvirt transfers the XML to libvirtd on the other host so that QEMU with the right parameters can be started there. Just to make sure, the disk images used by the domain are stored on a shared storage accessible from both hosts, right? Jirka
14.09.2018 16:15, Jiri Denemark пишет:> On Fri, Sep 14, 2018 at 16:00:43 +0400, Dmitry Melekhov wrote: >> 14.09.2018 15:43, Jiri Denemark пишет: >>> On Thu, Sep 13, 2018 at 19:37:00 +0400, Dmitry Melekhov wrote: >>>> 13.09.2018 18:57, Jiri Denemark пишет: >>>>> On Thu, Sep 13, 2018 at 18:38:57 +0400, Dmitry Melekhov wrote: >>>>>> 13.09.2018 17:47, Jiri Denemark пишет: >>>>>>> On Thu, Sep 13, 2018 at 10:35:09 +0400, Dmitry Melekhov wrote: >>>>>>>> After some mistakes yesterday we ( me and my colleague ) think that it >>>>>>>> will be wise for libvirt to check config file existence on remote side >>>>>>> Which config file? >>>>>>> >>>>>> VM config file, namely qemu. >>>>> Do you mean the domain XML definition found in /etc/libvirt/qemu/*.xml ? >>>> Yes and no, xml, but at another location. >>>> >>>>>> We forgot to mount shared storage (namely gluster volume), on which we >>>>>> have config files, twice... >>>>> If you're talking about the domain XML file, then you should never store >>>>> them on a shared storage. All XML files in /etc/libvirt are supposed to >>>>> be local to each host. >>>> They are not stored in /etc/libvirt , but in other location, which is >>>> suitable for sharing. >>> I'm even more confused now. How could libvirt even know about such >>> files? Unless you use transient domains, i.e., point libvirt to the >>> right XML when starting the domain. >> Yes, it transient, really, we use pacemaker's VirtualDomain >> http://www.linux-ha.org/wiki/VirtualDomain_(resource_agent) >> >> pacemaker creates domain and destroys it. >> >> As I can see from sources migration looks like: >> virsh ${VIRSH_OPTIONS} migrate --live $migrate_opts $DOMAIN_NAME >> $remoteuri $migrateuri & >> >> Don't know which exact options it uses though. >> >> Do we need xml on another host if domain is transient for live migration? > No, the XML is only needed when starting the domain. At the beginning of > migration libvirt transfers the XML to libvirtd on the other host so > that QEMU with the right parameters can be started there.Well, then it is really strange...> Just to make sure, the disk images used by the domain are stored on a > shared storage accessible from both hosts, right? >Yes, we use gluster and connection over gfapi, so VM storage is accessible from all hosts. Usually, when xml are available, migration works without problems. Just thought - it is possible that problem is in pacemaker's VirtualDomain and it needs xml on all hosts, there is such comment : # During a probe, it is permissible for the config file to not be # readable (it might be on shared storage not available during the # probe). In that case, we're # unable to get the domain name. Thus, we also can't check whether the # domain is running. The only thing we can do here is to assume that # it is not running. We have to check, but looks like we have to ask in pacemaker mail list, it has to check if xml is available. Thank you!