Matt Wilson
2013-Aug-30 23:40 UTC
[PATCH] xend: handle extended PCI configuration space when saving state
From: Steven Noonan <snoonan@amazon.com> Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended configuration space which is larger than 256 bytes. This patch uses stat() to determine the amount of space used to correctly save all of the PCI configuration space. Resets handled by the xen-pciback driver don''t have this problem, as that code correctly handles saving extended configuration space. Signed-off-by: Steven Noonan <snoonan@amazon.com> Reviewed-by: Matt Wilson <msw@amazon.com> [msw: adjusted commit message] Cc: Ian Jackson <ian.jackson@eu.citrix.com> Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com> Cc: Ian Campbell <ian.campbell@citrix.com> Signed-off-by: Matt Wilson <msw@amazon.com> --- tools/python/xen/util/pci.py | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/tools/python/xen/util/pci.py b/tools/python/xen/util/pci.py index 792fb69..adeca4b 100644 --- a/tools/python/xen/util/pci.py +++ b/tools/python/xen/util/pci.py @@ -521,8 +521,9 @@ def save_pci_conf_space(devs_string): pci_path = sysfs_mnt + SYSFS_PCI_DEVS_PATH + ''/'' + pci_str + \ SYSFS_PCI_DEV_CONFIG_PATH fd = os.open(pci_path, os.O_RDONLY) + size = os.fstat(fd).st_size configs = [] - for i in range(0, 256, 4): + for i in range(0, size, 4): configs = configs + [os.read(fd,4)] os.close(fd) pci_list = pci_list + [pci_path] -- 1.7.9.5
George Dunlap
2013-Sep-02 09:53 UTC
Re: [PATCH] xend: handle extended PCI configuration space when saving state
On Sat, Aug 31, 2013 at 12:40 AM, Matt Wilson <msw@linux.com> wrote:> From: Steven Noonan <snoonan@amazon.com> > > Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended > configuration space which is larger than 256 bytes. This patch uses > stat() to determine the amount of space used to correctly save all of > the PCI configuration space. Resets handled by the xen-pciback driver > don''t have this problem, as that code correctly handles saving > extended configuration space. > > Signed-off-by: Steven Noonan <snoonan@amazon.com> > Reviewed-by: Matt Wilson <msw@amazon.com>Do you guys have plans for a migration over to xl? xend has been deprecated for two release cycles now, and I personally think 4.4 is a good time to put it out of its misery. If there is xend functionality you use which is missing from libxl, we need to know so that we can re-implement it. -George
Matt Wilson
2013-Sep-02 20:32 UTC
xend deprecation [Was: Re: [PATCH] xend: handle extended PCI configuration space when saving state]
On Mon, Sep 02, 2013 at 10:53:25AM +0100, George Dunlap wrote:> > Do you guys have plans for a migration over to xl? xend has been > deprecated for two release cycles now, and I personally think 4.4 is a > good time to put it out of its misery.Work is underway, but there''s lots to do to make that change. Making the xend build optional in 4.4 is the first step, but I don''t think we should start removing it altogether yet.> If there is xend functionality you use which is missing from libxl, we > need to know so that we can re-implement it.We''ll need another sweep to verify that xl sets up everything in a way that''s compatible with all the OS images that run in EC2. E.g., other problems like: http://lists.xen.org/archives/html/xen-devel/2013-04/msg02362.html We''ll need to look closely at the xenstore permissions for any problems like XSA-57. Live migration between a system running xend to a system that doesn''t support xend would be a nice-to-have. That probably means some support for SXP parsing either in libxl or in some compatibility tool. --msw
Ian Campbell
2013-Sep-03 08:03 UTC
Re: xend deprecation [Was: Re: [PATCH] xend: handle extended PCI configuration space when saving state]
On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:> Live migration between a system running xend to a system that doesn''t > support xend would be a nice-to-have. That probably means some support > for SXP parsing either in libxl or in some compatibility tool.This already works by using xl on the source side, e.g. a guest started with xend on 4.1 can be migrated with 4.1''s xl to 4.2. http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html Ian.
Ian Campbell
2013-Sep-03 16:33 UTC
Re: [PATCH] xend: handle extended PCI configuration space when saving state
On Fri, 2013-08-30 at 16:40 -0700, Matt Wilson wrote:> From: Steven Noonan <snoonan@amazon.com> > > Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended > configuration space which is larger than 256 bytes. This patch uses > stat() to determine the amount of space used to correctly save all of > the PCI configuration space. Resets handled by the xen-pciback driver > don''t have this problem, as that code correctly handles saving > extended configuration space. > > Signed-off-by: Steven Noonan <snoonan@amazon.com> > Reviewed-by: Matt Wilson <msw@amazon.com> > [msw: adjusted commit message] > Cc: Ian Jackson <ian.jackson@eu.citrix.com> > Cc: Stefano Stabellini <stefano.stabellini@eu.citrix.com> > Cc: Ian Campbell <ian.campbell@citrix.com> > Signed-off-by: Matt Wilson <msw@amazon.com>I''ve applied this and "xend: fix file descriptor leak in pci utilities" on the basis that they looked pretty simple, however in general I''m not prepared to review xend patches so I am relying on whoever wants to keep xend alive to co-review each others patches and provide the appropriate reviewed-by etc (and to take responsibility for the fallout etc).> --- > tools/python/xen/util/pci.py | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/tools/python/xen/util/pci.py b/tools/python/xen/util/pci.py > index 792fb69..adeca4b 100644 > --- a/tools/python/xen/util/pci.py > +++ b/tools/python/xen/util/pci.py > @@ -521,8 +521,9 @@ def save_pci_conf_space(devs_string): > pci_path = sysfs_mnt + SYSFS_PCI_DEVS_PATH + ''/'' + pci_str + \ > SYSFS_PCI_DEV_CONFIG_PATH > fd = os.open(pci_path, os.O_RDONLY) > + size = os.fstat(fd).st_size > configs = [] > - for i in range(0, 256, 4): > + for i in range(0, size, 4): > configs = configs + [os.read(fd,4)] > os.close(fd) > pci_list = pci_list + [pci_path]
On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote:> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > Live migration between a system running xend to a system that doesn''t > > support xend would be a nice-to-have. That probably means some support > > for SXP parsing either in libxl or in some compatibility tool. > > This already works by using xl on the source side, e.g. a guest started > with xend on 4.1 can be migrated with 4.1''s xl to 4.2. > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.htmlxl doesn''t run everywhere that we need it to. --msw
Matt Wilson
2013-Sep-04 06:12 UTC
Re: [PATCH] xend: handle extended PCI configuration space when saving state
On Tue, Sep 03, 2013 at 05:33:13PM +0100, Ian Campbell wrote:> > I''ve applied this and "xend: fix file descriptor leak in pci utilities" > on the basis that they looked pretty simple, however in general I''m not > prepared to review xend patches so I am relying on whoever wants to keep > xend alive to co-review each others patches and provide the appropriate > reviewed-by etc (and to take responsibility for the fallout etc).We (Amazon) can take on some of the xend life support maintenance. I''m happy to review xend code from C/Python bindings to the Python code itself. If you''d like I''ll submit a patch to MAINTAINERS for that. --msw
Noonan, Steven
2013-Sep-04 06:47 UTC
Re: [PATCH] xend: handle extended PCI configuration space when saving state
On Tue, Sep 03, 2013 at 11:12:44PM -0700, Matt Wilson wrote:> On Tue, Sep 03, 2013 at 05:33:13PM +0100, Ian Campbell wrote: > > > > I''ve applied this and "xend: fix file descriptor leak in pci utilities" > > on the basis that they looked pretty simple, however in general I''m not > > prepared to review xend patches so I am relying on whoever wants to keep > > xend alive to co-review each others patches and provide the appropriate > > reviewed-by etc (and to take responsibility for the fallout etc). > > We (Amazon) can take on some of the xend life support maintenance. I''m > happy to review xend code from C/Python bindings to the Python code > itself. > > If you''d like I''ll submit a patch to MAINTAINERS for that. > > --mswAgreed, I''m equally happy to maintain xend alongside Matt. - Steven
On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote:> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > > Live migration between a system running xend to a system that doesn''t > > > support xend would be a nice-to-have. That probably means some support > > > for SXP parsing either in libxl or in some compatibility tool. > > > > This already works by using xl on the source side, e.g. a guest started > > with xend on 4.1 can be migrated with 4.1''s xl to 4.2. > > > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > > xl doesn''t run everywhere that we need it to.Details please, or we aren''t going to be able to even think about how to solve this for you. BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis in 4.2) and the use 4.2''s xl to do a localhost migrate or a second remote migrate to another 4.2 system. xend''s deprecation has been on the cards, and widely known, for several releases now. It is time to quit stalling and start moving. I intend to post a patch deleting it as soon as 4.4 is out the door. Of course you are also free to pick up xend maintenance if you really want to keep it alive. We could for example move it out of tree. And by "We" I mean "You and those others who are interested in continuing with xend". Ian.
Ian Campbell
2013-Sep-04 07:30 UTC
Re: [PATCH] xend: handle extended PCI configuration space when saving state
On Tue, 2013-09-03 at 23:47 -0700, Noonan, Steven wrote:> On Tue, Sep 03, 2013 at 11:12:44PM -0700, Matt Wilson wrote: > > On Tue, Sep 03, 2013 at 05:33:13PM +0100, Ian Campbell wrote: > > > > > > I''ve applied this and "xend: fix file descriptor leak in pci utilities" > > > on the basis that they looked pretty simple, however in general I''m not > > > prepared to review xend patches so I am relying on whoever wants to keep > > > xend alive to co-review each others patches and provide the appropriate > > > reviewed-by etc (and to take responsibility for the fallout etc). > > > > We (Amazon) can take on some of the xend life support maintenance. I''m > > happy to review xend code from C/Python bindings to the Python code > > itself. > > > > If you''d like I''ll submit a patch to MAINTAINERS for that. > > > > --msw > > Agreed, I''m equally happy to maintain xend alongside Matt.Please do send a patch to MAINTAINERS. Try and pick the right S:tatus from the list at the top to indicate your level of support. Ian
Ian Jackson
2013-Sep-04 10:14 UTC
Re: [PATCH] xend: handle extended PCI configuration space when saving state
Ian Campbell writes ("Re: [PATCH] xend: handle extended PCI configuration space when saving state"):> On Fri, 2013-08-30 at 16:40 -0700, Matt Wilson wrote: > > Newer PCI standards (e.g., PCI-X 2.0 and PCIe) introduce extended > > configuration space which is larger than 256 bytes. This patch uses > > stat() to determine the amount of space used to correctly save all of > > the PCI configuration space. Resets handled by the xen-pciback driver > > don''t have this problem, as that code correctly handles saving > > extended configuration space....> I''ve applied this and "xend: fix file descriptor leak in pci utilities" > on the basis that they looked pretty simple, however in general I''m not > prepared to review xend patches so I am relying on whoever wants to keep > xend alive to co-review each others patches and provide the appropriate > reviewed-by etc (and to take responsibility for the fallout etc).I think we should be planning to drop xend in 4.4. Ian.
On Tue, 3 Sep 2013, Matt Wilson wrote:> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > > Live migration between a system running xend to a system that doesn''t > > > support xend would be a nice-to-have. That probably means some support > > > for SXP parsing either in libxl or in some compatibility tool. > > > > This already works by using xl on the source side, e.g. a guest started > > with xend on 4.1 can be migrated with 4.1''s xl to 4.2. > > > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > > xl doesn''t run everywhere that we need it to.Really? How is that possible? I am not aware of any specific hardware or software dependencies in the xl/libxl code. If there are any, it would good to know them, because we certainly need to address them before removing xend.
On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote:> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: > > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > > > Live migration between a system running xend to a system that doesn't > > > > support xend would be a nice-to-have. That probably means some support > > > > for SXP parsing either in libxl or in some compatibility tool. > > > > > > This already works by using xl on the source side, e.g. a guest started > > > with xend on 4.1 can be migrated with 4.1's xl to 4.2. > > > > > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > > > > xl doesn't run everywhere that we need it to. > > Details please, or we aren't going to be able to even think about how to > solve this for you. > > BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis > in 4.2) and the use 4.2's xl to do a localhost migrate or a second > remote migrate to another 4.2 system. > > xend's deprecation has been on the cards, and widely known, for several > releases now. It is time to quit stalling and start moving. I intend to > post a patch deleting it as soon as 4.4 is out the door.And I think I might have to lobby hard against removing it. There are issues that are stopping us from using it: - No status in xl list -l when only dom0 is present. - 'xl list -l' says 'Domain name must be specified' even though the '-l' is for 'Output all VM details' - xl list -l for PV guest has bug: it’s not formated S-expression text. - No console in xl list -l, so can't get tty console port. - xl uses SSH for migration. We need SSL daemon or socat like xend has. Those should be fixed at some point :-) Problem is that other things keep on popping up on the radar and hence we end up using xend. CC-ing Zhigang here as he might have a more up-to-date list. I think there were some statistics too, but I can't recall.> > Of course you are also free to pick up xend maintenance if you really > want to keep it alive. We could for example move it out of tree. And by > "We" I mean "You and those others who are interested in continuing with > xend".Hahahah. I think we all want to move away but there are things that are stopping us and we would like them to magically get fixed. And I also want a pony. With a barrel of guinness attached to it. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: >> > > > Live migration between a system running xend to a system that doesn''t >> > > > support xend would be a nice-to-have. That probably means some support >> > > > for SXP parsing either in libxl or in some compatibility tool. >> > > >> > > This already works by using xl on the source side, e.g. a guest started >> > > with xend on 4.1 can be migrated with 4.1''s xl to 4.2. >> > > >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html >> > >> > xl doesn''t run everywhere that we need it to. >> >> Details please, or we aren''t going to be able to even think about how to >> solve this for you. >> >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis >> in 4.2) and the use 4.2''s xl to do a localhost migrate or a second >> remote migrate to another 4.2 system. >> >> xend''s deprecation has been on the cards, and widely known, for several >> releases now. It is time to quit stalling and start moving. I intend to >> post a patch deleting it as soon as 4.4 is out the door. > > And I think I might have to lobby hard against removing it. > > There are issues that are stopping us from using it: > > - No status in xl list -l when only dom0 is present. > - ''xl list -l'' says ''Domain name must be specified'' even though the ''-l'' > is for ''Output all VM details'' > - xl list -l for PV guest has bug: it’s not formated S-expression text. > - No console in xl list -l, so can''t get tty console port. > - xl uses SSH for migration. We need SSL daemon or socat like xend has. > > Those should be fixed at some point :-) Problem is that other things > keep on popping up on the radar and hence we end up using xend.Uum, when were you going to report these? A lot of us were under the impression that xl behaved sufficiently like xend for anyone who cared. Maybe we should make an announcement (on blog, xen-users, xen-devel, and xen-announce) that there has been a proposal to make 4.4 the last release with xend, and so it''s important to report all bugs / missing features of xl so we can get them fixed by the 4.4 release? -George
On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote:> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk > <konrad.wilk@oracle.com> wrote: > > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: > >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: > >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > >> > > > Live migration between a system running xend to a system that doesn't > >> > > > support xend would be a nice-to-have. That probably means some support > >> > > > for SXP parsing either in libxl or in some compatibility tool. > >> > > > >> > > This already works by using xl on the source side, e.g. a guest started > >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2. > >> > > > >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > >> > > >> > xl doesn't run everywhere that we need it to. > >> > >> Details please, or we aren't going to be able to even think about how to > >> solve this for you. > >> > >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis > >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second > >> remote migrate to another 4.2 system. > >> > >> xend's deprecation has been on the cards, and widely known, for several > >> releases now. It is time to quit stalling and start moving. I intend to > >> post a patch deleting it as soon as 4.4 is out the door. > > > > And I think I might have to lobby hard against removing it.If by lobby you mean "step up to maintain" then please do.> > There are issues that are stopping us from using it: > > > > - No status in xl list -l when only dom0 is present.xl list -l produces a tonne of output here when only dom0 is present.> > - 'xl list -l' says 'Domain name must be specified' even though the '-l' > > is for 'Output all VM details'Not here it doesn't. IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed.> > - xl list -l for PV guest has bug: it’s not formated S-expression text.This is a feature not a bug. However you can request an sxp compatibility mode (I forget how, check the manpage). Note that this doesn't include new features, so you would be advised to make plans to switch to consuming the json variant.> > - No console in xl list -l, so can't get tty console port.That one should be easily fixable> > - xl uses SSH for migration. We need SSL daemon or socat like xend has.You can provide any tool you like to run instead of ssh, including an ssl daemon/client or socat. See the manpage.> > > > Those should be fixed at some point :-) Problem is that other things > > keep on popping up on the radar and hence we end up using xend. > > Uum, when were you going to report these? A lot of us were under the > impression that xl behaved sufficiently like xend for anyone who > cared. > > Maybe we should make an announcement (on blog, xen-users, xen-devel, > and xen-announce) that there has been a proposal to make 4.4 the last > release with xend, and so it's important to report all bugs / missing > features of xl so we can get them fixed by the 4.4 release?Didn't we do this for 4.2 or 4.3? We didn't get much feedback. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Ian Campbell writes ("Re: [Xen-devel] xend deprecation"):> > On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk > > <konrad.wilk@oracle.com> wrote: > > > - xl uses SSH for migration. We need SSL daemon or socat like xend has. > > You can provide any tool you like to run instead of ssh, including an > ssl daemon/client or socat. See the manpage.If there is some migration mode that you''d like to see natively supported in xl, besides "fork this program which looks a bit like ssh", we''d be very happy to take patches or even write them. Ian.
At 16:34 +0100 on 04 Sep (1378312470), Ian Campbell wrote:> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: > > Maybe we should make an announcement (on blog, xen-users, xen-devel, > > and xen-announce) that there has been a proposal to make 4.4 the last > > release with xend, and so it''s important to report all bugs / missing > > features of xl so we can get them fixed by the 4.4 release? > > Didn''t we do this for 4.2 or 4.3? We didn''t get much feedback.Indeed. The 4.2 release notes say: The XEND toolstack is now deprecated and should be expected to be removed in the 4.3 timescale. Therefore it is strongly recommended that users evaluate Xen 4.2 with XL for their use cases and report any omissions or blockers such that they can be addressed for 4.3. Fixes for serious functionality gaps will also be considered for backport to the 4.2.x releases. I guess we''re now on "expected to be removed in the 4.5 timescale"? Tim.
Tim Deegan writes ("Re: [Xen-devel] xend deprecation"):> At 16:34 +0100 on 04 Sep (1378312470), Ian Campbell wrote:...> > Didn''t we do this for 4.2 or 4.3? We didn''t get much feedback. > > Indeed. The 4.2 release notes say: > > The XEND toolstack is now deprecated and should be expected to be > removed in the 4.3 timescale. Therefore it is strongly recommended > that users evaluate Xen 4.2 with XL for their use cases and report any > omissions or blockers such that they can be addressed for 4.3. Fixes > for serious functionality gaps will also be considered for backport to > the 4.2.x releases. > > I guess we''re now on "expected to be removed in the 4.5 timescale"?Surely we can still remove it in 4.4, having advertised that we were going to remove it in 4.3. Ian.
On Wed, 2013-09-04 at 17:17 +0100, Ian Jackson wrote:> Tim Deegan writes ("Re: [Xen-devel] xend deprecation"): > > At 16:34 +0100 on 04 Sep (1378312470), Ian Campbell wrote: > ... > > > Didn''t we do this for 4.2 or 4.3? We didn''t get much feedback. > > > > Indeed. The 4.2 release notes say: > > > > The XEND toolstack is now deprecated and should be expected to be > > removed in the 4.3 timescale. Therefore it is strongly recommended > > that users evaluate Xen 4.2 with XL for their use cases and report any > > omissions or blockers such that they can be addressed for 4.3. Fixes > > for serious functionality gaps will also be considered for backport to > > the 4.2.x releases. > > > > I guess we''re now on "expected to be removed in the 4.5 timescale"? > > Surely we can still remove it in 4.4, having advertised that we were > going to remove it in 4.3.Well, either that or the people insisting it remain need to step up and take over maintenance. Amazon appear to be offering to do that, at least to some extent. I nearly said "remain Supported" above, but the reality is that it effectively hasn''t been Supported by this community for many releases now. I don''t know to what extent Matt and Steven are offering to maintain it. Depending on the answer to that we should consider moving xend to a separate tree. Ian.
On Wed, Sep 04, 2013 at 04:18:49PM +0100, George Dunlap wrote:> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk > <konrad.wilk@oracle.com> wrote: > > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: > >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: > >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > >> > > > Live migration between a system running xend to a system that doesn't > >> > > > support xend would be a nice-to-have. That probably means some support > >> > > > for SXP parsing either in libxl or in some compatibility tool. > >> > > > >> > > This already works by using xl on the source side, e.g. a guest started > >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2. > >> > > > >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > >> > > >> > xl doesn't run everywhere that we need it to. > >> > >> Details please, or we aren't going to be able to even think about how to > >> solve this for you. > >> > >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis > >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second > >> remote migrate to another 4.2 system. > >> > >> xend's deprecation has been on the cards, and widely known, for several > >> releases now. It is time to quit stalling and start moving. I intend to > >> post a patch deleting it as soon as 4.4 is out the door. > > > > And I think I might have to lobby hard against removing it. > > > > There are issues that are stopping us from using it: > > > > - No status in xl list -l when only dom0 is present. > > - 'xl list -l' says 'Domain name must be specified' even though the '-l' > > is for 'Output all VM details' > > - xl list -l for PV guest has bug: it’s not formated S-expression text. > > - No console in xl list -l, so can't get tty console port. > > - xl uses SSH for migration. We need SSL daemon or socat like xend has. > > > > Those should be fixed at some point :-) Problem is that other things > > keep on popping up on the radar and hence we end up using xend. > > Uum, when were you going to report these? A lot of us were under the > impression that xl behaved sufficiently like xend for anyone who > cared.I got overwhelmed with other things and this dropped from my radar. This thread got me digging through my notes.> > Maybe we should make an announcement (on blog, xen-users, xen-devel, > and xen-announce) that there has been a proposal to make 4.4 the last > release with xend, and so it's important to report all bugs / missing > features of xl so we can get them fixed by the 4.4 release?Sounds sensible.> > -George_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote:> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: > > On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk > > <konrad.wilk@oracle.com> wrote: > > > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: > > >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: > > >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > > >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > >> > > > Live migration between a system running xend to a system that doesn't > > >> > > > support xend would be a nice-to-have. That probably means some support > > >> > > > for SXP parsing either in libxl or in some compatibility tool. > > >> > > > > >> > > This already works by using xl on the source side, e.g. a guest started > > >> > > with xend on 4.1 can be migrated with 4.1's xl to 4.2. > > >> > > > > >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > > >> > > > >> > xl doesn't run everywhere that we need it to. > > >> > > >> Details please, or we aren't going to be able to even think about how to > > >> solve this for you. > > >> > > >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis > > >> in 4.2) and the use 4.2's xl to do a localhost migrate or a second > > >> remote migrate to another 4.2 system. > > >> > > >> xend's deprecation has been on the cards, and widely known, for several > > >> releases now. It is time to quit stalling and start moving. I intend to > > >> post a patch deleting it as soon as 4.4 is out the door. > > > > > > And I think I might have to lobby hard against removing it. > > If by lobby you mean "step up to maintain" then please do.I can surely do that, but I would prefer to have the issues fixed.> > > > There are issues that are stopping us from using it: > > > > > > - No status in xl list -l when only dom0 is present. > > xl list -l produces a tonne of output here when only dom0 is present. > > > > - 'xl list -l' says 'Domain name must be specified' even though the '-l' > > > is for 'Output all VM details' > > Not here it doesn't. > > IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed.Odd, I am running Xen 4.3. Hm, time to rebase.> > > > - xl list -l for PV guest has bug: it’s not formated S-expression text. > > This is a feature not a bug. However you can request an sxp > compatibility mode (I forget how, check the manpage). Note that this > doesn't include new features, so you would be advised to make plans to > switch to consuming the json variant. > > > > - No console in xl list -l, so can't get tty console port. > > That one should be easily fixable > > > > - xl uses SSH for migration. We need SSL daemon or socat like xend has. > > You can provide any tool you like to run instead of ssh, including an > ssl daemon/client or socat. See the manpage.Right, except that the socat part does not work as the 'ssh' arguments are encoded in the code. I am not saying that these are 'earth-shattering hard to fix' it just that they should be done before Xend is gone.> > > > > > > Those should be fixed at some point :-) Problem is that other things > > > keep on popping up on the radar and hence we end up using xend. > > > > Uum, when were you going to report these? A lot of us were under the > > impression that xl behaved sufficiently like xend for anyone who > > cared. > > > > Maybe we should make an announcement (on blog, xen-users, xen-devel, > > and xen-announce) that there has been a proposal to make 4.4 the last > > release with xend, and so it's important to report all bugs / missing > > features of xl so we can get them fixed by the 4.4 release? > > Didn't we do this for 4.2 or 4.3? We didn't get much feedback.No idea.> > Ian. >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 04/09/13 17:33, Konrad Rzeszutek Wilk wrote:> On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote: >> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: >>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk >>> <konrad.wilk@oracle.com> wrote: >>>> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: >>>>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: >>>>>> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: >>>>>>> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: >>>>>>>> Live migration between a system running xend to a system that doesn''t >>>>>>>> support xend would be a nice-to-have. That probably means some support >>>>>>>> for SXP parsing either in libxl or in some compatibility tool. >>>>>>> This already works by using xl on the source side, e.g. a guest started >>>>>>> with xend on 4.1 can be migrated with 4.1''s xl to 4.2. >>>>>>> >>>>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html >>>>>> xl doesn''t run everywhere that we need it to. >>>>> Details please, or we aren''t going to be able to even think about how to >>>>> solve this for you. >>>>> >>>>> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis >>>>> in 4.2) and the use 4.2''s xl to do a localhost migrate or a second >>>>> remote migrate to another 4.2 system. >>>>> >>>>> xend''s deprecation has been on the cards, and widely known, for several >>>>> releases now. It is time to quit stalling and start moving. I intend to >>>>> post a patch deleting it as soon as 4.4 is out the door. >>>> And I think I might have to lobby hard against removing it. >> If by lobby you mean "step up to maintain" then please do. > I can surely do that, but I would prefer to have the issues fixed.You mean you prefer to have the xl issues fixed? I think we all do; so instead of lobbying hard to remove xend, you can lobby hard to get the outstanding issues fixed. :-) -George
Il 04/09/2013 18:33, Konrad Rzeszutek Wilk ha scritto:> On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote: >> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: >>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk >>> <konrad.wilk@oracle.com> wrote: >>>> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: >>>>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: >>>>>> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: >>>>>>> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: >>>>>>>> Live migration between a system running xend to a system that doesn't >>>>>>>> support xend would be a nice-to-have. That probably means some support >>>>>>>> for SXP parsing either in libxl or in some compatibility tool. >>>>>>> This already works by using xl on the source side, e.g. a guest started >>>>>>> with xend on 4.1 can be migrated with 4.1's xl to 4.2. >>>>>>> >>>>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html >>>>>> xl doesn't run everywhere that we need it to. >>>>> Details please, or we aren't going to be able to even think about how to >>>>> solve this for you. >>>>> >>>>> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis >>>>> in 4.2) and the use 4.2's xl to do a localhost migrate or a second >>>>> remote migrate to another 4.2 system. >>>>> >>>>> xend's deprecation has been on the cards, and widely known, for several >>>>> releases now. It is time to quit stalling and start moving. I intend to >>>>> post a patch deleting it as soon as 4.4 is out the door. >>>> And I think I might have to lobby hard against removing it. >> If by lobby you mean "step up to maintain" then please do. > I can surely do that, but I would prefer to have the issues fixed. > >>>> There are issues that are stopping us from using it: >>>> >>>> - No status in xl list -l when only dom0 is present. >> xl list -l produces a tonne of output here when only dom0 is present. >> >>>> - 'xl list -l' says 'Domain name must be specified' even though the '-l' >>>> is for 'Output all VM details' >> Not here it doesn't. >> >> IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed. > Odd, I am running Xen 4.3. Hm, time to rebase. > >>>> - xl list -l for PV guest has bug: it’s not formated S-expression text. >> This is a feature not a bug. However you can request an sxp >> compatibility mode (I forget how, check the manpage). Note that this >> doesn't include new features, so you would be advised to make plans to >> switch to consuming the json variant. >> >>>> - No console in xl list -l, so can't get tty console port. >> That one should be easily fixable >> >>>> - xl uses SSH for migration. We need SSL daemon or socat like xend has. >> You can provide any tool you like to run instead of ssh, including an >> ssl daemon/client or socat. See the manpage. > Right, except that the socat part does not work as the 'ssh' arguments > are encoded in the code. I am not saying that these are 'earth-shattering hard > to fix' it just that they should be done before Xend is gone. > >>>> Those should be fixed at some point :-) Problem is that other things >>>> keep on popping up on the radar and hence we end up using xend. >>> Uum, when were you going to report these? A lot of us were under the >>> impression that xl behaved sufficiently like xend for anyone who >>> cared. >>> >>> Maybe we should make an announcement (on blog, xen-users, xen-devel, >>> and xen-announce) that there has been a proposal to make 4.4 the last >>> release with xend, and so it's important to report all bugs / missing >>> features of xl so we can get them fixed by the 4.4 release?I think is a good idea, if will not removed on xen 4.4 at least disabled it by default to get more visible the new xl to the users and get more feedback. About very small thing present in xend and not in xl not reported for now is the autocompletion name of domU in commandsmissed. I think is good do a similar thing with qemu traditional with deprecation and massive comunication of a certain deadline. Also in this case will be good disabling it by default to accentuate its deprecation. Upstream qemu needs more testing/support to have hvm domUs performance/features/bugfixes improved.>> Didn't we do this for 4.2 or 4.3? We didn't get much feedback. > No idea. >> Ian. >> > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 05/09/13 11:18, Fabio Fantoni wrote:> Il 04/09/2013 18:33, Konrad Rzeszutek Wilk ha scritto: >> On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote: >>> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: >>>> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk >>>> <konrad.wilk@oracle.com> wrote: >>>>> On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: >>>>>> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: >>>>>>> On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: >>>>>>>> On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: >>>>>>>>> Live migration between a system running xend to a system that >>>>>>>>> doesn't >>>>>>>>> support xend would be a nice-to-have. That probably means some >>>>>>>>> support >>>>>>>>> for SXP parsing either in libxl or in some compatibility tool. >>>>>>>> This already works by using xl on the source side, e.g. a guest >>>>>>>> started >>>>>>>> with xend on 4.1 can be migrated with 4.1's xl to 4.2. >>>>>>>> >>>>>>>> http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html >>>>>>> xl doesn't run everywhere that we need it to. >>>>>> Details please, or we aren't going to be able to even think about >>>>>> how to >>>>>> solve this for you. >>>>>> >>>>>> BTW You can also do a xend migration to e.g. 4.2 (using the xend >>>>>> whichis >>>>>> in 4.2) and the use 4.2's xl to do a localhost migrate or a second >>>>>> remote migrate to another 4.2 system. >>>>>> >>>>>> xend's deprecation has been on the cards, and widely known, for >>>>>> several >>>>>> releases now. It is time to quit stalling and start moving. I >>>>>> intend to >>>>>> post a patch deleting it as soon as 4.4 is out the door. >>>>> And I think I might have to lobby hard against removing it. >>> If by lobby you mean "step up to maintain" then please do. >> I can surely do that, but I would prefer to have the issues fixed. >> >>>>> There are issues that are stopping us from using it: >>>>> >>>>> - No status in xl list -l when only dom0 is present. >>> xl list -l produces a tonne of output here when only dom0 is present. >>> >>>>> - 'xl list -l' says 'Domain name must be specified' even though >>>>> the '-l' >>>>> is for 'Output all VM details' >>> Not here it doesn't. >>> >>> IIRC this was a bug in 4.1 or 4.2 timescales, which is now fixed. >> Odd, I am running Xen 4.3. Hm, time to rebase. >> >>>>> - xl list -l for PV guest has bug: it’s not formated >>>>> S-expression text. >>> This is a feature not a bug. However you can request an sxp >>> compatibility mode (I forget how, check the manpage). Note that this >>> doesn't include new features, so you would be advised to make plans to >>> switch to consuming the json variant. >>> >>>>> - No console in xl list -l, so can't get tty console port. >>> That one should be easily fixable >>> >>>>> - xl uses SSH for migration. We need SSL daemon or socat like >>>>> xend has. >>> You can provide any tool you like to run instead of ssh, including an >>> ssl daemon/client or socat. See the manpage. >> Right, except that the socat part does not work as the 'ssh' arguments >> are encoded in the code. I am not saying that these are >> 'earth-shattering hard >> to fix' it just that they should be done before Xend is gone. >> >>>>> Those should be fixed at some point :-) Problem is that other things >>>>> keep on popping up on the radar and hence we end up using xend. >>>> Uum, when were you going to report these? A lot of us were under the >>>> impression that xl behaved sufficiently like xend for anyone who >>>> cared. >>>> >>>> Maybe we should make an announcement (on blog, xen-users, xen-devel, >>>> and xen-announce) that there has been a proposal to make 4.4 the last >>>> release with xend, and so it's important to report all bugs / missing >>>> features of xl so we can get them fixed by the 4.4 release? > > I think is a good idea, if will not removed on xen 4.4 at least > disabled it by default to get more visible the new xl to the users and > get more feedback. > > About very small thing present in xend and not in xl not reported for > now is the autocompletion name of domU in commandsmissed. > > I think is good do a similar thing with qemu traditional with > deprecation and massive comunication of a certain deadline.qemu is a bit different than xend: it is entirely possible to migrate a VM started with xend to a system using only xl. I don't think the same is true for qemu; so we will probably be keeping qemu-traditional around for some time after qemu-xen has reached feature parity (which it still has yet to do). -George _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Wed, Sep 4, 2013 at 4:34 PM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: >> On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk >> <konrad.wilk@oracle.com> wrote: >> > On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: >> >> On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: >> >> > On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: >> >> > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: >> >> > > > Live migration between a system running xend to a system that doesn''t >> >> > > > support xend would be a nice-to-have. That probably means some support >> >> > > > for SXP parsing either in libxl or in some compatibility tool. >> >> > > >> >> > > This already works by using xl on the source side, e.g. a guest started >> >> > > with xend on 4.1 can be migrated with 4.1''s xl to 4.2. >> >> > > >> >> > > http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html >> >> > >> >> > xl doesn''t run everywhere that we need it to. >> >> >> >> Details please, or we aren''t going to be able to even think about how to >> >> solve this for you. >> >> >> >> BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis >> >> in 4.2) and the use 4.2''s xl to do a localhost migrate or a second >> >> remote migrate to another 4.2 system. >> >> >> >> xend''s deprecation has been on the cards, and widely known, for several >> >> releases now. It is time to quit stalling and start moving. I intend to >> >> post a patch deleting it as soon as 4.4 is out the door. >> > >> > And I think I might have to lobby hard against removing it. > > If by lobby you mean "step up to maintain" then please do. > >> > There are issues that are stopping us from using it: >> > >> > - No status in xl list -l when only dom0 is present. > > xl list -l produces a tonne of output here when only dom0 is present.FWIW: # xl list -l [ ] This is from c/s 0c006b41a283a0a569c863d44abde5aa5750ae01, dated August 13. -George
On Thu, Sep 05, 2013 at 10:12:51AM +0100, George Dunlap wrote:> On 04/09/13 17:33, Konrad Rzeszutek Wilk wrote: > >On Wed, Sep 04, 2013 at 04:34:30PM +0100, Ian Campbell wrote: > >>On Wed, 2013-09-04 at 16:18 +0100, George Dunlap wrote: > >>>On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk > >>><konrad.wilk@oracle.com> wrote: > >>>>On Wed, Sep 04, 2013 at 08:29:07AM +0100, Ian Campbell wrote: > >>>>>On Tue, 2013-09-03 at 23:06 -0700, Matt Wilson wrote: > >>>>>>On Tue, Sep 03, 2013 at 09:03:46AM +0100, Ian Campbell wrote: > >>>>>>>On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > >>>>>>>>Live migration between a system running xend to a system that doesn''t > >>>>>>>>support xend would be a nice-to-have. That probably means some support > >>>>>>>>for SXP parsing either in libxl or in some compatibility tool. > >>>>>>>This already works by using xl on the source side, e.g. a guest started > >>>>>>>with xend on 4.1 can be migrated with 4.1''s xl to 4.2. > >>>>>>> > >>>>>>>http://lists.xen.org/archives/html/xen-devel/2012-08/msg02191.html > >>>>>>xl doesn''t run everywhere that we need it to. > >>>>>Details please, or we aren''t going to be able to even think about how to > >>>>>solve this for you. > >>>>> > >>>>>BTW You can also do a xend migration to e.g. 4.2 (using the xend whichis > >>>>>in 4.2) and the use 4.2''s xl to do a localhost migrate or a second > >>>>>remote migrate to another 4.2 system. > >>>>> > >>>>>xend''s deprecation has been on the cards, and widely known, for several > >>>>>releases now. It is time to quit stalling and start moving. I intend to > >>>>>post a patch deleting it as soon as 4.4 is out the door. > >>>>And I think I might have to lobby hard against removing it. > >>If by lobby you mean "step up to maintain" then please do. > >I can surely do that, but I would prefer to have the issues fixed. > > You mean you prefer to have the xl issues fixed? I think we all do; > so instead of lobbying hard to remove xend, you can lobby hard to > get the outstanding issues fixed. :-)Of course I would love to have the issues fixes. But I am running in a time management problem and I don''t thing we will get them all (along with the other changes we (Oracle) would like to get done) by the deadline. Hence as a safety net I am proposing that if we don''t get them done, we postpone the killing of xend.
On Fri, 2013-09-06 at 09:36 -0400, Konrad Rzeszutek Wilk wrote:> Hence as a safety net I am proposing that if we don''t get them done, > we postpone the killing of xend.It''s been two releases now, how many more do you want? We can''t just be keeping unmaintained code around indefinitely. Are you volunteering to add your name to MAINTAINERS file next to Matt and Steven? I would also note that xend being removed from upstream xen does not in any way preclude your continuing to use and develop it. However removing it does remove a bunch of confusion from our end users about what they should use and removes a burden from the project as a whole. Ian
>>> On 06.09.13 at 15:49, Ian Campbell <Ian.Campbell@citrix.com> wrote: > On Fri, 2013-09-06 at 09:36 -0400, Konrad Rzeszutek Wilk wrote: >> Hence as a safety net I am proposing that if we don''t get them done, >> we postpone the killing of xend. > > It''s been two releases now, how many more do you want? > > We can''t just be keeping unmaintained code around indefinitely. Are you > volunteering to add your name to MAINTAINERS file next to Matt and > Steven? > > I would also note that xend being removed from upstream xen does not in > any way preclude your continuing to use and develop it. However removing > it does remove a bunch of confusion from our end users about what they > should use and removes a burden from the project as a whole.Can''t we, for 4.4 at least, simply disable CONFIG_XEND by default, to make the earlier verbal announcement more obvious to consumers of the tree? (That''s not to say that I''d like it kept; I''m just saying that knowing that verbal announcements of planned feature removal is usually less efficient than forcing people to re-enable their feature, thus - hopefully - grokking the news.) Jan
On Fri, 2013-09-06 at 15:01 +0100, Jan Beulich wrote:> >>> On 06.09.13 at 15:49, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Fri, 2013-09-06 at 09:36 -0400, Konrad Rzeszutek Wilk wrote: > >> Hence as a safety net I am proposing that if we don''t get them done, > >> we postpone the killing of xend. > > > > It''s been two releases now, how many more do you want? > > > > We can''t just be keeping unmaintained code around indefinitely. Are you > > volunteering to add your name to MAINTAINERS file next to Matt and > > Steven? > > > > I would also note that xend being removed from upstream xen does not in > > any way preclude your continuing to use and develop it. However removing > > it does remove a bunch of confusion from our end users about what they > > should use and removes a burden from the project as a whole. > > Can''t we, for 4.4 at least, simply disable CONFIG_XEND by default, > to make the earlier verbal announcement more obvious to consumers > of the tree?I think this is the minimum we should do. Ian.
create ! title it xl lacks support for xm create -F (sxp config file syntax) On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:> > Live migration between a system running xend to a system that doesn''t > support xend would be a nice-to-have. That probably means some support > for SXP parsing either in libxl or in some compatibility tool.Matt mentioned elsewhere that they use "xm create -F" which takes a configuration file in the xend sxp format (I had no idea this option existed). I don''t think we really want to be adding SXP code to xm but perhaps we can produce a conversion tool? Matt, can you provide some examples of the actual SXP you care about, we obviously don''t want to be worrying about the bits which aren''t actually used. Ian.
xen@bugs.xenproject.org
2013-Sep-12 10:00 UTC
Processed: Support for xm create -F (sxp configuration files)
Processing commands for xen@bugs.xenproject.org:> create !Created new bug #17 rooted at `! '' Title: `Support for xm create -F (sxp configuration files)''> title it xl lacks support for xm create -F (sxp config file syntax)Set title for #17 to `xl lacks support for xm create -F (sxp config file syntax)''> > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote:Command failed: Unknown command `On''. at /srv/xen-devel-bugs/lib/emesinae/control.pl line 437, <M> line 42. Stop processing here. Modified/created Bugs: - 17: http://bugs.xenproject.org/xen/bug/17 (new) --- Xen Hypervisor Bug Tracker See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues
create ! title it xl improve support for migration over non-sshlike tunnels thanks On Wed, 2013-09-04 at 16:43 +0100, Ian Jackson wrote:> Ian Campbell writes ("Re: [Xen-devel] xend deprecation"): > > > On Wed, Sep 4, 2013 at 3:04 PM, Konrad Rzeszutek Wilk > > > <konrad.wilk@oracle.com> wrote: > > > > - xl uses SSH for migration. We need SSL daemon or socat like xend has. > > > > You can provide any tool you like to run instead of ssh, including an > > ssl daemon/client or socat. See the manpage. > > If there is some migration mode that you''d like to see natively > supported in xl, besides "fork this program which looks a bit like > ssh", we''d be very happy to take patches or even write them.Indeed. Konrad, what is the exact requirement here? You mention socat which just takes an IP address and connects to it and pipes stuff back and forth (crypto optional). I suspect the sending side of that can be done with a thin wrapper which provides the ''ssh like'' command line interface, so what is needed is a daemon listening on a known port which can fork + exec xl migrate-receive for each connection? Is socat actually what you want or was that just an example? Ian.
Processing commands for xen@bugs.xenproject.org:> create !Created new bug #18 rooted at `<1378980129.10076.33.camel@kazak.uk.xensource.com>'' Title: `Re: [Xen-devel] xend deprecation''> title it xl improve support for migration over non-sshlike tunnelsSet title for #18 to `xl improve support for migration over non-sshlike tunnels''> thanksFinished processing. Modified/created Bugs: - 18: http://bugs.xenproject.org/xen/bug/18 (new) --- Xen Hypervisor Bug Tracker See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues
Ian Campbell writes ("Re: [Xen-devel] xend deprecation"):> create ! > title it xl improve support for migration over non-sshlike tunnels > thanks...> Konrad, what is the exact requirement here? You mention socat which just > takes an IP address and connects to it and pipes stuff back and forth > (crypto optional). I suspect the sending side of that can be done with a > thin wrapper which provides the ''ssh like'' command line interface,I think it can be done with the existing xl interface, TBH, but it''s probably ugly. I would have no problem with an option which made xl make the network connection itself.> so > what is needed is a daemon listening on a known port which can fork + > exec xl migrate-receive for each connection?I think this should be doable by running xl migrate-receive out of inetd, or some kind of socket-activation-based feature of an init-replacement (eg upstart). If not this should be fixed. Ian.
On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:> - No status in xl list -l when only dom0 is present.Which bits of status are you interested in? I''m thinking something like the following, which will include the content of libxl_dominfo for every domain. 8<---------------------------------- From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001 From: Ian Campbell <ian.campbell@citrix.com> Date: Thu, 12 Sep 2013 13:50:33 +0100 Subject: [PATCH] xl: print runtime info in "xl list -l" Include dom0 runtime information, but not domain config. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> --- tools/libxl/xl_cmdimpl.c | 74 ++++++++++++++++++++++++++++------------------ 1 file changed, 46 insertions(+), 28 deletions(-) diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 884f050..46fdafc 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -298,7 +298,8 @@ static void dolog(const char *file, int line, const char *func, char *fmt, ...) } static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid, - libxl_domain_config *d_config) + libxl_domain_config *d_config, + libxl_dominfo *d_info) { yajl_gen_status s; @@ -317,13 +318,25 @@ static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid, if (s != yajl_gen_status_ok) goto out; - s = yajl_gen_string(hand, (const unsigned char *)"config", - sizeof("config")-1); - if (s != yajl_gen_status_ok) - goto out; - s = libxl_domain_config_gen_json(hand, d_config); - if (s != yajl_gen_status_ok) - goto out; + if (d_config) { + s = yajl_gen_string(hand, (const unsigned char *)"config", + sizeof("config")-1); + if (s != yajl_gen_status_ok) + goto out; + s = libxl_domain_config_gen_json(hand, d_config); + if (s != yajl_gen_status_ok) + goto out; + } + + if (d_info) { + s = yajl_gen_string(hand, (const unsigned char *)"info", + sizeof("info")-1); + if (s != yajl_gen_status_ok) + goto out; + s = libxl_dominfo_gen_json(hand, d_info); + if (s != yajl_gen_status_ok) + goto out; + } s = yajl_gen_map_close(hand); if (s != yajl_gen_status_ok) @@ -350,7 +363,7 @@ static void printf_info(enum output_format output_format, return; } - s = printf_info_one_json(hand, domid, d_config); + s = printf_info_one_json(hand, domid, d_config, NULL); if (s != yajl_gen_status_ok) goto out; @@ -2990,7 +3003,7 @@ static void reboot_domain(uint32_t domid, libxl_evgen_domain_death **deathw, } } -static void list_domains_details(const libxl_dominfo *info, int nb_domain) +static void list_domains_details(libxl_dominfo *info, int nb_domain) { libxl_domain_config d_config; @@ -3017,26 +3030,31 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain) s = yajl_gen_status_ok; for (i = 0; i < nb_domain; i++) { - /* no detailed info available on dom0 */ - if (info[i].domid == 0) - continue; - rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len); - if (rc) - continue; - CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid)); - libxl_domain_config_init(&d_config); - parse_config_data(config_source, (char *)data, len, &d_config, NULL); - if (default_output_format == OUTPUT_FORMAT_JSON) - s = printf_info_one_json(hand, info[i].domid, &d_config); - else + fprintf(stderr, "dom%d\n", info[i].domid); + + if ( info[i].domid > 0 ) { + fprintf(stderr, "parse d_config for dom%d\n", info[i].domid); + rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len); + if (rc) + continue; + CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid)); + libxl_domain_config_init(&d_config); + parse_config_data(config_source, (char *)data, len, &d_config, NULL); + } else { fprintf(stderr, "no d_config\n"); } + if (default_output_format == OUTPUT_FORMAT_JSON) { + s = printf_info_one_json(hand, info[i].domid, + i == 0 ? NULL : &d_config, &info[i]); + if (s != yajl_gen_status_ok) + goto out; + } else printf_info_sexp(info[i].domid, &d_config); - libxl_domain_config_dispose(&d_config); - free(data); - free(config_source); - if (s != yajl_gen_status_ok) - goto out; - } + if ( info[i].domid > 0 ) { + libxl_domain_config_dispose(&d_config); + free(data); + free(config_source); + } + } if (default_output_format == OUTPUT_FORMAT_JSON) { s = yajl_gen_array_close(hand); if (s != yajl_gen_status_ok) -- 1.7.10.4
On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote:> > - No console in xl list -l, so can''t get tty console port.Together with the previous patch this can be resoled with the following, I think. I''m not sure this is the best way to do it though, although dominfo is mostly "runtime" info and this fits.... 8<---------------------------- From 4cffdf9517c668a176f37fa58236f4eb2fd4bbb0 Mon Sep 17 00:00:00 2001 From: Ian Campbell <ian.campbell@citrix.com> Date: Thu, 12 Sep 2013 14:41:06 +0100 Subject: [PATCH] libxl: expose the POV console tty in the dominfo. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> --- tools/libxl/libxl.c | 38 ++++++++++++++++++++++++++++---------- tools/libxl/libxl_types.idl | 2 ++ 2 files changed, 30 insertions(+), 10 deletions(-) diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 1bce4bb..8dea15f 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -510,9 +510,12 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid, return 0; } -static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo, +static void xcinfo2xlinfo(libxl__gc *gc, + const xc_domaininfo_t *xcinfo, libxl_dominfo *xlinfo) { + libxl_dominfo_init(xlinfo); + memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t)); xlinfo->domid = xcinfo->domain; xlinfo->ssidref = xcinfo->ssidref; @@ -539,10 +542,17 @@ static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo, xlinfo->cpupool = xcinfo->cpupool; xlinfo->domain_type = (xcinfo->flags & XEN_DOMINF_hvm_guest) ? LIBXL_DOMAIN_TYPE_HVM : LIBXL_DOMAIN_TYPE_PV; + + if (xcinfo->domaindomid && libxl_console_get_tty(CTX, xcinfo->domain, 0, + LIBXL_CONSOLE_TYPE_PV, + &xlinfo->console_tty) < 0) + LOG(WARN, "unable to get dom%d pv console tty\n", xcinfo->domain); + } libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out) { + GC_INIT(ctx); libxl_dominfo *ptr; int i, ret; xc_domaininfo_t info[1024]; @@ -551,38 +561,46 @@ libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out) ptr = calloc(size, sizeof(libxl_dominfo)); if (!ptr) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "allocating domain info"); - return NULL; + goto err; } ret = xc_domain_getinfolist(ctx->xch, 0, 1024, info); if (ret<0) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list"); free(ptr); - return NULL; + ptr = NULL; + goto err; } for (i = 0; i < ret; i++) { - xcinfo2xlinfo(&info[i], &ptr[i]); + xcinfo2xlinfo(gc, &info[i], &ptr[i]); } *nb_domain_out = ret; +err: + GC_FREE; return ptr; } int libxl_domain_info(libxl_ctx *ctx, libxl_dominfo *info_r, - uint32_t domid) { + uint32_t domid) +{ + GC_INIT(ctx); xc_domaininfo_t xcinfo; - int ret; + int ret, rc = ERROR_INVAL; ret = xc_domain_getinfolist(ctx->xch, domid, 1, &xcinfo); if (ret<0) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list"); return ERROR_FAIL; } - if (ret==0 || xcinfo.domain != domid) return ERROR_INVAL; + if (ret==0 || xcinfo.domain != domid) goto err;; if (info_r) - xcinfo2xlinfo(&xcinfo, info_r); - return 0; + xcinfo2xlinfo(gc, &xcinfo, info_r); + rc = 0; +err: + GC_FREE; + return rc; } static int cpupool_info(libxl__gc *gc, @@ -3764,7 +3782,7 @@ retry_transaction: abort_transaction = 1; goto out; } - xcinfo2xlinfo(&info, &ptr); + xcinfo2xlinfo(gc, &info, &ptr); uuid = libxl__uuid2string(gc, ptr.uuid); libxl__xs_write(gc, t, libxl__sprintf(gc, "/vm/%s/memory", uuid), "%"PRIu32, new_target_memkb / 1024); diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 10f95f4..6de305b 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -200,6 +200,8 @@ libxl_dominfo = Struct("dominfo",[ ("shutdown", bool), ("dying", bool), + ("console_tty", string), + # Valid iff (shutdown||dying). # # Otherwise set to a value guaranteed not to clash with any valid -- 1.7.10.4
On Thu, Sep 12, 2013 at 10:50:49AM +0100, Ian Campbell wrote:> create ! > title it xl lacks support for xm create -F (sxp config file syntax) > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > > > Live migration between a system running xend to a system that doesn''t > > support xend would be a nice-to-have. That probably means some support > > for SXP parsing either in libxl or in some compatibility tool. > > Matt mentioned elsewhere that they use "xm create -F" which takes a > configuration file in the xend sxp format (I had no idea this option > existed). > > I don''t think we really want to be adding SXP code to xm but perhaps we > can produce a conversion tool?Indeed, also mentioned elsewhere it''s probably not in the best interest of the project to go adding SXP support when we already have more well adopted options like json.> Matt, can you provide some examples of the actual SXP you care about, we > obviously don''t want to be worrying about the bits which aren''t actually > used.Honestly I think it''d be less total work to avoid SXP altogether. There are some other differences that need to be carefully considered, and shifting config file formats is a forcing function for looking closely for those differences. --msw
Konrad Rzeszutek Wilk
2013-Sep-25 17:41 UTC
Re: [PATCH] xl: print runtime info in "xl list -l"
On Thu, Sep 12, 2013 at 01:52:09PM +0100, Ian Campbell wrote:> On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote: > > - No status in xl list -l when only dom0 is present. > > Which bits of status are you interested in?The (status 2) field.> > I''m thinking something like the following, which will include the > content of libxl_dominfo for every domain. > > 8<---------------------------------- > > >From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001 > From: Ian Campbell <ian.campbell@citrix.com> > Date: Thu, 12 Sep 2013 13:50:33 +0100 > Subject: [PATCH] xl: print runtime info in "xl list -l" > > Include dom0 runtime information, but not domain config.With that I get (with /etc/xen/xl.conf having output_mode=sxp) -bash-4.1# xl list -l dom0 no d_config (domain (domid 0) (create_info) (hvm 1) (hap True) (oos <default>) (ssidref 0) (name (null)) (uuid 00000000-0000-0000-0000-000000000000) (xsdata (null)) (platformdata (null)) (build_info) (max_vcpus 0) (tsc_mode (null)) (max_memkb 0) (target_memkb 45887743) (nomigrate True) (image (hvm (firmware ) (video_memkb 140277926854668) (shadow_memkb 140280103390500) (pae True) (apic True) (acpi True) (nx False) (viridian True) (hpet False) (vpt_align <default>) (timer_mode (null)) (nestedhvm True) (stdvga False) (vnc <default>) Segmentation fault While under xm: -bash-4.1# xm list -l (domain (domid 0) (cpu_weight 256) (cpu_cap 0) (pool_name Pool-0) (bootloader '''') (vcpus 4) (cpus ((0 1 2 3) (0 1 2 3) (0 1 2 3) (0 1 2 3))) (on_poweroff destroy) (on_crash restart) (uuid 00000000-0000-0000-0000-000000000000) (bootloader_args '''') (name Domain-0) (on_reboot restart) (maxmem 6144) (memory 2048) (shadow_memory 0) (features '''') (on_xend_start ignore) (on_xend_stop ignore) (cpu_time 11.138272406) (online_vcpus 4) (image (linux (kernel '''') (superpages 0) (nomigrate 0) (tsc_mode 0))) (status 2) (state r-----) ) Let me (once I am done with this email backlog) look at this and see if I can make the segfault go away. Thanks for the prototype!> > Signed-off-by: Ian Campbell <ian.campbell@citrix.com> > --- > tools/libxl/xl_cmdimpl.c | 74 ++++++++++++++++++++++++++++------------------ > 1 file changed, 46 insertions(+), 28 deletions(-) > > diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c > index 884f050..46fdafc 100644 > --- a/tools/libxl/xl_cmdimpl.c > +++ b/tools/libxl/xl_cmdimpl.c > @@ -298,7 +298,8 @@ static void dolog(const char *file, int line, const char *func, char *fmt, ...) > } > > static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid, > - libxl_domain_config *d_config) > + libxl_domain_config *d_config, > + libxl_dominfo *d_info) > { > yajl_gen_status s; > > @@ -317,13 +318,25 @@ static yajl_gen_status printf_info_one_json(yajl_gen hand, int domid, > if (s != yajl_gen_status_ok) > goto out; > > - s = yajl_gen_string(hand, (const unsigned char *)"config", > - sizeof("config")-1); > - if (s != yajl_gen_status_ok) > - goto out; > - s = libxl_domain_config_gen_json(hand, d_config); > - if (s != yajl_gen_status_ok) > - goto out; > + if (d_config) { > + s = yajl_gen_string(hand, (const unsigned char *)"config", > + sizeof("config")-1); > + if (s != yajl_gen_status_ok) > + goto out; > + s = libxl_domain_config_gen_json(hand, d_config); > + if (s != yajl_gen_status_ok) > + goto out; > + } > + > + if (d_info) { > + s = yajl_gen_string(hand, (const unsigned char *)"info", > + sizeof("info")-1); > + if (s != yajl_gen_status_ok) > + goto out; > + s = libxl_dominfo_gen_json(hand, d_info); > + if (s != yajl_gen_status_ok) > + goto out; > + } > > s = yajl_gen_map_close(hand); > if (s != yajl_gen_status_ok) > @@ -350,7 +363,7 @@ static void printf_info(enum output_format output_format, > return; > } > > - s = printf_info_one_json(hand, domid, d_config); > + s = printf_info_one_json(hand, domid, d_config, NULL); > if (s != yajl_gen_status_ok) > goto out; > > @@ -2990,7 +3003,7 @@ static void reboot_domain(uint32_t domid, libxl_evgen_domain_death **deathw, > } > } > > -static void list_domains_details(const libxl_dominfo *info, int nb_domain) > +static void list_domains_details(libxl_dominfo *info, int nb_domain) > { > libxl_domain_config d_config; > > @@ -3017,26 +3030,31 @@ static void list_domains_details(const libxl_dominfo *info, int nb_domain) > s = yajl_gen_status_ok; > > for (i = 0; i < nb_domain; i++) { > - /* no detailed info available on dom0 */ > - if (info[i].domid == 0) > - continue; > - rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len); > - if (rc) > - continue; > - CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid)); > - libxl_domain_config_init(&d_config); > - parse_config_data(config_source, (char *)data, len, &d_config, NULL); > - if (default_output_format == OUTPUT_FORMAT_JSON) > - s = printf_info_one_json(hand, info[i].domid, &d_config); > - else > + fprintf(stderr, "dom%d\n", info[i].domid); > + > + if ( info[i].domid > 0 ) { > + fprintf(stderr, "parse d_config for dom%d\n", info[i].domid); > + rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len); > + if (rc) > + continue; > + CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid)); > + libxl_domain_config_init(&d_config); > + parse_config_data(config_source, (char *)data, len, &d_config, NULL); > + } else { fprintf(stderr, "no d_config\n"); } > + if (default_output_format == OUTPUT_FORMAT_JSON) { > + s = printf_info_one_json(hand, info[i].domid, > + i == 0 ? NULL : &d_config, &info[i]); > + if (s != yajl_gen_status_ok) > + goto out; > + } else > printf_info_sexp(info[i].domid, &d_config); > - libxl_domain_config_dispose(&d_config); > - free(data); > - free(config_source); > - if (s != yajl_gen_status_ok) > - goto out; > - } > > + if ( info[i].domid > 0 ) { > + libxl_domain_config_dispose(&d_config); > + free(data); > + free(config_source); > + } > + } > if (default_output_format == OUTPUT_FORMAT_JSON) { > s = yajl_gen_array_close(hand); > if (s != yajl_gen_status_ok) > -- > 1.7.10.4 > > >
On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:> On Thu, Sep 12, 2013 at 01:52:09PM +0100, Ian Campbell wrote: > > On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote: > > > - No status in xl list -l when only dom0 is present. > > > > Which bits of status are you interested in? > > The > (status 2)Oh, I thought you might mean the "(state r-----)" bit which I know and understand. I''ve no idea what this 2 status is -- any idea what it means/relates to?> field. > > > > I''m thinking something like the following, which will include the > > content of libxl_dominfo for every domain. > > > > 8<---------------------------------- > > > > >From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001 > > From: Ian Campbell <ian.campbell@citrix.com> > > Date: Thu, 12 Sep 2013 13:50:33 +0100 > > Subject: [PATCH] xl: print runtime info in "xl list -l" > > > > Include dom0 runtime information, but not domain config. > > With that I get (with /etc/xen/xl.conf having output_mode=sxp)Actually, I probably didn''t hook up the sxp case, oops! Should be easy enough to add though. Ian.
On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote:> While under xm: > -bash-4.1# xm list -lBTW, if you can gather a representative set of such outputs from the sorts of configurations you support we can look at heading straight there instead of faulting in the interesting fields one at a time... Ian.
Konrad Rzeszutek Wilk
2013-Sep-25 18:25 UTC
Re: [PATCH] xl: print runtime info in "xl list -l"
On Wed, Sep 25, 2013 at 06:56:36PM +0100, Ian Campbell wrote:> On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote: > > On Thu, Sep 12, 2013 at 01:52:09PM +0100, Ian Campbell wrote: > > > On Wed, 2013-09-04 at 10:04 -0400, Konrad Rzeszutek Wilk wrote: > > > > - No status in xl list -l when only dom0 is present. > > > > > > Which bits of status are you interested in? > > > > The > > (status 2) > > Oh, I thought you might mean the "(state r-----)" bit which I know and > understand. I''ve no idea what this 2 status is -- any idea what it > means/relates to?It looks to be (status 2) (state -b----) def _stateGet(self): # Lets try and reconsitute the state from xc # first lets try and get the domain info # from xc - this will tell us if the domain # exists info = dom_get(self.getDomid()) if info is None or info[''shutdown'']: # We are either HALTED or SUSPENDED # check saved image exists from xen.xend import XendDomain managed_config_path = \ XendDomain.instance()._managed_check_point_path( \ self.get_uuid()) if os.path.exists(managed_config_path): return XEN_API_VM_POWER_STATE_SUSPENDED else: return XEN_API_VM_POWER_STATE_HALTED elif info[''crashed'']: # Crashed return XEN_API_VM_POWER_STATE_CRASHED else: # We are either RUNNING or PAUSED if info[''paused'']: return XEN_API_VM_POWER_STATE_PAUSED else: return XEN_API_VM_POWER_STATE_RUNNING which is XEN_API_VM_POWER_STATE_HALTED = 0 XEN_API_VM_POWER_STATE_PAUSED = 1 XEN_API_VM_POWER_STATE_RUNNING = 2 XEN_API_VM_POWER_STATE_SUSPENDED = 3 XEN_API_VM_POWER_STATE_SHUTTINGDOWN = 4 XEN_API_VM_POWER_STATE_CRASHED = 5 XEN_API_VM_POWER_STATE_UNKNOWN = 6> > > field. > > > > > > I''m thinking something like the following, which will include the > > > content of libxl_dominfo for every domain. > > > > > > 8<---------------------------------- > > > > > > >From ee9fb6c41cd53afd4983fd2e21ad0e39f178c066 Mon Sep 17 00:00:00 2001 > > > From: Ian Campbell <ian.campbell@citrix.com> > > > Date: Thu, 12 Sep 2013 13:50:33 +0100 > > > Subject: [PATCH] xl: print runtime info in "xl list -l" > > > > > > Include dom0 runtime information, but not domain config. > > > > With that I get (with /etc/xen/xl.conf having output_mode=sxp) > > Actually, I probably didn''t hook up the sxp case, oops!Ah.> > Should be easy enough to add though. > > Ian. >
Konrad Rzeszutek Wilk
2013-Sep-25 18:27 UTC
Re: [PATCH] xl: print runtime info in "xl list -l"
On Wed, Sep 25, 2013 at 06:59:38PM +0100, Ian Campbell wrote:> On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote: > > While under xm: > > -bash-4.1# xm list -l > > BTW, if you can gather a representative set of such outputs from the > sorts of configurations you support we can look at heading straight > there instead of faulting in the interesting fields one at a time...Let me talk to Zhigang on how to make that possible. Thank you!> > Ian. > >
On Wed, Sep 04, 2013 at 10:04:14AM -0400, Konrad Rzeszutek Wilk wrote: [...]> There are issues that are stopping us from using it: >I look at these xl items from time to time and see if I can fix them but they come out a little bit more complex than I expect. Xl will store DomU configs in its private storage location and retrievs them as it see fit. This behavior is causing a series of issues. For ''xl list -l'', the detail info of a domain is retrieved from the stored config file.> - No status in xl list -l when only dom0 is present.This would not work because there is no config file for Dom0.> - ''xl list -l'' says ''Domain name must be specified'' even though the ''-l'' > is for ''Output all VM details''Haven''t traced this one down, but I presume this is caused by the same thing.> - No console in xl list -l, so can''t get tty console port.This would not work because apparently you cannot specify a tty in DomU''s config file -- that''s allcoated by xenconsoled. Along with the above issues, xl save/restored will only use the stored config file which makes it impossible to persist any runtime info across save/restore -- one significant defect is that VM''s MAC address changes every time unless it is specified in the original config file. These issues are all connected in a way -- runtime info is not retrieved / updated / preserved. What I have in mind is that we might need to introduce a mechanism to pull info from Xenstore, serialize / deserialize config info / update stored config file when necessary. Comments? Wei.
On 10/21/2013 10:09 AM, Wei Liu wrote:> On Wed, Sep 04, 2013 at 10:04:14AM -0400, Konrad Rzeszutek Wilk wrote: > [...] >> There are issues that are stopping us from using it: >> > > I look at these xl items from time to time and see if I can fix them but > they come out a little bit more complex than I expect. > > Xl will store DomU configs in its private storage location and retrievs > them as it see fit. This behavior is causing a series of issues. > > For ''xl list -l'', the detail info of a domain is retrieved from the > stored config file. > >> - No status in xl list -l when only dom0 is present. > > This would not work because there is no config file for Dom0. > >> - ''xl list -l'' says ''Domain name must be specified'' even though the ''-l'' >> is for ''Output all VM details'' > > Haven''t traced this one down, but I presume this is caused by the same > thing. > >> - No console in xl list -l, so can''t get tty console port. > > This would not work because apparently you cannot specify a tty in > DomU''s config file -- that''s allcoated by xenconsoled. > > Along with the above issues, xl save/restored will only use the stored > config file which makes it impossible to persist any runtime info across > save/restore -- one significant defect is that VM''s MAC address changes > every time unless it is specified in the original config file. > > These issues are all connected in a way -- runtime info is not retrieved > / updated / preserved. What I have in mind is that we might need to > introduce a mechanism to pull info from Xenstore, serialize / > deserialize config info / update stored config file when necessary. > > Comments?Long time ago, when I work on it, I have the same feeling (I haven''t confirm the latest code yet). xend is a daemon: it handles all VM state change. With xl, a private storage location for each VM is the corresponding solution for now. My suggestion with my limited knowledge is: remove this private storage location and get all the info from xenstore/hypercall. Thanks, Zhigang
On Mon, 2013-10-21 at 15:09 +0100, Wei Liu wrote:> > - No status in xl list -l when only dom0 is present. > > This would not work because there is no config file for Dom0.I sent a PoC patch which adds the content of libxl_dominfo (ie. the runtime info) to the listing alongside the domain config. For dom0 only the runtime info is included, but that is what is needed here.> > - ''xl list -l'' says ''Domain name must be specified'' even though the ''-l'' > > is for ''Output all VM details'' > > Haven''t traced this one down, but I presume this is caused by the same > thing.Yes, I think dom0 can be special cased to only care about libxl_dominfo and not libxl_domain_config. I think I might have included that in the PoC. I thought I''d added a bug to track the thread with that PoC in it, but apparently not. See the subthread starting at <1378990329.10076.54.camel@kazak.uk.xensource.com> I thought I''d posted the tty thing too, apparently not. It was like this and hooked into the patch above to actually get printed. Ian. commit 4cffdf9517c668a176f37fa58236f4eb2fd4bbb0 Author: Ian Campbell <ian.campbell@citrix.com> Date: Thu Sep 12 14:41:06 2013 +0100 libxl: expose the POV console tty in the dominfo. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 1bce4bb..8dea15f 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -510,9 +510,12 @@ int libxl_domain_preserve(libxl_ctx *ctx, uint32_t domid, return 0; } -static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo, +static void xcinfo2xlinfo(libxl__gc *gc, + const xc_domaininfo_t *xcinfo, libxl_dominfo *xlinfo) { + libxl_dominfo_init(xlinfo); + memcpy(&(xlinfo->uuid), xcinfo->handle, sizeof(xen_domain_handle_t)); xlinfo->domid = xcinfo->domain; xlinfo->ssidref = xcinfo->ssidref; @@ -539,10 +542,17 @@ static void xcinfo2xlinfo(const xc_domaininfo_t *xcinfo, xlinfo->cpupool = xcinfo->cpupool; xlinfo->domain_type = (xcinfo->flags & XEN_DOMINF_hvm_guest) ? LIBXL_DOMAIN_TYPE_HVM : LIBXL_DOMAIN_TYPE_PV; + + if (xcinfo->domaindomid && libxl_console_get_tty(CTX, xcinfo->domain, 0, + LIBXL_CONSOLE_TYPE_PV, + &xlinfo->console_tty) < 0) + LOG(WARN, "unable to get dom%d pv console tty\n", xcinfo->domain); + } libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out) { + GC_INIT(ctx); libxl_dominfo *ptr; int i, ret; xc_domaininfo_t info[1024]; @@ -551,38 +561,46 @@ libxl_dominfo * libxl_list_domain(libxl_ctx *ctx, int *nb_domain_out) ptr = calloc(size, sizeof(libxl_dominfo)); if (!ptr) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "allocating domain info"); - return NULL; + goto err; } ret = xc_domain_getinfolist(ctx->xch, 0, 1024, info); if (ret<0) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list"); free(ptr); - return NULL; + ptr = NULL; + goto err; } for (i = 0; i < ret; i++) { - xcinfo2xlinfo(&info[i], &ptr[i]); + xcinfo2xlinfo(gc, &info[i], &ptr[i]); } *nb_domain_out = ret; +err: + GC_FREE; return ptr; } int libxl_domain_info(libxl_ctx *ctx, libxl_dominfo *info_r, - uint32_t domid) { + uint32_t domid) +{ + GC_INIT(ctx); xc_domaininfo_t xcinfo; - int ret; + int ret, rc = ERROR_INVAL; ret = xc_domain_getinfolist(ctx->xch, domid, 1, &xcinfo); if (ret<0) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR, "geting domain info list"); return ERROR_FAIL; } - if (ret==0 || xcinfo.domain != domid) return ERROR_INVAL; + if (ret==0 || xcinfo.domain != domid) goto err;; if (info_r) - xcinfo2xlinfo(&xcinfo, info_r); - return 0; + xcinfo2xlinfo(gc, &xcinfo, info_r); + rc = 0; +err: + GC_FREE; + return rc; } static int cpupool_info(libxl__gc *gc, @@ -3764,7 +3782,7 @@ retry_transaction: abort_transaction = 1; goto out; } - xcinfo2xlinfo(&info, &ptr); + xcinfo2xlinfo(gc, &info, &ptr); uuid = libxl__uuid2string(gc, ptr.uuid); libxl__xs_write(gc, t, libxl__sprintf(gc, "/vm/%s/memory", uuid), "%"PRIu32, new_target_memkb / 1024); diff --git a/tools/libxl/libxl_types.idl b/tools/libxl/libxl_types.idl index 10f95f4..6de305b 100644 --- a/tools/libxl/libxl_types.idl +++ b/tools/libxl/libxl_types.idl @@ -200,6 +200,8 @@ libxl_dominfo = Struct("dominfo",[ ("shutdown", bool), ("dying", bool), + ("console_tty", string), + # Valid iff (shutdown||dying). # # Otherwise set to a value guaranteed not to clash with any valid
On Mon, 2013-10-21 at 15:09 +0100, Wei Liu wrote:> These issues are all connected in a way -- runtime info is not retrieved > / updated / preserved. What I have in mind is that we might need to > introduce a mechanism to pull info from Xenstore, serialize / > deserialize config info / update stored config file when necessary. > > Comments?I agree. I have long thought that we need a way to resynthesize a libxl_domain_config from a running domain directly instead of by reparsing the config, which is likely to be stale. This could also be used as part of migration which also suffers from issues where e.g. the migrated domain gets a new random mac or reverts to the original memory target. In this case instead of sending the config file over the migration wire we''d want to switch to using the JSON generation and add autogenerated JSON parsing for use on the other end. I started to hack on such a thing (see below, as you can see it is a bit WIP!). If you wanted to pick it up that would be awesome. Ian. commit 47a05fa3fa53761f889bec0bc2884fe927ac988f Author: Ian Campbell <ian.campbell@citrix.com> Date: Mon Sep 23 12:52:07 2013 +0100 begin to implement libxl_domain_running_config diff --git a/tools/libxl/libxl.c b/tools/libxl/libxl.c index 8dea15f..0ae145f 100644 --- a/tools/libxl/libxl.c +++ b/tools/libxl/libxl.c @@ -543,9 +543,9 @@ static void xcinfo2xlinfo(libxl__gc *gc, xlinfo->domain_type = (xcinfo->flags & XEN_DOMINF_hvm_guest) ? LIBXL_DOMAIN_TYPE_HVM : LIBXL_DOMAIN_TYPE_PV; - if (xcinfo->domaindomid && libxl_console_get_tty(CTX, xcinfo->domain, 0, - LIBXL_CONSOLE_TYPE_PV, - &xlinfo->console_tty) < 0) + if (xcinfo->domain && libxl_console_get_tty(CTX, xcinfo->domain, 0, + LIBXL_CONSOLE_TYPE_PV, + &xlinfo->console_tty) < 0) LOG(WARN, "unable to get dom%d pv console tty\n", xcinfo->domain); } diff --git a/tools/libxl/libxl.h b/tools/libxl/libxl.h index be19bf5..c824ad6 100644 --- a/tools/libxl/libxl.h +++ b/tools/libxl/libxl.h @@ -677,6 +677,17 @@ int libxl_primary_console_get_tty(libxl_ctx *ctx, uint32_t domid_vm, char **path int libxl_domain_info(libxl_ctx*, libxl_dominfo *info_r, uint32_t domid); +/* Returns the running configuration of a domain. If + * preserve_defaults is true then, where it makes sense, fields which + * were initially set to the default will be returned as defaults + * rather than presenting the actual value. e.g. this will be done for + * disk backend but not NIC MAC addresses. This can be useful if this + * call is to be used to recreate a domain, in which case you want to + * allow libxl to select the appropriate disk backend again but not + * change the MAC address. */ +int libxl_domain_running_config(libxl_ctx*, libxl_domain_config *dconfig_r, + uint32_t domid, bool preserve_defaults); + /* These functions each return (on success) an array of elements, * and the length via the int* out parameter. These arrays and * their contents come from malloc, and must be freed with the diff --git a/tools/libxl/libxl_dom.c b/tools/libxl/libxl_dom.c index 6e2252a..bf11b30 100644 --- a/tools/libxl/libxl_dom.c +++ b/tools/libxl/libxl_dom.c @@ -1668,6 +1668,153 @@ out: return rc; } +int libxl_domain_running_config(libxl_ctx *ctx, libxl_domain_config *dconfig_r, + uint32_t domid, bool preserve_defaults) +{ + GC_INIT(ctx); + int rc; + xc_domaininfo_t info; + int ret; + + /* Convenient aliases */ + libxl_domain_create_info *c_info = &dconfig_r->c_info; + libxl_domain_build_info *b_info = &dconfig_r->b_info; + + libxl_domain_config_init(dconfig_r); + + + ret = xc_domain_getinfolist(CTX->xch, domid, 1, &info); + if (ret != 1) + { + LOGE(ERROR, "getinfolist failed %d\n", ret); + rc = ERROR_FAIL; + goto out; + } + if (info.domain != domid) + { + LOGE(ERROR, "got info for dom%d, wanted dom%d\n", info.domain, domid); + rc = ERROR_FAIL; + goto out; + } + + c_info->type = libxl__domain_type(gc, domid); + if (c_info->type == LIBXL_DOMAIN_TYPE_INVALID) { + rc = ERROR_INVAL; + goto out; + } + + //c_info->hap + //c_info->oos + c_info->ssidref = info.ssidref; + //c_info->name + memcpy(&c_info->uuid, info.handle, sizeof(xen_domain_handle_t)); + //c_info->xsdata + //c_info->platformdata + c_info->poolid = info.cpupool; + //c_info->run_hotplug_scripts + + libxl_domain_build_info_init_type(b_info, c_info->type); + + b_info->max_vcpus = info.max_vcpu_id; + //b_info->avail_vcpus + //b_info->cpumap + //b_info->nodemap + //b_info->numa_placement + //b_info->tsc_mode + b_info->max_memkb = info.max_pages * XC_PAGE_SIZE / 1024; + //b_info->target_memkb + //b_info->video_memkb + //b_info->shadow_memkb + //b_info->rtc_timeoffset + //b_info->exec_ssidref + //b_info->localtime + //b_info->disable_migrate + //b_info->cpuid + //b_info->blkdev_start + + //b_info->device_model_version + //b_info->device_model_stubdomain + //b_info->device_model + //b_info->device_model_ssidref + + //b_info->extra + //b_info->extra_pv + //b_info->extra_hvm + //b_info->sched_params + + //b_info->ioports + //b_info->irqs + //b_info->iomem + //b_info->claim_mode + + switch (b_info->type) { + case LIBXL_DOMAIN_TYPE_HVM: + //b_info->firmware + //b_info->bios + //b_info->pae + //b_info->apic + //b_info->acpi + //b_info->acpi_s3 + //b_info->acpi_s4 + //b_info->nx + //b_info->viridian + //b_info->timeoffset + //b_info->hpet + //b_info->vpt_align + //b_info->timer_mode + //b_info->nested_hvm + //b_info->smbios_firmware + //b_info->acpi_firmware + //b_info->nographic + //b_info->vga + //b_info->vnc + //b_info->keymap + //b_info->sdl + //b_info->spice + //b_info->gfx_passthru + //b_info->serial + //b_info->boot + //b_info->usb + //b_info->usbdevice + //b_info->soundhw + //b_info->xen_platform_pci + //b_info->usbdevice_list + //b_info->vendor_device + break; + case LIBXL_DOMAIN_TYPE_PV: + //b_info->kernel + //b_info->slack_memkb + //b_info->bootloader + //b_info->bootloader_args + //b_info->cmdline + //b_info->ramdisk + //b_info->features + //b_info->e820_host + break; + case LIBXL_DOMAIN_TYPE_INVALID: abort(); + } + + dconfig_r->disks = libxl_device_disk_list(ctx, domid, + &dconfig_r->num_disks); + dconfig_r->nics = libxl_device_nic_list(ctx, domid, + &dconfig_r->num_nics); + dconfig_r->pcidevs = libxl_device_pci_list(ctx, domid, + &dconfig_r->num_pcidevs); + //dconfig_r->vfbs = libxl_device_vfb_list(ctx, domid, + // &dconfig_r->num_vfbs); + //dconfig_r->vkbs = libxl_device_vkb_list(ctx, domid, + // &dconfig_r->num_vkbs); + dconfig_r->vtpms = libxl_device_vtpm_list(ctx, domid, + &dconfig_r->num_vtpms); + + /* on_{poweroff,reboot,watchdog,crash} */ + rc = 0; + +out: + GC_FREE; + return rc; +} + /* * Local variables: * mode: C diff --git a/tools/libxl/xl_cmdimpl.c b/tools/libxl/xl_cmdimpl.c index 9f051f8..48815f0 100644 --- a/tools/libxl/xl_cmdimpl.c +++ b/tools/libxl/xl_cmdimpl.c @@ -3007,9 +3007,9 @@ static void list_domains_details(libxl_dominfo *info, int nb_domain) { libxl_domain_config d_config; - char *config_source; - uint8_t *data; - int i, len, rc; + //char *config_source; + //uint8_t *data; + int i, rc; yajl_gen hand = NULL; yajl_gen_status s; @@ -3032,12 +3032,17 @@ static void list_domains_details(libxl_dominfo *info, int nb_domain) for (i = 0; i < nb_domain; i++) { if ( info[i].domid > 0 ) { - rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len); + //int len; + //rc = libxl_userdata_retrieve(ctx, info[i].domid, "xl", &data, &len); + //if (rc) + // continue; + //CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid)); + //libxl_domain_config_init(&d_config); + //parse_config_data(config_source, (char *)data, len, &d_config, NULL); + rc = libxl_domain_running_config(ctx, &d_config, + info[i].domid, false); if (rc) continue; - CHK_ERRNO(asprintf(&config_source, "<domid %d data>", info[i].domid)); - libxl_domain_config_init(&d_config); - parse_config_data(config_source, (char *)data, len, &d_config, NULL); } if (default_output_format == OUTPUT_FORMAT_JSON) { s = printf_info_one_json(hand, info[i].domid, @@ -3049,8 +3054,8 @@ static void list_domains_details(libxl_dominfo *info, int nb_domain) if ( info[i].domid > 0 ) { libxl_domain_config_dispose(&d_config); - free(data); - free(config_source); + //free(data); + //free(config_source); } } if (default_output_format == OUTPUT_FORMAT_JSON) {
On Wed, Sep 25, 2013 at 02:27:40PM -0400, Konrad Rzeszutek Wilk wrote:> On Wed, Sep 25, 2013 at 06:59:38PM +0100, Ian Campbell wrote: > > On Wed, 2013-09-25 at 13:41 -0400, Konrad Rzeszutek Wilk wrote: > > > While under xm: > > > -bash-4.1# xm list -l > > > > BTW, if you can gather a representative set of such outputs from the > > sorts of configurations you support we can look at heading straight > > there instead of faulting in the interesting fields one at a time... > > Let me talk to Zhigang on how to make that possible. >How is this going?> Thank you! > > > > Ian. > > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
close 17 thanks On Mon, 2013-09-16 at 09:49 -0700, Matt Wilson wrote:> On Thu, Sep 12, 2013 at 10:50:49AM +0100, Ian Campbell wrote: > > create ! > > title it xl lacks support for xm create -F (sxp config file syntax) > > > > On Mon, 2013-09-02 at 13:32 -0700, Matt Wilson wrote: > > > > > > Live migration between a system running xend to a system that doesn''t > > > support xend would be a nice-to-have. That probably means some support > > > for SXP parsing either in libxl or in some compatibility tool. > > > > Matt mentioned elsewhere that they use "xm create -F" which takes a > > configuration file in the xend sxp format (I had no idea this option > > existed). > > > > I don''t think we really want to be adding SXP code to xm but perhaps we > > can produce a conversion tool? > > Indeed, also mentioned elsewhere it''s probably not in the best > interest of the project to go adding SXP support when we already have > more well adopted options like json. > > > Matt, can you provide some examples of the actual SXP you care about, we > > obviously don''t want to be worrying about the bits which aren''t actually > > used. > > Honestly I think it''d be less total work to avoid SXP altogether. > There are some other differences that need to be carefully considered, > and shifting config file formats is a forcing function for looking > closely for those differences.I suppose I should close this bug. If anyone else has a requirement for this stuff now would be a good time to say so. I''ll also add it to http://wiki.xen.org/wiki/XL#Anti-Features. Ian.
xen@bugs.xenproject.org
2013-Nov-05 14:45 UTC
Processed: Re: Support for xm create -F (sxp configuration files)
Processing commands for xen@bugs.xenproject.org:> close 17Closing bug #17> thanksFinished processing. Modified/created Bugs: - 17: http://bugs.xenproject.org/xen/bug/17 --- Xen Hypervisor Bug Tracker See http://wiki.xen.org/wiki/Reporting_Bugs_against_Xen for information on reporting bugs Contact xen-bugs-owner@bugs.xenproject.org with any infrastructure issues