So as discussed last week we now have a plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
The time line is as follows:
19 March -- TODO list locked down << WE ARE HERE
2 April -- Feature Freeze
Mid/Late April -- First release candidate
Weekly -- RCN+1 until it is ready
The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.
Things look pretty good on the hypervisor side of things.
The tools list is quite long but most stuff is known to be in progress
and in many cases preliminary patches are available. I think there is a
name next to everything.
I have gather various items under a top level "xl compatibility with
xm"
heading under both blocker and nice-to-have. I expect this is the area
where most bugs will be reported once we hit -rc and users start testing
this stuff in anger.
Ian.
hypervisor, blockers:
* NONE?
tools, blockers:
* libxl stable API -- we would like 4.2 to define a stable API
which downstream's can start to rely on not changing. Aspects of
this are:
* Safe fork vs. fd handling hooks. Involves API changes
(Ian J, patches posted)
* locking/serialization, e.g., for domain creation. As of
now, nothing is provided for this purpose, and
downstream toolstacks have to put their own mechanisms
in place. E.g., xl uses a fcntl() lock around the whole
process of domain creation. It should OTOH be libxl job
to offer a proper set of hooks --placed at the proper
spots during the domain creation process-- for the
downstreams to fill with the proper callbacks. (Dario
Faggioli)
* agree & document compatibility guarantees + associated
technical measures (Ian C)
* xl compatibility with xm:
* feature parity wrt driver domain support (George Dunlap)
* xl support for "rtc_timeoffset" and
"localtime" (Lin
Ming, Patches posted)
* More formally deprecate xm/xend. Manpage patches already in
tree. Needs release noting and communication around -rc1 to
remind people to test xl.
* Domain 0 block attach & general hotplug when using qdisk backend
(need to start qemu as necessary etc) (Stefano S)
* file:// backend performance. qemu-xen-tradition's qdisk is quite
slow & blktap2 not available in upstream kernels. Need to
consider our options:
* qemu-xen's qdisk is thought to be well performing but
qemu-xen is not yet the default. Complexity arising from
splitting qemu-for-qdisk out from qemu-for-dm and
running N qemu's.
* potentially fully userspace blktap could be ready for
4.2
* use /dev/loop+blkback. This requires loop driver AIO and
O_DIRECT patches which are not (AFAIK) yet upstream.
* Leverage XCP's blktap2 DKMS work.
* Other ideas?
* Improved Hotplug script support (Roger Pau Monné, patches
posted)
* Block script support -- follows on from hotplug script (Roger
Pau Monné)
hypervisor, nice to have:
* solid implementation of sharing/paging/mem-events (using work
queues) (Tim Deegan, Olaf Herring et al -- patches posted)
* "The last patch to use a waitqueue in
__get_gfn_type_access() from Tim works. However, there
are a few users who call __get_gfn_type_access with the
domain_lock held. This part needs to be addressed in
some way."
* Sharing support for AMD (Tim, Andres).
* PoD performance improvements (George Dunlap)
tools, nice to have:
* Configure/control paging via xl/libxl (Olaf Herring, lots of
discussion around interface, general consensus reached on what
it should look like)
* Upstream qemu feature patches:
* Upstream qemu PCI passthrough support (Anthony Perard,
patches sent)
* Upstream qemu save restore (Anthony Perard, Stefano
Stabellini, patches sent, waiting for upstream ack)
* Nested-virtualisation. Currently "experimental". Likely to
release that way.
* Nested SVM. Tested in a variety of configurations but
still some issues with the most important use case (w7
XP mode) [0] (Christoph Egger)
* Nested VMX. Needs nested EPT to be genuinely useful.
Need more data on testedness etc (Intel)
* Initial xl support for Remus (memory checkpoint, blackholing)
(Shriram, patches posted, blocked behind qemu save restore
patches)
* xl compatibility with xm:
* xl support for autospawning vncviewer (vncviewer=1 or
otherwise) (Goncalo Gomes)
* support for vif "rate" parameter (Mathieu Gagné)
[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
>>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote: > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen''s qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu''s. > * potentially fully userspace blktap could be ready for > 4.2 > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream.I meant to ask already when this was first mentioned: What''s the reason for this requirement? Didn''t we have blkback over loop running fine for years? Or is this just a performance consideration (in which case "requires" might be too strong a term)? Jan> * Leverage XCP''s blktap2 DKMS work. > * Other ideas?
On Mon, 2012-03-19 at 11:25 +0000, Jan Beulich wrote:> >>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > > slow & blktap2 not available in upstream kernels. Need to > > consider our options: > > * qemu-xen''s qdisk is thought to be well performing but > > qemu-xen is not yet the default. Complexity arising from > > splitting qemu-for-qdisk out from qemu-for-dm and > > running N qemu''s. > > * potentially fully userspace blktap could be ready for > > 4.2 > > * use /dev/loop+blkback. This requires loop driver AIO and > > O_DIRECT patches which are not (AFAIK) yet upstream. > > I meant to ask already when this was first mentioned: What''s the > reason for this requirement? Didn''t we have blkback over loop running > fine for years? Or is this just a performance consideration (in which > case "requires" might be too strong a term)?My understanding (which could well be totally bogus) was that the use of /dev/loop in this way was unsafe since pages were only committed to the dom0 page cache and not to the actual platter when success was reported to the guest. I think that is why many people used tap:aio: instead of file: (personally I use phy: almost exclusively so I could be talking rubbish). Unless there are some loop patches in the classic-Xen patchset? I don''t think there are though. I don''t know so much about the performance aspect. Stefano might be able to comment. Ian.> > Jan > > > * Leverage XCP''s blktap2 DKMS work. > > * Other ideas? > >
>>> On 19.03.12 at 12:33, Ian Campbell <Ian.Campbell@citrix.com> wrote: > On Mon, 2012-03-19 at 11:25 +0000, Jan Beulich wrote: >> >>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > * file:// backend performance. qemu-xen-tradition''s qdisk is quite >> > slow & blktap2 not available in upstream kernels. Need to >> > consider our options: >> > * qemu-xen''s qdisk is thought to be well performing but >> > qemu-xen is not yet the default. Complexity arising from >> > splitting qemu-for-qdisk out from qemu-for-dm and >> > running N qemu''s. >> > * potentially fully userspace blktap could be ready for >> > 4.2 >> > * use /dev/loop+blkback. This requires loop driver AIO and >> > O_DIRECT patches which are not (AFAIK) yet upstream. >> >> I meant to ask already when this was first mentioned: What''s the >> reason for this requirement? Didn''t we have blkback over loop running >> fine for years? Or is this just a performance consideration (in which >> case "requires" might be too strong a term)? > > My understanding (which could well be totally bogus) was that the use > of /dev/loop in this way was unsafe since pages were only committed to > the dom0 page cache and not to the actual platter when success was > reported to the guest. I think that is why many people used tap:aio: > instead of file: (personally I use phy: almost exclusively so I could be > talking rubbish). > > Unless there are some loop patches in the classic-Xen patchset? I don''t > think there are though.I know of none either. Jan> I don''t know so much about the performance aspect. Stefano might be able > to comment. > > Ian. > >> >> Jan >> >> > * Leverage XCP''s blktap2 DKMS work. >> > * Other ideas? >> >>
On Mon, 19 Mar 2012, Jan Beulich wrote:> >>> On 19.03.12 at 11:57, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > > slow & blktap2 not available in upstream kernels. Need to > > consider our options: > > * qemu-xen''s qdisk is thought to be well performing but > > qemu-xen is not yet the default. Complexity arising from > > splitting qemu-for-qdisk out from qemu-for-dm and > > running N qemu''s. > > * potentially fully userspace blktap could be ready for > > 4.2 > > * use /dev/loop+blkback. This requires loop driver AIO and > > O_DIRECT patches which are not (AFAIK) yet upstream. > > I meant to ask already when this was first mentioned: What''s the > reason for this requirement? Didn''t we have blkback over loop running > fine for years? Or is this just a performance consideration (in which > case "requires" might be too strong a term)?There are several problems here: - the usage of loop with blkback is unsafe because loop doesn''t support O_DIRECT, at least the version of loop present in upstream Linux. Also this means that the very good performance results that we get with loop are actually inflated, the real numbers could be very low. - Loop with blkback doesn''t work with anything but raw files, so it won''t work for qcow, qcow2 or vhd. - Using qdisk as backend, with or without AIO, is possible but from the performance measurements I have run so far is very slow. In theory this should be the best option, but in practice I cannot explain why I am getting 1/10 of the performances I am expecting.
On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> So as discussed last week we now have a plan for a 4.2 release: > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > > The time line is as follows: > > 19 March -- TODO list locked down << WE ARE HERE > 2 April -- Feature Freeze > Mid/Late April -- First release candidate > Weekly -- RCN+1 until it is ready > > The updated TODO list follows. Per the release plan a strong case will > now have to be made as to why new items should be added to the list, > especially as a blocker, rather than deferred to 4.3. > > Things look pretty good on the hypervisor side of things. > > The tools list is quite long but most stuff is known to be in progress > and in many cases preliminary patches are available. I think there is a > name next to everything. > > I have gather various items under a top level "xl compatibility with xm" > heading under both blocker and nice-to-have. I expect this is the area > where most bugs will be reported once we hit -rc and users start testing > this stuff in anger. > > Ian. > > hypervisor, blockers: > * NONE? > > tools, blockers: > * libxl stable API -- we would like 4.2 to define a stable API > which downstream''s can start to rely on not changing. Aspects of > this are: > * Safe fork vs. fd handling hooks. Involves API changes > (Ian J, patches posted) > * locking/serialization, e.g., for domain creation. As of > now, nothing is provided for this purpose, and > downstream toolstacks have to put their own mechanisms > in place. E.g., xl uses a fcntl() lock around the whole > process of domain creation. It should OTOH be libxl job > to offer a proper set of hooks --placed at the proper > spots during the domain creation process-- for the > downstreams to fill with the proper callbacks. (Dario > Faggioli) > * agree & document compatibility guarantees + associated > technical measures (Ian C) > * xl compatibility with xm: > * feature parity wrt driver domain support (George Dunlap) > * xl support for "rtc_timeoffset" and "localtime" (Lin > Ming, Patches posted) > * More formally deprecate xm/xend. Manpage patches already in > tree. Needs release noting and communication around -rc1 to > remind people to test xl.Nonetheless, we should still make basic functionality of xm a requirement for this release, right? I''ve just created a VM with a config file that worked in xl, and it complains that the hotplug scripts are not working, for example. Does that go in this todo list, or are we keeping track of bugs somewhere else?> * Domain 0 block attach & general hotplug when using qdisk backend > (need to start qemu as necessary etc) (Stefano S) > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen''s qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu''s. > * potentially fully userspace blktap could be ready for > 4.2 > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream. > * Leverage XCP''s blktap2 DKMS work. > * Other ideas? > * Improved Hotplug script support (Roger Pau Monné, patches > posted) > * Block script support -- follows on from hotplug script (Roger > Pau Monné) > > hypervisor, nice to have: > * solid implementation of sharing/paging/mem-events (using work > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > * "The last patch to use a waitqueue in > __get_gfn_type_access() from Tim works. However, there > are a few users who call __get_gfn_type_access with the > domain_lock held. This part needs to be addressed in > some way." > * Sharing support for AMD (Tim, Andres). > * PoD performance improvements (George Dunlap) > > tools, nice to have: > * Configure/control paging via xl/libxl (Olaf Herring, lots of > discussion around interface, general consensus reached on what > it should look like) > * Upstream qemu feature patches: > * Upstream qemu PCI passthrough support (Anthony Perard, > patches sent) > * Upstream qemu save restore (Anthony Perard, Stefano > Stabellini, patches sent, waiting for upstream ack) > * Nested-virtualisation. Currently "experimental". Likely to > release that way. > * Nested SVM. Tested in a variety of configurations but > still some issues with the most important use case (w7 > XP mode) [0] (Christoph Egger) > * Nested VMX. Needs nested EPT to be genuinely useful. > Need more data on testedness etc (Intel) > * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches posted, blocked behind qemu save restore > patches) > * xl compatibility with xm: > * xl support for autospawning vncviewer (vncviewer=1 or > otherwise) (Goncalo Gomes) > * support for vif "rate" parameter (Mathieu Gagné) > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Mon, 2012-03-19 at 12:13 +0000, George Dunlap wrote:> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > * More formally deprecate xm/xend. Manpage patches already in > > tree. Needs release noting and communication around -rc1 to > > remind people to test xl. > > Nonetheless, we should still make basic functionality of xm a > requirement for this release, right?Right, deprecated != known broken.> I''ve just created a VM with a > config file that worked in xl, and it complains that the hotplug > scripts are not working, for example. Does that go in this todo list, > or are we keeping track of bugs somewhere else?I''ve not been tracking bugs as such here, I suppose I could. I''d be inclined to wait until we get a bit further along the process (e.g. past feature freeze) before starting to keep a close eye on the bugs. Of course I would be more than happy (indeed, very appreciative) if someone wanted to independently take on that task. However xend is still tested by the automated test and appears to be passing -- are you sure you have a bug and not a configuration issue? Ian.
On Mon, Mar 19, 2012 at 03:57:25AM -0700, Ian Campbell wrote:> > The updated TODO list follows. Per the release plan a strong case will > now have to be made as to why new items should be added to the list, > especially as a blocker, rather than deferred to 4.3.The patches to PV-GRUB for ext4 [1] and btrfs [2] support are very low risk changes. I''ll happily repost them rebased against tip xen-unstable (and with no rejects created in the case of the btrfs patch), if that helps. Matt [1] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00651.html [2] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00649.html
On Tue, 2012-03-20 at 05:19 +0000, Matt Wilson wrote:> On Mon, Mar 19, 2012 at 03:57:25AM -0700, Ian Campbell wrote: > > > > The updated TODO list follows. Per the release plan a strong case will > > now have to be made as to why new items should be added to the list, > > especially as a blocker, rather than deferred to 4.3. > > The patches to PV-GRUB for ext4 [1] and btrfs [2] support are very low > risk changes. I''ll happily repost them rebased against tip > xen-unstable (and with no rejects created in the case of the btrfs > patch), if that helps.I think these would be good to have in, and it certainly isn''t too late for 4.2. I think these originally came in while Ian J (tools maintainer) was away so I expect they just fell through the cracks. Reposting would likely be very useful, thanks. Ian.> > Matt > > [1] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00651.html > [2] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00649.html
Someone pointed out that some people may not know that their name was on this list, so I've CC'd everyone who is mentioned by name. Please let me know if there is a problem with your entry. On Mon, 2012-03-19 at 10:57 +0000, Ian Campbell wrote:> So as discussed last week we now have a plan for a 4.2 release: > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > > The time line is as follows: > > 19 March -- TODO list locked down << WE ARE HERE > 2 April -- Feature Freeze > Mid/Late April -- First release candidate > Weekly -- RCN+1 until it is ready > > The updated TODO list follows. Per the release plan a strong case will > now have to be made as to why new items should be added to the list, > especially as a blocker, rather than deferred to 4.3. > > Things look pretty good on the hypervisor side of things. > > The tools list is quite long but most stuff is known to be in progress > and in many cases preliminary patches are available. I think there is a > name next to everything. > > I have gather various items under a top level "xl compatibility with xm" > heading under both blocker and nice-to-have. I expect this is the area > where most bugs will be reported once we hit -rc and users start testing > this stuff in anger. > > Ian. > > hypervisor, blockers: > * NONE? > > tools, blockers: > * libxl stable API -- we would like 4.2 to define a stable API > which downstream's can start to rely on not changing. Aspects of > this are: > * Safe fork vs. fd handling hooks. Involves API changes > (Ian J, patches posted) > * locking/serialization, e.g., for domain creation. As of > now, nothing is provided for this purpose, and > downstream toolstacks have to put their own mechanisms > in place. E.g., xl uses a fcntl() lock around the whole > process of domain creation. It should OTOH be libxl job > to offer a proper set of hooks --placed at the proper > spots during the domain creation process-- for the > downstreams to fill with the proper callbacks. (Dario > Faggioli) > * agree & document compatibility guarantees + associated > technical measures (Ian C) > * xl compatibility with xm: > * feature parity wrt driver domain support (George Dunlap) > * xl support for "rtc_timeoffset" and "localtime" (Lin > Ming, Patches posted) > * More formally deprecate xm/xend. Manpage patches already in > tree. Needs release noting and communication around -rc1 to > remind people to test xl. > * Domain 0 block attach & general hotplug when using qdisk backend > (need to start qemu as necessary etc) (Stefano S) > * file:// backend performance. qemu-xen-tradition's qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen's qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu's. > * potentially fully userspace blktap could be ready for > 4.2 > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream. > * Leverage XCP's blktap2 DKMS work. > * Other ideas? > * Improved Hotplug script support (Roger Pau Monné, patches > posted) > * Block script support -- follows on from hotplug script (Roger > Pau Monné) > > hypervisor, nice to have: > * solid implementation of sharing/paging/mem-events (using work > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > * "The last patch to use a waitqueue in > __get_gfn_type_access() from Tim works. However, there > are a few users who call __get_gfn_type_access with the > domain_lock held. This part needs to be addressed in > some way." > * Sharing support for AMD (Tim, Andres). > * PoD performance improvements (George Dunlap) > > tools, nice to have: > * Configure/control paging via xl/libxl (Olaf Herring, lots of > discussion around interface, general consensus reached on what > it should look like) > * Upstream qemu feature patches: > * Upstream qemu PCI passthrough support (Anthony Perard, > patches sent) > * Upstream qemu save restore (Anthony Perard, Stefano > Stabellini, patches sent, waiting for upstream ack) > * Nested-virtualisation. Currently "experimental". Likely to > release that way. > * Nested SVM. Tested in a variety of configurations but > still some issues with the most important use case (w7 > XP mode) [0] (Christoph Egger) > * Nested VMX. Needs nested EPT to be genuinely useful. > Need more data on testedness etc (Intel) > * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches posted, blocked behind qemu save restore > patches) > * xl compatibility with xm: > * xl support for autospawning vncviewer (vncviewer=1 or > otherwise) (Goncalo Gomes) > * support for vif "rate" parameter (Mathieu Gagné) > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Mon, 2012-03-19 at 10:57 +0000, Ian Campbell wrote:> tools, nice to have:[...]> * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches posted, blocked behind qemu save restore > patches)AIUI the qemu save/restore patches have now been accepted. Not sure of the status of the xl/libxl side of that, perhaps time for a repost of both that series and the Remus one? Ian.
On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> * xl compatibility with xm: > * feature parity wrt driver domain support (George Dunlap)I just discovered (while playing with driver domains) that xl is missing one bit of feature parity with xm for pci passthrough for PV guests -- and that''s the "pci quirk" config file support. I''m going to ask Intel if they have an interest in porting it over; I think it should at least be a "nice-to-have", and it may be a low-level blocker, as a lot of devices won''t work passed through without it.> * xl support for "rtc_timeoffset" and "localtime" (Lin > Ming, Patches posted) > * More formally deprecate xm/xend. Manpage patches already in > tree. Needs release noting and communication around -rc1 to > remind people to test xl. > * Domain 0 block attach & general hotplug when using qdisk backend > (need to start qemu as necessary etc) (Stefano S) > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen''s qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu''s. > * potentially fully userspace blktap could be ready for > 4.2 > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream. > * Leverage XCP''s blktap2 DKMS work. > * Other ideas? > * Improved Hotplug script support (Roger Pau Monné, patches > posted) > * Block script support -- follows on from hotplug script (Roger > Pau Monné) > > hypervisor, nice to have: > * solid implementation of sharing/paging/mem-events (using work > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > * "The last patch to use a waitqueue in > __get_gfn_type_access() from Tim works. However, there > are a few users who call __get_gfn_type_access with the > domain_lock held. This part needs to be addressed in > some way." > * Sharing support for AMD (Tim, Andres). > * PoD performance improvements (George Dunlap) > > tools, nice to have: > * Configure/control paging via xl/libxl (Olaf Herring, lots of > discussion around interface, general consensus reached on what > it should look like) > * Upstream qemu feature patches: > * Upstream qemu PCI passthrough support (Anthony Perard, > patches sent) > * Upstream qemu save restore (Anthony Perard, Stefano > Stabellini, patches sent, waiting for upstream ack) > * Nested-virtualisation. Currently "experimental". Likely to > release that way. > * Nested SVM. Tested in a variety of configurations but > still some issues with the most important use case (w7 > XP mode) [0] (Christoph Egger) > * Nested VMX. Needs nested EPT to be genuinely useful. > Need more data on testedness etc (Intel) > * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches posted, blocked behind qemu save restore > patches) > * xl compatibility with xm: > * xl support for autospawning vncviewer (vncviewer=1 or > otherwise) (Goncalo Gomes) > * support for vif "rate" parameter (Mathieu Gagné) > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote:> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > * xl compatibility with xm: > > * feature parity wrt driver domain support (George Dunlap) > I just discovered (while playing with driver domains) that xl is > missing one bit of feature parity with xm for pci passthrough for PV > guests -- and that's the "pci quirk" config file support. I'm going > to ask Intel if they have an interest in porting it over; I think it > should at least be a "nice-to-have", and it may be a low-level > blocker, as a lot of devices won't work passed through without it.This is the stuff in tools/python/xen/xend/server/pciquirk.py ? pciback in upstream doesn't mention "quirk" which suggests no support for the necessary sysfs node either? tools/examples/xend-pci-quirks.sxp seems to only have a quirk for a single card? I don't think we want to implement an SXP parser for xl/libxl so if this is reimplemented I think a different format should be used. Anyway, I'll put this onto the list. Ian> > > * xl support for "rtc_timeoffset" and "localtime" (Lin > > Ming, Patches posted) > > * More formally deprecate xm/xend. Manpage patches already in > > tree. Needs release noting and communication around -rc1 to > > remind people to test xl. > > * Domain 0 block attach & general hotplug when using qdisk backend > > (need to start qemu as necessary etc) (Stefano S) > > * file:// backend performance. qemu-xen-tradition's qdisk is quite > > slow & blktap2 not available in upstream kernels. Need to > > consider our options: > > * qemu-xen's qdisk is thought to be well performing but > > qemu-xen is not yet the default. Complexity arising from > > splitting qemu-for-qdisk out from qemu-for-dm and > > running N qemu's. > > * potentially fully userspace blktap could be ready for > > 4.2 > > * use /dev/loop+blkback. This requires loop driver AIO and > > O_DIRECT patches which are not (AFAIK) yet upstream. > > * Leverage XCP's blktap2 DKMS work. > > * Other ideas? > > * Improved Hotplug script support (Roger Pau Monné, patches > > posted) > > * Block script support -- follows on from hotplug script (Roger > > Pau Monné) > > > > hypervisor, nice to have: > > * solid implementation of sharing/paging/mem-events (using work > > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > > * "The last patch to use a waitqueue in > > __get_gfn_type_access() from Tim works. However, there > > are a few users who call __get_gfn_type_access with the > > domain_lock held. This part needs to be addressed in > > some way." > > * Sharing support for AMD (Tim, Andres). > > * PoD performance improvements (George Dunlap) > > > > tools, nice to have: > > * Configure/control paging via xl/libxl (Olaf Herring, lots of > > discussion around interface, general consensus reached on what > > it should look like) > > * Upstream qemu feature patches: > > * Upstream qemu PCI passthrough support (Anthony Perard, > > patches sent) > > * Upstream qemu save restore (Anthony Perard, Stefano > > Stabellini, patches sent, waiting for upstream ack) > > * Nested-virtualisation. Currently "experimental". Likely to > > release that way. > > * Nested SVM. Tested in a variety of configurations but > > still some issues with the most important use case (w7 > > XP mode) [0] (Christoph Egger) > > * Nested VMX. Needs nested EPT to be genuinely useful. > > Need more data on testedness etc (Intel) > > * Initial xl support for Remus (memory checkpoint, blackholing) > > (Shriram, patches posted, blocked behind qemu save restore > > patches) > > * xl compatibility with xm: > > * xl support for autospawning vncviewer (vncviewer=1 or > > otherwise) (Goncalo Gomes) > > * support for vif "rate" parameter (Mathieu Gagné) > > > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xen.org > > http://lists.xen.org/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Thu, Mar 22, 2012 at 9:53 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote: >> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > * xl compatibility with xm: >> > * feature parity wrt driver domain support (George Dunlap) >> I just discovered (while playing with driver domains) that xl is >> missing one bit of feature parity with xm for pci passthrough for PV >> guests -- and that''s the "pci quirk" config file support. I''m going >> to ask Intel if they have an interest in porting it over; I think it >> should at least be a "nice-to-have", and it may be a low-level >> blocker, as a lot of devices won''t work passed through without it. > > This is the stuff in tools/python/xen/xend/server/pciquirk.py ? > > pciback in upstream doesn''t mention "quirk" which suggests no support > for the necessary sysfs node either?Ah, interesting -- that''s worth tracking down. Maybe there''s a better way to deal with quirks? Or maybe it just hasn''t been upstreamed yet (or perhaps even implemented in pvops?). I''m using the Debian squeeze 2.6.32-5-xen-686 kernel.> tools/examples/xend-pci-quirks.sxp seems to only have a quirk for a > single card?Yes, well I could add two more cards just from experience w/ one of my test boxen. :-)> I don''t think we want to implement an SXP parser for xl/libxl so if this > is reimplemented I think a different format should be used.Since we''re using yajl anyway, JSON might not be a bad option. Anyway, I''ll ping the Intel guy who recently posted a patch to libxl_pci.c. -George> > Anyway, I''ll put this onto the list. > > Ian > >> >> > * xl support for "rtc_timeoffset" and "localtime" (Lin >> > Ming, Patches posted) >> > * More formally deprecate xm/xend. Manpage patches already in >> > tree. Needs release noting and communication around -rc1 to >> > remind people to test xl. >> > * Domain 0 block attach & general hotplug when using qdisk backend >> > (need to start qemu as necessary etc) (Stefano S) >> > * file:// backend performance. qemu-xen-tradition''s qdisk is quite >> > slow & blktap2 not available in upstream kernels. Need to >> > consider our options: >> > * qemu-xen''s qdisk is thought to be well performing but >> > qemu-xen is not yet the default. Complexity arising from >> > splitting qemu-for-qdisk out from qemu-for-dm and >> > running N qemu''s. >> > * potentially fully userspace blktap could be ready for >> > 4.2 >> > * use /dev/loop+blkback. This requires loop driver AIO and >> > O_DIRECT patches which are not (AFAIK) yet upstream. >> > * Leverage XCP''s blktap2 DKMS work. >> > * Other ideas? >> > * Improved Hotplug script support (Roger Pau Monné, patches >> > posted) >> > * Block script support -- follows on from hotplug script (Roger >> > Pau Monné) >> > >> > hypervisor, nice to have: >> > * solid implementation of sharing/paging/mem-events (using work >> > queues) (Tim Deegan, Olaf Herring et al -- patches posted) >> > * "The last patch to use a waitqueue in >> > __get_gfn_type_access() from Tim works. However, there >> > are a few users who call __get_gfn_type_access with the >> > domain_lock held. This part needs to be addressed in >> > some way." >> > * Sharing support for AMD (Tim, Andres). >> > * PoD performance improvements (George Dunlap) >> > >> > tools, nice to have: >> > * Configure/control paging via xl/libxl (Olaf Herring, lots of >> > discussion around interface, general consensus reached on what >> > it should look like) >> > * Upstream qemu feature patches: >> > * Upstream qemu PCI passthrough support (Anthony Perard, >> > patches sent) >> > * Upstream qemu save restore (Anthony Perard, Stefano >> > Stabellini, patches sent, waiting for upstream ack) >> > * Nested-virtualisation. Currently "experimental". Likely to >> > release that way. >> > * Nested SVM. Tested in a variety of configurations but >> > still some issues with the most important use case (w7 >> > XP mode) [0] (Christoph Egger) >> > * Nested VMX. Needs nested EPT to be genuinely useful. >> > Need more data on testedness etc (Intel) >> > * Initial xl support for Remus (memory checkpoint, blackholing) >> > (Shriram, patches posted, blocked behind qemu save restore >> > patches) >> > * xl compatibility with xm: >> > * xl support for autospawning vncviewer (vncviewer=1 or >> > otherwise) (Goncalo Gomes) >> > * support for vif "rate" parameter (Mathieu Gagné) >> > >> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html >> > >> > >> > _______________________________________________ >> > Xen-devel mailing list >> > Xen-devel@lists.xen.org >> > http://lists.xen.org/xen-devel > >
On Thu, 2012-03-22 at 10:08 +0000, George Dunlap wrote:> On Thu, Mar 22, 2012 at 9:53 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > > On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote: > >> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: > >> > * xl compatibility with xm: > >> > * feature parity wrt driver domain support (George Dunlap) > >> I just discovered (while playing with driver domains) that xl is > >> missing one bit of feature parity with xm for pci passthrough for PV > >> guests -- and that's the "pci quirk" config file support. I'm going > >> to ask Intel if they have an interest in porting it over; I think it > >> should at least be a "nice-to-have", and it may be a low-level > >> blocker, as a lot of devices won't work passed through without it. > > > > This is the stuff in tools/python/xen/xend/server/pciquirk.py ? > > > > pciback in upstream doesn't mention "quirk" which suggests no support > > for the necessary sysfs node either? > > Ah, interesting -- that's worth tracking down. Maybe there's a better > way to deal with quirks? Or maybe it just hasn't been upstreamed yet > (or perhaps even implemented in pvops?). I'm using the Debian squeeze > 2.6.32-5-xen-686 kernel.I told a lie -- the code does seem to be there in mainline (drivers/xen/xen-pciback/conf_space_quirks.c et al). Not sure how grep missed it. Does anyone know what the actual purpose/function of the single defined quirk is? 10845:df80de098d15 which introduces it doesn't really say, it's just a bunch of magic register frobbing as far as I'm concerned. I guess you have a tg3 and are suffering from this exact quirk? It's an awful lot of scaffolding on both the userspace and kernel side to support a generic quirks system which has had exactly one quirk since it was introduced in mid 2006. Perhaps we should just address the specific tg3 issue directly?> > > tools/examples/xend-pci-quirks.sxp seems to only have a quirk for a > > single card? > > Yes, well I could add two more cards just from experience w/ one of my > test boxen. :-) > > > I don't think we want to implement an SXP parser for xl/libxl so if this > > is reimplemented I think a different format should be used. > > Since we're using yajl anyway, JSON might not be a bad option. > > Anyway, I'll ping the Intel guy who recently posted a patch to libxl_pci.c. > > -George > > > > > Anyway, I'll put this onto the list. > > > > Ian > > > >> > >> > * xl support for "rtc_timeoffset" and "localtime" (Lin > >> > Ming, Patches posted) > >> > * More formally deprecate xm/xend. Manpage patches already in > >> > tree. Needs release noting and communication around -rc1 to > >> > remind people to test xl. > >> > * Domain 0 block attach & general hotplug when using qdisk backend > >> > (need to start qemu as necessary etc) (Stefano S) > >> > * file:// backend performance. qemu-xen-tradition's qdisk is quite > >> > slow & blktap2 not available in upstream kernels. Need to > >> > consider our options: > >> > * qemu-xen's qdisk is thought to be well performing but > >> > qemu-xen is not yet the default. Complexity arising from > >> > splitting qemu-for-qdisk out from qemu-for-dm and > >> > running N qemu's. > >> > * potentially fully userspace blktap could be ready for > >> > 4.2 > >> > * use /dev/loop+blkback. This requires loop driver AIO and > >> > O_DIRECT patches which are not (AFAIK) yet upstream. > >> > * Leverage XCP's blktap2 DKMS work. > >> > * Other ideas? > >> > * Improved Hotplug script support (Roger Pau Monné, patches > >> > posted) > >> > * Block script support -- follows on from hotplug script (Roger > >> > Pau Monné) > >> > > >> > hypervisor, nice to have: > >> > * solid implementation of sharing/paging/mem-events (using work > >> > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > >> > * "The last patch to use a waitqueue in > >> > __get_gfn_type_access() from Tim works. However, there > >> > are a few users who call __get_gfn_type_access with the > >> > domain_lock held. This part needs to be addressed in > >> > some way." > >> > * Sharing support for AMD (Tim, Andres). > >> > * PoD performance improvements (George Dunlap) > >> > > >> > tools, nice to have: > >> > * Configure/control paging via xl/libxl (Olaf Herring, lots of > >> > discussion around interface, general consensus reached on what > >> > it should look like) > >> > * Upstream qemu feature patches: > >> > * Upstream qemu PCI passthrough support (Anthony Perard, > >> > patches sent) > >> > * Upstream qemu save restore (Anthony Perard, Stefano > >> > Stabellini, patches sent, waiting for upstream ack) > >> > * Nested-virtualisation. Currently "experimental". Likely to > >> > release that way. > >> > * Nested SVM. Tested in a variety of configurations but > >> > still some issues with the most important use case (w7 > >> > XP mode) [0] (Christoph Egger) > >> > * Nested VMX. Needs nested EPT to be genuinely useful. > >> > Need more data on testedness etc (Intel) > >> > * Initial xl support for Remus (memory checkpoint, blackholing) > >> > (Shriram, patches posted, blocked behind qemu save restore > >> > patches) > >> > * xl compatibility with xm: > >> > * xl support for autospawning vncviewer (vncviewer=1 or > >> > otherwise) (Goncalo Gomes) > >> > * support for vif "rate" parameter (Mathieu Gagné) > >> > > >> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > >> > > >> > > >> > _______________________________________________ > >> > Xen-devel mailing list > >> > Xen-devel@lists.xen.org > >> > http://lists.xen.org/xen-devel > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Thu, 22 Mar 2012, Ian Campbell wrote:> On Mon, 2012-03-19 at 10:57 +0000, Ian Campbell wrote: > > > tools, nice to have: > [...] > > * Initial xl support for Remus (memory checkpoint, blackholing) > > (Shriram, patches posted, blocked behind qemu save restore > > patches) > > AIUI the qemu save/restore patches have now been accepted. Not sure of > the status of the xl/libxl side of that, perhaps time for a repost of > both that series and the Remus one?I have reposted my series already: http://marc.info/?l=xen-devel&m=133224577008104
On 22/03/2012 10:19, "Ian Campbell" <Ian.Campbell@citrix.com> wrote:> I told a lie -- the code does seem to be there in > mainline(drivers/xen/xen-pciback/conf_space_quirks.c et al). Not sure how> grepmissed it. Does anyone know what the actual purpose/function of the> single definedquirk is? 10845:df80de098d15 which introduces it doesn''t really> say,it''s just a bunch of magic register frobbing as far as I''m concerned. I> guess you have a tg3 and are suffering from this exact quirk?It''s an awful> lot of scaffolding on both the userspace and kernel sideto support a generic> quirks system which has had exactly one quirk sinceit was introduced in mid> 2006. Perhaps we should just address thespecific tg3 issue directly? The issue is that ''quirk'' is something of a misnomer. Many devices may have device-specific registers exposed via their PCI config space. The default pciback policy is set to allow only known generic PCI config space registers to be accessed by a guest. This will fail for likely a fair few devices, and the most common solution is to set the pciback.permissive flag and just allow guest access to all of the device''s PCI config space. This is far less hassle than actually setting up a proper ''quirk'' and frankly most people trust the device drivers they run, even if they are in a domU. I wonder how many people really give a crap about the protection of !pciback.permissive. -- Keir
On Thu, Mar 22, 2012 at 10:19 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Thu, 2012-03-22 at 10:08 +0000, George Dunlap wrote: >> On Thu, Mar 22, 2012 at 9:53 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> > On Thu, 2012-03-22 at 09:35 +0000, George Dunlap wrote: >> >> On Mon, Mar 19, 2012 at 10:57 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote: >> >> > * xl compatibility with xm: >> >> > * feature parity wrt driver domain support (George Dunlap) >> >> I just discovered (while playing with driver domains) that xl is >> >> missing one bit of feature parity with xm for pci passthrough for PV >> >> guests -- and that''s the "pci quirk" config file support. I''m going >> >> to ask Intel if they have an interest in porting it over; I think it >> >> should at least be a "nice-to-have", and it may be a low-level >> >> blocker, as a lot of devices won''t work passed through without it. >> > >> > This is the stuff in tools/python/xen/xend/server/pciquirk.py ? >> > >> > pciback in upstream doesn''t mention "quirk" which suggests no support >> > for the necessary sysfs node either? >> >> Ah, interesting -- that''s worth tracking down. Maybe there''s a better >> way to deal with quirks? Or maybe it just hasn''t been upstreamed yet >> (or perhaps even implemented in pvops?). I''m using the Debian squeeze >> 2.6.32-5-xen-686 kernel. > > I told a lie -- the code does seem to be there in mainline > (drivers/xen/xen-pciback/conf_space_quirks.c et al). Not sure how grep > missed it. > > Does anyone know what the actual purpose/function of the single defined > quirk is? 10845:df80de098d15 which introduces it doesn''t really say, > it''s just a bunch of magic register frobbing as far as I''m concerned. > > I guess you have a tg3 and are suffering from this exact quirk? > > It''s an awful lot of scaffolding on both the userspace and kernel side > to support a generic quirks system which has had exactly one quirk since > it was introduced in mid 2006. Perhaps we should just address the > specific tg3 issue directly?On the contrary, I don''t have a tg3, but an Intel nic that uses Linux''s igd driver, and another that uses the bnx2 driver. When I pass those through to a PV guest, I get the following messages printed from dom0''s pciback, respectively: (for igd) [ 77.619293] pciback 0000:07:00.0: PCI INT A -> GSI 45 (level, low) -> IRQ 45^M [ 77.626683] pciback 0000:07:00.0: Driver tried to write to a read-only configuration space field at offset 0xa8, size 2. This may be harmless, but if you have problems with your device:^M [ 77.626685] 1) see permissive attribute in sysfs^M [ 77.626687] 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci.^M (for bnx2) [ 363.582059] pciback 0000:02:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32^M [ 363.590050] pciback 0000:02:00.0: Driver tried to write to a read-only configuration space field at offset 0x68, size 4. This may be harmless, but if you have problems with your device:^M [ 363.590054] 1) see permissive attribute in sysfs^M [ 363.590055] 2) report problems to the xen-devel mailing list along with details of your device obtained from lspci.^M And at least one person has solved this by adding something to the "quirks" file (search for "PCI permissions"): http://technical.bestgrid.org/index.php/Xen:_assigning_PCI_devices_to_a_domain The devices in fact don''t work quite right, AFAICT. So I''m gathering that the "quirks" lists areas of PCI configuration space which it is safe for pciback to allow the guest to modify. (I''m going to hack libxl to set "permissive" to test this theory.) -George> >> >> > tools/examples/xend-pci-quirks.sxp seems to only have a quirk for a >> > single card? >> >> Yes, well I could add two more cards just from experience w/ one of my >> test boxen. :-) >> >> > I don''t think we want to implement an SXP parser for xl/libxl so if this >> > is reimplemented I think a different format should be used. >> >> Since we''re using yajl anyway, JSON might not be a bad option. >> >> Anyway, I''ll ping the Intel guy who recently posted a patch to libxl_pci.c. >> >> -George >> >> > >> > Anyway, I''ll put this onto the list. >> > >> > Ian >> > >> >> >> >> > * xl support for "rtc_timeoffset" and "localtime" (Lin >> >> > Ming, Patches posted) >> >> > * More formally deprecate xm/xend. Manpage patches already in >> >> > tree. Needs release noting and communication around -rc1 to >> >> > remind people to test xl. >> >> > * Domain 0 block attach & general hotplug when using qdisk backend >> >> > (need to start qemu as necessary etc) (Stefano S) >> >> > * file:// backend performance. qemu-xen-tradition''s qdisk is quite >> >> > slow & blktap2 not available in upstream kernels. Need to >> >> > consider our options: >> >> > * qemu-xen''s qdisk is thought to be well performing but >> >> > qemu-xen is not yet the default. Complexity arising from >> >> > splitting qemu-for-qdisk out from qemu-for-dm and >> >> > running N qemu''s. >> >> > * potentially fully userspace blktap could be ready for >> >> > 4.2 >> >> > * use /dev/loop+blkback. This requires loop driver AIO and >> >> > O_DIRECT patches which are not (AFAIK) yet upstream. >> >> > * Leverage XCP''s blktap2 DKMS work. >> >> > * Other ideas? >> >> > * Improved Hotplug script support (Roger Pau Monné, patches >> >> > posted) >> >> > * Block script support -- follows on from hotplug script (Roger >> >> > Pau Monné) >> >> > >> >> > hypervisor, nice to have: >> >> > * solid implementation of sharing/paging/mem-events (using work >> >> > queues) (Tim Deegan, Olaf Herring et al -- patches posted) >> >> > * "The last patch to use a waitqueue in >> >> > __get_gfn_type_access() from Tim works. However, there >> >> > are a few users who call __get_gfn_type_access with the >> >> > domain_lock held. This part needs to be addressed in >> >> > some way." >> >> > * Sharing support for AMD (Tim, Andres). >> >> > * PoD performance improvements (George Dunlap) >> >> > >> >> > tools, nice to have: >> >> > * Configure/control paging via xl/libxl (Olaf Herring, lots of >> >> > discussion around interface, general consensus reached on what >> >> > it should look like) >> >> > * Upstream qemu feature patches: >> >> > * Upstream qemu PCI passthrough support (Anthony Perard, >> >> > patches sent) >> >> > * Upstream qemu save restore (Anthony Perard, Stefano >> >> > Stabellini, patches sent, waiting for upstream ack) >> >> > * Nested-virtualisation. Currently "experimental". Likely to >> >> > release that way. >> >> > * Nested SVM. Tested in a variety of configurations but >> >> > still some issues with the most important use case (w7 >> >> > XP mode) [0] (Christoph Egger) >> >> > * Nested VMX. Needs nested EPT to be genuinely useful. >> >> > Need more data on testedness etc (Intel) >> >> > * Initial xl support for Remus (memory checkpoint, blackholing) >> >> > (Shriram, patches posted, blocked behind qemu save restore >> >> > patches) >> >> > * xl compatibility with xm: >> >> > * xl support for autospawning vncviewer (vncviewer=1 or >> >> > otherwise) (Goncalo Gomes) >> >> > * support for vif "rate" parameter (Mathieu Gagné) >> >> > >> >> > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html >> >> > >> >> > >> >> > _______________________________________________ >> >> > Xen-devel mailing list >> >> > Xen-devel@lists.xen.org >> >> > http://lists.xen.org/xen-devel >> > >> > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Thu, 2012-03-22 at 10:34 +0000, George Dunlap wrote:> The devices in fact don''t work quite right, AFAICT. So I''m gathering > that the "quirks" lists areas of PCI configuration space which it is > safe for pciback to allow the guest to modify. (I''m going to hack > libxl to set "permissive" to test this theory.)I think that based on what Keir says we should just expose the permissive setting in xl/libxl for the time being, that''s a far simpler patch. No need for the full power of per-register whitelisting Ian.
On Thu, 2012-03-22 at 10:38 +0000, Ian Campbell wrote:> On Thu, 2012-03-22 at 10:34 +0000, George Dunlap wrote: > > The devices in fact don''t work quite right, AFAICT. So I''m gathering > > that the "quirks" lists areas of PCI configuration space which it is > > safe for pciback to allow the guest to modify. (I''m going to hack > > libxl to set "permissive" to test this theory.) > > I think that based on what Keir says we should just expose the > permissive setting in xl/libxl for the time being, that''s a far simpler > patch. No need for the full power of per-register whitelistingLooks like this is just a case of writing the BDF of a device to a sysfs node or booting with xen-pciback.permissive=1 to turn it on globally. Possibly this could be "fixed" purely by documentation? I''ve added it to the TODO as a nice to have with George''s name next to it, hope that''s OK. Ian.
A day late due to the document day yesterday.
Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
The time line is as follows:
19 March -- TODO list locked down
<< WE ARE HERE
2 April -- Feature Freeze << THIS IS NEXT WEEK
Mid/Late April -- First release candidate
Weekly -- RCN+1 until it is ready
The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.
Feature Freeze is this Monday. At this point we will begin deferring
patches which have not previously been posted and which are not on the
list below until Xen 4.3.
Ian.
hypervisor, blockers:
* NONE?
tools, blockers:
* libxl stable API -- we would like 4.2 to define a stable API
which downstream's can start to rely on not changing. Aspects of
this are:
* Safe fork vs. fd handling hooks. Involves API changes
(Ian J, patches posted)
* locking/serialization, e.g., for domain creation. As of
now, nothing is provided for this purpose, and
downstream toolstacks have to put their own mechanisms
in place. E.g., xl uses a fcntl() lock around the whole
process of domain creation. It should OTOH be libxl job
to offer a proper set of hooks --placed at the proper
spots during the domain creation process-- for the
downstreams to fill with the proper callbacks. (Dario
Faggioli)
* agree & document compatibility guarantees + associated
technical measures (Ian C, patches posted)
* xl compatibility with xm:
* feature parity wrt driver domain support (George Dunlap)
* xl support for "rtc_timeoffset" and
"localtime" (Lin
Ming, Patches posted)
* More formally deprecate xm/xend. Manpage patches already in
tree. Needs release noting and communication around -rc1 to
remind people to test xl.
* Domain 0 block attach & general hotplug when using qdisk backend
(need to start qemu as necessary etc) (Stefano S)
* file:// backend performance. qemu-xen-tradition's qdisk is quite
slow & blktap2 not available in upstream kernels. Need to
consider our options:
* qemu-xen's qdisk is thought to be well
performing but qemu-xen is not yet the default.
Complexity arising from splitting qemu-for-qdisk
out from qemu-for-dm and running N qemu's.
* potentially fully userspace blktap could be
ready for 4.2
* use /dev/loop+blkback. This requires loop driver
AIO and O_DIRECT patches which are not (AFAIK)
yet upstream.
* Leverage XCP's blktap2 DKMS work.
* Other ideas?
* In general we should recommend blkback
(e.g. with LVM) since it always out
performs other solutions, although at
the expense of flexibility regarding
image formats.
Stefano has done a lot of work here and has proposed some
performance improvements for qdisk in both qemu-xen and
qemu-xen-traditional which make them a plausible alternative for
Xen 4.2.
* Improved Hotplug script support (Roger Pau Monné, patches
posted)
* Block script support -- follows on from hotplug script (Roger
Pau Monné)
hypervisor, nice to have:
* solid implementation of sharing/paging/mem-events (using work
queues) (Tim Deegan, Olaf Herring et al -- patches posted)
* "The last patch to use a waitqueue in
__get_gfn_type_access() from Tim works. However, there
are a few users who call __get_gfn_type_access with the
domain_lock held. This part needs to be addressed in
some way."
* Sharing support for AMD (Tim, Andres).
* PoD performance improvements (George Dunlap)
tools, nice to have:
* Configure/control paging via xl/libxl (Olaf Herring, lots of
discussion around interface, general consensus reached on what
it should look like. Olaf has let me know that he is probably
not going to have time to do this for 4.2, will likely slip to
4.3 unless someone else want to pick up that work)
* Upstream qemu feature patches:
* Upstream qemu PCI passthrough support (Anthony Perard,
patches sent)
* Upstream qemu save restore (Anthony Perard, Stefano
Stabellini, qemu patches applied, waiting for libxl etc
side which has been recently reposted)
* Nested-virtualisation. Currently "experimental". Likely to
release that way.
* Nested SVM. Tested in a variety of configurations but
still some issues with the most important use case (w7
XP mode) [0] (Christoph Egger)
* Nested VMX. Needs nested EPT to be genuinely useful.
Need more data on testedness etc (Intel)
* Initial xl support for Remus (memory checkpoint, blackholing)
(Shriram, patches reposted recently, was blocked behind qemu
save restore patches which are now in)
* xl compatibility with xm:
* xl support for autospawning vncviewer (vncviewer=1 or
otherwise) (Goncalo Gomes, patches posted)
* support for vif "rate" parameter (Mathieu Gagné,
patches
posted)
* expose PCI back "permissive" parameter (George Dunlap)
[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
On Tue, Mar 27, 2012 at 10:33 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Thu, 2012-03-22 at 10:38 +0000, Ian Campbell wrote: >> On Thu, 2012-03-22 at 10:34 +0000, George Dunlap wrote: >> > The devices in fact don''t work quite right, AFAICT. So I''m gathering >> > that the "quirks" lists areas of PCI configuration space which it is >> > safe for pciback to allow the guest to modify. (I''m going to hack >> > libxl to set "permissive" to test this theory.) >> >> I think that based on what Keir says we should just expose the >> permissive setting in xl/libxl for the time being, that''s a far simpler >> patch. No need for the full power of per-register whitelisting > > Looks like this is just a case of writing the BDF of a device to a sysfs > node or booting with xen-pciback.permissive=1 to turn it on globally. > Possibly this could be "fixed" purely by documentation? I''ve added it to > the TODO as a nice to have with George''s name next to it, hope that''s > OK.It seems like having xl/libxl capable of doing the sysfs writing is a better thing, and shouldn''t be too hard really. I''ll try to hack that up this week. Going forward, it would be nice if xl could do hot unplug and pciback binding, so people wouldn''t have to muck about with rmmod and manual sysfs twiddling; but that will have to be a 4.3 thing I guess. -George> > Ian. > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Tue, Mar 27, 2012 at 2:34 AM, Ian Campbell <Ian.Campbell@citrix.com>wrote:> A day late due to the document day yesterday. > > Plan for a 4.2 release: > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > > The time line is as follows: > > 19 March -- TODO list locked down > << WE ARE HERE > 2 April -- Feature Freeze << THIS IS NEXT WEEK > Mid/Late April -- First release candidate > Weekly -- RCN+1 until it is ready > * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches reposted recently, was blocked behind qemu > save restore patches which are now in) >I reposted the patches but I dont see any of them in the staging tree (nor do I see stefano''s patches).> * xl compatibility with xm: > * xl support for autospawning vncviewer (vncviewer=1 or > otherwise) (Goncalo Gomes, patches posted) > * support for vif "rate" parameter (Mathieu Gagné, patches > posted) > * expose PCI back "permissive" parameter (George Dunlap) > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
The time line is as follows:
19 March -- TODO list locked down
2 April -- Feature Freeze << WE ARE HERE
Mid/Late April -- First release candidate
Weekly -- RCN+1 until it is ready
The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.
We have now entered Feature Freeze. Patches which have been posted
before or which address something on the list below are still acceptable
(for now, we will gradually be getting stricter about this), everything
else will be deferred until 4.3 by default.
hypervisor, blockers:
* NONE?
tools, blockers:
* libxl stable API -- we would like 4.2 to define a stable API
which downstream's can start to rely on not changing. Aspects of
this are:
* Safe fork vs. fd handling hooks. Involves API changes
(Ian J, patches posted)
* locking/serialization, e.g., for domain creation. As of
now, nothing is provided for this purpose, and
downstream toolstacks have to put their own mechanisms
in place. E.g., xl uses a fcntl() lock around the whole
process of domain creation. It should OTOH be libxl job
to offer a proper set of hooks --placed at the proper
spots during the domain creation process-- for the
downstreams to fill with the proper callbacks. (Dario
Faggioli)
* agree & document compatibility guarantees + associated
technical measures (Ian C, patches posted)
* xl compatibility with xm:
* feature parity wrt driver domain support (George Dunlap)
* xl support for "rtc_timeoffset" and
"localtime" (Lin
Ming, Patches posted)
* More formally deprecate xm/xend. Manpage patches already in
tree. Needs release noting and communication around -rc1 to
remind people to test xl.
* Domain 0 block attach & general hotplug when using qdisk backend
(need to start qemu as necessary etc) (Stefano S)
* file:// backend performance. qemu-xen-tradition's qdisk is quite
slow & blktap2 not available in upstream kernels. Need to
consider our options:
* qemu-xen's qdisk is thought to be well performing but
qemu-xen is not yet the default. Complexity arising from
splitting qemu-for-qdisk out from qemu-for-dm and
running N qemu's.
* potentially fully userspace blktap could be ready for
4.2
* use /dev/loop+blkback. This requires loop driver AIO and
O_DIRECT patches which are not (AFAIK) yet upstream.
* Leverage XCP's blktap2 DKMS work.
* Other ideas?
* In general we should recommend blkback (e.g.
with LVM) since it always out performs other
solutions, although at the expense of
flexibility regarding image formats.
Stefano has done a lot of work here and has proposed some
performance improvements for qdisk in both qemu-xen and
qemu-xen-traditional which make them a plausible alternative for
Xen 4.2.
* Improved Hotplug script support (Roger Pau Monné, patches
posted)
* Block script support -- follows on from hotplug script (Roger
Pau Monné)
hypervisor, nice to have:
* solid implementation of sharing/paging/mem-events (using work
queues) (Tim Deegan, Olaf Herring et al -- patches posted)
* "The last patch to use a waitqueue in
__get_gfn_type_access() from Tim works. However, there
are a few users who call __get_gfn_type_access with the
domain_lock held. This part needs to be addressed in
some way."
* Sharing support for AMD (Tim, Andres).
* PoD performance improvements (George Dunlap)
tools, nice to have:
* Configure/control paging via xl/libxl (Olaf Herring, lots of
discussion around interface, general consensus reached on what
it should look like. Olaf has let me know that he is probably
not going to have time to do this for 4.2, will likely slip to
4.3 unless someone else want to pick up that work)
* Upstream qemu feature patches:
* Upstream qemu PCI passthrough support (Anthony Perard,
patches sent)
* Upstream qemu save restore (Anthony Perard, Stefano
Stabellini, qemu patches applied, waiting for libxl etc
side which has been recently reposted)
* Nested-virtualisation. Currently "experimental". Likely to
release that way.
* Nested SVM. Tested in a variety of configurations but
still some issues with the most important use case (w7
XP mode) [0] (Christoph Egger)
* Nested VMX. Needs nested EPT to be genuinely useful.
Need more data on testedness etc (Intel)
* Initial xl support for Remus (memory checkpoint, blackholing)
(Shriram, patches reposted recently, was blocked behind qemu
save restore patches which are now in)
* xl compatibility with xm:
* xl support for autospawning vncviewer (vncviewer=1 or
otherwise) (Goncalo Gomes, patches posted)
* support for vif "rate" parameter (Mathieu Gagné,
patches
posted)
* expose PCI back "permissive" parameter (George Dunlap)
[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel
On 02/04/12 11:26, Ian Campbell wrote:> > We have now entered Feature Freeze. Patches which have been posted > before or which address something on the list below are still acceptable > (for now, we will gradually be getting stricter about this), everything > else will be deferred until 4.3 by default.How does this affect ARM patches? Are they similarly restricted? David
On Mon, 2012-04-02 at 11:39 +0100, David Vrabel wrote:> On 02/04/12 11:26, Ian Campbell wrote: > > > > We have now entered Feature Freeze. Patches which have been posted > > before or which address something on the list below are still acceptable > > (for now, we will gradually be getting stricter about this), everything > > else will be deferred until 4.3 by default. > > How does this affect ARM patches? Are they similarly restricted?Given that ARM support in 4.2 is going to be experimental in nature I suggest that, at least for the time being, ARM patches which do not have an impact on generic or non-ARM code continue to be accepted. Any objections? Ian.
On Mon, Apr 2, 2012 at 11:26 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> Plan for a 4.2 release: > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > > The time line is as follows: > > 19 March -- TODO list locked down > 2 April -- Feature Freeze << WE ARE HERE > Mid/Late April -- First release candidate > Weekly -- RCN+1 until it is ready > > The updated TODO list follows. Per the release plan a strong case will > now have to be made as to why new items should be added to the list, > especially as a blocker, rather than deferred to 4.3. > > We have now entered Feature Freeze. Patches which have been posted > before or which address something on the list below are still acceptable > (for now, we will gradually be getting stricter about this), everything > else will be deferred until 4.3 by default. > > hypervisor, blockers: > * NONE? > > tools, blockers: > * libxl stable API -- we would like 4.2 to define a stable API > which downstream''s can start to rely on not changing. Aspects of > this are: > * Safe fork vs. fd handling hooks. Involves API changes > (Ian J, patches posted) > * locking/serialization, e.g., for domain creation. As of > now, nothing is provided for this purpose, and > downstream toolstacks have to put their own mechanisms > in place. E.g., xl uses a fcntl() lock around the whole > process of domain creation. It should OTOH be libxl job > to offer a proper set of hooks --placed at the proper > spots during the domain creation process-- for the > downstreams to fill with the proper callbacks. (Dario > Faggioli) > * agree & document compatibility guarantees + associated > technical measures (Ian C, patches posted) > * xl compatibility with xm: > * feature parity wrt driver domain support (George Dunlap)The only thing missing is the pci "permissive" flag. Patches posted.> * xl support for "rtc_timeoffset" and "localtime" (Lin > Ming, Patches posted) > * More formally deprecate xm/xend. Manpage patches already in > tree. Needs release noting and communication around -rc1 to > remind people to test xl. > * Domain 0 block attach & general hotplug when using qdisk backend > (need to start qemu as necessary etc) (Stefano S) > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen''s qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu''s. > * potentially fully userspace blktap could be ready for > 4.2 > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream. > * Leverage XCP''s blktap2 DKMS work. > * Other ideas? > * In general we should recommend blkback (e.g. > with LVM) since it always out performs other > solutions, although at the expense of > flexibility regarding image formats. > Stefano has done a lot of work here and has proposed some > performance improvements for qdisk in both qemu-xen and > qemu-xen-traditional which make them a plausible alternative for > Xen 4.2. > * Improved Hotplug script support (Roger Pau Monné, patches > posted) > * Block script support -- follows on from hotplug script (Roger > Pau Monné) > > hypervisor, nice to have: > * solid implementation of sharing/paging/mem-events (using work > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > * "The last patch to use a waitqueue in > __get_gfn_type_access() from Tim works. However, there > are a few users who call __get_gfn_type_access with the > domain_lock held. This part needs to be addressed in > some way." > * Sharing support for AMD (Tim, Andres). > * PoD performance improvements (George Dunlap) > > tools, nice to have: > * Configure/control paging via xl/libxl (Olaf Herring, lots of > discussion around interface, general consensus reached on what > it should look like. Olaf has let me know that he is probably > not going to have time to do this for 4.2, will likely slip to > 4.3 unless someone else want to pick up that work) > * Upstream qemu feature patches: > * Upstream qemu PCI passthrough support (Anthony Perard, > patches sent) > * Upstream qemu save restore (Anthony Perard, Stefano > Stabellini, qemu patches applied, waiting for libxl etc > side which has been recently reposted) > * Nested-virtualisation. Currently "experimental". Likely to > release that way. > * Nested SVM. Tested in a variety of configurations but > still some issues with the most important use case (w7 > XP mode) [0] (Christoph Egger) > * Nested VMX. Needs nested EPT to be genuinely useful. > Need more data on testedness etc (Intel) > * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches reposted recently, was blocked behind qemu > save restore patches which are now in) > * xl compatibility with xm: > * xl support for autospawning vncviewer (vncviewer=1 or > otherwise) (Goncalo Gomes, patches posted) > * support for vif "rate" parameter (Mathieu Gagné, patches > posted) > * expose PCI back "permissive" parameter (George Dunlap) > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Mon, 2 Apr 2012, Ian Campbell wrote:> * file:// backend performance. qemu-xen-tradition''s qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen''s qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu''s. > * potentially fully userspace blktap could be ready for > 4.2 > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream. > * Leverage XCP''s blktap2 DKMS work. > * Other ideas? > * In general we should recommend blkback (e.g. > with LVM) since it always out performs other > solutions, although at the expense of > flexibility regarding image formats. > Stefano has done a lot of work here and has proposed some > performance improvements for qdisk in both qemu-xen and > qemu-xen-traditional which make them a plausible alternative for > Xen 4.2.Using O_DIRECT rather than the default write-through cache policy solves the performance problem in QEMU. I sent patches to xen-devel to enable O_DIRECT for QDISK on qemu-xen-traditional. I also sent patches to qemu-devel to enable O_DIRECT and native AIO for QDISK on upstream QEMU. Upstream QEMU PV backends are as good as the ones in qemu-xen-traditional, but upstream QDISK performs better because it can use native AIO. Thus I sent a patch to xen-devel to enable upstream QEMU as default to provide userspace backends to PV guests. I wrote and sent a patch series to fix the m2p override in Linux in case the frontend and backend are both in the same domain. Then I sent a patch series to xen-devel to make libxl__device_disk_local_attach work with QDISK: the implementation uses a frontend/backend pair in dom0. As a result, with all these patches applied, the disk performances with file based disk images are good and one can use a qcow2 file to store a PV guest disk image and boot from it using pygrub.
Plan for a 4.2 release:
http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
The time line is as follows:
19 March -- TODO list locked down
2 April -- Feature Freeze
<< WE ARE HERE
Mid/Late April -- First release candidate
Weekly -- RCN+1 until it is ready
The updated TODO list follows. Per the release plan a strong case will
now have to be made as to why new items should be added to the list,
especially as a blocker, rather than deferred to 4.3.
We have now entered Feature Freeze. Patches which have been posted
before or which address something on the list below are still acceptable
(for now, we will gradually be getting stricter about this), everything
else will be deferred until 4.3 by default.
Lots more DONE this week, still a few big ticket items or items with no
progress (that I''ve noticed) please let me know if the status of
anything has changed.
From next week I think I will start also tracking bugs on these lists.
Please let me know if you have something which you feel should be listed
here, as well as your justification for it being a blocker rather than
"nice to fix".
hypervisor, blockers:
* NONE?
tools, blockers:
* libxl stable API -- we would like 4.2 to define a stable API
which downstream''s can start to rely on not changing. Aspects
of
this are:
* Safe fork vs. fd handling hooks. Involves API changes
(Ian J, patches posted)
* locking/serialization, e.g., for domain creation. As of
now, nothing is provided for this purpose, and
downstream toolstacks have to put their own mechanisms
in place. E.g., xl uses a fcntl() lock around the whole
process of domain creation. It should OTOH be libxl job
to offer a proper set of hooks --placed at the proper
spots during the domain creation process-- for the
downstreams to fill with the proper callbacks. (Dario
Faggioli)
* Thinking to defer this until 4.3?
* agree & document compatibility guarantees + associated
technical measures (Ian C, DONE)
* xl compatibility with xm:
* feature parity wrt driver domain support (George Dunlap,
DONE)
* xl support for "rtc_timeoffset" and
"localtime" (Lin
Ming, DONE)
* More formally deprecate xm/xend. Manpage patches already in
tree. Needs release noting and communication around -rc1 to
remind people to test xl.
* Domain 0 block attach & general hotplug when using qdisk backend
(need to start qemu as necessary etc) (Stefano S, patches
posted)
* file:// backend performance. qemu-xen-tradition''s qdisk is
quite
slow & blktap2 not available in upstream kernels. Need to
consider our options:
* qemu-xen''s qdisk is thought to be well performing but
qemu-xen is not yet the default. Complexity arising from
splitting qemu-for-qdisk out from qemu-for-dm and
running N qemu''s.
* potentially fully userspace blktap could be ready for
4.2 (unlikely to be available in 4.2)
* use /dev/loop+blkback. This requires loop driver AIO and
O_DIRECT patches which are not (AFAIK) yet upstream.
* Leverage XCP''s blktap2 DKMS work. (Useful fallback for
some usecases
Stefano has done a lot of work here and has proposed some
performance improvements for qdisk in both qemu-xen and
qemu-xen-traditional which make them a plausible alternative for
Xen 4.2. This is likely the route we will now take. Need to
mention in release notes?
* Improved Hotplug script support (Roger Pau Monné, patches
posted)
* Block script support -- follows on from hotplug script (Roger
Pau Monné)
hypervisor, nice to have:
* solid implementation of sharing/paging/mem-events (using work
queues) (Tim Deegan, Olaf Herring et al -- patches posted)
* "The last patch to use a waitqueue in
__get_gfn_type_access() from Tim works. However, there
are a few users who call __get_gfn_type_access with the
domain_lock held. This part needs to be addressed in
some way."
* Sharing support for AMD (Tim, Andres).
* PoD performance improvements (George Dunlap)
tools, nice to have:
* Configure/control paging via xl/libxl (Olaf Herring, lots of
discussion around interface, general consensus reached on what
it should look like. Olaf has let me know that he is probably
not going to have time to do this for 4.2, will likely slip to
4.3 unless someone else want to pick up that work)
* Upstream qemu feature patches:
* Upstream qemu PCI passthrough support (Anthony Perard,
patches sent)
* Upstream qemu save restore (Anthony Perard, Stefano
Stabellini, qemu patches applied, waiting for libxl etc
side which has been recently reposted)
* Nested-virtualisation. Currently "experimental". Likely to
release that way.
* Nested SVM. Tested in a variety of configurations but
still some issues with the most important use case (w7
XP mode) [0] (Christoph Egger)
* Nested VMX. Needs nested EPT to be genuinely useful.
Need more data on testedness etc (Intel)
* Initial xl support for Remus (memory checkpoint, blackholing)
(Shriram, patches reposted recently, was blocked behind qemu
save restore patches which are now in)
* xl compatibility with xm:
* xl support for autospawning vncviewer (vncviewer=1 or
otherwise) (Goncalo Gomes, patches posted)
* support for vif "rate" parameter (Mathieu Gagné,
patches
posted)
* expose PCI back "permissive" parameter (George Dunlap)
[0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html
Ian Campbell writes ("Xen 4.2 Release Plan /
TODO"):> Plan for a 4.2 release:
> http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
...> tools, blockers:
> * libxl stable API -- we would like 4.2 to define a stable API
> which downstream''s can start to rely on not changing.
Aspects of
> this are:
I took a look at libxl.h and came up with the comments below. Firstly
general things I tripped over, and then the list of things which need
converting to the new event system.
Ian.
Other:
=====
] int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, uint32_t
memory_kb, int wait_secs);
] /* wait for the memory target of a domain to be reached */
] int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int
wait_secs);
This whole memory interface is a bit of a dog''s breakfast.
] int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass);
] int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num,
libxl_console_type type);
] /* libxl_primary_console_exec finds the domid and console number
] * corresponding to the primary console of the given vm, then calls
] * libxl_console_exec with the right arguments (domid might be different
] * if the guest is using stubdoms).
] * This function can be called after creating the device model, in
] * case of HVM guests, and before libxl_run_bootloader in case of PV
] * guests using pygrub. */
] int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm);
These functions should not exec things for you; they should tell you
the parameters. Any execing helpers should be in libxlu.
] /* common paths */
] const char *libxl_sbindir_path(void);
] const char *libxl_bindir_path(void);
] const char *libxl_libexec_path(void);
] const char *libxl_libdir_path(void);
] const char *libxl_sharedir_path(void);
] const char *libxl_private_bindir_path(void);
] const char *libxl_xenfirmwaredir_path(void);
] const char *libxl_xen_config_dir_path(void);
] const char *libxl_xen_script_dir_path(void);
] const char *libxl_lock_dir_path(void);
] const char *libxl_run_dir_path(void);
] const char *libxl_xenpaging_dir_path(void);
Surely these should be private ?
Need to be ao/eventified:
========================
] typedef struct {
] #define XL_SUSPEND_DEBUG 1
] #define XL_SUSPEND_LIVE 2
] int flags;
] int (*suspend_callback)(void *, int);
] } libxl_domain_suspend_info;
...
] int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_info *info,
] uint32_t domid, int fd);
] typedef int (*libxl_console_ready)(libxl_ctx *ctx, uint32_t domid, void
*priv);
] int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config,
libxl_console_ready cb, void *priv, uint32_t *domid);
] int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config
*d_config, libxl_console_ready cb, void *priv, uint32_t *domid, int restore_fd);
] int libxl_domain_shutdown(libxl_ctx *ctx, uint32_t domid);
] int libxl_domain_reboot(libxl_ctx *ctx, uint32_t domid);
Are these now merely initiations ?
] int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const char
*filename);
Might become long-running in the future.
] int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl_device_disk
*disk);
] /*
] * Insert a CD-ROM device. A device corresponding to disk must already
] * be attached to the guest.
] */
] int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk
*disk);
] /*
] * Make a disk available in this (the control) domain. Returns path to
] * a device.
] */
] char * libxl_device_disk_local_attach(libxl_ctx *ctx, libxl_device_disk
*disk);
] int libxl_device_disk_local_detach(libxl_ctx *ctx, libxl_device_disk *disk);
Does this even need to be public at this stage ?
] /* Network Interfaces */
] int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic
*nic);
] /* Keyboard */
] int libxl_device_vkb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vkb
*vkb);
] /* Framebuffer */
] int libxl_device_vfb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vfb
*vfb);
] /* PCI Passthrough */
] int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, libxl_device_pci
*pcidev);
] int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_pci
*pcidev);
] typedef struct libxl__xen_console_reader libxl_xen_console_reader;
]
] libxl_xen_console_reader *
] libxl_xen_console_read_start(libxl_ctx *ctx, int clear);
] int libxl_xen_console_read_line(libxl_ctx *ctx,
] libxl_xen_console_reader *cr,
] char **line_r);
] void libxl_xen_console_read_finish(libxl_ctx *ctx,
] libxl_xen_console_reader *cr);
] char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int use_long);
] int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid);
] int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid);
] int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid);
] int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name,
] uint32_t set);
] int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* uuid,
] int auth);
] int libxl_tmem_freeable(libxl_ctx *ctx);
Not sure about the tmem calls.
And from libxl_utils.h:
] pid_t libxl_fork(libxl_ctx *ctx);
This function is going to have to go away.
--
Ian Jackson writes ("Re: Xen 4.2 Release Plan /
TODO"):> Ian Campbell writes ("Xen 4.2 Release Plan / TODO"):
> > Plan for a 4.2 release:
> > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html
> ...
> > tools, blockers:
> > * libxl stable API -- we would like 4.2 to define a stable API
> > which downstream''s can start to rely on not changing.
Aspects of
> > this are:
>
> I took a look at libxl.h and came up with the comments below. Firstly
> general things I tripped over, and then the list of things which need
> converting to the new event system.
And I should mention that in my event series I have two
as-yet-unposted half-backed patches to rewrite libxl__spawn_* and
libxl_domain_create_*.
It may be that we can add dummy ao_hows to these functions which might
be good enough for now, although particularly for domain creation
(which includes spawning) it might complicate the efforts of users to
use the new API.
Currently any use of subprocesses inside libxl which is not dealt with
by the new event machinery will experience problems if the event loop
is also used, because the event loop will reap the children.
Ian.
On Wed, 2012-04-11 at 17:11 +0100, Ian Jackson wrote:> Ian Campbell writes ("Xen 4.2 Release Plan / TODO"): > > Plan for a 4.2 release: > > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > ... > > tools, blockers: > > * libxl stable API -- we would like 4.2 to define a stable API > > which downstream''s can start to rely on not changing. Aspects of > > this are: > > I took a look at libxl.h and came up with the comments below. Firstly > general things I tripped over, and then the list of things which need > converting to the new event system.A slightly worrying list at this stage in the game.> > Ian. > > > Other: > =====> > ] int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, uint32_t memory_kb, int wait_secs); > ] /* wait for the memory target of a domain to be reached */ > ] int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid, int wait_secs); > > This whole memory interface is a bit of a dog''s breakfast.I think we can defer this to 4.3? The existing interface may be pants but at least the name is pretty explicit that it will block. I think this should then be easy enough to sort out in a backward compatible manner in 4.3 since I expect the name of the function would change and we could retain the old name in terms of the new for compat.> ] int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass); > ] int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_type type); > ] /* libxl_primary_console_exec finds the domid and console number > ] * corresponding to the primary console of the given vm, then calls > ] * libxl_console_exec with the right arguments (domid might be different > ] * if the guest is using stubdoms). > ] * This function can be called after creating the device model, in > ] * case of HVM guests, and before libxl_run_bootloader in case of PV > ] * guests using pygrub. */ > ] int libxl_primary_console_exec(libxl_ctx *ctx, uint32_t domid_vm); > > These functions should not exec things for you; they should tell you > the parameters. Any execing helpers should be in libxlu.It''s not enough for them to just use libxl__exec? It''d be reasonably easy to make this return a libxl_string_list or similar and to write a libxlu function which takes one of those.> ] /* common paths */ > ] const char *libxl_sbindir_path(void); > ] const char *libxl_bindir_path(void); > ] const char *libxl_libexec_path(void); > ] const char *libxl_libdir_path(void); > ] const char *libxl_sharedir_path(void); > ] const char *libxl_private_bindir_path(void); > ] const char *libxl_xenfirmwaredir_path(void); > ] const char *libxl_xen_config_dir_path(void); > ] const char *libxl_xen_script_dir_path(void); > ] const char *libxl_lock_dir_path(void); > ] const char *libxl_run_dir_path(void); > ] const char *libxl_xenpaging_dir_path(void); > > Surely these should be private ?As far as I can grep, yes.> Need to be ao/eventified: > ========================> > ] typedef struct { > ] #define XL_SUSPEND_DEBUG 1 > ] #define XL_SUSPEND_LIVE 2 > ] int flags; > ] int (*suspend_callback)(void *, int); > ] } libxl_domain_suspend_info; > ... > ] int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_info *info, > ] uint32_t domid, int fd); > > ] typedef int (*libxl_console_ready)(libxl_ctx *ctx, uint32_t domid, void *priv); > ] int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_console_ready cb, void *priv, uint32_t *domid); > ] int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_config *d_config, libxl_console_ready cb, void *priv, uint32_t *domid, int restore_fd);You are on the case with these?> ] int libxl_domain_shutdown(libxl_ctx *ctx, uint32_t domid); > ] int libxl_domain_reboot(libxl_ctx *ctx, uint32_t domid); > > Are these now merely initiations ?I think so yes. Does a non-transaction write make a function "slow"? That''s all these actually do. If they are currently "fast" then we could likely get away with a dummy ao_how. (I think it is valid for a function which is "fast" to take an ao_how and always run sync?)> ] int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const char *filename); > > Might become long-running in the future.But is currently fast? Dummy ao_how?> ] int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk);Roger makes this async in his hotplug series.> ] /* > ] * Insert a CD-ROM device. A device corresponding to disk must already > ] * be attached to the guest. > ] */ > ] int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_device_disk *disk);Were you looking at this one? I know you mentioned it at one point.> ] /* > ] * Make a disk available in this (the control) domain. Returns path to > ] * a device. > ] */ > ] char * libxl_device_disk_local_attach(libxl_ctx *ctx, libxl_device_disk *disk); > ] int libxl_device_disk_local_detach(libxl_ctx *ctx, libxl_device_disk *disk); > > Does this even need to be public at this stage ?I think Stefano internalises them in his qdisk/dom0-attach series.> ] /* Network Interfaces */ > ] int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_device_nic *nic); > > ] /* Keyboard */ > ] int libxl_device_vkb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vkb *vkb); > > ] /* Framebuffer */ > ] int libxl_device_vfb_add(libxl_ctx *ctx, uint32_t domid, libxl_device_vfb *vfb);These ought to be pretty trivial conversions?> ] /* PCI Passthrough */ > ] int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, libxl_device_pci *pcidev); > ] int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid, libxl_device_pci *pcidev);I''m less confident that this one will be trivial to make async :-(> ] typedef struct libxl__xen_console_reader libxl_xen_console_reader; > ] > ] libxl_xen_console_reader * > ] libxl_xen_console_read_start(libxl_ctx *ctx, int clear); > ] int libxl_xen_console_read_line(libxl_ctx *ctx, > ] libxl_xen_console_reader *cr, > ] char **line_r); > ] void libxl_xen_console_read_finish(libxl_ctx *ctx, > ] libxl_xen_console_reader *cr);This is basically "xl dmesg". I''m not sure what interface makes sense here, really it is just dumping the current ring, so each call is "fast". I''m not sure there is a need for an event driven "new-line-in-log" callback style thing, i.e. a need to implement a "tail -f" style thing. Firstly I''m not sure that Xen actually produces an event which would allow this to be implemented without polling and secondly if you want that you can configure xenconsoled to log the hypervisor output and then tail the logfile. Perhaps this interface is OK?> ] char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int use_long); > ] int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid); > ] int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid); > ] int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid); > ] int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name, > ] uint32_t set); > ] int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* uuid, > ] int auth); > ] int libxl_tmem_freeable(libxl_ctx *ctx); > > Not sure about the tmem calls.Me neither.> And from libxl_utils.h: > > ] pid_t libxl_fork(libxl_ctx *ctx); > > This function is going to have to go away.Great. Maybe things aren''t as bad as I feared.
On Wed, 2012-04-11 at 17:13 +0100, Ian Jackson wrote:> Ian Jackson writes ("Re: Xen 4.2 Release Plan / TODO"): > > Ian Campbell writes ("Xen 4.2 Release Plan / TODO"): > > > Plan for a 4.2 release: > > > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > > ... > > > tools, blockers: > > > * libxl stable API -- we would like 4.2 to define a stable API > > > which downstream''s can start to rely on not changing. Aspects of > > > this are: > > > > I took a look at libxl.h and came up with the comments below. Firstly > > general things I tripped over, and then the list of things which need > > converting to the new event system. > > And I should mention that in my event series I have two > as-yet-unposted half-backed patches to rewrite libxl__spawn_* and > libxl_domain_create_*. > > It may be that we can add dummy ao_hows to these functions which might > be good enough for now, although particularly for domain creation > (which includes spawning) it might complicate the efforts of users to > use the new API.How close is your half baked series to do it properly?> Currently any use of subprocesses inside libxl which is not dealt with > by the new event machinery will experience problems if the event loop > is also used, because the event loop will reap the children.You''ve covered the bootloader one in existing patches and mentioned the libxl_$CONSOLE_exec style ones in your last mail. The only other one is the DM fork which is covered by your rewrite of libxl__spawn? Ian.
On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote:> > > ] char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int > use_long); > > ] int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid); > > ] int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid); > > ] int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid); > > ] int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name, > > ] uint32_t set); > > ] int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* > uuid, > > ] int auth); > > ] int libxl_tmem_freeable(libxl_ctx *ctx); > > > > Not sure about the tmem calls. > > Me neither.Dan, We want to declare the libxl 4.2 API as "stable" so we are trying to determine whether any of these functions need to be made potentially asynchronous or not, i.e. if they may be "slow" per the definition under the comment "Machinery for asynchronous operations ("ao")" in tools/libxl/libxl_internal.h, effectively if they may block for extended periods. If they were then we would possibly want to change the API to take an "ao_how" as described under "Some libxl operations can take a long time" in tools/libxl/libxl.h If they are "fast" today but could potentially be slow in the future then we may be able to make the trivial API change but keep the synchronous implementation (depending on the specifics). It''s quite late in the day so if the functions are "slow" then this would be the preferred option at this stage. Otherwise the alternative is that we have to maintain these interfaces going forward (for compat) and perhaps be forced introduce a new parallel async interface in the future. Annoying but not the end of the world. Ian.
On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote:> > ] /* common paths */ > > ] const char *libxl_sbindir_path(void); > > ] const char *libxl_bindir_path(void); > > ] const char *libxl_libexec_path(void); > > ] const char *libxl_libdir_path(void); > > ] const char *libxl_sharedir_path(void); > > ] const char *libxl_private_bindir_path(void); > > ] const char *libxl_xenfirmwaredir_path(void); > > ] const char *libxl_xen_config_dir_path(void); > > ] const char *libxl_xen_script_dir_path(void); > > ] const char *libxl_lock_dir_path(void); > > ] const char *libxl_run_dir_path(void); > > ] const char *libxl_xenpaging_dir_path(void); > > > > Surely these should be private ? > > As far as I can grep, yes.All but two it turns out. Not sure about those. The following patch moves the rest. libxl_lock_dir_path seems like it ought to be part of xl not libxl, or at a stretch libxlu. config_dir_path is only actually used by xl but an argument could be made that it is more generally useful? 8<----------------------------------------------- # HG changeset patch # User Ian Campbell <ian.campbell@citrix.com> # Date 1334218378 -3600 # Node ID 14cac8170e122115ef090cb390775b8c356ae643 # Parent 86b4ff3e201dc81c76ac91f8a9f134669294b3ba libxl: make most libxl_FOO_path() functions internal. Only libxl_xen_config_dir_path and libxl_lock_dir_path are used outside the library. Also bindir, sbindir, sharedir and xenpagingdir appeared to be completely unused so nuke them. Signed-off-by: Ian Campbell <ian.campbell@citrix.com> diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl.c --- a/tools/libxl/libxl.c Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl.c Thu Apr 12 09:12:58 2012 +0100 @@ -1126,7 +1126,7 @@ out: int libxl_console_exec(libxl_ctx *ctx, uint32_t domid, int cons_num, libxl_console_type type) { GC_INIT(ctx); - char *p = libxl__sprintf(gc, "%s/xenconsole", libxl_private_bindir_path()); + char *p = libxl__sprintf(gc, "%s/xenconsole", libxl__private_bindir_path()); char *domid_s = libxl__sprintf(gc, "%d", domid); char *cons_num_s = libxl__sprintf(gc, "%d", cons_num); char *cons_type_s; @@ -1767,7 +1767,7 @@ int libxl__device_nic_setdefault(libxl__ if (!nic->bridge) return ERROR_NOMEM; } if ( !nic->script && asprintf(&nic->script, "%s/vif-bridge", - libxl_xen_script_dir_path()) < 0 ) + libxl__xen_script_dir_path()) < 0 ) return ERROR_FAIL; if (!nic->nictype) nic->nictype = LIBXL_NIC_TYPE_IOEMU; @@ -1837,7 +1837,7 @@ int libxl_device_nic_add(libxl_ctx *ctx, flexarray_append(back, "script"); flexarray_append(back, nic->script[0]==''/'' ? nic->script : libxl__sprintf(gc, "%s/%s", - libxl_xen_script_dir_path(), + libxl__xen_script_dir_path(), nic->script)); } diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl.h --- a/tools/libxl/libxl.h Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl.h Thu Apr 12 09:12:58 2012 +0100 @@ -787,18 +787,8 @@ int libxl_flask_setenforce(libxl_ctx *ct int libxl_flask_loadpolicy(libxl_ctx *ctx, void *policy, uint32_t size); /* common paths */ -const char *libxl_sbindir_path(void); -const char *libxl_bindir_path(void); -const char *libxl_libexec_path(void); -const char *libxl_libdir_path(void); -const char *libxl_sharedir_path(void); -const char *libxl_private_bindir_path(void); -const char *libxl_xenfirmwaredir_path(void); const char *libxl_xen_config_dir_path(void); -const char *libxl_xen_script_dir_path(void); const char *libxl_lock_dir_path(void); -const char *libxl_run_dir_path(void); -const char *libxl_xenpaging_dir_path(void); /* misc */ diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_dm.c --- a/tools/libxl/libxl_dm.c Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl_dm.c Thu Apr 12 09:12:58 2012 +0100 @@ -24,7 +24,7 @@ static const char *libxl_tapif_script(li #ifdef __linux__ return libxl__strdup(gc, "no"); #else - return libxl__sprintf(gc, "%s/qemu-ifup", libxl_xen_script_dir_path()); + return libxl__sprintf(gc, "%s/qemu-ifup", libxl__xen_script_dir_path()); #endif } @@ -47,10 +47,10 @@ const char *libxl__domain_device_model(l } else { switch (info->device_model_version) { case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN_TRADITIONAL: - dm = libxl__abs_path(gc, "qemu-dm", libxl_libexec_path()); + dm = libxl__abs_path(gc, "qemu-dm", libxl__libexec_path()); break; case LIBXL_DEVICE_MODEL_VERSION_QEMU_XEN: - dm = libxl__abs_path(gc, "qemu-system-i386", libxl_libexec_path()); + dm = libxl__abs_path(gc, "qemu-system-i386", libxl__libexec_path()); break; default: LIBXL__LOG(ctx, LIBXL__LOG_ERROR, @@ -337,7 +337,7 @@ static char ** libxl__build_device_model flexarray_append(dm_args, libxl__sprintf(gc, "socket,id=libxl-cmd," "path=%s/qmp-libxl-%d,server,nowait", - libxl_run_dir_path(), guest_domid)); + libxl__run_dir_path(), guest_domid)); flexarray_append(dm_args, "-mon"); flexarray_append(dm_args, "chardev=libxl-cmd,mode=control"); @@ -708,7 +708,7 @@ static int libxl__create_stubdom(libxl__ dm_config.b_info.target_memkb = dm_config.b_info.max_memkb; dm_config.b_info.u.pv.kernel.path = libxl__abs_path(gc, "ioemu-stubdom.gz", - libxl_xenfirmwaredir_path()); + libxl__xenfirmwaredir_path()); dm_config.b_info.u.pv.cmdline = libxl__sprintf(gc, " -d %d", guest_domid); dm_config.b_info.u.pv.ramdisk.path = ""; dm_config.b_info.u.pv.features = ""; diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_dom.c --- a/tools/libxl/libxl_dom.c Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl_dom.c Thu Apr 12 09:12:58 2012 +0100 @@ -349,7 +349,7 @@ static const char *libxl__domain_firmwar break; } } - return libxl__abs_path(gc, firmware, libxl_xenfirmwaredir_path()); + return libxl__abs_path(gc, firmware, libxl__xenfirmwaredir_path()); } int libxl__build_hvm(libxl__gc *gc, uint32_t domid, diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_internal.h --- a/tools/libxl/libxl_internal.h Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl_internal.h Thu Apr 12 09:12:58 2012 +0100 @@ -1374,6 +1374,14 @@ int libxl__carefd_close(libxl__carefd*); /* You may pass NULL in which case the answer is -1. */ int libxl__carefd_fd(const libxl__carefd*); +/* common paths */ +_hidden const char *libxl__libexec_path(void); +_hidden const char *libxl__private_bindir_path(void); +_hidden const char *libxl__xenfirmwaredir_path(void); +_hidden const char *libxl__xen_config_dir_path(void); +_hidden const char *libxl__xen_script_dir_path(void); +_hidden const char *libxl__lock_dir_path(void); +_hidden const char *libxl__run_dir_path(void); /* * Convenience macros. diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_paths.c --- a/tools/libxl/libxl_paths.c Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl_paths.c Thu Apr 12 09:12:58 2012 +0100 @@ -15,37 +15,17 @@ #include "libxl_osdeps.h" /* must come before any other headers */ #include "libxl_internal.h" -const char *libxl_sbindir_path(void) -{ - return SBINDIR; -} - -const char *libxl_bindir_path(void) -{ - return BINDIR; -} - -const char *libxl_libexec_path(void) +const char *libxl__libexec_path(void) { return LIBEXEC; } -const char *libxl_libdir_path(void) -{ - return LIBDIR; -} - -const char *libxl_sharedir_path(void) -{ - return SHAREDIR; -} - -const char *libxl_private_bindir_path(void) +const char *libxl__private_bindir_path(void) { return PRIVATE_BINDIR; } -const char *libxl_xenfirmwaredir_path(void) +const char *libxl__xenfirmwaredir_path(void) { return XENFIRMWAREDIR; } @@ -55,7 +35,7 @@ const char *libxl_xen_config_dir_path(vo return XEN_CONFIG_DIR; } -const char *libxl_xen_script_dir_path(void) +const char *libxl__xen_script_dir_path(void) { return XEN_SCRIPT_DIR; } @@ -65,16 +45,11 @@ const char *libxl_lock_dir_path(void) return XEN_LOCK_DIR; } -const char *libxl_run_dir_path(void) +const char *libxl__run_dir_path(void) { return XEN_RUN_DIR; } -const char *libxl_xenpaging_dir_path(void) -{ - return XEN_PAGING_DIR; -} - /* * Local variables: * mode: C diff -r 86b4ff3e201d -r 14cac8170e12 tools/libxl/libxl_qmp.c --- a/tools/libxl/libxl_qmp.c Thu Apr 12 09:02:05 2012 +0100 +++ b/tools/libxl/libxl_qmp.c Thu Apr 12 09:12:58 2012 +0100 @@ -633,7 +633,7 @@ libxl__qmp_handler *libxl__qmp_initializ qmp = qmp_init_handler(gc, domid); qmp_socket = libxl__sprintf(gc, "%s/qmp-libxl-%d", - libxl_run_dir_path(), domid); + libxl__run_dir_path(), domid); if ((ret = qmp_open(qmp, qmp_socket, QMP_SOCKET_CONNECT_TIMEOUT)) < 0) { LIBXL__LOG_ERRNO(qmp->ctx, LIBXL__LOG_ERROR, "Connection error"); qmp_free_handler(qmp); @@ -671,7 +671,7 @@ void libxl__qmp_cleanup(libxl__gc *gc, u char *qmp_socket; qmp_socket = libxl__sprintf(gc, "%s/qmp-libxl-%d", - libxl_run_dir_path(), domid); + libxl__run_dir_path(), domid); if (unlink(qmp_socket) == -1) { if (errno != ENOENT) { LIBXL__LOG_ERRNO(ctx, LIBXL__LOG_ERROR,
On Tue, Apr 10, 2012 at 11:24 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> Plan for a 4.2 release: > http://lists.xen.org/archives/html/xen-devel/2012-03/msg00793.html > > The time line is as follows: > > 19 March -- TODO list locked down > 2 April -- Feature Freeze > << WE ARE HERE > Mid/Late April -- First release candidate > Weekly -- RCN+1 until it is ready > > The updated TODO list follows. Per the release plan a strong case will > now have to be made as to why new items should be added to the list, > especially as a blocker, rather than deferred to 4.3. > > We have now entered Feature Freeze. Patches which have been posted > before or which address something on the list below are still acceptable > (for now, we will gradually be getting stricter about this), everything > else will be deferred until 4.3 by default. > > Lots more DONE this week, still a few big ticket items or items with no > progress (that I''ve noticed) please let me know if the status of > anything has changed. > > From next week I think I will start also tracking bugs on these lists. > Please let me know if you have something which you feel should be listed > here, as well as your justification for it being a blocker rather than > "nice to fix".Here are bugs I''ve found: * Zombie driver domains. There''s a bug where PV guests with devices passed through with xl hang around as "zombies", with one outstanding page reference. I think this should really be a blocker. I''m working on this at the moment. * Manually ballooning dom0. xl mem-set 0 [foo] fails with "libxl: error: libxl.c:2569:libxl_set_memory_target: cannot get memory info from /local/domain/0/memory/static-max: No such file or directory". I''d make this a blocker just because it should be easy to fix and it''s pretty embarassing. This might be suitable for an enthusiastic on-looker. Also, since there have been other situations / reports of zombie domains, it would be good if we could get a zombie domain detector into the testing system to see how widespread they are. -George> > hypervisor, blockers: > * NONE? > > tools, blockers: > * libxl stable API -- we would like 4.2 to define a stable API > which downstream''s can start to rely on not changing. Aspects of > this are: > * Safe fork vs. fd handling hooks. Involves API changes > (Ian J, patches posted) > * locking/serialization, e.g., for domain creation. As of > now, nothing is provided for this purpose, and > downstream toolstacks have to put their own mechanisms > in place. E.g., xl uses a fcntl() lock around the whole > process of domain creation. It should OTOH be libxl job > to offer a proper set of hooks --placed at the proper > spots during the domain creation process-- for the > downstreams to fill with the proper callbacks. (Dario > Faggioli) > * Thinking to defer this until 4.3? > * agree & document compatibility guarantees + associated > technical measures (Ian C, DONE) > * xl compatibility with xm: > * feature parity wrt driver domain support (George Dunlap, > DONE) > * xl support for "rtc_timeoffset" and "localtime" (Lin > Ming, DONE) > * More formally deprecate xm/xend. Manpage patches already in > tree. Needs release noting and communication around -rc1 to > remind people to test xl. > * Domain 0 block attach & general hotplug when using qdisk backend > (need to start qemu as necessary etc) (Stefano S, patches > posted) > * file:// backend performance. qemu-xen-tradition''s qdisk is quite > slow & blktap2 not available in upstream kernels. Need to > consider our options: > * qemu-xen''s qdisk is thought to be well performing but > qemu-xen is not yet the default. Complexity arising from > splitting qemu-for-qdisk out from qemu-for-dm and > running N qemu''s. > * potentially fully userspace blktap could be ready for > 4.2 (unlikely to be available in 4.2) > * use /dev/loop+blkback. This requires loop driver AIO and > O_DIRECT patches which are not (AFAIK) yet upstream. > * Leverage XCP''s blktap2 DKMS work. (Useful fallback for > some usecases > Stefano has done a lot of work here and has proposed some > performance improvements for qdisk in both qemu-xen and > qemu-xen-traditional which make them a plausible alternative for > Xen 4.2. This is likely the route we will now take. Need to > mention in release notes? > * Improved Hotplug script support (Roger Pau Monné, patches > posted) > * Block script support -- follows on from hotplug script (Roger > Pau Monné) > > hypervisor, nice to have: > * solid implementation of sharing/paging/mem-events (using work > queues) (Tim Deegan, Olaf Herring et al -- patches posted) > * "The last patch to use a waitqueue in > __get_gfn_type_access() from Tim works. However, there > are a few users who call __get_gfn_type_access with the > domain_lock held. This part needs to be addressed in > some way." > * Sharing support for AMD (Tim, Andres). > * PoD performance improvements (George Dunlap) > > tools, nice to have: > * Configure/control paging via xl/libxl (Olaf Herring, lots of > discussion around interface, general consensus reached on what > it should look like. Olaf has let me know that he is probably > not going to have time to do this for 4.2, will likely slip to > 4.3 unless someone else want to pick up that work) > * Upstream qemu feature patches: > * Upstream qemu PCI passthrough support (Anthony Perard, > patches sent) > * Upstream qemu save restore (Anthony Perard, Stefano > Stabellini, qemu patches applied, waiting for libxl etc > side which has been recently reposted) > * Nested-virtualisation. Currently "experimental". Likely to > release that way. > * Nested SVM. Tested in a variety of configurations but > still some issues with the most important use case (w7 > XP mode) [0] (Christoph Egger) > * Nested VMX. Needs nested EPT to be genuinely useful. > Need more data on testedness etc (Intel) > * Initial xl support for Remus (memory checkpoint, blackholing) > (Shriram, patches reposted recently, was blocked behind qemu > save restore patches which are now in) > * xl compatibility with xm: > * xl support for autospawning vncviewer (vncviewer=1 or > otherwise) (Goncalo Gomes, patches posted) > * support for vif "rate" parameter (Mathieu Gagné, patches > posted) > * expose PCI back "permissive" parameter (George Dunlap) > > [0] http://lists.xen.org/archives/html/xen-devel/2012-03/msg00883.html > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Thu, 2012-04-12 at 11:24 +0100, Dario Faggioli wrote:> On Tue, 2012-04-10 at 10:24 +0000, Ian Campbell wrote: > > Lots more DONE this week, still a few big ticket items or items with no > > progress (that I''ve noticed) please let me know if the status of > > anything has changed. > > > Here I am. :-) > > > tools, blockers: > > * libxl stable API -- we would like 4.2 to define a stable API > > which downstream''s can start to rely on not changing. Aspects of > > this are: > > * Safe fork vs. fd handling hooks. Involves API changes > > (Ian J, patches posted) > > * locking/serialization, e.g., for domain creation. As of > > now, nothing is provided for this purpose, and > > downstream toolstacks have to put their own mechanisms > > in place. E.g., xl uses a fcntl() lock around the whole > > process of domain creation. It should OTOH be libxl job > > to offer a proper set of hooks --placed at the proper > > spots during the domain creation process-- for the > > downstreams to fill with the proper callbacks. (Dario > > Faggioli) > > * Thinking to defer this until 4.3? > > > Here''s the point. This was raised for the following main reasons: > 1. xl holds a lock during domain creation for way too long > 2. checking for free memory in xl when it is Xen that will actually > allocate it after a while is prone to races (the classical TOCTTOU > condition) > > We thought both these things to be addressable by messing around with > locks, but a more careful analysis revealed we can''t solve 2. in a sane > enough way by just doing that (i.e., messing with lock, moving it > around, etc.). > > That being the case, yes, I think we can move this forward toward 4.3 > without much problems, and I propose doing so. Thoughts?I think this is OK, it will effectively be a new API in 4.3 so it should be reasonably easy to do in a compatible manner. Ian.
Ian Campbell writes ("Re: Xen 4.2 Release Plan /
TODO"):> On Wed, 2012-04-11 at 17:13 +0100, Ian Jackson wrote:
> > It may be that we can add dummy ao_hows to these functions which might
> > be good enough for now, although particularly for domain creation
> > (which includes spawning) it might complicate the efforts of users to
> > use the new API.
>
> How close is your half baked series to do it properly?
Another weeks or two maybe, if I don''t get too badly sucked into doing
anything else.
> > Currently any use of subprocesses inside libxl which is not dealt with
> > by the new event machinery will experience problems if the event loop
> > is also used, because the event loop will reap the children.
>
> You''ve covered the bootloader one in existing patches and
mentioned the
> libxl_$CONSOLE_exec style ones in your last mail. The only other one is
> the DM fork which is covered by your rewrite of libxl__spawn?
Yes, I think so. That''s why I''ve been targeting those.
Ian Campbell writes ("Re: Xen 4.2 Release Plan /
TODO"):> On Wed, 2012-04-11 at 17:11 +0100, Ian Jackson wrote:
> > I took a look at libxl.h and came up with the comments below. Firstly
> > general things I tripped over, and then the list of things which need
> > converting to the new event system.
>
> A slightly worrying list at this stage in the game.
Yes.
> > Other:
> > =====> >
> > ] int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid,
> > ] /* wait for the memory target of a domain to be reached */
> > ] int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid
> >
> > This whole memory interface is a bit of a dog''s breakfast.
>
> I think we can defer this to 4.3? The existing interface may be pants
> but at least the name is pretty explicit that it will block. I think
> this should then be easy enough to sort out in a backward compatible
> manner in 4.3 since I expect the name of the function would change and
> we could retain the old name in terms of the new for compat.
The problem isn''t just that it blocks but also that the semantics are
wrong. This is all related to the problems of allocating memory. We
had a recent conversation where we concluded that we needed hypervisor
changes to do memory assignment in a race-free way. This is not 4.3
material.
I suggest that the right answer is to make a note that this interface
is considered substandard and not to rely on it too heavily; we expect
to replace it in the future and at that point we will provide an
emulation which we intend will works by and large well enough for
existing callers.
> > ] int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int
autopass);
...> > These functions should not exec things for you; they should tell you
> > the parameters. Any execing helpers should be in libxlu.
>
> It''s not enough for them to just use libxl__exec?
>
> It''d be reasonably easy to make this return a libxl_string_list or
> similar and to write a libxlu function which takes one of those.
But what if your vnc viewer doesn''t have the command line options
these functions expect ? libxl_vncviewer_* should give you an idl
struct containing the ip address (or perhaps the domain name), port
number, and password.
The command lines should be built in xlu.
> > Need to be ao/eventified:
> > ========================...
> > ] typedef int (*libxl_console_ready)(libxl_ctx *ctx, uint32_t domi
> > ] int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config
> > ] int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_con
>
> You are on the case with these?
Yes for the creation.
> > ] typedef struct {
> > ] #define XL_SUSPEND_DEBUG 1
> > ] #define XL_SUSPEND_LIVE 2
> > ] int flags;
> > ] int (*suspend_callback)(void *, int);
> > ] } libxl_domain_suspend_info;
> > ...
> > ] int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_i
> > ] uint32_t domid, int fd);
No for these, which I haven''t looked at at all.
> > ] int libxl_domain_shutdown(libxl_ctx *ctx, uint32_t domid);
> > ] int libxl_domain_reboot(libxl_ctx *ctx, uint32_t domid);
> >
> > Are these now merely initiations ?
>
> I think so yes.
>
> Does a non-transaction write make a function "slow"?
That''s all these
> actually do. If they are currently "fast" then we could likely
get away
> with a dummy ao_how. (I think it is valid for a function which is
"fast"
> to take an ao_how and always run sync?)
All xenstore operations apart from waiting for watches are fast, even
transactional read/modify/writes. So these are OK.
> > ] int libxl_domain_core_dump(libxl_ctx *ctx, uint32_t domid, const
char *filename);
> >
> > Might become long-running in the future.
>
> But is currently fast? Dummy ao_how?
That''s probably correct, yes.
> > ] int libxl_device_disk_add(libxl_ctx *ctx, uint32_t domid, libxl
>
> Roger makes this async in his hotplug series.
Yes.
> > ] /*
> > ] * Insert a CD-ROM device. A device corresponding to disk must
already
> > ] * be attached to the guest.
> > ] */
> > ] int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_de
>
> Were you looking at this one? I know you mentioned it at one point.
No, I haven''t, but it shouldn''t be too hard.
> > ] /*
> > ] * Make a disk available in this (the control) domain. Returns
path to
> > ] * a device.
> > ] */
> > ] char * libxl_device_disk_local_attach(libxl_ctx *ctx, libxl_dev
> > ] int libxl_device_disk_local_detach(libxl_ctx *ctx, libxl_device
> >
> > Does this even need to be public at this stage ?
>
> I think Stefano internalises them in his qdisk/dom0-attach series.
Oh good.
> > ] /* Network Interfaces */
> > ] int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_
> >
> > ] /* Keyboard */
> > ] int libxl_device_vkb_add(libxl_ctx *ctx, uint32_t domid, libxl_
> >
> > ] /* Framebuffer */
> > ] int libxl_device_vfb_add(libxl_ctx *ctx, uint32_t domid, libxl_
>
> These ought to be pretty trivial conversions?
Yes. For the NICs I expect Roger will end up adding an ao_how for
hotplug script invocation.
> > ] /* PCI Passthrough */
> > ] int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, libxl_de
> > ] int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid, libxl
>
> I''m less confident that this one will be trivial to make async :-(
It''s not trivial but it doesn''t seem to contain any
significant
difficulties. It''s all just xenstore stuff. You split something like
do_pci_remove into a setup and a callback.
> > ] typedef struct libxl__xen_console_reader libxl_xen_console_reader;
> > ]
> > ] libxl_xen_console_reader *
> > ] libxl_xen_console_read_start(libxl_ctx *ctx, int clear);
> > ] int libxl_xen_console_read_line(libxl_ctx *ctx,
> > ] libxl_xen_console_reader *cr,
> > ] char **line_r);
> > ] void libxl_xen_console_read_finish(libxl_ctx *ctx,
> > ] libxl_xen_console_reader *cr);
>
> This is basically "xl dmesg". I''m not sure what
interface makes sense
> here, really it is just dumping the current ring, so each call is
> "fast".
OK, good.
Is it possible to wait for more input ?
> I''m not sure there is a need for an event driven
"new-line-in-log"
> callback style thing, i.e. a need to implement a "tail -f" style
thing.
> Firstly I''m not sure that Xen actually produces an event which
would
> allow this to be implemented without polling and secondly if you want
> that you can configure xenconsoled to log the hypervisor output and then
> tail the logfile.
Right.
> Perhaps this interface is OK?
I think I''m sold on it being supportable.
Ian.
On Thu, 2012-04-12 at 12:10 +0100, Ian Jackson wrote:> Ian Campbell writes ("Re: Xen 4.2 Release Plan / TODO"): > > On Wed, 2012-04-11 at 17:13 +0100, Ian Jackson wrote: > > > It may be that we can add dummy ao_hows to these functions which might > > > be good enough for now, although particularly for domain creation > > > (which includes spawning) it might complicate the efforts of users to > > > use the new API. > > > > How close is your half baked series to do it properly? > > Another weeks or two maybe, if I don''t get too badly sucked into doing > anything else.OK, great.> Ian Campbell writes ("Re: Xen 4.2 Release Plan / TODO"): > > On Wed, 2012-04-11 at 17:11 +0100, Ian Jackson wrote: > > > I took a look at libxl.h and came up with the comments below. Firstly > > > general things I tripped over, and then the list of things which need > > > converting to the new event system. > > > > A slightly worrying list at this stage in the game. > > Yes. > > > > Other: > > > =====> > > > > > ] int libxl_wait_for_free_memory(libxl_ctx *ctx, uint32_t domid, > > > ] /* wait for the memory target of a domain to be reached */ > > > ] int libxl_wait_for_memory_target(libxl_ctx *ctx, uint32_t domid > > > > > > This whole memory interface is a bit of a dog''s breakfast. > > > > I think we can defer this to 4.3? The existing interface may be pants > > but at least the name is pretty explicit that it will block. I think > > this should then be easy enough to sort out in a backward compatible > > manner in 4.3 since I expect the name of the function would change and > > we could retain the old name in terms of the new for compat. > > The problem isn''t just that it blocks but also that the semantics are > wrong. This is all related to the problems of allocating memory. We > had a recent conversation where we concluded that we needed hypervisor > changes to do memory assignment in a race-free way. This is not 4.3 > material. > > I suggest that the right answer is to make a note that this interface > is considered substandard and not to rely on it too heavily; we expect > to replace it in the future and at that point we will provide an > emulation which we intend will works by and large well enough for > existing callers.That sounds fine, do you want to propose text which you are happy with as a patch adding a comment?> > > > ] int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass); > ... > > > These functions should not exec things for you; they should tell you > > > the parameters. Any execing helpers should be in libxlu. > > > > It''s not enough for them to just use libxl__exec? > > > > It''d be reasonably easy to make this return a libxl_string_list or > > similar and to write a libxlu function which takes one of those. > > But what if your vnc viewer doesn''t have the command line options > these functions expect ? libxl_vncviewer_* should give you an idl > struct containing the ip address (or perhaps the domain name), port > number, and password. > > The command lines should be built in xlu.Hrm, this seems like 4.3 material at this stage. I''d expect that the functions which behaved this way would not be called ..._exec (perhaps ..._get_params or so) so implementing the proper interface in 4.3 won''t cause a compat issue.> > > Need to be ao/eventified: > > > ========================> ... > > > ] typedef int (*libxl_console_ready)(libxl_ctx *ctx, uint32_t domi > > > ] int libxl_domain_create_new(libxl_ctx *ctx, libxl_domain_config > > > ] int libxl_domain_create_restore(libxl_ctx *ctx, libxl_domain_con > > > > You are on the case with these? > > Yes for the creation. > > > > ] typedef struct { > > > ] #define XL_SUSPEND_DEBUG 1 > > > ] #define XL_SUSPEND_LIVE 2 > > > ] int flags; > > > ] int (*suspend_callback)(void *, int); > > > ] } libxl_domain_suspend_info; > > > ... > > > ] int libxl_domain_suspend(libxl_ctx *ctx, libxl_domain_suspend_i > > > ] uint32_t domid, int fd); > > No for these, which I haven''t looked at at all.Any volunteers for this one? I suspect that a dummy ao_how isn''t going to cut it in this case. What''s the pattern for handling long running things which actually happen in our process? I suppose libxl needs to start a thread or fork or something?> > > ] /* > > > ] * Insert a CD-ROM device. A device corresponding to disk must already > > > ] * be attached to the guest. > > > ] */ > > > ] int libxl_cdrom_insert(libxl_ctx *ctx, uint32_t domid, libxl_de > > > > Were you looking at this one? I know you mentioned it at one point. > > No, I haven''t, but it shouldn''t be too hard.I''ll leave this to you then.> > > ] /* Network Interfaces */ > > > ] int libxl_device_nic_add(libxl_ctx *ctx, uint32_t domid, libxl_ > > > > > > ] /* Keyboard */ > > > ] int libxl_device_vkb_add(libxl_ctx *ctx, uint32_t domid, libxl_ > > > > > > ] /* Framebuffer */ > > > ] int libxl_device_vfb_add(libxl_ctx *ctx, uint32_t domid, libxl_ > > > > These ought to be pretty trivial conversions? > > Yes. For the NICs I expect Roger will end up adding an ao_how for > hotplug script invocation.He also adds libxl__device_add_initiate (or so) which makes the others look pretty trivial to convert also.> > > ] /* PCI Passthrough */ > > > ] int libxl_device_pci_add(libxl_ctx *ctx, uint32_t domid, libxl_de > > > ] int libxl_device_pci_remove(libxl_ctx *ctx, uint32_t domid, libxl > > > > I''m less confident that this one will be trivial to make async :-( > > It''s not trivial but it doesn''t seem to contain any significant > difficulties. It''s all just xenstore stuff. You split something like > do_pci_remove into a setup and a callback.I''ll have a poke at this then I think.> > > ] typedef struct libxl__xen_console_reader libxl_xen_console_reader; > > > ] > > > ] libxl_xen_console_reader * > > > ] libxl_xen_console_read_start(libxl_ctx *ctx, int clear); > > > ] int libxl_xen_console_read_line(libxl_ctx *ctx, > > > ] libxl_xen_console_reader *cr, > > > ] char **line_r); > > > ] void libxl_xen_console_read_finish(libxl_ctx *ctx, > > > ] libxl_xen_console_reader *cr); > > > > This is basically "xl dmesg". I''m not sure what interface makes sense > > here, really it is just dumping the current ring, so each call is > > "fast". > > OK, good. > > Is it possible to wait for more input ?I don''t believe so.> > > I''m not sure there is a need for an event driven "new-line-in-log" > > callback style thing, i.e. a need to implement a "tail -f" style thing. > > Firstly I''m not sure that Xen actually produces an event which would > > allow this to be implemented without polling and secondly if you want > > that you can configure xenconsoled to log the hypervisor output and then > > tail the logfile. > > Right. > > > Perhaps this interface is OK? > > I think I''m sold on it being supportable.Phew! Ian.
Ian Campbell writes ("Re: Xen 4.2 Release Plan / TODO [and 1 more
messages]"):> > > > ] int libxl_domain_suspend(libxl_ctx *ctx,
libxl_domain_suspend_i
> > > > ] uint32_t domid, int fd);
> >
> > No for these, which I haven''t looked at at all.
>
> Any volunteers for this one? I suspect that a dummy ao_how isn''t
going
> to cut it in this case.
>
> What''s the pattern for handling long running things which actually
> happen in our process? I suppose libxl needs to start a thread or fork
> or something?
The intended pattern in the new libxl event universe is to use event
callbacks. So for example for outgoing migration you''d have an fd
writeable event which called into xc. But that''s not really doable in
this case because the libxc function is entirely monolithic and
synchronous.
The options for libxl are to fork, or to start a thread.
Ian.
> From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > Sent: Thursday, April 12, 2012 1:59 AM > To: Ian Jackson; Dan Magenheimer > Cc: xen-devel > Subject: Re: [Xen-devel] Xen 4.2 Release Plan / TODO > > On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote: > > > > > ] char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int > > use_long); > > > ] int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid); > > > ] int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid); > > > ] int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid); > > > ] int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name, > > > ] uint32_t set); > > > ] int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* > > uuid, > > > ] int auth); > > > ] int libxl_tmem_freeable(libxl_ctx *ctx); > > > > > > Not sure about the tmem calls. > > > > Me neither. > > Dan, > > We want to declare the libxl 4.2 API as "stable" so we are trying to > determine whether any of these functions need to be made potentially > asynchronous or not, i.e. if they may be "slow" per the definition under > the comment "Machinery for asynchronous operations ("ao")" in > tools/libxl/libxl_internal.h, effectively if they may block for extended > periods. > > If they were then we would possibly want to change the API to take an > "ao_how" as described under "Some libxl operations can take a long time" > in tools/libxl/libxl.h > > If they are "fast" today but could potentially be slow in the future > then we may be able to make the trivial API change but keep the > synchronous implementation (depending on the specifics). It''s quite late > in the day so if the functions are "slow" then this would be the > preferred option at this stage. > > Otherwise the alternative is that we have to maintain these interfaces > going forward (for compat) and perhaps be forced introduce a new > parallel async interface in the future. Annoying but not the end of the > world.Hi Ian(s) -- After reading libxl.h, I''m not absolutely positive I understand all the conditions that would cause you to label a function as "slow" but I believe all the libxl_tmem_* functions are "fast". All of them are strictly "call the hypervisor, wait for it to return" and none of the hypercalls (actually which are variations of the one tmem hypercall) require a callback to dom0 or to the calling guest... they all complete entirely inside the hypervisor. Libxl_tmem_destroy may take a long time as it has to walk through and free some potentially very large data structures, but it is only used at domain destruction. Libxl_tmem_list does allocate some memory in userland that the hypercall fills synchronously (with ascii-formatted statistics/counters maintained entirely by the tmem code in the hypervisor). If any of the above raises any alarms/concerns, let me know, else no need to asynchronizify any of the tmem functions in libxl. (Zhigang cc''ed since he''s more familiar with the libxl layer than I.) Dan
On Thu, 2012-04-12 at 17:37 +0100, Dan Magenheimer wrote:> > From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > > Sent: Thursday, April 12, 2012 1:59 AM > > To: Ian Jackson; Dan Magenheimer > > Cc: xen-devel > > Subject: Re: [Xen-devel] Xen 4.2 Release Plan / TODO > > > > On Thu, 2012-04-12 at 08:35 +0100, Ian Campbell wrote: > > > > > > > ] char *libxl_tmem_list(libxl_ctx *ctx, uint32_t domid, int > > > use_long); > > > > ] int libxl_tmem_freeze(libxl_ctx *ctx, uint32_t domid); > > > > ] int libxl_tmem_destroy(libxl_ctx *ctx, uint32_t domid); > > > > ] int libxl_tmem_thaw(libxl_ctx *ctx, uint32_t domid); > > > > ] int libxl_tmem_set(libxl_ctx *ctx, uint32_t domid, char* name, > > > > ] uint32_t set); > > > > ] int libxl_tmem_shared_auth(libxl_ctx *ctx, uint32_t domid, char* > > > uuid, > > > > ] int auth); > > > > ] int libxl_tmem_freeable(libxl_ctx *ctx); > > > > > > > > Not sure about the tmem calls. > > > > > > Me neither. > > > > Dan, > > > > We want to declare the libxl 4.2 API as "stable" so we are trying to > > determine whether any of these functions need to be made potentially > > asynchronous or not, i.e. if they may be "slow" per the definition under > > the comment "Machinery for asynchronous operations ("ao")" in > > tools/libxl/libxl_internal.h, effectively if they may block for extended > > periods. > > > > If they were then we would possibly want to change the API to take an > > "ao_how" as described under "Some libxl operations can take a long time" > > in tools/libxl/libxl.h > > > > If they are "fast" today but could potentially be slow in the future > > then we may be able to make the trivial API change but keep the > > synchronous implementation (depending on the specifics). It''s quite late > > in the day so if the functions are "slow" then this would be the > > preferred option at this stage. > > > > Otherwise the alternative is that we have to maintain these interfaces > > going forward (for compat) and perhaps be forced introduce a new > > parallel async interface in the future. Annoying but not the end of the > > world. > > Hi Ian(s) -- > > After reading libxl.h, I''m not absolutely positive I understand > all the conditions that would cause you to label a function as > "slow" but I believe all the libxl_tmem_* functions are "fast". > All of them are strictly "call the hypervisor, wait for it to > return" and none of the hypercalls (actually which are variations of > the one tmem hypercall) require a callback to dom0 or to the > calling guest... they all complete entirely inside the hypervisor.OK, this sounds good, thanks.> Libxl_tmem_destroy may take a long time as it has to walk > through and free some potentially very large data structures, > but it is only used at domain destruction.Should libxl_tmem_destroy be a public function or should it solely be called from libxl_domain_destroy? I notice that there is an xl tmem-destroy so I presume the former? We might consider adding a dummy ao_how to this one I suppose.> Libxl_tmem_list does allocate some memory in userland that the > hypercall fills synchronously (with ascii-formatted statistics/counters > maintained entirely by the tmem code in the hypervisor). > > If any of the above raises any alarms/concerns, let me know, > else no need to asynchronizify any of the tmem functions in libxl.It all sounds fine, I more curious about tmem_destroy than worried. Ian.> > (Zhigang cc''ed since he''s more familiar with the libxl layer than I.) > > Dan
On Thu, 2012-04-12 at 22:48 +0100, Dario Faggioli wrote:> On Thu, 2012-04-12 at 12:10 +0100, Ian Jackson wrote:> > This is not 4.3 > > material. > > > Just to be sure I get it, you think we need to have the > creation-vs-ballooning potential race solved *before* 4.2 ?I initially misread it as "This is not 4.2 material", which I agreed with... Ian.
Dario Faggioli writes ("Re: [Xen-devel] Xen 4.2 Release Plan / TODO [and 1
more messages]"):> On Thu, 2012-04-12 at 12:10 +0100, Ian Jackson wrote:
> > This is not 4.3
> > material.
>
> Just to be sure I get it, you think we need to have the
> creation-vs-ballooning potential race solved *before* 4.2 ?
No, I meant the opposite. Half time I say 4.2 I mean 4.3 and vice
versa :-/.
Ian.
Dan Magenheimer writes ("RE: [Xen-devel] Xen 4.2 Release Plan /
TODO"):> After reading libxl.h, I''m not absolutely positive I understand
> all the conditions that would cause you to label a function as
> "slow" but I believe all the libxl_tmem_* functions are
"fast".
Thanks for your reply.
There are a few operations that make a function necessarily have to be
slow in the libxl api sense. These are: xenstore watches; spawning
subprocesses; anything with a timeout.
More broadly any function which is sufficiently slow that a caller
might reasonably want to initiate it, and then carry on doing
something else while the function completes. So this includes any
operation which a toolstack might want to parallelise.
> All of them are strictly "call the hypervisor, wait for it to
> return" and none of the hypercalls (actually which are variations of
> the one tmem hypercall) require a callback to dom0 or to the
> calling guest... they all complete entirely inside the hypervisor.
Right, that sounds good. I guess you also mean that this will always
be the case.
> Libxl_tmem_destroy may take a long time as it has to walk
> through and free some potentially very large data structures,
> but it is only used at domain destruction.
How long a time are we talking about ? Would it be a scalability or
performance problem if an entire host''s management toolstack had to
block, and no other management operations could be performed on any
domain for any reason, while the tmem destroy takes place ?
> Libxl_tmem_list does allocate some memory in userland that the
> hypercall fills synchronously (with ascii-formatted statistics/counters
> maintained entirely by the tmem code in the hypervisor).
Memory allocation in userland is fine. I guess we''re not talking
about megabytes here.
Ian.
> From: Ian Campbell [mailto:Ian.Campbell@citrix.com] > > > > > > Libxl_tmem_destroy may take a long time as it has to walk > > through and free some potentially very large data structures, > > but it is only used at domain destruction. > > Should libxl_tmem_destroy be a public function or should it solely be > called from libxl_domain_destroy? I notice that there is an xl > tmem-destroy so I presume the former? > > We might consider adding a dummy ao_how to this one I suppose.Bearing in mind that I haven''t poked around in this code for over 2 years, I *think* the only reason tmem_destroy needs to exist at all is if tmem is broken. The normal domain termination process destroys any persistent tmem pools and any ephemeral tmem pools will eventually "age out" as the last of the pools'' pages fall off the end of the LRU. I think the tmem_destroy functionality pre-dates the existence of tmem "freeable" memory* and was a way for a toolset to force the hypervisor to free up the hypervisor memory used by some or all ephemeral tmem pools. Once the tmem allocation/free process was directly linked into alloc_heap_pages() in the hypervisor (see call to tmem_relinquish_pages()), this forcing function was no longer needed. So, bottom line, I *think* it can be ripped out, or at least for now removed from the definition of the stable xl API/UI. The libxl.c routine libxl_tmem_destroy() could also be removed if you like, but I guess I''d prefer to leave the lower level droppings in xc.c and in the hypervisor in case I am misremembering. * http://xenbits.xensource.com/hg/xen-unstable.hg/rev/03063e309356
> From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com] > Subject: RE: [Xen-devel] Xen 4.2 Release Plan / TODO > > Dan Magenheimer writes ("RE: [Xen-devel] Xen 4.2 Release Plan / TODO"): > > After reading libxl.h, I''m not absolutely positive I understand > > all the conditions that would cause you to label a function as > > "slow" but I believe all the libxl_tmem_* functions are "fast". > > There are a few operations that make a function necessarily have to be > slow in the libxl api sense. These are: xenstore watches; spawning > subprocesses; anything with a timeout. > > More broadly any function which is sufficiently slow that a caller > might reasonably want to initiate it, and then carry on doing > something else while the function completes. So this includes any > operation which a toolstack might want to parallelise.Got it. Thanks. This is a bit clearer than the comment in libxl.h.> > All of them are strictly "call the hypervisor, wait for it to > > return" and none of the hypercalls (actually which are variations of > > the one tmem hypercall) require a callback to dom0 or to the > > calling guest... they all complete entirely inside the hypervisor. > > Right, that sounds good. I guess you also mean that this will always > be the case.Yes AFAICT.> > Libxl_tmem_destroy may take a long time as it has to walk > > through and free some potentially very large data structures, > > but it is only used at domain destruction. > > How long a time are we talking about ? Would it be a scalability or > performance problem if an entire host''s management toolstack had to > block, and no other management operations could be performed on any > domain for any reason, while the tmem destroy takes place ?See previous reply to IanC... this is moot since (I think) tmem_destroy will go away.> > Libxl_tmem_list does allocate some memory in userland that the > > hypercall fills synchronously (with ascii-formatted statistics/counters > > maintained entirely by the tmem code in the hypervisor). > > Memory allocation in userland is fine. I guess we''re not talking > about megabytes here.A reasonable bound would be on the order of 1K per tmem-enabled guest. The current code in pyxc_tmem_control enforces a 32K buffer limit. Dan
Dan Magenheimer writes ("RE: [Xen-devel] Xen 4.2 Release Plan /
TODO"):> > From: Ian Jackson [mailto:Ian.Jackson@eu.citrix.com]
...> > There are a few operations that make a function necessarily have to be
> > slow in the libxl api sense. These are: xenstore watches; spawning
> > subprocesses; anything with a timeout.
> >
> > More broadly any function which is sufficiently slow that a caller
> > might reasonably want to initiate it, and then carry on doing
> > something else while the function completes. So this includes any
> > operation which a toolstack might want to parallelise.
>
> Got it. Thanks. This is a bit clearer than the comment in libxl.h.
The internal-facing documentation, which is for people who might be
implementing libxl facilities and API designers, is in
libxl_internal.h. But the comment there is less comprehensive. I
will prepare a patch to update it.
Ian.
> > > > ] int libxl_vncviewer_exec(libxl_ctx *ctx, uint32_t domid, int autopass); > > ... > > > > These functions should not exec things for you; they should tell you > > > > the parameters. Any execing helpers should be in libxlu. > > > > > > It''s not enough for them to just use libxl__exec? > > > > > > It''d be reasonably easy to make this return a libxl_string_list or > > > similar and to write a libxlu function which takes one of those. > > > > But what if your vnc viewer doesn''t have the command line options > > these functions expect ? libxl_vncviewer_* should give you an idl > > struct containing the ip address (or perhaps the domain name), port > > number, and password. > > > > The command lines should be built in xlu. > > Hrm, this seems like 4.3 material at this stage. > > I''d expect that the functions which behaved this way would not be > called ..._exec (perhaps ..._get_params or so) so implementing the > proper interface in 4.3 won''t cause a compat issue.Just looking at this again, does this argument only really apply to vncviewer (which is a 3rd party component)? The other libxl_FOO_exec are both variations of execing xenconsole which is supplied as part of the xen install and is not really subject to 3rd party replacements (at least we can decree that they are cmdline compatible).
Ian Campbell writes ("Re: [Xen-devel] Xen 4.2 Release Plan /
TODO"):> libxl: make most libxl_FOO_path() functions internal.
Acked-by: Ian Jackson <ian.jackson@eu.citrix.com>
Committed-by: Ian Jackson <ian.jackson@eu.citrix.com>