Xen devel - Nov 2013 - Xen 4.4 development update: Code freezing point reached

This information will be mirrored on the Xen 4.4 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.4

(And I actually updated the wiki this time.)

The code "freezing point" is today; which means that starting today
non-bug fixes need a freeze exception to be included.

Remember our goal for the release:
 1. A bug-free release
 2. An awesome release
 3. An on-time release

Accepting a new feature may make Xen more awesome; but it also
introduces a risk that it will introduce more bugs.  That bug may be
found before the release (threatening #3), or it may not be found
until after the release (threatening #1).  Each freeze exception
request will attempt to balance the benefits (how awesome the
exception is) vs the risks (will it cause the release to slip, or
worse, cause a bug which goes un-noticed into the final release).

The idea is that today we will be pretty permissive, but that we will
become progressively more conservative until the first RC, which is
scheduled for 3 weeks'' time (6 December).  After that, we will only
accept bug fixes.

Bug fixes can be checked in without a freeze exception throughout the
code freeze, unless the maintianer thinks they are particularly high
risk.  In later RC''s, we may even begin rejecting bug fixes if the
broken functionality is small and the risk to other functionality is
high.

Features which are currently marked "experimental" or do not at the
moment work at all cannot be broken really; so changes to code only
used by those features should be able to get a freeze exception
easily.  (Tianocore is something which would probably fall under
this.)

Features which change or add new interfaces which will need to be
supported in a backwards-compatible way (for instance, vNUMA) will
need freeze exceptions to make sure that the interface itself has
enough time to be considered stable.

These are guidelines and principles to give you an idea where we''re
coming from; if you think there''s a good reason why making an
exception for you will help us achieve goals 1-3 above better than not
doing so, feel free to make your case.

= Timeline 
Here is our current timeline based on a 6-month release:

* Feature freeze: 18 October 2013
* Code freezing point: 18 November 2013 <== WE ARE HERE
* First RC: 6 December 2013
* Release: 21 January 2014

Last updated: 18 November 2016

== Completed  =
* Event channel scalability (FIFO event channels)

* Non-udev scripts for driver domains (non-Linux driver domains)

* Multi-vector PCI MSI (Hypervisor side)

* Improved Spice support on libxl
 - Added Spice vdagent support
 - Added Spice clipboard sharing support

* PHV domU (experimental only)

* Guest EFI booting (tianocore)

* kexec

* Testing: Xen on ARM

* Update to SeaBIOS 1.7.3.1

* pvgrub2 checked into grub upstream

== Resolved since last update =
* credit scheduler doesn''t update other fields when tslice updated from
sysctl

== Open =
* qemu-upstream not freeing pirq
 > http://www.gossamer-threads.com/lists/xen/devel/281498
 status: patches posted; latest patches need testing

* Race in PV shutdown between tool detection and shutdown watch
 > http://www.gossamer-threads.com/lists/xen/devel/282467
 > Nothing to do with ACPI
 status: Patches posted

* Supposed regression from a3513737 ("x86: allow guest to set/clear
 > MSI-X mask bit (try 2)"), as per
 > http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.

* qemu-traditional mis-parses host bus 8 as 0
 > http://bugs.xenproject.org/xen/bug/15

* xen_platform_pci=0 doesn''t work with qemu-xen
 > http://bugs.xenproject.org/xen/bug/20

* xl does not support specifying virtual function for passthrough device
 > http://bugs.xenproject.org/xen/bug/22

* xl does not handle migrate interruption gracefully
  > If you start a localhost migrate, and press "Ctrl-C" in the
middle,
  > you get two hung domains

* libxl / xl does not handle failure of remote qemu gracefully
  > Easiest way to reproduce:
  >  - set "vncunused=0" and do a local migrate
  >  - The "remote" qemu will fail because the vnc port is in use
  > The failure isn''t the problem, but everything being stuck
afterwards is

* HPET interrupt stack overflow (when using hpet_broadcast mode and MSI
capable HPETs)
  owner: andyh@citrix
  status: patches posted, undergoing review iteration.

== Backlog =
=== Testing coverage ==
* new libxl w/ previous versions of xl
 @IanJ

* Host S3 suspend
 @bguthro, @dariof

* Default [example] XSM policy
 @Stefano to ask Daniel D

* Storage driver domains
 @roger

* HVM pci passthrough
 @anthony

* PV pci passthrough
 @konrad (or @george if he gets to it first)

* Network driver domains
 @George

* Nested virt?
 @intel (chased by George)

* Fix SRIOV test (chase intel)
 @ianj

* Fix bisector to e-mail blame-worthy parties
 @ianj

* Fix xl shutdown
  @ianj

* stub domains
  @athony

* performance benchmarks
  @dario

=== Meta-items (composed of other items) ==
* Meta: PVIO NUMA improvements
 - NUMA affinity for vcpus (4.4 possible)
 - PV guest NUMA interface (4.4 possible)
 - Sensible dom0 NUMA layout
 - Toolstack pinning backend thread / virq to appropraite d0 vcpu
 - NUMA-aware ballooning

* xend still in tree (x)
 - xl list -l on a dom0-only system
 - xl list -l doesn''t contain tty console port
 - xl Alternate transport support for migration*
 - xl PVSCSI support
 - xl PVUSB support

=== Big ticket items ==
* PVH mode (w/ Linux)
  owner: mukesh@oracle, george@citrix
  status (Linux): Acked, waiting for ABI to be nailed down
  status (Xen): Initial version checked in, still nailing down interface

* Update to qemu 1.6
  owner: Anthonyper
  status: In staging, still working out a bug in the VMX code

* Live Migration Support
  owner: Jaeyong Yoo <jaeyong.yoo@samsung.com>
  status: v5 posted, looking good for code freeze

* ARM64 guest
  owner: IanC
  status: v3 posted, v4 in progress. looking good.

* SWIOTLB (kernel side thing)
  owner: Stefano
  status: Pull request sent.

* soft affinity for vcpus (was NUMA affinity for vcpus)
    owner: Dario
    status: v2 posted

* PV guest NUMA interface
    owner: Elena
    status: v3 posted

* libvirt/libxl integration (external)
 - owner: jfehlig@suse, dario@citrix
 - patches posted (should be released before 4.4)
  - migration
  - PCI pass-through
 - In progress
  - integration w/ libvirt''s lock manager
  - improved concurrency

* xl USB pass-through for HVM guests using Qemu USB emulation
  prognosis: Good if extended
  owner: George
  status: v6 patch series posted

* libxl: Spice usbredirection support for upstream qemu
 owner: fabio@M2R
 status: I''ll post new patch version shortly

* libxl: usb2 and usb3 controller support for upstream qemu
 owner: fabio@M2R
 status: patch v5 posted, tested and working, awaiting reviews

* libxl network buffering support for Remus
   @shriram
   status: patches posted
   prognosis: fair

* Disk: indirect descriptors
   owner: roger@citrix
   status: Linux side in 3.11, Xen-side patch posted

On lun, 2013-11-18 at 18:19 +0000, George Dunlap wrote:
> [..]
> = Timeline > 
> Here is our current timeline based on a 6-month release:
> 
> * Feature freeze: 18 October 2013
> * Code freezing point: 18 November 2013 <== WE ARE HERE
> * First RC: 6 December 2013
> * Release: 21 January 2014
> 
> Last updated: 18 November 2016
> [..]
> === Meta-items (composed of other items) ==> 
> * Meta: PVIO NUMA improvements
>  - NUMA affinity for vcpus (4.4 possible)
>
v3 of the series posted right now. I don'' think is that intrusive and
likely to cause bugs _but_ it introduces new a couple of libxl API
calls. I therefore think that the decision of whether or not to make
this a 4.4 thing would mostly be whether or not we think such interface
is good enough.

Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://about.me/dario.faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap
wrote:
> 
> = Timeline > 
> Here is our current timeline based on a 6-month release:
> 
> * Feature freeze: 18 October 2013
> * Code freezing point: 18 November 2013 <== WE ARE HERE
> * First RC: 6 December 2013
> * Release: 21 January 2014
> 
> Last updated: 18 November 2016
>
2016 :) 

> 
> == Backlog => 
> 
> === Big ticket items ==> 
 * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with PCI/GPU
passthrough
   http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
   Where Stefano writes:

   1) modify upstream QEMU to start the PCI hole at 0xe0000000, to match
   qemu-xen-unstable in terms of configuration and not to introduce any
   regressions. Do this for the Xen 4.3 release.

   2) for Xen 4.4 rework the two patches above and improve
   i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
   enough, it also needs to be able to resize the system memory region
   (xen.ram) to make room for the bigger pci_hole



-- Pasi

On 11/18/13 13:19, George Dunlap wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
[snip]

I realize that I did not send anything to you before this.  However:

http://lists.xen.org/archives/html/xen-devel/2013-10/msg01117.html
http://lists.xen.org/archives/html/xen-devel/2013-11/msg02352.html
http://lists.xen.org/archives/html/xen-devel/2013-11/msg02569.html

Is about a new minor feature.  Clearly I would like it in 4.4.  This is 
close to:

    xl dump-core <Domain> <filename>
    crash <filename>

But does not require the disk space, nor the delay while writing the 
file.  For me it took ~2 minutes to write out a 3G core file.
     -Don Slutz
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On 18/11/13 19:19, George Dunlap wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> 
> (And I actually updated the wiki this time.)
> 
> The code "freezing point" is today; which means that starting
today
> non-bug fixes need a freeze exception to be included.
> 
> Remember our goal for the release:
>  1. A bug-free release
>  2. An awesome release
>  3. An on-time release
> 
> Accepting a new feature may make Xen more awesome; but it also
> introduces a risk that it will introduce more bugs.  That bug may be
> found before the release (threatening #3), or it may not be found
> until after the release (threatening #1).  Each freeze exception
> request will attempt to balance the benefits (how awesome the
> exception is) vs the risks (will it cause the release to slip, or
> worse, cause a bug which goes un-noticed into the final release).
> 
> The idea is that today we will be pretty permissive, but that we will
> become progressively more conservative until the first RC, which is
> scheduled for 3 weeks'' time (6 December).  After that, we will
only
> accept bug fixes.
> 
> Bug fixes can be checked in without a freeze exception throughout the
> code freeze, unless the maintianer thinks they are particularly high
> risk.  In later RC''s, we may even begin rejecting bug fixes if the
> broken functionality is small and the risk to other functionality is
> high.
> 
> Features which are currently marked "experimental" or do not at
the
> moment work at all cannot be broken really; so changes to code only
> used by those features should be able to get a freeze exception
> easily.  (Tianocore is something which would probably fall under
> this.)
> 
> Features which change or add new interfaces which will need to be
> supported in a backwards-compatible way (for instance, vNUMA) will
> need freeze exceptions to make sure that the interface itself has
> enough time to be considered stable.
> 
> These are guidelines and principles to give you an idea where
we''re
> coming from; if you think there''s a good reason why making an
> exception for you will help us achieve goals 1-3 above better than not
> doing so, feel free to make your case.
> 
> = Timeline > 
> Here is our current timeline based on a 6-month release:
> 
> * Feature freeze: 18 October 2013
> * Code freezing point: 18 November 2013 <== WE ARE HERE
> * First RC: 6 December 2013
> * Release: 21 January 2014
> 
> Last updated: 18 November 2016
> 
> == Completed  => 
> * Event channel scalability (FIFO event channels)
> 
> * Non-udev scripts for driver domains (non-Linux driver domains)
> 
> * Multi-vector PCI MSI (Hypervisor side)
> 
> * Improved Spice support on libxl
>  - Added Spice vdagent support
>  - Added Spice clipboard sharing support
> 
> * PHV domU (experimental only)
> 
> * Guest EFI booting (tianocore)
> 
> * kexec
> 
> * Testing: Xen on ARM
> 
> * Update to SeaBIOS 1.7.3.1
> 
> * pvgrub2 checked into grub upstream
* Driver domain userspace daemon.
> 
> == Resolved since last update => 
> * credit scheduler doesn''t update other fields when tslice updated
from sysctl
> 
> == Open => 
> * qemu-upstream not freeing pirq
>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>  status: patches posted; latest patches need testing
> 
> * Race in PV shutdown between tool detection and shutdown watch
>  > http://www.gossamer-threads.com/lists/xen/devel/282467
>  > Nothing to do with ACPI
>  status: Patches posted
> 
> * Supposed regression from a3513737 ("x86: allow guest to set/clear
>  > MSI-X mask bit (try 2)"), as per
>  >
http://lists.xenproject.org/archives/html/xen-devel/2013-09/msg01589.html.
> 
> * qemu-traditional mis-parses host bus 8 as 0
>  > http://bugs.xenproject.org/xen/bug/15
> 
> * xen_platform_pci=0 doesn''t work with qemu-xen
>  > http://bugs.xenproject.org/xen/bug/20
> 
> * xl does not support specifying virtual function for passthrough device
>  > http://bugs.xenproject.org/xen/bug/22
> 
> * xl does not handle migrate interruption gracefully
>   > If you start a localhost migrate, and press "Ctrl-C" in
the middle,
>   > you get two hung domains
> 
> * libxl / xl does not handle failure of remote qemu gracefully
>   > Easiest way to reproduce:
>   >  - set "vncunused=0" and do a local migrate
>   >  - The "remote" qemu will fail because the vnc port is in
use
>   > The failure isn''t the problem, but everything being stuck
afterwards is
> 
> * HPET interrupt stack overflow (when using hpet_broadcast mode and MSI
> capable HPETs)
>   owner: andyh@citrix
>   status: patches posted, undergoing review iteration.
> 
> == Backlog => 
> === Testing coverage ==> 
> * new libxl w/ previous versions of xl
>  @IanJ
> 
> * Host S3 suspend
>  @bguthro, @dariof
> 
> * Default [example] XSM policy
>  @Stefano to ask Daniel D
> 
> * Storage driver domains
>  @roger
> 
> * HVM pci passthrough
>  @anthony
> 
> * PV pci passthrough
>  @konrad (or @george if he gets to it first)
> 
> * Network driver domains
>  @George
> 
> * Nested virt?
>  @intel (chased by George)
> 
> * Fix SRIOV test (chase intel)
>  @ianj
> 
> * Fix bisector to e-mail blame-worthy parties
>  @ianj
> 
> * Fix xl shutdown
>   @ianj
> 
> * stub domains
>   @athony
> 
> * performance benchmarks
>   @dario
> 
> === Meta-items (composed of other items) ==> 
> * Meta: PVIO NUMA improvements
>  - NUMA affinity for vcpus (4.4 possible)
>  - PV guest NUMA interface (4.4 possible)
>  - Sensible dom0 NUMA layout
>  - Toolstack pinning backend thread / virq to appropraite d0 vcpu
>  - NUMA-aware ballooning
> 
> * xend still in tree (x)
>  - xl list -l on a dom0-only system
>  - xl list -l doesn''t contain tty console port
>  - xl Alternate transport support for migration*
>  - xl PVSCSI support
>  - xl PVUSB support
> 
> === Big ticket items ==> 
> * PVH mode (w/ Linux)
>   owner: mukesh@oracle, george@citrix
>   status (Linux): Acked, waiting for ABI to be nailed down
>   status (Xen): Initial version checked in, still nailing down interface
> 
> * Update to qemu 1.6
>   owner: Anthonyper
>   status: In staging, still working out a bug in the VMX code
> 
> * Live Migration Support
>   owner: Jaeyong Yoo <jaeyong.yoo@samsung.com>
>   status: v5 posted, looking good for code freeze
> 
> * ARM64 guest
>   owner: IanC
>   status: v3 posted, v4 in progress. looking good.
> 
> * SWIOTLB (kernel side thing)
>   owner: Stefano
>   status: Pull request sent.
> 
> * soft affinity for vcpus (was NUMA affinity for vcpus)
>     owner: Dario
>     status: v2 posted
> 
> * PV guest NUMA interface
>     owner: Elena
>     status: v3 posted
> 
> * libvirt/libxl integration (external)
>  - owner: jfehlig@suse, dario@citrix
>  - patches posted (should be released before 4.4)
>   - migration
>   - PCI pass-through
>  - In progress
>   - integration w/ libvirt''s lock manager
>   - improved concurrency
> 
> * xl USB pass-through for HVM guests using Qemu USB emulation
>   prognosis: Good if extended
>   owner: George
>   status: v6 patch series posted
> 
> * libxl: Spice usbredirection support for upstream qemu
>  owner: fabio@M2R
>  status: I''ll post new patch version shortly
> 
> * libxl: usb2 and usb3 controller support for upstream qemu
>  owner: fabio@M2R
>  status: patch v5 posted, tested and working, awaiting reviews
> 
> * libxl network buffering support for Remus
>    @shriram
>    status: patches posted
>    prognosis: fair
> 
> * Disk: indirect descriptors
>    owner: roger@citrix
>    status: Linux side in 3.11, Xen-side patch posted
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

On 11/19/2013 10:50 AM, Roger Pau Monné wrote:
>> == Completed  =>>
>
> * Driver domain userspace daemon.
That''s what I was trying to get at with this one:

 >>
 >> * Non-udev scripts for driver domains (non-Linux driver domains)
 >>

Sorry, I put it near the top because I thought it was among one of the 
more important user-visible features. :-)

  -George

On 19/11/13 12:06, George Dunlap wrote:
> On 11/19/2013 10:50 AM, Roger Pau Monné wrote:
>>> == Completed  =>>>
>>
>> * Driver domain userspace daemon.
> 
> That''s what I was trying to get at with this one:
> 
>>>
>>> * Non-udev scripts for driver domains (non-Linux driver domains)
>>>
> 
> Sorry, I put it near the top because I thought it was among one of the
> more important user-visible features. :-)
My bad, I didn''t recognize it :)

On Mon, 2013-11-18 at 18:19 +0000, George Dunlap wrote:
> === Big ticket items ==
> * Live Migration Support
NB:^ARM
>   owner: Jaeyong Yoo <jaeyong.yoo@samsung.com>
>   status: v5 posted, looking good for code freeze
I think I was terribly confused about the release timeline when I gave
this prognosis.

Things are certainly looking good but given that we are now past feature
freeze and into code freeze I think we might be better off targeting
4.5. The patch is reasonably deeply involved with the lowlevel memory
mgmt stuff and the risk is that we regress the basic functionality.

Sorry for the earlier bum steer.
> * ARM64 guest
>   owner: IanC
>   status: v3 posted, v4 in progress. looking good.
v6 posted just now, I hope this can go in soon.
> * Disk: indirect descriptors
>    owner: roger@citrix
>    status: Linux side in 3.11, Xen-side patch posted
NB this is just an interface documentation update -- there is no
functionality on the Xen side.

Ian.

On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap
wrote:
> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> 
> (And I actually updated the wiki this time.)
> 
> The code "freezing point" is today; which means that starting
today
> non-bug fixes need a freeze exception to be included.
> 
> Remember our goal for the release:
>  1. A bug-free release
>  2. An awesome release
>  3. An on-time release
> 
> Accepting a new feature may make Xen more awesome; but it also
> introduces a risk that it will introduce more bugs.  That bug may be
> found before the release (threatening #3), or it may not be found
> until after the release (threatening #1).  Each freeze exception
> request will attempt to balance the benefits (how awesome the
> exception is) vs the risks (will it cause the release to slip, or
> worse, cause a bug which goes un-noticed into the final release).
> 
> The idea is that today we will be pretty permissive, but that we will
> become progressively more conservative until the first RC, which is
> scheduled for 3 weeks'' time (6 December).  After that, we will
only
> accept bug fixes.
> 
> Bug fixes can be checked in without a freeze exception throughout the
> code freeze, unless the maintianer thinks they are particularly high
> risk.  In later RC''s, we may even begin rejecting bug fixes if the
> broken functionality is small and the risk to other functionality is
> high.
> 
> Features which are currently marked "experimental" or do not at
the
> moment work at all cannot be broken really; so changes to code only
> used by those features should be able to get a freeze exception
> easily.  (Tianocore is something which would probably fall under
> this.)
> 
> Features which change or add new interfaces which will need to be
> supported in a backwards-compatible way (for instance, vNUMA) will
> need freeze exceptions to make sure that the interface itself has
> enough time to be considered stable.
> 
> These are guidelines and principles to give you an idea where
we''re
> coming from; if you think there''s a good reason why making an
> exception for you will help us achieve goals 1-3 above better than not
> doing so, feel free to make your case.
I am wondering in which category the tmem cleanup patches fall?

They aren''t bug-fixes, they could be considered a feature. They were
posted before the deadline. I posted the GIT PULL (see
http://article.gmane.org/gmane.comp.emulators.xen.devel/178043)
to one of the folks who has write access to the repository (as
documented in http://www.xenproject.org/governance.html)?

> == Open => 
> * qemu-upstream not freeing pirq
>  > http://www.gossamer-threads.com/lists/xen/devel/281498
>  status: patches posted; latest patches need testing
Duan, ping?
> 
> * Race in PV shutdown between tool detection and shutdown watch
>  > http://www.gossamer-threads.com/lists/xen/devel/282467
>  > Nothing to do with ACPI
>  status: Patches posted
I think I am going to slurp that one for v3.13-rc1
> * xend still in tree (x)
>  - xl list -l on a dom0-only system
>  - xl list -l doesn''t contain tty console port
>  - xl Alternate transport support for migration*
>  - xl PVSCSI support
>  - xl PVUSB support
Add this one please:
  -xl needs to disallow PoD with PCI passthrough
   (see
http://xen.1045712.n5.nabble.com/PATCH-VT-d-Dis-allow-PCI-device-assignment-if-PoD-is-enabled-td2547788.html)

I believe some of these were tacked by Wei - but he has
been doing other things. And I am busy right now tackling
bugs.
> * SWIOTLB (kernel side thing)
>   owner: Stefano
>   status: Pull request sent.
In v3.13.
> * Disk: indirect descriptors
>    owner: roger@citrix
>    status: Linux side in 3.11, Xen-side patch posted
I think you can drop this from your list.

>>> On 19.11.13 at 15:40, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap wrote:
>> The idea is that today we will be pretty permissive, but that we will
>> become progressively more conservative until the first RC, which is
>> scheduled for 3 weeks'' time (6 December).  After that, we will
only
>> accept bug fixes.
>> 
>> Bug fixes can be checked in without a freeze exception throughout the
>> code freeze, unless the maintianer thinks they are particularly high
>> risk.  In later RC''s, we may even begin rejecting bug fixes if
the
>> broken functionality is small and the risk to other functionality is
>> high.
>> 
>> Features which are currently marked "experimental" or do not
at the
>> moment work at all cannot be broken really; so changes to code only
>> used by those features should be able to get a freeze exception
>> easily.  (Tianocore is something which would probably fall under
>> this.)
>> 
>> Features which change or add new interfaces which will need to be
>> supported in a backwards-compatible way (for instance, vNUMA) will
>> need freeze exceptions to make sure that the interface itself has
>> enough time to be considered stable.
>> 
>> These are guidelines and principles to give you an idea where
we''re
>> coming from; if you think there''s a good reason why making an
>> exception for you will help us achieve goals 1-3 above better than not
>> doing so, feel free to make your case.
> 
> I am wondering in which category the tmem cleanup patches fall?
> 
> They aren''t bug-fixes, they could be considered a feature. They
were
> posted before the deadline. I posted the GIT PULL (see
> http://article.gmane.org/gmane.comp.emulators.xen.devel/178043)
> to one of the folks who has write access to the repository (as
> documented in http://www.xenproject.org/governance.html)? 
Considering the state of TMEM, I think these could go in at almost
any time. (Btw, I would have committed them already, but wasn''t
up to trying out my first git pull from a foreign tree, due to my
expectation of it not going to work as I expect the first time
through, and there''s other more important work that needs my
attention first. And of course you sent the pull request to Keir
only anyway...)

Jan

On Tue, Nov 19, 2013 at 02:54:19PM +0000, Jan Beulich
wrote:
> >>> On 19.11.13 at 15:40, Konrad Rzeszutek Wilk
<konrad.wilk@oracle.com> wrote:
> > On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap wrote:
> >> The idea is that today we will be pretty permissive, but that we
will
> >> become progressively more conservative until the first RC, which
is
> >> scheduled for 3 weeks'' time (6 December).  After that, we
will only
> >> accept bug fixes.
> >> 
> >> Bug fixes can be checked in without a freeze exception throughout
the
> >> code freeze, unless the maintianer thinks they are particularly
high
> >> risk.  In later RC''s, we may even begin rejecting bug
fixes if the
> >> broken functionality is small and the risk to other functionality
is
> >> high.
> >> 
> >> Features which are currently marked "experimental" or do
not at the
> >> moment work at all cannot be broken really; so changes to code
only
> >> used by those features should be able to get a freeze exception
> >> easily.  (Tianocore is something which would probably fall under
> >> this.)
> >> 
> >> Features which change or add new interfaces which will need to be
> >> supported in a backwards-compatible way (for instance, vNUMA) will
> >> need freeze exceptions to make sure that the interface itself has
> >> enough time to be considered stable.
> >> 
> >> These are guidelines and principles to give you an idea where
we''re
> >> coming from; if you think there''s a good reason why
making an
> >> exception for you will help us achieve goals 1-3 above better than
not
> >> doing so, feel free to make your case.
> > 
> > I am wondering in which category the tmem cleanup patches fall?
> > 
> > They aren''t bug-fixes, they could be considered a feature.
They were
> > posted before the deadline. I posted the GIT PULL (see
> > http://article.gmane.org/gmane.comp.emulators.xen.devel/178043)
> > to one of the folks who has write access to the repository (as
> > documented in http://www.xenproject.org/governance.html)? 
> 
> Considering the state of TMEM, I think these could go in at almost
> any time. (Btw, I would have committed them already, but wasn''t
> up to trying out my first git pull from a foreign tree, due to my
> expectation of it not going to work as I expect the first time
:-)
> through, and there''s other more important work that needs my
> attention first. And of course you sent the pull request to Keir
> only anyway...)
Duh! I will make sure to send it all committers in the hypervisor
next time.

Thanks!

On 11/18/2013 07:22 PM, Pasi Kärkkäinen wrote:
> On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap wrote:
>>
>> = Timeline >>
>> Here is our current timeline based on a 6-month release:
>>
>> * Feature freeze: 18 October 2013
>> * Code freezing point: 18 November 2013 <== WE ARE HERE
>> * First RC: 6 December 2013
>> * Release: 21 January 2014
>>
>> Last updated: 18 November 2016
>>
>
> 2016 :)
>
>
>>
>> == Backlog =>>
>>
>> === Big ticket items ==>>
>
>   * PCI hole resize support hvmloader/qemu-traditional/qemu-upstream with
PCI/GPU passthrough
>     http://lists.xen.org/archives/html/xen-devel/2013-05/msg02813.html
>     Where Stefano writes:
>
>     1) modify upstream QEMU to start the PCI hole at 0xe0000000, to match
>     qemu-xen-unstable in terms of configuration and not to introduce any
>     regressions. Do this for the Xen 4.3 release.
>
>     2) for Xen 4.4 rework the two patches above and improve
>     i440fx_update_pci_mem_hole: resizing the pci_hole subregion is not
>     enough, it also needs to be able to resize the system memory region
>     (xen.ram) to make room for the bigger pci_hole
Right -- I''ll put it on my list; but given that it needs a new
interface
into qemu, and work hasn''t even begun on that yet, I rather doubt
it''s
going to make it for 4.4, unfortunately. :-(

  -George

On 11/19/2013 12:59 AM, Don Slutz wrote:
> On 11/18/13 13:19, George Dunlap wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> [snip]
>
> I realize that I did not send anything to you before this.  However:
>
> http://lists.xen.org/archives/html/xen-devel/2013-10/msg01117.html
> http://lists.xen.org/archives/html/xen-devel/2013-11/msg02352.html
> http://lists.xen.org/archives/html/xen-devel/2013-11/msg02569.html
>
> Is about a new minor feature.  Clearly I would like it in 4.4.  This is
> close to:
>
>     xl dump-core <Domain> <filename>
>     crash <filename>
>
> But does not require the disk space, nor the delay while writing the
> file.  For me it took ~2 minutes to write out a 3G core file.
>      -Don Slutz
Got it, thanks.

  -George

On 11/19/2013 02:40 PM, Konrad Rzeszutek Wilk wrote:
> On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>
>> (And I actually updated the wiki this time.)
>>
>> The code "freezing point" is today; which means that starting
today
>> non-bug fixes need a freeze exception to be included.
>>
>> Remember our goal for the release:
>>   1. A bug-free release
>>   2. An awesome release
>>   3. An on-time release
>>
>> Accepting a new feature may make Xen more awesome; but it also
>> introduces a risk that it will introduce more bugs.  That bug may be
>> found before the release (threatening #3), or it may not be found
>> until after the release (threatening #1).  Each freeze exception
>> request will attempt to balance the benefits (how awesome the
>> exception is) vs the risks (will it cause the release to slip, or
>> worse, cause a bug which goes un-noticed into the final release).
>>
>> The idea is that today we will be pretty permissive, but that we will
>> become progressively more conservative until the first RC, which is
>> scheduled for 3 weeks'' time (6 December).  After that, we will
only
>> accept bug fixes.
>>
>> Bug fixes can be checked in without a freeze exception throughout the
>> code freeze, unless the maintianer thinks they are particularly high
>> risk.  In later RC''s, we may even begin rejecting bug fixes if
the
>> broken functionality is small and the risk to other functionality is
>> high.
>>
>> Features which are currently marked "experimental" or do not
at the
>> moment work at all cannot be broken really; so changes to code only
>> used by those features should be able to get a freeze exception
>> easily.  (Tianocore is something which would probably fall under
>> this.)
>>
>> Features which change or add new interfaces which will need to be
>> supported in a backwards-compatible way (for instance, vNUMA) will
>> need freeze exceptions to make sure that the interface itself has
>> enough time to be considered stable.
>>
>> These are guidelines and principles to give you an idea where
we''re
>> coming from; if you think there''s a good reason why making an
>> exception for you will help us achieve goals 1-3 above better than not
>> doing so, feel free to make your case.
>
> I am wondering in which category the tmem cleanup patches fall?
>
> They aren''t bug-fixes, they could be considered a feature. They
were
> posted before the deadline. I posted the GIT PULL (see
> http://article.gmane.org/gmane.comp.emulators.xen.devel/178043)
> to one of the folks who has write access to the repository (as
> documented in http://www.xenproject.org/governance.html)?
But it''s still marked "experimental", right?  As long as it
only touches
tmem code, it should be OK until fairly late.
>
>
>> == Open =>>
>> * qemu-upstream not freeing pirq
>>   > http://www.gossamer-threads.com/lists/xen/devel/281498
>>   status: patches posted; latest patches need testing
>
> Duan, ping?
>
>>
>> * Race in PV shutdown between tool detection and shutdown watch
>>   > http://www.gossamer-threads.com/lists/xen/devel/282467
>>   > Nothing to do with ACPI
>>   status: Patches posted
>
> I think I am going to slurp that one for v3.13-rc1
Cool, let me know when it''s in.
>
>> * xend still in tree (x)
>>   - xl list -l on a dom0-only system
>>   - xl list -l doesn''t contain tty console port
>>   - xl Alternate transport support for migration*
>>   - xl PVSCSI support
>>   - xl PVUSB support
>
> Add this one please:
>    -xl needs to disallow PoD with PCI passthrough
>     (see
http://xen.1045712.n5.nabble.com/PATCH-VT-d-Dis-allow-PCI-device-assignment-if-PoD-is-enabled-td2547788.html)
Done.
>
> I believe some of these were tacked by Wei - but he has
> been doing other things. And I am busy right now tackling
> bugs.
>
>> * SWIOTLB (kernel side thing)
>>    owner: Stefano
>>    status: Pull request sent.
>
> In v3.13.
Sweet!
>
>> * Disk: indirect descriptors
>>     owner: roger@citrix
>>     status: Linux side in 3.11, Xen-side patch posted
>
> I think you can drop this from your list.
I''ve moved it to the "completed" section.

Thanks!
  -George

On Tue, Nov 19, 2013 at 05:52:39PM +0000, George Dunlap
wrote:
> On 11/19/2013 02:40 PM, Konrad Rzeszutek Wilk wrote:
> >On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap wrote:
> >>This information will be mirrored on the Xen 4.4 Roadmap wiki page:
> >>  http://wiki.xen.org/wiki/Xen_Roadmap/4.4
> >>
> >>(And I actually updated the wiki this time.)
> >>
> >>The code "freezing point" is today; which means that
starting today
> >>non-bug fixes need a freeze exception to be included.
> >>
> >>Remember our goal for the release:
> >>  1. A bug-free release
> >>  2. An awesome release
> >>  3. An on-time release
> >>
> >>Accepting a new feature may make Xen more awesome; but it also
> >>introduces a risk that it will introduce more bugs.  That bug may
be
> >>found before the release (threatening #3), or it may not be found
> >>until after the release (threatening #1).  Each freeze exception
> >>request will attempt to balance the benefits (how awesome the
> >>exception is) vs the risks (will it cause the release to slip, or
> >>worse, cause a bug which goes un-noticed into the final release).
> >>
> >>The idea is that today we will be pretty permissive, but that we
will
> >>become progressively more conservative until the first RC, which is
> >>scheduled for 3 weeks'' time (6 December).  After that, we
will only
> >>accept bug fixes.
> >>
> >>Bug fixes can be checked in without a freeze exception throughout
the
> >>code freeze, unless the maintianer thinks they are particularly
high
> >>risk.  In later RC''s, we may even begin rejecting bug
fixes if the
> >>broken functionality is small and the risk to other functionality
is
> >>high.
> >>
> >>Features which are currently marked "experimental" or do
not at the
> >>moment work at all cannot be broken really; so changes to code only
> >>used by those features should be able to get a freeze exception
> >>easily.  (Tianocore is something which would probably fall under
> >>this.)
> >>
> >>Features which change or add new interfaces which will need to be
> >>supported in a backwards-compatible way (for instance, vNUMA) will
> >>need freeze exceptions to make sure that the interface itself has
> >>enough time to be considered stable.
> >>
> >>These are guidelines and principles to give you an idea where
we''re
> >>coming from; if you think there''s a good reason why making
an
> >>exception for you will help us achieve goals 1-3 above better than
not
> >>doing so, feel free to make your case.
> >
> >I am wondering in which category the tmem cleanup patches fall?
> >
> >They aren''t bug-fixes, they could be considered a feature.
They were
> >posted before the deadline. I posted the GIT PULL (see
> >http://article.gmane.org/gmane.comp.emulators.xen.devel/178043)
> >to one of the folks who has write access to the repository (as
> >documented in http://www.xenproject.org/governance.html)?
> 
> But it''s still marked "experimental", right?  As long as
it only
No idea what it is marked as. You have to use ''tmem=1'' to
actually
enable it - so it is no by default enabled.
> touches tmem code, it should be OK until fairly late.
OK.
> 
> >
> >
> >>== Open => >>
> >>* qemu-upstream not freeing pirq
> >>  > http://www.gossamer-threads.com/lists/xen/devel/281498
> >>  status: patches posted; latest patches need testing
> >
> >Duan, ping?
> >
> >>
> >>* Race in PV shutdown between tool detection and shutdown watch
> >>  > http://www.gossamer-threads.com/lists/xen/devel/282467
> >>  > Nothing to do with ACPI
> >>  status: Patches posted
> >
> >I think I am going to slurp that one for v3.13-rc1
> 
> Cool, let me know when it''s in.
Will do.

On 2013-11-19 22:40, Konrad Rzeszutek Wilk wrote:
> On Mon, Nov 18, 2013 at 06:19:46PM +0000, George Dunlap wrote:
>> This information will be mirrored on the Xen 4.4 Roadmap wiki page:
>>   http://wiki.xen.org/wiki/Xen_Roadmap/4.4
>>
>> (And I actually updated the wiki this time.)
>>
>> The code "freezing point" is today; which means that starting
today
>> non-bug fixes need a freeze exception to be included.
>>
>> Remember our goal for the release:
>>   1. A bug-free release
>>   2. An awesome release
>>   3. An on-time release
>>
>> Accepting a new feature may make Xen more awesome; but it also
>> introduces a risk that it will introduce more bugs.  That bug may be
>> found before the release (threatening #3), or it may not be found
>> until after the release (threatening #1).  Each freeze exception
>> request will attempt to balance the benefits (how awesome the
>> exception is) vs the risks (will it cause the release to slip, or
>> worse, cause a bug which goes un-noticed into the final release).
>>
>> The idea is that today we will be pretty permissive, but that we will
>> become progressively more conservative until the first RC, which is
>> scheduled for 3 weeks'' time (6 December).  After that, we will
only
>> accept bug fixes.
>>
>> Bug fixes can be checked in without a freeze exception throughout the
>> code freeze, unless the maintianer thinks they are particularly high
>> risk.  In later RC''s, we may even begin rejecting bug fixes if
the
>> broken functionality is small and the risk to other functionality is
>> high.
>>
>> Features which are currently marked "experimental" or do not
at the
>> moment work at all cannot be broken really; so changes to code only
>> used by those features should be able to get a freeze exception
>> easily.  (Tianocore is something which would probably fall under
>> this.)
>>
>> Features which change or add new interfaces which will need to be
>> supported in a backwards-compatible way (for instance, vNUMA) will
>> need freeze exceptions to make sure that the interface itself has
>> enough time to be considered stable.
>>
>> These are guidelines and principles to give you an idea where
we''re
>> coming from; if you think there''s a good reason why making an
>> exception for you will help us achieve goals 1-3 above better than not
>> doing so, feel free to make your case.
> I am wondering in which category the tmem cleanup patches fall?
>
> They aren''t bug-fixes, they could be considered a feature. They
were
> posted before the deadline. I posted the GIT PULL (see
> http://article.gmane.org/gmane.comp.emulators.xen.devel/178043)
> to one of the folks who has write access to the repository (as
> documented in http://www.xenproject.org/governance.html)?
>
>
>> == Open =>>
>> * qemu-upstream not freeing pirq
>>   > http://www.gossamer-threads.com/lists/xen/devel/281498
>>   status: patches posted; latest patches need testing
> Duan, ping?
I didn''t have a test env to test this. This need to reimage the env.

Regards
zduan