Xen devel - Sep 2012 - Xen 4.3 development update

= Timeline 
We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:
* Feature Freeze: 1 March, 2013
* First RC: 15 April 2013
* Release: 1 June 2013

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Feature tracking 
Below is a list of features we''re tracking for this release.  Please
respond to this mail with any updates to the status.  If the status
is "?", as most of them are, please respond with the status!  Example
statuses include "not started", "in progress", "initial
implementation
submitted for review", "patches submitted".

There are a number of items whose owners are marked as "?".  If you
are working on this, or know who is working on it, please respond and
let me know.  Alternately, if you would *like* to work on it, please
let me know as well.

And if there is something you''re working on you''d like
tracked, please
respond, and I will add it to the list.


* PVH mode, domU (w/ Linux)
  owner: mukesh@oracle
  status: ?

* PVH mode, dom0 (w/ Linux)
  owner: mukesh@oracle
  status: ?

* Event channel scalability
  owner: attilio@citrix
  status: ?
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* ARM server port
  owner: ijc@citrix
  status: Core hypervisor patches accepted; Linux paches pending

* NUMA scheduler affinity
  critical
  owner: dario@citrix
  status: ?

* NUMA Memory migration
  owner: dario@citrix
  status: ?

* blktap3
  owner: thanos@citrix
  status: ?

* Default to QEMU upstream
 - qemu-based stubdom (Linux or BSD libc)
   owner: anthony@citrix
   status: ?
   qemu-upstream needs a more fully-featured libc than exists in
   minios.  Either work on a minimalist linux-based stubdom with
   glibc, or port one of the BSD libcs to minios.

 - pci pass-thru
   owner: anthony@citrix
   status: ?

* Persistent grants
  owner: @citrix
  status: ?

* Multi-page blk rings
 - blkback in kernel (konrad@oracle, ?@intel)
 - qemu blkback
  status: ?

* Multi-page net protocol
  owner: ?
  status: ?
  expand the network ring protocol to allow multiple pages for
  increased throughput

* Scalability: 16TiB of RAM
  owner: jan@suse
  status: ?

* libvirt integration
  owner: ?
  status: ?
  To begin with, we need someone to go and make some lists:
  - Features available in libvirt/KVM not available in libvirt/Xen
  - Features available in xl/Xen but not available in libvirt/Xen

* xl vm-{export,import}
  owner: ?
  status: ?
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.


* xl USB pass-through for PV guests
  owner: ?
  status: ?
  - Port the xend PV pass-through functionality to xl.
  - Make sure qemu-based USB with qemu-upstream works
  - Upstream the Linux frontend/backend drivers

* openvswitch toostack integration
  owner: roger@citrix
  status: Sample script posted by Bastian ("[RFC] openvswitch support
script")

* Rationalized backend scripts (incl. driver domains)
  owner: roger@citrix
  status: ?

* Linux console improvements
  owner: jan@suse
  -EHCI debug port (done, to be submitted)
  -xHCI debug port
  -Firewire

* CPUID-based idle (don''t rely on ACPI info f/ dom0)
  owner: jan@suse
  status: done, to be submitted

* Remove hardcoded mobprobe''s in xencommons
  owner: ?
  status: ?

* Make storage migration possible
  owner: ?
  status: ?
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* Full-VM snapshotting
  owner: ?
  status: ?
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  status: May need review
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* Memory: Replace PoD with paging mechanism
  owner: george@citrix
  status: May need review

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix
  status: ?

* Managed domains?

Hello,

On Wednesday 19 September 2012 18:58:02 George Dunlap
wrote:
> * libvirt integration
>   owner: ?
>   status: ?
>   To begin with, we need someone to go and make some lists:
>   - Features available in libvirt/KVM not available in libvirt/Xen
>   - Features available in xl/Xen but not available in libvirt/Xen
Libvirt already has a comparison matrix at 
<http://libvirt.org/hvsupport.html>. That file is actually generated from
by
docs/hvsupport.pl by looking at the implemented functions.

Sincerely
Philipp
-- 
Philipp Hahn           Open Source Software Engineer      hahn@univention.de
Univention GmbH        be open.                       fon: +49 421 22 232- 0
Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
                                                   http://www.univention.de/


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

> * Persistent grants
>   owner: @citrix
>   status: ?
on network side:
annie li <annie.li@oracle.com>
>
> * Multi-page blk rings
>  - blkback in kernel (konrad@oracle, ?@intel)
>  - qemu blkback
>   status: ?
>
> * Multi-page net protocol
>   owner: ?
If Ian Campbell doesn''t get to it, then annie li
<annie.li@oracle.com>
will look at the patches that Wei Liu posted and work the kinks out.
>   status: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput
>

>>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> = Timeline > 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 1 March, 2013
> * First RC: 15 April 2013
> * Release: 1 June 2013
While it''s only slightly over two weeks - didn''t we agree to
count
from the release date (rather than the intended one)?
> * Persistent grants
>   owner: @citrix
>   status: ?
Isn''t that a guest side only thing (i.e. not really to be tracked
here)? Or does that mean migrating active grants?
> * Multi-page blk rings
>  - blkback in kernel (konrad@oracle, ?@intel)
This part certainly is a guest side only thing (apart from the
interface definition living in the our tree).
>  - qemu blkback
>   status: ?
> 
> * Multi-page net protocol
>   owner: ?
>   status: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput
Same here.
> * Scalability: 16TiB of RAM
>   owner: jan@suse
>   status: ?
Not started.
> * Linux console improvements
>   owner: jan@suse
>   -EHCI debug port (done, to be submitted)
Committed.
>   -xHCI debug port
This one needs an owner having access to suitable hardware _and_
knowledge of the protocol since, other than for EHCI, there isn''t
even a reference implementation (e.g. in Linux) to clone from. I''d
therefore rather consider this nice-to-have than a firm plan to have
such functionality.
>   -Firewire
Largely same here - while I''m not suffering from lack of hardware
(apart from - afaict - cables), I''ve never done anything with
Firewire, and hence would consider this one as well nice-to-have
only.

Jan

On Wed, 2012-09-19 at 17:58 +0100, George Dunlap wrote:
> = Timeline > 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 1 March, 2013
> * First RC: 15 April 2013
> * Release: 1 June 2013
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> = Feature tracking > 
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status
> is "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
> 
> There are a number of items whose owners are marked as "?".  If
you
> are working on this, or know who is working on it, please respond and
> let me know.  Alternately, if you would *like* to work on it, please
> let me know as well.
> 
> And if there is something you''re working on you''d like
tracked, please
> respond, and I will add it to the list.
> 
> 
> * PVH mode, domU (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
> 
> * PVH mode, dom0 (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
> 
> * Event channel scalability
>   owner: attilio@citrix
>   status: ?
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)
> 
> * ARM server port
>   owner: ijc@citrix
>   status: Core hypervisor patches accepted; Linux paches pending
> 
> * NUMA scheduler affinity
>   critical
>   owner: dario@citrix
>   status: ?
> 
> * NUMA Memory migration
>   owner: dario@citrix
>   status: ?
> 
> * blktap3
>   owner: thanos@citrix
>   status: ?
> 
> * Default to QEMU upstream
>  - qemu-based stubdom (Linux or BSD libc)
>    owner: anthony@citrix
>    status: ?
>    qemu-upstream needs a more fully-featured libc than exists in
>    minios.  Either work on a minimalist linux-based stubdom with
>    glibc, or port one of the BSD libcs to minios.
> 
>  - pci pass-thru
>    owner: anthony@citrix
>    status: ?
> 
> * Persistent grants
>   owner: @citrix
>   status:
Initial implementation of persistent grants for block devices submitted
for review
> 
> * Multi-page blk rings
>  - blkback in kernel (konrad@oracle, ?@intel)
>  - qemu blkback
>   status: ?
> 
> * Multi-page net protocol
>   owner: ?
>   status: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput
> 
> * Scalability: 16TiB of RAM
>   owner: jan@suse
>   status: ?
> 
> * libvirt integration
>   owner: ?
>   status: ?
>   To begin with, we need someone to go and make some lists:
>   - Features available in libvirt/KVM not available in libvirt/Xen
>   - Features available in xl/Xen but not available in libvirt/Xen
> 
> * xl vm-{export,import}
>   owner: ?
>   status: ?
>   Allow xl to import and export VMs to other formats; particularly
>   ovf, perhaps the XenServer format, or more.
> 
> 
> * xl USB pass-through for PV guests
>   owner: ?
>   status: ?
>   - Port the xend PV pass-through functionality to xl.
>   - Make sure qemu-based USB with qemu-upstream works
>   - Upstream the Linux frontend/backend drivers
> 
> * openvswitch toostack integration
>   owner: roger@citrix
>   status: Sample script posted by Bastian ("[RFC] openvswitch support
script")
> 
> * Rationalized backend scripts (incl. driver domains)
>   owner: roger@citrix
>   status: ?
> 
> * Linux console improvements
>   owner: jan@suse
>   -EHCI debug port (done, to be submitted)
>   -xHCI debug port
>   -Firewire
> 
> * CPUID-based idle (don''t rely on ACPI info f/ dom0)
>   owner: jan@suse
>   status: done, to be submitted
> 
> * Remove hardcoded mobprobe''s in xencommons
>   owner: ?
>   status: ?
> 
> * Make storage migration possible
>   owner: ?
>   status: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of
libxl
>   or xl.
> 
> * Full-VM snapshotting
>   owner: ?
>   status: ?
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
> 
> * VM Cloning
>   owner: ?
>   status: May need review
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
> 
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
>   status: May need review
> 
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix
>   status: ?
> 
> * Managed domains?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

On 19 September 2012 17:58, George Dunlap <George.Dunlap@eu.citrix.com>
wrote:
> = Timeline >
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 1 March, 2013
> * First RC: 15 April 2013
> * Release: 1 June 2013
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
>
> = Feature tracking >
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status
> is "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
>
> There are a number of items whose owners are marked as "?".  If
you
> are working on this, or know who is working on it, please respond and
> let me know.  Alternately, if you would *like* to work on it, please
> let me know as well.
>
> And if there is something you''re working on you''d like
tracked, please
> respond, and I will add it to the list.
>
>
> * PVH mode, domU (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
>
> * PVH mode, dom0 (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
>
> * Event channel scalability
>   owner: attilio@citrix
>   status: ?
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)
>
> * ARM server port
>   owner: ijc@citrix
>   status: Core hypervisor patches accepted; Linux paches pending
>
> * NUMA scheduler affinity
>   critical
>   owner: dario@citrix
>   status: ?
>
> * NUMA Memory migration
>   owner: dario@citrix
>   status: ?
>
> * blktap3
>   owner: thanos@citrix
>   status: ?
>
> * Default to QEMU upstream
>  - qemu-based stubdom (Linux or BSD libc)
>    owner: anthony@citrix
>    status: ?
>    qemu-upstream needs a more fully-featured libc than exists in
>    minios.  Either work on a minimalist linux-based stubdom with
>    glibc, or port one of the BSD libcs to minios.
>
>  - pci pass-thru
>    owner: anthony@citrix
>    status: ?
>
> * Persistent grants
>   owner: @citrix
>   status: ?
>
> * Multi-page blk rings
>  - blkback in kernel (konrad@oracle, ?@intel)
>  - qemu blkback
>   status: ?
>
> * Multi-page net protocol
>   owner: ?
>   status: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput
>
> * Scalability: 16TiB of RAM
>   owner: jan@suse
>   status: ?
>
> * libvirt integration
>   owner: ?
>   status: ?
>   To begin with, we need someone to go and make some lists:
>   - Features available in libvirt/KVM not available in libvirt/Xen
>   - Features available in xl/Xen but not available in libvirt/Xen
>
> * xl vm-{export,import}
>   owner: ?
>   status: ?
>   Allow xl to import and export VMs to other formats; particularly
>   ovf, perhaps the XenServer format, or more.
>
>
> * xl USB pass-through for PV guests
>   owner: ?
>   status: ?
>   - Port the xend PV pass-through functionality to xl.
>   - Make sure qemu-based USB with qemu-upstream works
>   - Upstream the Linux frontend/backend drivers
>
> * openvswitch toostack integration
>   owner: roger@citrix
>   status: Sample script posted by Bastian ("[RFC] openvswitch support
script")
>
> * Rationalized backend scripts (incl. driver domains)
>   owner: roger@citrix
>   status: ?
>
> * Linux console improvements
>   owner: jan@suse
>   -EHCI debug port (done, to be submitted)
>   -xHCI debug port
>   -Firewire
>
> * CPUID-based idle (don''t rely on ACPI info f/ dom0)
>   owner: jan@suse
>   status: done, to be submitted
>
> * Remove hardcoded mobprobe''s in xencommons
>   owner: ?
>   status: ?
>
> * Make storage migration possible
>   owner: ?
>   status: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of
libxl
>   or xl.
>
> * Full-VM snapshotting
>   owner: ?
>   status: ?
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
>
> * VM Cloning
>   owner: ?
>   status: May need review
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
>
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
>   status: May need review
>
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix
>   status: ?
>
> * Managed domains?
>
Hi George,

Maybe we could had V4V in the list.

* V4V: Inter-domain communication
  owner (Xen): jean.guyader@citrix.com
  status (Xen): patches submitted
  owner (Linux driver):  stefano.panella@citrix
  status (Linux driver): in progress

Thanks,
Jean

On Wed, Sep 19, 2012 at 6:18 PM, Philipp Hahn <hahn@univention.de>
wrote:
> Hello,
>
> On Wednesday 19 September 2012 18:58:02 George Dunlap wrote:
>> * libvirt integration
>>   owner: ?
>>   status: ?
>>   To begin with, we need someone to go and make some lists:
>>   - Features available in libvirt/KVM not available in libvirt/Xen
>>   - Features available in xl/Xen but not available in libvirt/Xen
>
> Libvirt already has a comparison matrix at
> <http://libvirt.org/hvsupport.html>. That file is actually generated
from by
> docs/hvsupport.pl by looking at the implemented functions.
Ah, very interesting -- so "libxl" is the one we particularly care
about.  "Xen" is presumably "xm"?  Would KVM fall under
"qemu"?

I guess the next step would be for someone to go through and identify
user-level features that would be enabled by the various functions, so
that we can figure out which functions are important to us.

 -George

On 09/20/2012 10:53 AM, Jean Guyader wrote:
> On 19 September 2012 17:58, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
>> = Timeline >>
>> We are planning on a 9-month release cycle.  Based on that, below are
>> our estimated dates:
>> * Feature Freeze: 1 March, 2013
>> * First RC: 15 April 2013
>> * Release: 1 June 2013
>>
>> The RCs and release will of course depend on stability and bugs, and
>> will therefore be fairly unpredictable.  The feature freeze may be
>> slipped for especially important features which are near completion.
>>
>> = Feature tracking >>
>> Below is a list of features we''re tracking for this release. 
Please
>> respond to this mail with any updates to the status.  If the status
>> is "?", as most of them are, please respond with the status! 
Example
>> statuses include "not started", "in progress",
"initial implementation
>> submitted for review", "patches submitted".
>>
>> There are a number of items whose owners are marked as "?". 
If you
>> are working on this, or know who is working on it, please respond and
>> let me know.  Alternately, if you would *like* to work on it, please
>> let me know as well.
>>
>> And if there is something you''re working on you''d
like tracked, please
>> respond, and I will add it to the list.
>>
>>
>> * PVH mode, domU (w/ Linux)
>>    owner: mukesh@oracle
>>    status: ?
>>
>> * PVH mode, dom0 (w/ Linux)
>>    owner: mukesh@oracle
>>    status: ?
>>
>> * Event channel scalability
>>    owner: attilio@citrix
>>    status: ?
>>    Increase limit on event channels (currently 1024 for 32-bit guests,
>>    4096 for 64-bit guests)
>>
>> * ARM server port
>>    owner: ijc@citrix
>>    status: Core hypervisor patches accepted; Linux paches pending
>>
>> * NUMA scheduler affinity
>>    critical
>>    owner: dario@citrix
>>    status: ?
>>
>> * NUMA Memory migration
>>    owner: dario@citrix
>>    status: ?
>>
>> * blktap3
>>    owner: thanos@citrix
>>    status: ?
>>
>> * Default to QEMU upstream
>>   - qemu-based stubdom (Linux or BSD libc)
>>     owner: anthony@citrix
>>     status: ?
>>     qemu-upstream needs a more fully-featured libc than exists in
>>     minios.  Either work on a minimalist linux-based stubdom with
>>     glibc, or port one of the BSD libcs to minios.
>>
>>   - pci pass-thru
>>     owner: anthony@citrix
>>     status: ?
>>
>> * Persistent grants
>>    owner: @citrix
>>    status: ?
>>
>> * Multi-page blk rings
>>   - blkback in kernel (konrad@oracle, ?@intel)
>>   - qemu blkback
>>    status: ?
>>
>> * Multi-page net protocol
>>    owner: ?
>>    status: ?
>>    expand the network ring protocol to allow multiple pages for
>>    increased throughput
>>
>> * Scalability: 16TiB of RAM
>>    owner: jan@suse
>>    status: ?
>>
>> * libvirt integration
>>    owner: ?
>>    status: ?
>>    To begin with, we need someone to go and make some lists:
>>    - Features available in libvirt/KVM not available in libvirt/Xen
>>    - Features available in xl/Xen but not available in libvirt/Xen
>>
>> * xl vm-{export,import}
>>    owner: ?
>>    status: ?
>>    Allow xl to import and export VMs to other formats; particularly
>>    ovf, perhaps the XenServer format, or more.
>>
>>
>> * xl USB pass-through for PV guests
>>    owner: ?
>>    status: ?
>>    - Port the xend PV pass-through functionality to xl.
>>    - Make sure qemu-based USB with qemu-upstream works
>>    - Upstream the Linux frontend/backend drivers
>>
>> * openvswitch toostack integration
>>    owner: roger@citrix
>>    status: Sample script posted by Bastian ("[RFC] openvswitch
support script")
>>
>> * Rationalized backend scripts (incl. driver domains)
>>    owner: roger@citrix
>>    status: ?
>>
>> * Linux console improvements
>>    owner: jan@suse
>>    -EHCI debug port (done, to be submitted)
>>    -xHCI debug port
>>    -Firewire
>>
>> * CPUID-based idle (don''t rely on ACPI info f/ dom0)
>>    owner: jan@suse
>>    status: done, to be submitted
>>
>> * Remove hardcoded mobprobe''s in xencommons
>>    owner: ?
>>    status: ?
>>
>> * Make storage migration possible
>>    owner: ?
>>    status: ?
>>    There needs to be a way, either via command-line or via some hooks,
>>    that someone can build a "storage migration" feature on
top of libxl
>>    or xl.
>>
>> * Full-VM snapshotting
>>    owner: ?
>>    status: ?
>>    Have a way of coordinating the taking and restoring of VM memory and
>>    disk snapshots.  This would involve some investigation into the best
>>    way to accomplish this.
>>
>> * VM Cloning
>>    owner: ?
>>    status: May need review
>>    Again, a way of coordinating the memory and disk aspects.  Research
>>    into the best way to do this would probably go along with the
>>    snapshotting feature.
>>
>> * Memory: Replace PoD with paging mechanism
>>    owner: george@citrix
>>    status: May need review
>>
>> * PV audio (audio for stubdom qemu)
>>    owner: stefano.panella@citrix
>>    status: ?
>>
>> * Managed domains?
>>
> Hi George,
>
> Maybe we could had V4V in the list.
>
> * V4V: Inter-domain communication
>    owner (Xen): jean.guyader@citrix.com
>    status (Xen): patches submitte
>    owner (Linux driver):  stefano.panella@citrix
>    status (Linux driver): in progress
As Jean has mentioned, I agree to take ownership of the linux driver for 
v4v.
>
> Thanks,
> Jean
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

On Thu, Sep 20, 2012 at 8:03 AM, Jan Beulich <JBeulich@suse.com>
wrote:
>>>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
>> = Timeline >>
>> We are planning on a 9-month release cycle.  Based on that, below are
>> our estimated dates:
>> * Feature Freeze: 1 March, 2013
>> * First RC: 15 April 2013
>> * Release: 1 June 2013
>
> While it''s only slightly over two weeks - didn''t we agree
to count
> from the release date (rather than the intended one)?
So we did.  And if we actually do 6 weeks for each milestone, that gives us:

* Feature Freeze: 25 March 2013
* First RC: 6 May 2013
* Release: 17 June 2013

Sound reasonable?
>> * Persistent grants
>>   owner: @citrix
>>   status: ?
>
> Isn''t that a guest side only thing (i.e. not really to be tracked
> here)? Or does that mean migrating active grants?
It is a bit funny to have Linux-side stuff tracked here, since it has
its own release schedule; but there''s not really a better place to
track it.  For both a "look at the great stuff happening" aspect, and
a "what needs to be done and who is going to do it when" aspect,
having all of the various development activities in one place is an
advantage.
>> * Scalability: 16TiB of RAM
>>   owner: jan@suse
>>   status: ?
>
> Not started.
>
>> * Linux console improvements
>>   owner: jan@suse
>>   -EHCI debug port (done, to be submitted)
>
> Committed.
Excellent; first to go under the (newly-created) "completed" category.
>>   -xHCI debug port
>
> This one needs an owner having access to suitable hardware _and_
> knowledge of the protocol since, other than for EHCI, there isn''t
> even a reference implementation (e.g. in Linux) to clone from. I''d
> therefore rather consider this nice-to-have than a firm plan to have
> such functionality.
>
>>   -Firewire
>
> Largely same here - while I''m not suffering from lack of hardware
> (apart from - afaict - cables), I''ve never done anything with
> Firewire, and hence would consider this one as well nice-to-have
> only.
"Nice-to-have" vs "blocker" is more something that will need
to be
considered near the feature freeze.  There are a handful of things
that the Citrix team, internally, have deemed "critical" -- that is,
if they seem to be in danger of missing, we will divert resources in
order to help them make 4.3; but other than those, I think basically
all the features in this list would be "nice-to-have".

Perhaps what we really need is a way to track the probability of
success, so that those who want a particular feature to make it in can
take action if they need to; for example:
* Green: Owned, and no reason to think it won''t make it in
* Yellow: At-risk; may need intervention to make it in
* Red: High probability of failure; possibly avert-able by immediate action
* Grey: Not owned, or something blocking; but no reason it couldn''t
make it in if someone picks it up
* Black: Very unlikely to make it in; expected to be held off until
next release.

So in this scheme, the xHCI and Firewire would be "grey" -- no reason
to think they couldn''t make it by the deadline, but it''s
blocked on
something.

Or, maybe that''s more than we need at the moment. :-)  Thoughts?

 -George

On Thu, Sep 20, 2012 at 10:53 AM, Jean Guyader <jean.guyader@gmail.com>
wrote:
> Hi George,
>
> Maybe we could had V4V in the list.
>
> * V4V: Inter-domain communication
>   owner (Xen): jean.guyader@citrix.com
>   status (Xen): patches submitted
>   owner (Linux driver):  stefano.panella@citrix
>   status (Linux driver): in progress
Added.

Just as an aside, in the future, could you trim the quotes from the
reply?  Since I use gmail it doesn''t bother me much, but many other
people have complained about having to scroll through hundreds of
lines of quote to get to the new message part. :-)

 -George

>>> On 20.09.12 at 13:17, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> On Thu, Sep 20, 2012 at 8:03 AM, Jan Beulich <JBeulich@suse.com>
wrote:
>>>>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
>>> = Timeline >>>
>>> We are planning on a 9-month release cycle.  Based on that, below
are
>>> our estimated dates:
>>> * Feature Freeze: 1 March, 2013
>>> * First RC: 15 April 2013
>>> * Release: 1 June 2013
>>
>> While it''s only slightly over two weeks - didn''t we
agree to count
>> from the release date (rather than the intended one)?
> 
> So we did.  And if we actually do 6 weeks for each milestone, that gives
us:
> 
> * Feature Freeze: 25 March 2013
> * First RC: 6 May 2013
> * Release: 17 June 2013
> 
> Sound reasonable?
Absolutely.
>>> * Persistent grants
>>>   owner: @citrix
>>>   status: ?
>>
>> Isn''t that a guest side only thing (i.e. not really to be
tracked
>> here)? Or does that mean migrating active grants?
> 
> It is a bit funny to have Linux-side stuff tracked here, since it has
> its own release schedule; but there''s not really a better place to
> track it.  For both a "look at the great stuff happening" aspect,
and
> a "what needs to be done and who is going to do it when" aspect,
> having all of the various development activities in one place is an
> advantage.
Understood.
>>>   -xHCI debug port
>>
>> This one needs an owner having access to suitable hardware _and_
>> knowledge of the protocol since, other than for EHCI, there
isn''t
>> even a reference implementation (e.g. in Linux) to clone from.
I''d
>> therefore rather consider this nice-to-have than a firm plan to have
>> such functionality.
>>
>>>   -Firewire
>>
>> Largely same here - while I''m not suffering from lack of
hardware
>> (apart from - afaict - cables), I''ve never done anything with
>> Firewire, and hence would consider this one as well nice-to-have
>> only.
> 
> "Nice-to-have" vs "blocker" is more something that will
need to be
> considered near the feature freeze.  There are a handful of things
> that the Citrix team, internally, have deemed "critical" -- that
is,
> if they seem to be in danger of missing, we will divert resources in
> order to help them make 4.3; but other than those, I think basically
> all the features in this list would be "nice-to-have".
> 
> Perhaps what we really need is a way to track the probability of
> success, so that those who want a particular feature to make it in can
> take action if they need to; for example:
> * Green: Owned, and no reason to think it won''t make it in
> * Yellow: At-risk; may need intervention to make it in
> * Red: High probability of failure; possibly avert-able by immediate action
> * Grey: Not owned, or something blocking; but no reason it
couldn''t
> make it in if someone picks it up
> * Black: Very unlikely to make it in; expected to be held off until
> next release.
> 
> So in this scheme, the xHCI and Firewire would be "grey" -- no
reason
> to think they couldn''t make it by the deadline, but it''s
blocked on
> something.
> 
> Or, maybe that''s more than we need at the moment. :-)  Thoughts?
Indeed this might be more than we want/need. With my response
I just wanted to clarify that while I would like to see these two things
to happen, it''s unlikely that I''m going to be able to do
anything about
them for the time being (so listing them as un-owned rather than
owned by me would presumably be the better choice).

Jan

On 20/09/12 12:26, Jan Beulich wrote:
>
> Indeed this might be more than we want/need. With my response
> I just wanted to clarify that while I would like to see these two things
> to happen, it''s unlikely that I''m going to be able to do
anything about
> them for the time being (so listing them as un-owned rather than
> owned by me would presumably be the better choice).
Right -- well they were added to the list because you said you were 
going to be working on them; so if you''re not, shall I just take them
off?

  -George

Hello George,

Am Donnerstag 20 September 2012 12:24:39 schrieb George
Dunlap:
> On Wed, Sep 19, 2012 at 6:18 PM, Philipp Hahn <hahn@univention.de>
wrote:
> > On Wednesday 19 September 2012 18:58:02 George Dunlap wrote:
> >> * libvirt integration
...
> > Libvirt already has a comparison matrix at
> > <http://libvirt.org/hvsupport.html>. That file is actually
generated from
> > by docs/hvsupport.pl by looking at the implemented functions.
>
> Ah, very interesting -- so "libxl" is the one we particularly
care
> about.  "Xen" is presumably "xm"?  Would KVM fall under
"qemu"?
Yes, that''s all correct.
The current "Xen" implementation is a mixture of Xend + direct
xenstore
accress + direct hypervisor access + inotify monitor on xm files.
> I guess the next step would be for someone to go through and identify
> user-level features that would be enabled by the various functions, so
> that we can figure out which functions are important to us.
Top on my wish list would be virDomainMigrate*, because migration is available 
with the old Xend driver, but not with the newer libxl.
It would probably help alot, if first those things already available via the 
old Xend would be added, since this would allow libvirt to deprecate the Xend 
part there as well.

Sincerely
Philipp
-- 
Philipp Hahn           Open Source Software Engineer      hahn@univention.de
Univention GmbH        be open.                       fon: +49 421 22 232- 0
Mary-Somerville-Str.1  D-28359 Bremen                 fax: +49 421 22 232-99
                                                   http://www.univention.de/


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

>>> On 20.09.12 at 13:33, George Dunlap
<george.dunlap@eu.citrix.com> wrote:
> On 20/09/12 12:26, Jan Beulich wrote:
>>
>> Indeed this might be more than we want/need. With my response
>> I just wanted to clarify that while I would like to see these two
things
>> to happen, it''s unlikely that I''m going to be able to
do anything about
>> them for the time being (so listing them as un-owned rather than
>> owned by me would presumably be the better choice).
> Right -- well they were added to the list because you said you were 
> going to be working on them; so if you''re not, shall I just take
them off?
I think keeping them un-owned would be preferable, to draw
people''s attention. The console situation really needs
improvement; the EHCI debug port thing was just a first step.
But in the end it''s you to decide, as you''ll have to maintain
the
list...

Jan

On Thu, Sep 20, 2012 at 12:44 PM, Jan Beulich <JBeulich@suse.com>
wrote:
>>>> On 20.09.12 at 13:33, George Dunlap
<george.dunlap@eu.citrix.com> wrote:
>> Right -- well they were added to the list because you said you were
>> going to be working on them; so if you''re not, shall I just
take them off?
>
> I think keeping them un-owned would be preferable, to draw
> people''s attention. The console situation really needs
> improvement; the EHCI debug port thing was just a first step.
> But in the end it''s you to decide, as you''ll have to
maintain the
> list...
OK, so what you mean is, "I think these are fairly important, and
Someone(TM) should do them sometime"?  It seems like more effective
than just having them on this list without an owner would be for you
to advertise that and ask if anyone can help, either by doing it
themselves or lending you the necessary hardware.

 -George

On 19/09/12 17:58, George Dunlap wrote:
> = Timeline >
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 1 March, 2013
> * First RC: 15 April 2013
> * Release: 1 June 2013
>
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
>
> = Feature tracking >
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status
> is "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
>
> There are a number of items whose owners are marked as "?".  If
you
> are working on this, or know who is working on it, please respond and
> let me know.  Alternately, if you would *like* to work on it, please
> let me know as well.
>
> And if there is something you''re working on you''d like
tracked, please
> respond, and I will add it to the list.
>
>
> * PVH mode, domU (w/ Linux)
>    owner: mukesh@oracle
>    status: ?
>
> * PVH mode, dom0 (w/ Linux)
>    owner: mukesh@oracle
>    status: ?
>
> * Event channel scalability
>    owner: attilio@citrix
>    status: ?
>    Increase limit on event channels (currently 1024 for 32-bit guests,
>    4096 for 64-bit guests)
>    
"in progress", green.

Attilio

> -----Original Message-----
> From: xen-devel-bounces@lists.xen.org [mailto:xen-devel-
> bounces@lists.xen.org] On Behalf Of George Dunlap
> Sent: 19 September 2012 17:58
> To: xen-devel@lists.xen.org
> Subject: [Xen-devel] Xen 4.3 development update
> 
> = Timeline > 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 1 March, 2013
> * First RC: 15 April 2013
> * Release: 1 June 2013
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> = Feature tracking > 
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status is
> "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
> 
> There are a number of items whose owners are marked as "?".  If
you are
> working on this, or know who is working on it, please respond and let
> me know.  Alternately, if you would *like* to work on it, please let me
> know as well.
> 
> And if there is something you''re working on you''d like
tracked, please
> respond, and I will add it to the list.
> 
> 
> * PVH mode, domU (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
> 
> * PVH mode, dom0 (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
> 
> * Event channel scalability
>   owner: attilio@citrix
>   status: ?
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)
> 
> * ARM server port
>   owner: ijc@citrix
>   status: Core hypervisor patches accepted; Linux paches pending
> 
> * NUMA scheduler affinity
>   critical
>   owner: dario@citrix
>   status: ?
> 
> * NUMA Memory migration
>   owner: dario@citrix
>   status: ?
> 
> * blktap3
>   owner: thanos@citrix
>   status: ?
In progress
> 
> * Default to QEMU upstream
>  - qemu-based stubdom (Linux or BSD libc)
>    owner: anthony@citrix
>    status: ?
>    qemu-upstream needs a more fully-featured libc than exists in
>    minios.  Either work on a minimalist linux-based stubdom with
>    glibc, or port one of the BSD libcs to minios.
> 
>  - pci pass-thru
>    owner: anthony@citrix
>    status: ?
> 
> * Persistent grants
>   owner: @citrix
>   status: ?
> 
> * Multi-page blk rings
>  - blkback in kernel (konrad@oracle, ?@intel)
>  - qemu blkback
>   status: ?
> 
> * Multi-page net protocol
>   owner: ?
>   status: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput
> 
> * Scalability: 16TiB of RAM
>   owner: jan@suse
>   status: ?
> 
> * libvirt integration
>   owner: ?
>   status: ?
>   To begin with, we need someone to go and make some lists:
>   - Features available in libvirt/KVM not available in libvirt/Xen
>   - Features available in xl/Xen but not available in libvirt/Xen
> 
> * xl vm-{export,import}
>   owner: ?
>   status: ?
>   Allow xl to import and export VMs to other formats; particularly
>   ovf, perhaps the XenServer format, or more.
> 
> 
> * xl USB pass-through for PV guests
>   owner: ?
>   status: ?
>   - Port the xend PV pass-through functionality to xl.
>   - Make sure qemu-based USB with qemu-upstream works
>   - Upstream the Linux frontend/backend drivers
> 
> * openvswitch toostack integration
>   owner: roger@citrix
>   status: Sample script posted by Bastian ("[RFC] openvswitch support
> script")
> 
> * Rationalized backend scripts (incl. driver domains)
>   owner: roger@citrix
>   status: ?
> 
> * Linux console improvements
>   owner: jan@suse
>   -EHCI debug port (done, to be submitted)
>   -xHCI debug port
>   -Firewire
> 
> * CPUID-based idle (don''t rely on ACPI info f/ dom0)
>   owner: jan@suse
>   status: done, to be submitted
> 
> * Remove hardcoded mobprobe''s in xencommons
>   owner: ?
>   status: ?
> 
> * Make storage migration possible
>   owner: ?
>   status: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of
libxl
>   or xl.
> 
> * Full-VM snapshotting
>   owner: ?
>   status: ?
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
> 
> * VM Cloning
>   owner: ?
>   status: May need review
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
> 
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
>   status: May need review
> 
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix
>   status: ?
> 
> * Managed domains?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

Philipp Hahn wrote:
> Hello George,
>
> Am Donnerstag 20 September 2012 12:24:39 schrieb George Dunlap:
>> On Wed, Sep 19, 2012 at 6:18 PM, Philipp Hahn
<hahn@univention.de> wrote:
>>> On Wednesday 19 September 2012 18:58:02 George Dunlap wrote:
>>>> * libvirt integration
> ...
>>> Libvirt already has a comparison matrix at
>>> <http://libvirt.org/hvsupport.html>. That file is actually
generated from
>>> by docs/hvsupport.pl by looking at the implemented functions.
>> Ah, very interesting -- so "libxl" is the one we particularly
care
>> about.  "Xen" is presumably "xm"?  Would KVM fall
under "qemu"?
> Yes, that''s all correct.
> The current "Xen" implementation is a mixture of Xend + direct
xenstore
> accress + direct hypervisor access + inotify monitor on xm files.
>
>> I guess the next step would be for someone to go through and identify
>> user-level features that would be enabled by the various functions, so
>> that we can figure out which functions are important to us.
> Top on my wish list would be virDomainMigrate*, because migration is
available
> with the old Xend driver, but not with the newer libxl.
> It would probably help alot, if first those things already available via
the
> old Xend would be added, since this would allow libvirt to deprecate the
Xend
> part there as well.
Chun Yan Liu was working on a patch to implement migration in the libvirt libxl 
driver, but she wasn''t able to finish tidying the patch before
departing on
maternity leave.  She recently returned to work and I''d expect a
refreshed patch
soon - after working through her backlog.

But why is libvirt integration listed as a Xen 4.3 feature?  AFAIK, all of this 
work is in libvirt and really doesn''t have much to do with the release
of 4.3.

Cheers,
Jim

On Thu, Sep 20, 2012 at 08:03:11AM +0100, Jan Beulich
wrote:
> > On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> > * Persistent grants
> >   owner: @citrix
> >   status: ?
> 
> Isn''t that a guest side only thing (i.e. not really to be tracked
> here)? Or does that mean migrating active grants?
I''d assume that the device would be renegotiated during migration, and
that new grants would be established during the re-connect sequence.

One question I have is how persistent grants intersect with
blktap3. Will a persistent grant-capable blktap3 implementation be in
scope for 4.3?
> > * Multi-page blk rings
> >  - blkback in kernel (konrad@oracle, ?@intel)
> 
> This part certainly is a guest side only thing (apart from the
> interface definition living in the our tree).
Same question here regarding blktap3.

Matt

On Thu, 2012-09-20 at 22:20 +0100, Jim Fehlig wrote:
> But why is libvirt integration listed as a Xen 4.3 feature?  AFAIK, all of
this
> work is in libvirt and really doesn''t have much to do with the
release of 4.3.
I think it''s useful to track activities undertaken by the community
during the 4.3 development timeframe even if it is on things which are
strictly speaking external and/or independent projects.

It makes a bit of sense if you also consider that we might want to
divert effort from stuff happening for Xen 4.3 "proper" to one of
these
external projects, which makes it useful to be able to prioritise/track
them next to each other.

Lastly, in this specific case the availability of libvirt bindings ties
into the xend->xl transition, which makes it interesting as a Xen 4.3
thing, IYSWIM.

George, since Jim isn''t the first to ask this question perhaps it would
be useful to include an "external" flag on these items and explain in
the legend why we are tracking them?

Ian.

On Wed, Sep 19, 2012 at 5:58 PM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> = Feature tracking >
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status
> is "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
> * Default to QEMU upstream
>  - qemu-based stubdom (Linux or BSD libc)
>    owner: anthony@citrix
>    status: ?
>    qemu-upstream needs a more fully-featured libc than exists in
>    minios.  Either work on a minimalist linux-based stubdom with
>    glibc, or port one of the BSD libcs to minios.
status: in progress
>  - pci pass-thru
>    owner: anthony@citrix
>    status: ?
Support for passthrough a PCI device is done, unless I miss something.
LibXL in Xen 4.2 support it, but qemu-xen in 4.2 does not. Right now,
only the last release of QEMU (1.2) support pci passthrough with Xen.

Also, to use QEMU upstream as default device model, we need implement
few more feature. I have:

- enable dirtybit tracking during migration
  status: patches submitted to QEMU and to LibXL.
- xl cd-{insert,eject}
  status: intilal implementation submitted

Thanks,

-- 
Anthony PERARD

On 21/09/12 09:48, Ian Campbell wrote:
> On Thu, 2012-09-20 at 22:20 +0100, Jim Fehlig wrote:
>> But why is libvirt integration listed as a Xen 4.3 feature?  AFAIK, all
of this
>> work is in libvirt and really doesn''t have much to do with the
release of 4.3.
> I think it''s useful to track activities undertaken by the
community
> during the 4.3 development timeframe even if it is on things which are
> strictly speaking external and/or independent projects.
>
> It makes a bit of sense if you also consider that we might want to
> divert effort from stuff happening for Xen 4.3 "proper" to one of
these
> external projects, which makes it useful to be able to prioritise/track
> them next to each other.
>
> Lastly, in this specific case the availability of libvirt bindings ties
> into the xend->xl transition, which makes it interesting as a Xen 4.3
> thing, IYSWIM.
>
> George, since Jim isn''t the first to ask this question perhaps it
would
> be useful to include an "external" flag on these items and
explain in
> the legend why we are tracking them?
Sounds good -- I''ll add this to the mail template:

NB: Several of the items on this list marked (external).  These are
not part of the Xen tree, but are directly related to our users''
experience (e.g., work in Linux or qemu) or to integration with other
important projects (e.g., libvirt bindings).  Since all of these are
part of the Xen community work, and comes from the same pool of labor,
it makes sense to track the progress here, even though they won''t
explicitly be released as part of 4.3.

  -George

On 21/09/12 11:38, Anthony PERARD wrote:
>   - pci pass-thru
>     owner: anthony@citrix
>     status: ?
> Support for passthrough a PCI device is done, unless I miss something.
> LibXL in Xen 4.2 support it, but qemu-xen in 4.2 does not. Right now,
> only the last release of QEMU (1.2) support pci passthrough with Xen.
I don''t really understand what you mean... do you mean qemu-traditional
doesn''t support pci pass-through in 4.2, or that qemu-upstream
doesn''t?
In any case, has everything been done that needs to be done for 
qemu-upstream to have passthrough support for 4.3?
> Also, to use QEMU upstream as default device model, we need implement
> few more feature. I have:
>
> - enable dirtybit tracking during migration
>    status: patches submitted to QEMU and to LibXL.
> - xl cd-{insert,eject}
>    status: intilal implementation submitted
Thanks, I''ve added these to the list.

  -George

On 20/09/12 22:20, Jim Fehlig wrote:
> Chun Yan Liu was working on a patch to implement migration in the libvirt
libxl
> driver, but she wasn''t able to finish tidying the patch before
departing on
> maternity leave.  She recently returned to work and I''d expect a
refreshed patch
> soon - after working through her backlog.
>
> But why is libvirt integration listed as a Xen 4.3 feature?  AFAIK, all of
this
> work is in libvirt and really doesn''t have much to do with the
release of 4.3.
For one, as Ian said, we want xl/libxl to be the default toolstack in 
4.3; as soon as possible we want to remove xend.

But from a broader perspective, the idea is for the Xen.org community to 
be thinking strategically about the success of the Xen project as a 
whole.  Things like distro support and libvirt integration are 
absolutely critical to Xen''s success in the open-source ecosystem.  So 
while we don''t have to do everything ourselves, we want to think about 
what things need to be done, track them to make sure that *someone* does 
them; and be prepared to pick them up ourselves if necessary.  And it 
just makes more sense to track everything on one list.

Does that make sense?

  -George

On Fri, Sep 21, 2012 at 11:52 AM, George Dunlap
<george.dunlap@eu.citrix.com> wrote:
> On 21/09/12 11:38, Anthony PERARD wrote:
>>
>>   - pci pass-thru
>>     owner: anthony@citrix
>>     status: ?
>> Support for passthrough a PCI device is done, unless I miss something.
>> LibXL in Xen 4.2 support it, but qemu-xen in 4.2 does not. Right now,
>> only the last release of QEMU (1.2) support pci passthrough with Xen.
>
> I don''t really understand what you mean... do you mean
qemu-traditional
> doesn''t support pci pass-through in 4.2, or that qemu-upstream
doesn''t?
I mean the qemu-upstream distributed with Xen 4.2.
> In any case, has everything been done that needs to be done for
qemu-upstream
> to have passthrough support for 4.3?
Yes.

-- 
Anthony PERARD

George Dunlap wrote:
> On 20/09/12 22:20, Jim Fehlig wrote:
>> Chun Yan Liu was working on a patch to implement migration in the
>> libvirt libxl
>> driver, but she wasn''t able to finish tidying the patch before
>> departing on
>> maternity leave.  She recently returned to work and I''d expect
a
>> refreshed patch
>> soon - after working through her backlog.
>>
>> But why is libvirt integration listed as a Xen 4.3 feature?  AFAIK,
>> all of this
>> work is in libvirt and really doesn''t have much to do with the
>> release of 4.3.
> For one, as Ian said, we want xl/libxl to be the default toolstack in
> 4.3; as soon as possible we want to remove xend.
Good :).
>
> But from a broader perspective, the idea is for the Xen.org community
> to be thinking strategically about the success of the Xen project as a
> whole.  Things like distro support and libvirt integration are
> absolutely critical to Xen''s success in the open-source ecosystem.
So
> while we don''t have to do everything ourselves, we want to think
about
> what things need to be done, track them to make sure that *someone*
> does them; and be prepared to pick them up ourselves if necessary. 
> And it just makes more sense to track everything on one list.
>
> Does that make sense?
Yes, it does.  I''m actually quite happy to hear this statement of
support for the broader ecosystem!

WRT libvirt support for libxl in Xen >= 4.2, SUSE will be contributing
some work.  Just today I had to disable the driver in our Factory
libvirt builds where we now have Xen 4.2.  This hack can''t last for
long.

Regards,
Jim

On 21/09/12 04:18, Matt Wilson wrote:
> On Thu, Sep 20, 2012 at 08:03:11AM +0100, Jan Beulich wrote:
>>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
>>> * Persistent grants
>>>    owner: @citrix
>>>    status: ?
>> Isn''t that a guest side only thing (i.e. not really to be
tracked
>> here)? Or does that mean migrating active grants?
> I''d assume that the device would be renegotiated during migration,
and
> that new grants would be established during the re-connect sequence.
>
> One question I have is how persistent grants intersect with
> blktap3. Will a persistent grant-capable blktap3 implementation be in
> scope for 4.3?
Thanos is working on blktap3 -- Thanos, can you comment?
>
>>> * Multi-page blk rings
>>>   - blkback in kernel (konrad@oracle, ?@intel)
>> This part certainly is a guest side only thing (apart from the
>> interface definition living in the our tree).
> Same question here regarding blktap3.
And here.

  -George

> -----Original Message-----
> From: George Dunlap [mailto:george.dunlap@eu.citrix.com]
> Sent: 26 September 2012 10:37
> To: Matt Wilson
> Cc: Jan Beulich; xen-devel@lists.xen.org; Thanos Makatos
> Subject: Re: [Xen-devel] Xen 4.3 development update
> 
> On 21/09/12 04:18, Matt Wilson wrote:
> > On Thu, Sep 20, 2012 at 08:03:11AM +0100, Jan Beulich wrote:
> >>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com>
> wrote:
> >>> * Persistent grants
> >>>    owner: @citrix
> >>>    status: ?
> >> Isn''t that a guest side only thing (i.e. not really to be
tracked
> >> here)? Or does that mean migrating active grants?
> > I''d assume that the device would be renegotiated during
migration,
> and
> > that new grants would be established during the re-connect sequence.
> >
> > One question I have is how persistent grants intersect with blktap3.
> > Will a persistent grant-capable blktap3 implementation be in scope
> for
> > 4.3?
> Thanos is working on blktap3 -- Thanos, can you comment?
Supporting this in 4.3 is not top priority; however I''ll do it if I
have time.
> >
> >>> * Multi-page blk rings
> >>>   - blkback in kernel (konrad@oracle, ?@intel)
> >> This part certainly is a guest side only thing (apart from the
> >> interface definition living in the our tree).
> > Same question here regarding blktap3.
> And here.
I''m not sure I understand the question, blktap3 doesn''t talk
to blkback at all, but to blkfront directly -- are you referring to blkfront?
> 
>   -George

On Wed, Sep 26, 2012 at 11:25:54AM +0100, Thanos Makatos
wrote:
> > -----Original Message-----
> > From: George Dunlap [mailto:george.dunlap@eu.citrix.com]
> > Sent: 26 September 2012 10:37
> > To: Matt Wilson
> > Cc: Jan Beulich; xen-devel@lists.xen.org; Thanos Makatos
> > Subject: Re: [Xen-devel] Xen 4.3 development update
> > 
> > On 21/09/12 04:18, Matt Wilson wrote:
> > > On Thu, Sep 20, 2012 at 08:03:11AM +0100, Jan Beulich wrote:
> > >>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com>
> > wrote:
> > >>> * Persistent grants
> > >>>    owner: @citrix
> > >>>    status: ?
> > >> Isn''t that a guest side only thing (i.e. not really
to be tracked
> > >> here)? Or does that mean migrating active grants?
> > > I''d assume that the device would be renegotiated during
migration,
> > and
> > > that new grants would be established during the re-connect
sequence.
> > >
> > > One question I have is how persistent grants intersect with
blktap3.
> > > Will a persistent grant-capable blktap3 implementation be in
scope
> > for
> > > 4.3?
> > Thanos is working on blktap3 -- Thanos, can you comment?
> 
> Supporting this in 4.3 is not top priority; however I''ll do it if
I
> have time.
I brought this up in response to Jan''s question as to if persistent
grants should be on the 4.3 release plan. With the introduction of
blktap3, we have a separate implementation of the protocol in a Xen
component. One worry I have about blktap3 is that it will lag behind
the in-kernel blkback implementation.
> > >>> * Multi-page blk rings
> > >>>   - blkback in kernel (konrad@oracle, ?@intel)
> > >> This part certainly is a guest side only thing (apart from
the
> > >> interface definition living in the our tree).
> > > Same question here regarding blktap3.
> > And here.
> 
> I''m not sure I understand the question, blktap3 doesn''t
talk to
> blkback at all, but to blkfront directly -- are you referring to
> blkfront?
blktap3 would need to implement the new multi-page blk ring protocol,
correct?

Matt

On Wed, 2012-09-26 at 18:15 +0100, Matt Wilson wrote:
> On Wed, Sep 26, 2012 at 11:25:54AM +0100, Thanos Makatos wrote:
> > > -----Original Message-----
> > > From: George Dunlap [mailto:george.dunlap@eu.citrix.com]
> > > Sent: 26 September 2012 10:37
> > > To: Matt Wilson
> > > Cc: Jan Beulich; xen-devel@lists.xen.org; Thanos Makatos
> > > Subject: Re: [Xen-devel] Xen 4.3 development update
> > > 
> > > On 21/09/12 04:18, Matt Wilson wrote:
> > > > On Thu, Sep 20, 2012 at 08:03:11AM +0100, Jan Beulich wrote:
> > > >>> On 19.09.12 at 18:58, George Dunlap
<George.Dunlap@eu.citrix.com>
> > > wrote:
> > > >>> * Persistent grants
> > > >>>    owner: @citrix
> > > >>>    status: ?
> > > >> Isn''t that a guest side only thing (i.e. not
really to be tracked
> > > >> here)? Or does that mean migrating active grants?
> > > > I''d assume that the device would be renegotiated
during migration,
> > > and
> > > > that new grants would be established during the re-connect
sequence.
> > > >
> > > > One question I have is how persistent grants intersect with
blktap3.
> > > > Will a persistent grant-capable blktap3 implementation be in
scope
> > > for
> > > > 4.3?
> > > Thanos is working on blktap3 -- Thanos, can you comment?
> > 
> > Supporting this in 4.3 is not top priority; however I''ll do
it if I
> > have time.
> 
> I brought this up in response to Jan''s question as to if
persistent
> grants should be on the 4.3 release plan. With the introduction of
> blktap3, we have a separate implementation of the protocol in a Xen
> component. One worry I have about blktap3 is that it will lag behind
> the in-kernel blkback implementation.
We already have multiple block backend implementations (blktap1,
blktap2, qdisk, non-Linux dom0s), so I don''t think that is a new
concern.

If anything blktap3 improves things by making blktap1 + 2 obsolete, plus
it comes with an active maintainer!

I think getting a working blktap3 implementation of the basic blk
protocol for 4.3 is a big enough chunk of work in its own right, it''s
more than enough to fill someone''s time for one release cycle IMHO.

So, I don''t think implementing extensions (persistent grants, multipage
rings etc) need to be a requirement for blktap3 in 4.3, there''s always
4.4 and beyond. Of course if someone wants to contribute patches I''m
sure Thanos would be very pleased ;-)

Ian.

On Wed, Sep 26, 2012 at 07:59:31PM +0100, Ian Campbell
wrote:
> On Wed, 2012-09-26 at 18:15 +0100, Matt Wilson wrote:
[...]
> > I brought this up in response to Jan''s question as to if
persistent
> > grants should be on the 4.3 release plan. With the introduction of
> > blktap3, we have a separate implementation of the protocol in a Xen
> > component. One worry I have about blktap3 is that it will lag behind
> > the in-kernel blkback implementation.
> 
> We already have multiple block backend implementations (blktap1,
> blktap2, qdisk, non-Linux dom0s), so I don''t think that is a new
> concern.
For blktap2 we use blkback, so blktap2 users should be able to take
advantage of the improvements made there.
> If anything blktap3 improves things by making blktap1 + 2 obsolete, plus
> it comes with an active maintainer!
I''m not saying that blktap3 is a bad thing! It''s definitely
exciting.
> I think getting a working blktap3 implementation of the basic blk
> protocol for 4.3 is a big enough chunk of work in its own right,
it''s
> more than enough to fill someone''s time for one release cycle
IMHO.
Indeed, I''m impressed that it can be done. :-)
> So, I don''t think implementing extensions (persistent grants,
multipage
> rings etc) need to be a requirement for blktap3 in 4.3, there''s
always
> 4.4 and beyond. Of course if someone wants to contribute patches
I''m
> sure Thanos would be very pleased ;-)
Definitely not a requirement, but a nice-to-have.

Matt

This information will be mirrored on the Xen 4.3 Roadmap wiki page:
 http://wiki.xen.org/wiki/Xen_Roadmap/4.3

= Timeline 
We are planning on a 9-month release cycle.  Based on that, below are
our estimated dates:
* Feature Freeze: 25 March 2013
* First RC: 6 May 2013
* Release: 17 June 2013

The RCs and release will of course depend on stability and bugs, and
will therefore be fairly unpredictable.  The feature freeze may be
slipped for especially important features which are near completion.

= Feature tracking 
Below is a list of features we''re tracking for this release. Please
respond to this mail with any updates to the status.

There are a number of items whose owners are marked as "?".  If you
are working on this, or know who is working on it, please respond and
let me know.  Alternately, if you would *like* to work on it, please
let me know as well.

And if there is something you''re working on you''d like
tracked, please
respond, and I will add it to the list.

NB: Several of the items on this list are from external projects:
linux, qemu, and libvirt.  These are not part of the Xen tree, but are
directly related to our users'' experience (e.g., work in Linux or
qemu) or to integration with other important projects (e.g., libvirt
bindings).  Since all of these are part of the Xen community work, and
comes from the same pool of labor, it makes sense to track the
progress here, even though they won''t explicitly be released as part
of 4.3.

== Completed =
* Linux console improvements
  -EHCI debug port

* Default to QEMU upstream
 - pci pass-thru (external)

== Not yet complete =
* PVH mode, domU (w/ Linux)
  owner: mukesh@oracle
  status: ?

* PVH mode, dom0 (w/ Linux)
  owner: mukesh@oracle
  status: ?

* Event channel scalability
  owner: attilio@citrix
  status: initial design proposed
  Increase limit on event channels (currently 1024 for 32-bit guests,
  4096 for 64-bit guests)

* ARM server port
  owner: ijc@citrix
  status: Core hypervisor patches accepted; Linux paches pending

* NUMA scheduler affinity
  critical
  owner: dario@citrix
  status: ?

* NUMA Memory migration
  owner: dario@citrix
  status: ?

* blktap3
  owner: thanos@citrix
  status: in progress

* Default to QEMU upstream
 - qemu-based stubdom (Linux or BSD libc)
   owner: anthony@citrix
   status: in progress
   qemu-upstream needs a more fully-featured libc than exists in
   minios.  Either work on a minimalist linux-based stubdom with
   glibc, or port one of the BSD libcs to minios.

 - enable dirtybit tracking during migration (external)
   status: patches submitted to QEMU and to LibXL.

 - xl cd-{insert,eject} (external)
   status: intilal implementation submitted

* Persistent grants (external)
  owner: @citrix
  status: Initial implementation posted

* Multi-page blk rings (external)
 - blkback in kernel (konrad@oracle, ?@intel)
 - qemu blkback
  status: ?

* Multi-page net protocol (external)
  owner: ijc@citrix or annie.li@oracle
  status: Initial patches posted (by Wei Liu)
  expand the network ring protocol to allow multiple pages for
  increased throughput

* Scalability: 16TiB of RAM
  owner: jan@suse
  status: Not started

* libvirt/libxl integration (external)
 - Migration
   owner: cyliu@suse (?)
   status: first draft implemented, not yet submitted
 - Itemize other things that need work
   To begin with, we need someone to go and make some lists:
   - Features available in libvirt/KVM not available in libvirt/libxl
     See http://libvirt.org/hvsupport.html
   - Features available in xl/Xen but not available in libvirt/Xen

* V4V: Inter-domain communication
  owner (Xen): jean.guyader@citrix.com
  status (Xen): patches submitted
  owner (Linux driver):  stefano.panella@citrix
  status (Linux driver): in progress

* xl vm-{export,import}
  owner: ?
  status: ?
  Allow xl to import and export VMs to other formats; particularly
  ovf, perhaps the XenServer format, or more.


* xl USB pass-through for PV guests
  owner: ?
  status: ?
  - Port the xend PV pass-through functionality to xl.
  - Make sure qemu-based USB with qemu-upstream works
  - Upstream the Linux frontend/backend drivers

* openvswitch toostack integration
  owner: roger@citrix
  status: Sample script posted by Bastian ("[RFC] openvswitch support
script")

* Rationalized backend scripts (incl. driver domains)
  owner: roger@citrix
  status: ?

* Linux console improvements
  owner: ?
  status: Stalled (see below)
  -xHCI debug port (Needs hardware)
  -Firewire (needs hardware)

* CPUID-based idle (don''t rely on ACPI info f/ dom0)
  owner: jan@suse
  status: done, to be submitted

* Remove hardcoded mobprobe''s in xencommons
  owner: ?
  status: ?

* Make storage migration possible
  owner: ?
  status: ?
  There needs to be a way, either via command-line or via some hooks,
  that someone can build a "storage migration" feature on top of libxl
  or xl.

* Full-VM snapshotting
  owner: ?
  status: ?
  Have a way of coordinating the taking and restoring of VM memory and
  disk snapshots.  This would involve some investigation into the best
  way to accomplish this.

* VM Cloning
  owner: ?
  status: May need review
  Again, a way of coordinating the memory and disk aspects.  Research
  into the best way to do this would probably go along with the
  snapshotting feature.

* Memory: Replace PoD with paging mechanism
  owner: george@citrix
  status: May need review

* PV audio (audio for stubdom qemu)
  owner: stefano.panella@citrix
  status: ?

* Managed domains?

On Mon, Oct 01, 2012 at 05:25:43PM +0100, George Dunlap
wrote:
> 
> * xl USB pass-through for PV guests
>   owner: ?
>   status: ?
>   - Port the xend PV pass-through functionality to xl.
>   - Make sure qemu-based USB with qemu-upstream works
>   - Upstream the Linux frontend/backend drivers
> 
I think this should be split into two separate tasks:

 * xl PVUSB pass-through for both PV and HVM guests
   owner: ? 
   status: ? 
   - xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both PV
and HVM guests).
   - port the xm/xend functionality to xl.
   - this PVUSB feature does not require support or emulation from Qemu.
   - upstream the Linux frontend/backend drivers. Current work-in-progress
versions are in Konrad''s git tree.
   - James Harper''s GPLPV drivers for Windows include PVUSB frontend
drivers.

 * xl USB pass-through for HVM guests using Qemu USB emulation
   owner: ? 
   status: ?
   - xm/xend with qemu-traditional supports USB passthrough to HVM guests using
the Qemu emulated USB controller.
   - port the xm/xend functionality to xl.
   - make sure USB passthrough with xl works with both qemu-traditional and
qemu-upstream.
   - The HVM guest does not need any special drivers for this feature.


-- Pasi

And maybe add:
* try to port xen-4.3 to illumos-based platform :)

I have some progress - but have panic.

---
Best regards,
Igor Kozhukhov
IRC# igork




On 10/1/12 8:42 PM, "Pasi Kärkkäinen" <pasik@iki.fi> wrote:
>On Mon, Oct 01, 2012 at 05:25:43PM +0100, George Dunlap wrote:
>> 
>> * xl USB pass-through for PV guests
>>   owner: ?
>>   status: ?
>>   - Port the xend PV pass-through functionality to xl.
>>   - Make sure qemu-based USB with qemu-upstream works
>>   - Upstream the Linux frontend/backend drivers
>> 
>
>I think this should be split into two separate tasks:
>
> * xl PVUSB pass-through for both PV and HVM guests
>   owner: ? 
>   status: ? 
>   - xm/xend supports PVUSB pass-through to guests with PVUSB drivers
>(both PV and HVM guests).
>   - port the xm/xend functionality to xl.
>   - this PVUSB feature does not require support or emulation from Qemu.
>   - upstream the Linux frontend/backend drivers. Current
>work-in-progress versions are in Konrad''s git tree.
>   - James Harper''s GPLPV drivers for Windows include PVUSB
frontend
>drivers.
>
> * xl USB pass-through for HVM guests using Qemu USB emulation
>   owner: ? 
>   status: ?
>   - xm/xend with qemu-traditional supports USB passthrough to HVM guests
>using the Qemu emulated USB controller.
>   - port the xm/xend functionality to xl.
>   - make sure USB passthrough with xl works with both qemu-traditional
>and qemu-upstream.
>   - The HVM guest does not need any special drivers for this feature.
>
>
>-- Pasi
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xen.org
>http://lists.xen.org/xen-devel

On Mon, Oct 1, 2012 at 5:42 PM, Pasi Kärkkäinen <pasik@iki.fi>
wrote:
> On Mon, Oct 01, 2012 at 05:25:43PM +0100, George Dunlap wrote:
>>
>> * xl USB pass-through for PV guests
>>   owner: ?
>>   status: ?
>>   - Port the xend PV pass-through functionality to xl.
>>   - Make sure qemu-based USB with qemu-upstream works
>>   - Upstream the Linux frontend/backend drivers
>>
>
> I think this should be split into two separate tasks:
That does make sense.  I''ve updated my notes accordingly.

Just to verify: xm/xend *already* supports both PVUSB and passthrough via QEMU?

 -George
>
>  * xl PVUSB pass-through for both PV and HVM guests
>    owner: ?
>    status: ?
>    - xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both
PV and HVM guests).
>    - port the xm/xend functionality to xl.
>    - this PVUSB feature does not require support or emulation from Qemu.
>    - upstream the Linux frontend/backend drivers. Current work-in-progress
versions are in Konrad''s git tree.
>    - James Harper''s GPLPV drivers for Windows include PVUSB
frontend drivers.
>
>  * xl USB pass-through for HVM guests using Qemu USB emulation
>    owner: ?
>    status: ?
>    - xm/xend with qemu-traditional supports USB passthrough to HVM guests
using the Qemu emulated USB controller.
>    - port the xm/xend functionality to xl.
>    - make sure USB passthrough with xl works with both qemu-traditional and
qemu-upstream.
>    - The HVM guest does not need any special drivers for this feature.
>
>
> -- Pasi
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

On Mon, Oct 1, 2012 at 5:25 PM, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> * PVH mode, domU (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
>
> * PVH mode, dom0 (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
Mukesh, have the xen-side patches been posted yet?  Any ETA on those?

Thanks,
 -George

>>> On 01.10.12 at 18:25, George Dunlap
<George.Dunlap@eu.citrix.com> wrote:
> * CPUID-based idle (don''t rely on ACPI info f/ dom0)
>   owner: jan@suse
>   status: done, to be submitted
Committed and hence done.

Jan

On Tue, Oct 02, 2012 at 10:26:45AM +0100, George Dunlap
wrote:
> On Mon, Oct 1, 2012 at 5:42 PM, Pasi Kärkkäinen <pasik@iki.fi> wrote:
> > On Mon, Oct 01, 2012 at 05:25:43PM +0100, George Dunlap wrote:
> >>
> >> * xl USB pass-through for PV guests
> >>   owner: ?
> >>   status: ?
> >>   - Port the xend PV pass-through functionality to xl.
> >>   - Make sure qemu-based USB with qemu-upstream works
> >>   - Upstream the Linux frontend/backend drivers
> >>
> >
> > I think this should be split into two separate tasks:
> 
> That does make sense.  I''ve updated my notes accordingly.
> 
Thanks.
> Just to verify: xm/xend *already* supports both PVUSB and passthrough via
QEMU?
>
Correct!

-- Pasi

 
>  -George
> 
> >
> >  * xl PVUSB pass-through for both PV and HVM guests
> >    owner: ?
> >    status: ?
> >    - xm/xend supports PVUSB pass-through to guests with PVUSB drivers
(both PV and HVM guests).
> >    - port the xm/xend functionality to xl.
> >    - this PVUSB feature does not require support or emulation from
Qemu.
> >    - upstream the Linux frontend/backend drivers. Current
work-in-progress versions are in Konrad''s git tree.
> >    - James Harper''s GPLPV drivers for Windows include PVUSB
frontend drivers.
> >
> >  * xl USB pass-through for HVM guests using Qemu USB emulation
> >    owner: ?
> >    status: ?
> >    - xm/xend with qemu-traditional supports USB passthrough to HVM
guests using the Qemu emulated USB controller.
> >    - port the xm/xend functionality to xl.
> >    - make sure USB passthrough with xl works with both
qemu-traditional and qemu-upstream.
> >    - The HVM guest does not need any special drivers for this feature.
> >
> >
> > -- Pasi
> >
> >
> > _______________________________________________
> > Xen-devel mailing list
> > Xen-devel@lists.xen.org
> > http://lists.xen.org/xen-devel

Pasi Kärkkäinen writes ("Re: [Xen-devel] Xen 4.3 development
update"):
>  * xl PVUSB pass-through for both PV and HVM guests
>    owner: ? 
>    status: ? 
>    - xm/xend supports PVUSB pass-through to guests with PVUSB drivers (both
PV and HVM guests).
>    - port the xm/xend functionality to xl.
>    - this PVUSB feature does not require support or emulation from Qemu.
>    - upstream the Linux frontend/backend drivers. Current work-in-progress
versions are in Konrad''s git tree.
>    - James Harper''s GPLPV drivers for Windows include PVUSB
frontend drivers.
Right.  I think upstreaming the frontend/backend is critical for
this.  Is this related to the usb-over-ip support that I found a few
years ago in linux-staging ?  At the time that was very buggy.
>  * xl USB pass-through for HVM guests using Qemu USB emulation
>    owner: ? 
>    status: ?
>    - xm/xend with qemu-traditional supports USB passthrough to HVM guests
using the Qemu emulated USB controller.
>    - port the xm/xend functionality to xl.
>    - make sure USB passthrough with xl works with both qemu-traditional and
qemu-upstream.
>    - The HVM guest does not need any special drivers for this feature.
I think this is just a matter of plumbing in libxl (presumably, now,
plumbing to qemu-upstream).

Ian.

On Tue, 2012-10-02 at 15:26 +0100, Ian Jackson wrote:
> Pasi Kärkkäinen writes ("Re: [Xen-devel] Xen 4.3 development
update"):
> >  * xl PVUSB pass-through for both PV and HVM guests
> >    owner: ? 
> >    status: ? 
> >    - xm/xend supports PVUSB pass-through to guests with PVUSB drivers
(both PV and HVM guests).
> >    - port the xm/xend functionality to xl.
> >    - this PVUSB feature does not require support or emulation from
Qemu.
> >    - upstream the Linux frontend/backend drivers. Current
work-in-progress versions are in Konrad's git tree.
> >    - James Harper's GPLPV drivers for Windows include PVUSB
frontend drivers.
> 
> Right.  I think upstreaming the frontend/backend is critical for
> this.  Is this related to the usb-over-ip support that I found a few
> years ago in linux-staging ?  At the time that was very buggy.
I don't think so, it's the Xen PV USB protocol defined in
xen/include/public/io/usbif.h.
> >  * xl USB pass-through for HVM guests using Qemu USB emulation
> >    owner: ? 
> >    status: ?
> >    - xm/xend with qemu-traditional supports USB passthrough to HVM
guests using the Qemu emulated USB controller.
> >    - port the xm/xend functionality to xl.
> >    - make sure USB passthrough with xl works with both
qemu-traditional and qemu-upstream.
> >    - The HVM guest does not need any special drivers for this feature.
> 
> I think this is just a matter of plumbing in libxl (presumably, now,
> plumbing to qemu-upstream).
Assuming qemu-upstream has the necessary feature...

Ian.



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Tue, Oct 02, 2012 at 03:26:11PM +0100, Ian Jackson
wrote:
> Pasi Kärkkäinen writes ("Re: [Xen-devel] Xen 4.3 development
update"):
> >  * xl PVUSB pass-through for both PV and HVM guests
> >    owner: ? 
> >    status: ? 
> >    - xm/xend supports PVUSB pass-through to guests with PVUSB drivers
(both PV and HVM guests).
> >    - port the xm/xend functionality to xl.
> >    - this PVUSB feature does not require support or emulation from
Qemu.
> >    - upstream the Linux frontend/backend drivers. Current
work-in-progress versions are in Konrad''s git tree.
> >    - James Harper''s GPLPV drivers for Windows include PVUSB
frontend drivers.
> 
> Right.  I think upstreaming the frontend/backend is critical for
> this.
>
Indeed!
>  Is this related to the usb-over-ip support that I found a few
> years ago in linux-staging ?  At the time that was very buggy.
>
I don''t think so. PVUSB is a separate protocol/feature afaik.

Xen PVUSB was originally developed by Fujitsu:
http://www.xen.org/files/xensummit_oracle09/PVUSB.pdf
http://www.xen.org/files/xensummit_intel09/PVUSBStatusUpdate.pdf

more info:
http://wiki.xen.org/wiki/Xen_USB_Passthrough

 
> >  * xl USB pass-through for HVM guests using Qemu USB emulation
> >    owner: ? 
> >    status: ?
> >    - xm/xend with qemu-traditional supports USB passthrough to HVM
guests using the Qemu emulated USB controller.
> >    - port the xm/xend functionality to xl.
> >    - make sure USB passthrough with xl works with both
qemu-traditional and qemu-upstream.
> >    - The HVM guest does not need any special drivers for this feature.
> 
> I think this is just a matter of plumbing in libxl (presumably, now,
> plumbing to qemu-upstream).
>
Yep, this one should be pretty easy!

-- Pasi

On Tue, Oct 02, 2012 at 03:32:41PM +0100, Ian Campbell
wrote:
> 
> > >  * xl USB pass-through for HVM guests using Qemu USB emulation
> > >    owner: ? 
> > >    status: ?
> > >    - xm/xend with qemu-traditional supports USB passthrough to
HVM guests using the Qemu emulated USB controller.
> > >    - port the xm/xend functionality to xl.
> > >    - make sure USB passthrough with xl works with both
qemu-traditional and qemu-upstream.
> > >    - The HVM guest does not need any special drivers for this
feature.
> > 
> > I think this is just a matter of plumbing in libxl (presumably, now,
> > plumbing to qemu-upstream).
> 
> Assuming qemu-upstream has the necessary feature...
> 
Yes, qemu-upstream supports host USB device pass-through.

references:
http://www.linux-kvm.org/page/USB_Host_Device_Assigned_to_Guest
http://david.wragg.org/blog/2009/03/using-usb-pass-through-under-libvirt.html
http://david.wragg.org/blog/2009/03/usb-pass-through-with-libvirt-and-kvm.html

So basicly the qemu cmdline needs to have:
-usb -usbdevice host:xxxx:yyyy


-- Pasi

On Tue, 2012-10-02 at 15:50 +0100, Pasi Kärkkäinen
wrote:
> On Tue, Oct 02, 2012 at 03:32:41PM +0100, Ian Campbell wrote:
> > 
> > > >  * xl USB pass-through for HVM guests using Qemu USB
emulation
> > > >    owner: ? 
> > > >    status: ?
> > > >    - xm/xend with qemu-traditional supports USB passthrough
to HVM guests using the Qemu emulated USB controller.
> > > >    - port the xm/xend functionality to xl.
> > > >    - make sure USB passthrough with xl works with both
qemu-traditional and qemu-upstream.
> > > >    - The HVM guest does not need any special drivers for
this feature.
> > > 
> > > I think this is just a matter of plumbing in libxl (presumably,
now,
> > > plumbing to qemu-upstream).
> > 
> > Assuming qemu-upstream has the necessary feature...
> > 
> 
> Yes, qemu-upstream supports host USB device pass-through.
> 
> references:
> http://www.linux-kvm.org/page/USB_Host_Device_Assigned_to_Guest
>
http://david.wragg.org/blog/2009/03/using-usb-pass-through-under-libvirt.html
>
http://david.wragg.org/blog/2009/03/usb-pass-through-with-libvirt-and-kvm.html
> 
> So basicly the qemu cmdline needs to have:
> -usb -usbdevice host:xxxx:yyyy
If you do this via xl's device_model_args= config option then does this
Just Work?

Ian.


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel

On Tue, Oct 02, 2012 at 03:56:02PM +0100, Ian Campbell
wrote:
> On Tue, 2012-10-02 at 15:50 +0100, Pasi Kärkkäinen wrote:
> > On Tue, Oct 02, 2012 at 03:32:41PM +0100, Ian Campbell wrote:
> > > 
> > > > >  * xl USB pass-through for HVM guests using Qemu USB
emulation
> > > > >    owner: ? 
> > > > >    status: ?
> > > > >    - xm/xend with qemu-traditional supports USB
passthrough to HVM guests using the Qemu emulated USB controller.
> > > > >    - port the xm/xend functionality to xl.
> > > > >    - make sure USB passthrough with xl works with both
qemu-traditional and qemu-upstream.
> > > > >    - The HVM guest does not need any special drivers
for this feature.
> > > > 
> > > > I think this is just a matter of plumbing in libxl
(presumably, now,
> > > > plumbing to qemu-upstream).
> > > 
> > > Assuming qemu-upstream has the necessary feature...
> > > 
> > 
> > Yes, qemu-upstream supports host USB device pass-through.
> > 
> > references:
> > http://www.linux-kvm.org/page/USB_Host_Device_Assigned_to_Guest
> >
http://david.wragg.org/blog/2009/03/using-usb-pass-through-under-libvirt.html
> >
http://david.wragg.org/blog/2009/03/usb-pass-through-with-libvirt-and-kvm.html
> > 
> > So basicly the qemu cmdline needs to have:
> > -usb -usbdevice host:xxxx:yyyy
> 
> If you do this via xl''s device_model_args= config option then does
this
> Just Work?
> 
It should work. Someone has to try it.. 
(I''m currently away from the machine where I could test this.)

-- Pasi

On Tue, 2 Oct 2012 10:29:21 +0100
George Dunlap <George.Dunlap@eu.citrix.com> wrote:
> On Mon, Oct 1, 2012 at 5:25 PM, George Dunlap
> <George.Dunlap@eu.citrix.com> wrote:
> > * PVH mode, domU (w/ Linux)
> >   owner: mukesh@oracle
> >   status: ?
> >
> > * PVH mode, dom0 (w/ Linux)
> >   owner: mukesh@oracle
> >   status: ?
> 
> Mukesh, have the xen-side patches been posted yet?  Any ETA on those?
> 
> Thanks,
>  -George
I had posted an informal patch few months ago. My xen tree is somewhat 
old now. So, as soon as I am done with linux patches, I''ll refresh my
xen tree, (I''m sure there''ll be some debugging) test it out,
and submit
the patch ASAP.

thanks,
Mukesh

On Wed, 2012-09-26 at 03:07 +0100, Jim Fehlig wrote:
> George Dunlap wrote:
> > On 20/09/12 22:20, Jim Fehlig wrote:
> >> Chun Yan Liu was working on a patch to implement migration in the
> >> libvirt libxl
> >> driver, but she wasn''t able to finish tidying the patch
before
> >> departing on
> >> maternity leave.  She recently returned to work and I''d
expect a
> >> refreshed patch
> >> soon - after working through her backlog.
> >>
> >> But why is libvirt integration listed as a Xen 4.3 feature? 
AFAIK,
> >> all of this
> >> work is in libvirt and really doesn''t have much to do
with the
> >> release of 4.3.
> > For one, as Ian said, we want xl/libxl to be the default toolstack in
> > 4.3; as soon as possible we want to remove xend.
> 
> Good :).
> 
> >
> > But from a broader perspective, the idea is for the Xen.org community
> > to be thinking strategically about the success of the Xen project as a
> > whole.  Things like distro support and libvirt integration are
> > absolutely critical to Xen''s success in the open-source
ecosystem.  So
> > while we don''t have to do everything ourselves, we want to
think about
> > what things need to be done, track them to make sure that *someone*
> > does them; and be prepared to pick them up ourselves if necessary. 
> > And it just makes more sense to track everything on one list.
> >
> > Does that make sense?
> 
> Yes, it does.  I''m actually quite happy to hear this statement of
> support for the broader ecosystem!
BTW, I''ve been meaning to ask, and it''s kind of the flipside
of this so
I''ll mention it here -- would it be possible for you (or whoever is
sending patches) to CC xen-devel with Xen related patches to libvirt?

Partly just to raise the visibility of the libvirt integration on the
Xen side but also at least I am interested to see how libxl is being
used outside of xl and what sorts of issues etc you guys are having. I
suppose there would also be other benefits to review from libxl folks of
patches which consume the APIs.

Thanks,
Ian.
> 
> WRT libvirt support for libxl in Xen >= 4.2, SUSE will be contributing
> some work.  Just today I had to disable the driver in our Factory
> libvirt builds where we now have Xen 4.2.  This hack can''t last
for long.
> 
> Regards,
> Jim
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel

On 19/09/12 18:58, George Dunlap wrote:
> = Timeline > 
> We are planning on a 9-month release cycle.  Based on that, below are
> our estimated dates:
> * Feature Freeze: 1 March, 2013
> * First RC: 15 April 2013
> * Release: 1 June 2013
> 
> The RCs and release will of course depend on stability and bugs, and
> will therefore be fairly unpredictable.  The feature freeze may be
> slipped for especially important features which are near completion.
> 
> = Feature tracking > 
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status
> is "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
> 
> There are a number of items whose owners are marked as "?".  If
you
> are working on this, or know who is working on it, please respond and
> let me know.  Alternately, if you would *like* to work on it, please
> let me know as well.
> 
> And if there is something you''re working on you''d like
tracked, please
> respond, and I will add it to the list.
> 
> 
> * PVH mode, domU (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
> 
> * PVH mode, dom0 (w/ Linux)
>   owner: mukesh@oracle
>   status: ?
> 
> * Event channel scalability
>   owner: attilio@citrix
>   status: ?
>   Increase limit on event channels (currently 1024 for 32-bit guests,
>   4096 for 64-bit guests)
> 
> * ARM server port
>   owner: ijc@citrix
>   status: Core hypervisor patches accepted; Linux paches pending
> 
> * NUMA scheduler affinity
>   critical
>   owner: dario@citrix
>   status: ?
> 
> * NUMA Memory migration
>   owner: dario@citrix
>   status: ?
> 
> * blktap3
>   owner: thanos@citrix
>   status: ?
> 
> * Default to QEMU upstream
>  - qemu-based stubdom (Linux or BSD libc)
>    owner: anthony@citrix
>    status: ?
>    qemu-upstream needs a more fully-featured libc than exists in
>    minios.  Either work on a minimalist linux-based stubdom with
>    glibc, or port one of the BSD libcs to minios.
> 
>  - pci pass-thru
>    owner: anthony@citrix
>    status: ?
> 
> * Persistent grants
>   owner: @citrix
>   status: ?
I''m taking Oliver''s patch, so:

owner: roger.pau@citrix
status: in progress
> 
> * Multi-page blk rings
>  - blkback in kernel (konrad@oracle, ?@intel)
>  - qemu blkback
>   status: ?
> 
> * Multi-page net protocol
>   owner: ?
>   status: ?
>   expand the network ring protocol to allow multiple pages for
>   increased throughput
> 
> * Scalability: 16TiB of RAM
>   owner: jan@suse
>   status: ?
> 
> * libvirt integration
>   owner: ?
>   status: ?
>   To begin with, we need someone to go and make some lists:
>   - Features available in libvirt/KVM not available in libvirt/Xen
>   - Features available in xl/Xen but not available in libvirt/Xen
> 
> * xl vm-{export,import}
>   owner: ?
>   status: ?
>   Allow xl to import and export VMs to other formats; particularly
>   ovf, perhaps the XenServer format, or more.
> 
> 
> * xl USB pass-through for PV guests
>   owner: ?
>   status: ?
>   - Port the xend PV pass-through functionality to xl.
>   - Make sure qemu-based USB with qemu-upstream works
>   - Upstream the Linux frontend/backend drivers
> 
> * openvswitch toostack integration
>   owner: roger@citrix
>   status: Sample script posted by Bastian ("[RFC] openvswitch support
script")
> 
> * Rationalized backend scripts (incl. driver domains)
>   owner: roger@citrix
>   status: ?
> 
> * Linux console improvements
>   owner: jan@suse
>   -EHCI debug port (done, to be submitted)
>   -xHCI debug port
>   -Firewire
> 
> * CPUID-based idle (don''t rely on ACPI info f/ dom0)
>   owner: jan@suse
>   status: done, to be submitted
> 
> * Remove hardcoded mobprobe''s in xencommons
>   owner: ?
>   status: ?
> 
> * Make storage migration possible
>   owner: ?
>   status: ?
>   There needs to be a way, either via command-line or via some hooks,
>   that someone can build a "storage migration" feature on top of
libxl
>   or xl.
> 
> * Full-VM snapshotting
>   owner: ?
>   status: ?
>   Have a way of coordinating the taking and restoring of VM memory and
>   disk snapshots.  This would involve some investigation into the best
>   way to accomplish this.
> 
> * VM Cloning
>   owner: ?
>   status: May need review
>   Again, a way of coordinating the memory and disk aspects.  Research
>   into the best way to do this would probably go along with the
>   snapshotting feature.
> 
> * Memory: Replace PoD with paging mechanism
>   owner: george@citrix
>   status: May need review
> 
> * PV audio (audio for stubdom qemu)
>   owner: stefano.panella@citrix
>   status: ?
> 
> * Managed domains?
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
>

Ian Campbell wrote:
> BTW, I''ve been meaning to ask, and it''s kind of the
flipside of this so
> I''ll mention it here -- would it be possible for you (or whoever
is
> sending patches) to CC xen-devel with Xen related patches to libvirt?
>   
Sure.
> Partly just to raise the visibility of the libvirt integration on the
> Xen side but also at least I am interested to see how libxl is being
> used outside of xl and what sorts of issues etc you guys are having. I
> suppose there would also be other benefits to review from libxl folks of
> patches which consume the APIs.
>   
Agreed.  You could set us straight on any misuse of the API :).

Regards,
Jim

On Thu, 2012-10-04 at 23:29 +0100, Jim Fehlig wrote:
> Ian Campbell wrote:
> > BTW, I''ve been meaning to ask, and it''s kind of the
flipside of this so
> > I''ll mention it here -- would it be possible for you (or
whoever is
> > sending patches) to CC xen-devel with Xen related patches to libvirt?
> >   
> 
> Sure.
Thanks!
> > Partly just to raise the visibility of the libvirt integration on the
> > Xen side but also at least I am interested to see how libxl is being
> > used outside of xl and what sorts of issues etc you guys are having. I
> > suppose there would also be other benefits to review from libxl folks
of
> > patches which consume the APIs.
> >   
> 
> Agreed.  You could set us straight on any misuse of the API :).
I think it''s equally likely that we''ll realise that our APIs
are prone
to easy misuse, but hopefully we''ll meet somewhere in the middle ;-)

Cheers,
Ian.

On Wed, 2012-09-19 at 17:58 +0100, George Dunlap wrote:
> = Feature tracking > 
> Below is a list of features we''re tracking for this release. 
Please
> respond to this mail with any updates to the status.  If the status
> is "?", as most of them are, please respond with the status! 
Example
> statuses include "not started", "in progress",
"initial implementation
> submitted for review", "patches submitted".
>
> ...
> * NUMA scheduler affinity
>   critical
>   owner: dario@citrix
>   status: ?
> 
Patches sent. Waiting for review.
> * NUMA Memory migration
>   owner: dario@citrix
>   status: ?
> 
Work starting in the upcoming weeks.

Thanks and Regards,
Dario

-- 
<<This happens because I choose it to happen!>> (Raistlin Majere)
-----------------------------------------------------------------
Dario Faggioli, Ph.D, http://retis.sssup.it/people/faggioli
Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK)



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
http://lists.xen.org/xen-devel