CentOS virt - Feb 2016 - Proposal: Version-specific centos-release-xen-NN packages

Thank you for those who raised the issue of upgrading across
potentially disruptive major versions (like the Xen 4.4 -> 4.6
update), and for everyone who weighed in.

After discussing options with users here on this list (as well as in
the IRC channel), further discussing things in the Virt SIG IRC
meeting, and also raising the issue with other SIGs on the
centos-devel list, here are my thoughts.  (Skip down to the "Proposal"
sections for the TLDR version.)

= The issue 
To summarize the issue:

* Some users want to get whatever the latest supported version of Xen
automatically via yum, without having to do anything but a "yum
update".  They want "yum update" to give them a new major version
when
one is available.  We can call this group "auto-upgrade", because they
expect a major version upgrade to happen automatically.

* Other users want "yum update" to have fairly strong guarantees not
to break their systems -- and thus want "yum update" *not* to give
them a new major version, but want to have to manually do something
specific (such as installing a different package) to get the updated
version.  Or at very least, they expect not to get a new major version
that is expected to break things.  We can call this group
"manual-upgrade", because they want to manually upgrade major
versions.

* At the moment, the Virt SIG is only committed to making security
updates to one version of Xen at a time

* The 4.4 -> 4.6 update may be particularly disruptive for some users,
since xend/xm will no longer be available.

* Many users may not follow this mailing list, so may not know about
the impending update

* There are new XSAs coming out in 2 days (17 February).

There are basically two "problems" to solve.

The first thing to look at is what we would like things to look
long-term, given that some users want auto-upgrade and some users want
manual-upgrade.

The second thing to look at is what to do in the current situation,
given that we have users from both camps with the same set of packages
installed, who may not read this mailing list.

A couple of goals (some of which are on conflict):

* We'd like to support both types of users

* We want to minimize breakage for people who do don't follow this
list, and do "yum update" without noticing the major upgrade that gets
pushed out.

* We want to minimize the security exposure for people who don't
follow this list, and who don't notice that there haven't been any
updates in response to XSAs.

Additionally, we'd like if possible to be able to make it possible for
anyone who wants to maintain older versions of Xen to do so in
parallel with the newer versions.

= Proposal: Long term 
Going forward, this is what I propose we aim for:

* Have separate repos for each version of Xen; at the present, this
means xen-44 and xen-46.

* Have version-specific centos-release-xen-NN pakcages which will
always point to that version of the repo.  So centos-release-xen-44
will point to xen-44, centos-release-xen-46 will point to xen-46.
Users in the "manual-upgrade" camp can install one of these versions,
knowing that a simple "yum update" will never move them across
versions.  When they're ready to upgrade, they can just install the
newer version; yum will choose the newest version from all the repos
available to it.

* Have a generic centos-release-xen package which will always point to
the most recent 'released' Virt SIG version of Xen.  Users in the
"auto-upgrade" camp can install this version, knowing that they'll
always get the version with the latest security fixes.

This will also make it simple for community members to step up and
support older versions if they desire.

= Discussion: Short term 
For those who do not follow the list, we have to essentially choose
one update policy for them.  Will the current users expecting
"manual-upgrade" have "auto-upgrade" imposed on them
(potentially
breaking on update)?  Or will the current users expecting
"auto-upgrade" have "manual-upgrade" imposed on them
(potentially
missing important security updates)?

For someone to be negatively affected by the "manual-upgrade" choice,
they only have to be running Xen in any configuration.

For someone to be negatively affected by the "auto-upgrade" choice,
they have to
* still be running xend (in spite of having a warning printed on every
xm command invocation for the last 18 months)
* update their systems without checking / noticing the new major version

Even then, they should have the option of downgrading with "yum
downgrade"; there should only be major carnage if they automatically
update large numbers of servers at once.

When I brought this question up with the other SIGs, the general
sentiment was that, as community-run projects, we do not have nearly
the testing effort required to make it possible for users to simply
run "yum update" without checking first.

So on the whole, I think the aggregate risk is minimized if we go with
the "auto-upgrade" option for those who don't specifically make a
choice.

= Proposal: Short term 
* I'll try to get the new xen-NN repos, along with
centos-release-xen-NN packages up and working tomorrow (Tuesday)

* Wednesday I'll push updates for the 4.6 packages to the xen-46 repos.

* centos-release-xen will point to the xen-44 repos for two weeks.
During that time, those who want the XSA updates will have to install
centos-release-xen-46.  Those who don't want to update to 4.6 should
install the centos-release-xen-44 package *and remove* the
centos-release-xen package.

* After those two weeks, centos-release-xen will switch to the xen-46
repos, giving the XSA updates to everyone who hasn't yet updated.

I know this doesn't make everyone happy, but I've tried to make the
best of a tricky situation.  Thank you all for the input you've given.

 -George