Xen users - Oct 2011 - debian squeeze, xen and a very big problem.

Hello, first of all sorry for my english.
Months ago I''ve installed a debian squeeze into two servers with a SAN
storage.
I''ve installed the package xen-linux-system, xen started and I could
create various domUs, live migrate them from one node to another and
everything without any problem.
The debian version was 6.0.2.
Today I''ve run aptitude and upgraded some packages as suggested.
Now the system has debian version 6.0.3.
Xen is 4.0.1.
I''ve upgraded also the domuS, all of them are debian squeeze machines.
By now the live migration doesn''t work anymore.
Here is what xend.log says:

[2011-10-13 17:59:53 11629] INFO (XendCheckpoint:423) ERROR Internal
error: Error when reading batch size
[2011-10-13 17:59:53 11629] INFO (XendCheckpoint:423) ERROR Internal
error: error when buffering batch, finishing

It''s a very big problem for me.
I can''t believe that a serious distribution like debian do not do
tests before suggest an upgrade.
There are others here with squeeze and xen that have the same problem?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

While I understand your position, you should have tested very
thoroughly before deploying any upgrades to a production system.
Screeching about Debian being bad is not going to garner you any
sympathy if you didn''t test your patches in a lab or QA environment
before upgrading your system.

That is my two cents.

On Thu, Oct 13, 2011 at 2:45 PM, Mauro <mrsanna1@gmail.com>
wrote:
> Hello, first of all sorry for my english.
> Months ago I''ve installed a debian squeeze into two servers with a
SAN storage.
> I''ve installed the package xen-linux-system, xen started and I
could
> create various domUs, live migrate them from one node to another and
> everything without any problem.
> The debian version was 6.0.2.
> Today I''ve run aptitude and upgraded some packages as suggested.
> Now the system has debian version 6.0.3.
> Xen is 4.0.1.
> I''ve upgraded also the domuS, all of them are debian squeeze
machines.
> By now the live migration doesn''t work anymore.
> Here is what xend.log says:
>
> [2011-10-13 17:59:53 11629] INFO (XendCheckpoint:423) ERROR Internal
> error: Error when reading batch size
> [2011-10-13 17:59:53 11629] INFO (XendCheckpoint:423) ERROR Internal
> error: error when buffering batch, finishing
>
> It''s a very big problem for me.
> I can''t believe that a serious distribution like debian do not do
> tests before suggest an upgrade.
> There are others here with squeeze and xen that have the same problem?
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On 13 October 2011 21:56, Scott Damron <sdamron@gmail.com>
wrote:
> While I understand your position, you should have tested very
> thoroughly before deploying any upgrades to a production system.
> Screeching about Debian being bad is not going to garner you any
> sympathy if you didn''t test your patches in a lab or QA
environment
> before upgrading your system.
How can I downgrade the xen linux system kernel?

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> How can I downgrade the xen linux system kernel?
Use http://snapshot.debian.org/
Guy

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mauro wrote:
>I can''t believe that a serious distribution like debian do not do
>tests before suggest an upgrade.
As Scott says, for production systems you either have to test things 
yourself, or you accept that possibly an upgrade may break your 
system. Debian (and the other distributions) do do a **LOT** of 
testing - they have a whole QA setup just to handle testing etc.

BUT they rely on the package maintainers/contributors to do most of 
the functional testing - the distribution testing is (I believe) 
mostly to ensure everything installs cleanly, there aren''t broken 
dependencies, etc.

The thing is, once you get past the trivial (and both the Linux 
kernel and Xen are very much non-trivial) it is simply impossible to 
test every possible combination of software and usage cases. That''s 
not "difficult", but "impossible" - unless you have a near
infinite
number of systems and people to run the test scenarios.

I understand how frustrating it is for you, I''ve been hit by this 
sort of thing before. You either have to have a test/QA setup and 
test all upgrades before going live, or you upgrade live systems and 
have a plan to backtrack what you upgraded. The last one I had was an 
incompatibility between (IIRC) Squirrelmail and (IIRC) one of the 
Perl elements it depends on - leading to inability to log in as a 
user. I had to downgrade the bits I''d upgraded, then upgrade them one 
at a time until I found the problem. Fortunately it got fixed at some 
point.


Guy Roussin wrote:
>Use http://snapshot.debian.org/
Wow, I wish I''d known about that before - several times I''ve
found
myself needing older packages and struggled to find them.

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

"You need to run your own test servers if you want a working update
since we don''t test stuff like live migration (that is most basic
functionality)" makes me chuckle a bit. 2 test servers can buy
helluvah support contract, where the issues would probably be
addressed quickly. Shouldn''t it be that the user tests anything that
is related to oddness of his own setup and a distro tests all that is
basic functionality?

If a upgrade from minor release A to minor release B fails like that
it''s perfectly OK to come forward and say "OK that must be a
regression, sorry. Please open a bug immediately" instead of yet
another round of
Q: Xen issue in Debian
A: "blame and flame the user"


More practical:
Is there any place a user could look up which tests *are* being run continuosly?
Then a end user would have a chance to see if something he would need
is missing and define the set of additional tests he needs to do, plus
also report the results back.


Florian

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Friday, October 14, 2011 11:59:23 PM Florian Heigl
wrote:
> "You need to run your own test servers if you want a working update
> since we don''t test stuff like live migration (that is most basic
> functionality)" makes me chuckle a bit. 2 test servers can buy
> helluvah support contract, where the issues would probably be
> addressed quickly. Shouldn''t it be that the user tests anything
that
> is related to oddness of his own setup and a distro tests all that is
> basic functionality?
> 
> If a upgrade from minor release A to minor release B fails like that
> it''s perfectly OK to come forward and say "OK that must be a
> regression, sorry. Please open a bug immediately" instead of yet
> another round of
> Q: Xen issue in Debian
> A: "blame and flame the user"
Totally agree. However, we cannot trust very much in this industry.
It''s just
the way it is. With my Xen cluster I would do so:

DomU upgrades:
* Clone the machine and perform the upgrade on the clone and test.
* If it goes well, clone the machine for backup, upgrade the production 
machine.
* If things go wrong, you have a back up.

There may be a problem with loss of data with the above method. If everything 
goes well for a few days on the new system. Data is modified. The backup is 
useless unless you also consider backups of just the data partition/volume and 
try use that with the old system.

Dom0 upgrades:
* Migrate DomUs off of Dom0 being upgraded.
* Copy them back possibly starting with a test machine.
* Test them.
* Migrate them back.

Xen itself empowers us to do safer upgrades to our machines. It is one of the 
reasons for using it.

If you can only afford one Dom0, then things get trickier, but it''s
still
possible to be safer than doing upgrades to a production server w/o testing.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users