Xen users - Sep 2012 - Severe dom0 performance issues after installing Xen

Hello,

I''m experiencing some severe performance issues running Xen 4.0.1 with
a
cleanly-installed Debian 6 dom0.  Prior to installation, the machine''s 
performance was fine, but after installation booting takes anywhere from 
5 to 20 times longer, processes take a long time to start, and SSH 
lags.  No domUs are running (or have been created yet).  I can switch 
back to running without Xen (but still using the Xen-compatible kernel) 
and the performance problems disappear.

I attempted to dig into the issue further, but haven''t been able to
find
any reason why the machine is so slow.  Some cursory CPU integer 
performance and memory bandwidth benchmarks I ran showed that there is 
almost no difference in performance, and disk only loses about 10% of 
its performance.  I also haven''t found anything particularly alarming
in
dmesg or xm dmesg (although this is my first time installing Xen, so I 
am not sure I would know what to look for).  top and xm top don''t show 
anything constantly consuming all of the system''s resources, and the 
only additional service I''ve installed so far is ntpd.  As an
additional
debugging step, I''ve attempted limiting the dom0 memory to 1536MB (the 
machine has 4GB of physical memory).  I haven''t managed to turn up 
anything about this on Google yet.

Has anyone seen a similar issue before or have ideas on how to fix/debug 
it?  I have no physical access to the machine, unfortunately, so I am 
unable to play with BIOS settings.  I''ve attached the contents of dmesg
and xm dmesg, should these be useful.

Thanks,
Keith



_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

On Tuesday 04 Sep 2012, Keith Buck wrote:
> I''m experiencing some severe performance issues running Xen 4.0.1
> with a cleanly-installed Debian 6 dom0.  Prior to installation, the
> machine''s performance was fine, but after installation booting
takes
> anywhere from 5 to 20 times longer, processes take a long time to
> start, and SSH lags.  No domUs are running (or have been created
> yet).  I can switch back to running without Xen (but still using the
> Xen-compatible kernel) and the performance problems disappear.
Probably a stupid question, but sometime we miss the obvious: did you 
make any networking changes?  Specifically, could this problem be due to 
not being able to reverse resolve IP addresses on your machine?

Regards,

-- Raj
-- 
Raj Mathur                          || raju@kandalaya.org   || GPG:
http://otheronepercent.blogspot.com || http://kandalaya.org || CC68
It is the mind that moves           || http://schizoid.in   || D17F

On Mon, Sep 3, 2012 at 7:00 PM, Raj Mathur (राज माथुर)
<raju@linux-delhi.org
> wrote:
> On Tuesday 04 Sep 2012, Keith Buck wrote:
> > I''m experiencing some severe performance issues running Xen
4.0.1
> > with a cleanly-installed Debian 6 dom0.  Prior to installation, the
> > machine''s performance was fine, but after installation
booting takes
> > anywhere from 5 to 20 times longer, processes take a long time to
> > start, and SSH lags.  No domUs are running (or have been created
> > yet).  I can switch back to running without Xen (but still using the
> > Xen-compatible kernel) and the performance problems disappear.
>
> Probably a stupid question, but sometime we miss the obvious: did you
> make any networking changes?  Specifically, could this problem be due to
> not being able to reverse resolve IP addresses on your machine?
>
> I haven''t made any networking changes.  The SSH latency
isn''t just logging
in, there''s also latency echoing back what I type at the prompt and
displaying the output of programs.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users

...not shure what's going on there, but would try following things:

- try to boot with kernel options noapic
- try to pin one CPU directly to Dom0 and set max cpu to 1 on Dom0 (xen and
kernel options)
- try to boot with disabled acpi etc.
- try to find out "what" is slow, i.e. I/O, CPU, net etc and what not

Is there any "point" to recognize during boot when the system
"got slow"?


Best regards,


Niels.



"Raj Mathur (राज माथुर)" <raju@linux-delhi.org> schrieb:
>On Tuesday 04 Sep 2012, Keith Buck wrote:
>> I'm experiencing some severe performance issues running Xen 4.0.1
>> with a cleanly-installed Debian 6 dom0.  Prior to installation, the
>> machine's performance was fine, but after installation booting
takes
>> anywhere from 5 to 20 times longer, processes take a long time to
>> start, and SSH lags.  No domUs are running (or have been created
>> yet).  I can switch back to running without Xen (but still using the
>> Xen-compatible kernel) and the performance problems disappear.
>
>Probably a stupid question, but sometime we miss the obvious: did you 
>make any networking changes?  Specifically, could this problem be due
>to 
>not being able to reverse resolve IP addresses on your machine?
>
>Regards,
>
>-- Raj
>-- 
>Raj Mathur                          || raju@kandalaya.org   || GPG:
>http://otheronepercent.blogspot.com || http://kandalaya.org || CC68
>It is the mind that moves           || http://schizoid.in   || D17F
>
>____________________________________________
-- 
Niels Dettenbach
Syndicat IT&Internet
http://www.syndicat.com


_______________________________________________
Xen-users mailing list
Xen-users@lists.xen.org
http://lists.xen.org/xen-users