Hi.
I entered one of our domU tonight and see following problem:
# iowait -k 5
avg-cpu: %user %nice %system %iowait %steal %idle
0.00 0.00 0.00 100.00 0.00 0.00
Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn
sda1 0.00 0.00 0.00 0 0
sda2 0.00 0.00 0.00 0 0
I check other domU and dom0 on same server - there were high iowait: from 20% to
50% with few transactions per second and about 0.1-1.5 Mb/sec storage bandwith
usage per domU. This domU have no disk IO at all but show LA 15 and possible
higher.
We have 6 domU on 8-core server with 7 FUJITSU MBB2147RC SAS HDD on DELL PERC
5/i RAID controller.
CentOS 5
Xen 3.1.0
Kernel 2.6.21
We suffer from this problem for a while on many our xen hosts, it is not a
single-time issue.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
Tomasz Chmielewski
2008-May-08 09:06 UTC
Re: [Xen-users] 100% iowait in domU with no IO tasks.
Bogdan B. Rudas schrieb:> Hi. > > I entered one of our domU tonight and see following problem: > > # iowait -k 5 > > avg-cpu: %user %nice %system %iowait %steal %idle > > 0.00 0.00 0.00 100.00 0.00 0.00 > > Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn > sda1 0.00 0.00 0.00 0 0 > sda2 0.00 0.00 0.00 0 0 > > I check other domU and dom0 on same server - there were high iowait: from 20% to 50% with few transactions per second and about 0.1-1.5 Mb/sec storage bandwith usage per domU. This domU have no disk IO at all but show LA 15 and possible higher.Did you check for processes in a "D" state? ps aux | grep " D" -- Tomasz Chmielewski http://wpkg.org _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maarten Vanraes
2008-May-08 09:15 UTC
Re: [Xen-users] 100% iowait in domU with no IO tasks.
Hi, This is also a problem we are having on our xen servers, but the iowait is considerably higher. i would like to reduce iowait as much as possible, i see iowait above 70% sometimes... Sincerely, Maarten Op donderdag 8 mei 2008 11:02, schreef Bogdan B. Rudas:> Hi. > > I entered one of our domU tonight and see following problem: > > # iowait -k 5 > > avg-cpu: %user %nice %system %iowait %steal %idle > > 0.00 0.00 0.00 100.00 0.00 0.00 > > Device: tps kB_read/s kB_wrtn/s kB_read kB_wrtn > sda1 0.00 0.00 0.00 0 0 > sda2 0.00 0.00 0.00 0 0 > > I check other domU and dom0 on same server - there were high iowait: from > 20% to 50% with few transactions per second and about 0.1-1.5 Mb/sec > storage bandwith usage per domU. This domU have no disk IO at all but show > LA 15 and possible higher. > > We have 6 domU on 8-core server with 7 FUJITSU MBB2147RC SAS HDD on DELL > PERC 5/i RAID controller. > > CentOS 5 > Xen 3.1.0 > Kernel 2.6.21 > > We suffer from this problem for a while on many our xen hosts, it is not a > single-time issue. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Maarten Vanraes BA NV: IT & Security _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maarten Vanraes wrote:> i would like to reduce iowait as much as possible, i see iowait above 70% > sometimes...What kind of services are you running inside your domUs? How many domUs are you running? -- Nick Anderson <nick@anders0n.net> http://www.cmdln.org http://www.anders0n.net _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Marco Strullato
2008-May-08 13:42 UTC
Re: [Xen-users] 100% iowait in domU with no IO tasks.
> > i would like to reduce iowait as much as possible, i see iowait above 70% > sometimes...maybe your domU are swapping... check the memory usage of domUs.. marco _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maarten Vanraes
2008-May-08 15:50 UTC
Re: [Xen-users] 100% iowait in domU with no IO tasks.
not much disk usage in any case, otherwise it''d be pointless to mention it here, and definately no swapping. This is of course the first thing to look for... Op donderdag 8 mei 2008 15:37, schreef Nick Anderson:> Maarten Vanraes wrote: > > i would like to reduce iowait as much as possible, i see iowait above 70% > > sometimes... > > What kind of services are you running inside your domUs? How many domUs > are you running?-- Maarten Vanraes BA NV: IT & Security _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Marco Strullato
2008-May-09 06:29 UTC
Re: [Xen-users] 100% iowait in domU with no IO tasks.
Did you see the send and receive queue? ( netstat -ne) Are you using drbd? marco 2008/5/8 Maarten Vanraes <maarten@ba.be>:> not much disk usage in any case, otherwise it''d be pointless to mention it > here, and definately no swapping. > > This is of course the first thing to look for... > > Op donderdag 8 mei 2008 15:37, schreef Nick Anderson: >> Maarten Vanraes wrote: >> > i would like to reduce iowait as much as possible, i see iowait above 70% >> > sometimes... >> >> What kind of services are you running inside your domUs? How many domUs >> are you running? > > -- > Maarten Vanraes > BA NV: IT & Security > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users >_______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Maarten Vanraes
2008-May-09 07:10 UTC
Re: [Xen-users] 100% iowait in domU with no IO tasks.
Op vrijdag 9 mei 2008 08:29, schreef Marco Strullato:> Did you see the send and receive queue? ( netstat -ne) > Are you using drbd?no drbd, i did not see the queue, is much of traffic the cause for high iowait? I don''t think this is the problem since there''s bandwidth to spare at the moment.> marco > > 2008/5/8 Maarten Vanraes <maarten@ba.be>: > > not much disk usage in any case, otherwise it''d be pointless to mention > > it here, and definately no swapping. > > > > This is of course the first thing to look for... > > > > Op donderdag 8 mei 2008 15:37, schreef Nick Anderson: > >> Maarten Vanraes wrote: > >> > i would like to reduce iowait as much as possible, i see iowait above > >> > 70% sometimes... > >> > >> What kind of services are you running inside your domUs? How many domUs > >> are you running? > > > > -- > > Maarten Vanraes > > BA NV: IT & Security > > > > _______________________________________________ > > Xen-users mailing list > > Xen-users@lists.xensource.com > > http://lists.xensource.com/xen-users > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- Maarten Vanraes BA NV: IT & Security _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Possibly Parallel Threads
- Slow terminal response Centos 7.7 1908
- Slow terminal response Centos 7.7 1908
- Slow RAID Check/high %iowait during check after updgrade from CentOS 6.5 -> CentOS 7.2
- Slow RAID Check/high %iowait during check after updgrade from CentOS 6.5 -> CentOS 7.2
- DRBD + Remus High IO load frozen