Hello! Will disk i/o limiting be added in any future versions? I see it in the 4.0 feature requests, but not in any changelogs. Best regards, Tõnis Bramanis Server administrator / client advisor Elkdata OÜ www.veebimajutus.ee Info: abi@veebimajutus.ee +372 683 5188 Directly: tonis@elkdata.ee +372 56 960 910 ------------------------------------------------------- _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2011-Sep-27 19:58 UTC
Re: [Xen-devel] Limit I/O for individual disks of VM
On 09/27/2011 01:33 AM, Tõnis Bramanis wrote:> Hello! > > Will disk i/o limiting be added in any future versions? I see it in > the 4.0 feature requests, but not in any changelogs.I think the hope is that we can use the generic Linux mechanisms for this rather than having to add anything Xen-specific. J _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hello and thank You for the answer. Has anyone of You tried to limit i/o on the VMs? I have about 40 containers and when the i/o gets too high, the host will crash with this error: [23959.739698] sd 4:0:0:0: rejecting I/O to offline device 5 [23959.771158] Buffer I/O error on device dm-0, logical block 40259 This is not a support request! I am just referring why i would like to limit the disks i/o. Regards, Tõnis Bramanis On 27.09.2011 22:58, Jeremy Fitzhardinge wrote:> On 09/27/2011 01:33 AM, Tõnis Bramanis wrote: >> Hello! >> >> Will disk i/o limiting be added in any future versions? I see it in >> the 4.0 feature requests, but not in any changelogs. > > I think the hope is that we can use the generic Linux mechanisms for > this rather than having to add anything Xen-specific. > > J_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Jeremy Fitzhardinge
2011-Sep-28 06:35 UTC
Re: [Xen-devel] Limit I/O for individual disks of VM
On 09/27/2011 11:25 PM, Tõnis Bramanis wrote:> Hello > > and thank You for the answer. Has anyone of You tried to limit i/o on > the VMs? > > I have about 40 containers and when the i/o gets too high, the host > will crash with this error: > [23959.739698] sd 4:0:0:0: rejecting I/O to offline device > 5 [23959.771158] Buffer I/O error on device dm-0, logical block 40259 > > This is not a support request! I am just referring why i would like to > limit the disks i/o.I think that IO error is a real problem that you should try to solve, rather than work around it by rate-limiting IO. J> > Regards, > Tõnis Bramanis > > On 27.09.2011 22:58, Jeremy Fitzhardinge wrote: >> On 09/27/2011 01:33 AM, Tõnis Bramanis wrote: >>> Hello! >>> >>> Will disk i/o limiting be added in any future versions? I see it in >>> the 4.0 feature requests, but not in any changelogs. >> >> I think the hope is that we can use the generic Linux mechanisms for >> this rather than having to add anything Xen-specific. >> >> J >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I haven''t met anyone yet who had ever seen something like that before. Guess it''s time to change the hardware and see what happens then. Thanks, Tõnis On 28.09.2011 9:35, Jeremy Fitzhardinge wrote:> On 09/27/2011 11:25 PM, Tõnis Bramanis wrote: >> Hello >> >> and thank You for the answer. Has anyone of You tried to limit i/o on >> the VMs? >> >> I have about 40 containers and when the i/o gets too high, the host >> will crash with this error: >> [23959.739698] sd 4:0:0:0: rejecting I/O to offline device >> 5 [23959.771158] Buffer I/O error on device dm-0, logical block 40259 >> >> This is not a support request! I am just referring why i would like to >> limit the disks i/o. > > I think that IO error is a real problem that you should try to solve, > rather than work around it by rate-limiting IO. > > J > >> >> Regards, >> Tõnis Bramanis >> >> On 27.09.2011 22:58, Jeremy Fitzhardinge wrote: >>> On 09/27/2011 01:33 AM, Tõnis Bramanis wrote: >>>> Hello! >>>> >>>> Will disk i/o limiting be added in any future versions? I see it in >>>> the 4.0 feature requests, but not in any changelogs. >>> >>> I think the hope is that we can use the generic Linux mechanisms for >>> this rather than having to add anything Xen-specific. >>> >>> J >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Joseph Glanville
2011-Sep-28 07:38 UTC
Re: [Xen-devel] Limit I/O for individual disks of VM
Hi, To add to the above it looks like you have a device driver or hardware issue with your harddrive or raid controller. I/O ratelimiting might hide the symptoms but not correct the problem. After you have addressed that you can look at implementing rate limiting via the methods described below: cgroups blk-io controller: You can find documentation etc in the linux git repo at /documentation/cgroups/blkio-controller.txt dm-ioband: Wiki is here http://sourceforge.net/apps/trac/ioband/wiki/dm-ioband Both of these are external to Xen for the reasons outlined by previous posts. Kind regards, Joseph. 2011/9/28 Jeremy Fitzhardinge <jeremy@goop.org>> On 09/27/2011 11:25 PM, Tõnis Bramanis wrote: > > Hello > > > > and thank You for the answer. Has anyone of You tried to limit i/o on > > the VMs? > > > > I have about 40 containers and when the i/o gets too high, the host > > will crash with this error: > > [23959.739698] sd 4:0:0:0: rejecting I/O to offline device > > 5 [23959.771158] Buffer I/O error on device dm-0, logical block 40259 > > > > This is not a support request! I am just referring why i would like to > > limit the disks i/o. > > I think that IO error is a real problem that you should try to solve, > rather than work around it by rate-limiting IO. > > J > > > > > Regards, > > Tõnis Bramanis > > > > On 27.09.2011 22:58, Jeremy Fitzhardinge wrote: > >> On 09/27/2011 01:33 AM, Tõnis Bramanis wrote: > >>> Hello! > >>> > >>> Will disk i/o limiting be added in any future versions? I see it in > >>> the 4.0 feature requests, but not in any changelogs. > >> > >> I think the hope is that we can use the generic Linux mechanisms for > >> this rather than having to add anything Xen-specific. > >> > >> J > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >-- * Founder | Director | VP Research Orion Virtualisation Solutions* | www.orionvm.com.au | Phone: 1300 56 99 52 | Mobile: 0428 754 846 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
If i run the exact same kernel without Xen and test the i/o, it won''t crash. I just tried to look up an old post to refer it to You and seems it has been updated just yesterday with a fix: http://xen.1045712.n5.nabble.com/Xen-4-1-3ware-9690SA-rejecting-I-O-to-offline-device-td3208156.html Still, i will try to change the raid controller and see what happens then. Regards, Tõnis Bramanis On 28.09.2011 10:38, Joseph Glanville wrote:> Hi, > > To add to the above it looks like you have a device driver or hardware > issue with your harddrive or raid controller. > I/O ratelimiting might hide the symptoms but not correct the problem. > After you have addressed that you can look at implementing rate limiting > via the methods described below: > > cgroups blk-io controller: > You can find documentation etc in the linux git repo at > /documentation/cgroups/blkio-controller.txt > > dm-ioband: > Wiki is here http://sourceforge.net/apps/trac/ioband/wiki/dm-ioband > > Both of these are external to Xen for the reasons outlined by previous > posts. > > Kind regards, > Joseph. > > 2011/9/28 Jeremy Fitzhardinge <jeremy@goop.org <mailto:jeremy@goop.org>> > > On 09/27/2011 11:25 PM, Tõnis Bramanis wrote: > > Hello > > > > and thank You for the answer. Has anyone of You tried to limit i/o on > > the VMs? > > > > I have about 40 containers and when the i/o gets too high, the host > > will crash with this error: > > [23959.739698] sd 4:0:0:0: rejecting I/O to offline device > > 5 [23959.771158] Buffer I/O error on device dm-0, logical > block 40259 > > > > This is not a support request! I am just referring why i would > like to > > limit the disks i/o. > > I think that IO error is a real problem that you should try to solve, > rather than work around it by rate-limiting IO. > > J > > > > > Regards, > > Tõnis Bramanis > > > > On 27.09.2011 22 <tel:27.09.2011%2022>:58, Jeremy Fitzhardinge wrote: > >> On 09/27/2011 01:33 AM, Tõnis Bramanis wrote: > >>> Hello! > >>> > >>> Will disk i/o limiting be added in any future versions? I see it in > >>> the 4.0 feature requests, but not in any changelogs. > >> > >> I think the hope is that we can use the generic Linux mechanisms for > >> this rather than having to add anything Xen-specific. > >> > >> J > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com <mailto:Xen-devel@lists.xensource.com> > http://lists.xensource.com/xen-devel > > > > > -- > */ > Founder | Director | VP Research > Orion Virtualisation Solutions/* | www.orionvm.com.au > <http://www.orionvm.com.au/> | Phone: 1300 56 99 52 | Mobile: 0428 754 846_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 28/09/11 07:35, Jeremy Fitzhardinge wrote:> On 09/27/2011 11:25 PM, Tõnis Bramanis wrote: >> Hello >> >> and thank You for the answer. Has anyone of You tried to limit i/o on >> the VMs? >> >> I have about 40 containers and when the i/o gets too high, the host >> will crash with this error: >> [23959.739698] sd 4:0:0:0: rejecting I/O to offline device >> 5 [23959.771158] Buffer I/O error on device dm-0, logical block 40259 >> >> This is not a support request! I am just referring why i would like to >> limit the disks i/o. > I think that IO error is a real problem that you should try to solve, > rather than work around it by rate-limiting IO. > > JThis is looking suspiciously like the line level interrupt migration bug. Can you confirm the version of Xen, the raid controller you are using, and paste /proc/interrupts from dom0 please. ~Andrew>> Regards, >> Tõnis Bramanis >> >> On 27.09.2011 22:58, Jeremy Fitzhardinge wrote: >>> On 09/27/2011 01:33 AM, Tõnis Bramanis wrote: >>>> Hello! >>>> >>>> Will disk i/o limiting be added in any future versions? I see it in >>>> the 4.0 feature requests, but not in any changelogs. >>> I think the hope is that we can use the generic Linux mechanisms for >>> this rather than having to add anything Xen-specific. >>> >>> J > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel-- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel