yf chu
2021-Mar-12 01:29 UTC
[CentOS] Disk read io very high, but no process perform io read
yes. I suspect it has something to do with swapping. but swap is turned off on this server. here is the result of free -m. total used free shared buff/cache available Mem: 128174 97449 24400 4158 6325 25232 Swap: 0 0 0 We have other servers. The processes running on these servers are same. but on other servers, the size of buff/cache is larger than the size on the server which experienced the problem and the size of "free" is smaller than the size on the server which experienced the problem. ? 2021-03-10 14:30:00?"Thomas Stephen Lee" <lee.iitb at gmail.com> ???>On Sat, Mar 6, 2021 at 6:37 AM yf chu <cyflhn at 163.com> wrote: >> >> no, not vm >> >> >> >> >> | | >> yf chu >> | >> | >> ???cyflhn at 163.com >> | >> >> ??? ?????? ?? >> >> On 03/05/2021 23:37, Matthew Miller wrote: >> On Fri, Mar 05, 2021 at 03:54:14PM +0800, yf chu wrote: >> > We have experienced a very weird problem. The load of the server machine is very high. We use "pidstat" and find that the disk read io is very high. >> >> Is this system a VM? >> >> -- >> Matthew Miller >> <mattdm at fedoraproject.org> >> Fedora Project Leader > >Hi, > >Is this due to swapping? > >what is the output of > >free -m > >--- >Lee >_______________________________________________ >CentOS mailing list >CentOS at centos.org >https://lists.centos.org/mailman/listinfo/centos
J Martin Rushton
2021-Mar-12 07:27 UTC
[CentOS] Disk read io very high, but no process perform io read
I haven't been following this thread closely, so may be off target. When pages are moved out of the working set they are either "clean" or "dirty". Clean pages have not been modified since they were originally moved into memory whereas dirty pages have been changed. A dirty page can become clean if it is written back to disk. Typically (though not always) this is a write to swap. Read only pages, such as code or data will always be clean, so can be dropped when required. When a hard fault occurs then pages have to be read from disk, somewhere. That somewhere could be swap, but can also be program images or files. It may be that what you observe is this latter process. HTH, Martin On 12/03/2021 01:29, yf chu wrote:> > yes. I suspect it has something to do with swapping. but swap is turned off on this server. > here is the result of free -m. > total used free shared buff/cache available > Mem: 128174 97449 24400 4158 6325 25232 > Swap: 0 0 0 > > > We have other servers. The processes running on these servers are same. but on other servers, the size of buff/cache is larger than the size on the server which experienced the problem and the size of "free" is smaller than the size on the server which experienced the problem. ><snip> -- J Martin Rushton MBCS
Simon Matter
2021-Mar-12 08:35 UTC
[CentOS] Disk read io very high, but no process perform io read
Hi, You said that you have multiple systems running this same application. But, do they work with the same data on disk or are there big differences?>From how I understand the figures below, your buff/cache seems a bit lowif you read a lot of data from disk. If you read a lot of data and filesystem caches are slow, it will result in heavy reading from disk directly. One more thing comes to mind: is your hardware behaving fine? I mean, if your storag has difficulties to read data from disk due to hardware issues, this could also lead to such problems you're facing. Regards, Simon> > yes. I suspect it has something to do with swapping. but swap is turned > off on this server. > here is the result of free -m. > total used free shared buff/cache available > Mem: 128174 97449 24400 4158 6325 > 25232 > Swap: 0 0 0 > > > We have other servers. The processes running on these servers are same. > but on other servers, the size of buff/cache is larger than the size on > the server which experienced the problem and the size of "free" is smaller > than the size on the server which experienced the problem. > > > > > > > > > > > > > > > ?? 2021-03-10 14:30:00??"Thomas Stephen Lee" <lee.iitb at gmail.com> ????? >>On Sat, Mar 6, 2021 at 6:37 AM yf chu <cyflhn at 163.com> wrote: >>> >>> no, not vm >>> >>> >>> >>> >>> | | >>> yf chu >>> | >>> | >>> ????cyflhn at 163.com >>> | >>> >>> ????? ?????????? ???? >>> >>> On 03/05/2021 23:37, Matthew Miller wrote: >>> On Fri, Mar 05, 2021 at 03:54:14PM +0800, yf chu wrote: >>> > We have experienced a very weird problem. The load of the server >>> machine is very high. We use "pidstat" and find that the disk read io >>> is very high. >>> >>> Is this system a VM? >>> >>> -- >>> Matthew Miller >>> <mattdm at fedoraproject.org> >>> Fedora Project Leader >> >>Hi, >> >>Is this due to swapping? >> >>what is the output of >> >>free -m >> >>--- >>Lee >>_______________________________________________ >>CentOS mailing list >>CentOS at centos.org >>https://lists.centos.org/mailman/listinfo/centos > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos >