search for: pagecache

...d sizes, but Tyan recommended making them all the same, and the vendor made the substitutions. We have also clocked the memory down from 400 MHz to 266 MHz, also on the advice of Tyan. The symptom is that some large (700MB to >1GB) files opened for read and then closed show corruption in the pagecache. One or more 4k blocks in a file will be completely trashed. It's as if a random page of other data is substituted. A reboot or a flush of the pagecache fixes the problem, so it's only in the pagecache, not on disk. We are doing regular MD5 checksums of the files, which shows up the pro...

There are cases that users want to shrink balloon pages after the pagecache depleted. The conservative_shrinker lets the shrinker shrink balloon pages when all the pagecache has been reclaimed. Signed-off-by: Wei Wang <wei.w.wang at intel.com> --- drivers/virtio/virtio_balloon.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/driv...

There are cases that users want to shrink balloon pages after the pagecache depleted. The conservative_shrinker lets the shrinker shrink balloon pages when all the pagecache has been reclaimed. Signed-off-by: Wei Wang <wei.w.wang at intel.com> --- drivers/virtio/virtio_balloon.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/driv...

...9;m hoping some of the smart folks here will know more about this problem... We have noticed when running some HPC applications on our OpenStack (libvirt+KVM) cloud that the same application occasionally performs much worse (4-5x slowdown) than normal. We can reproduce this quite easily by filling pagecache (i.e. dd-ing a single large file to /dev/null) before running the application. The problem seems to be that the kernel is not freeing (or has some trouble freeing) the non-dirty and (presumably immediately) reclaimable pagecache in order to allocate THPs for the application. This behaviour is also...

...y, February 10, 2020 11:57 AM, Tetsuo Handa wrote: > Then, "node-A's NR_FILE_PAGES is already 0 and node-B's NR_FILE_PAGES is > not 0, but allocation request which triggered this shrinker wants to allocate > from only node-A" > would be confused by this change, for the pagecache pages for allocating > thread's interested node are already depleted but the balloon cannot shrink > when it should because the pagecache pages for allocating thread's > uninterested nodes are not yet depleted. The existing balloon isn't numa aware. "but the balloon canno...

...y, February 10, 2020 11:57 AM, Tetsuo Handa wrote: > Then, "node-A's NR_FILE_PAGES is already 0 and node-B's NR_FILE_PAGES is > not 0, but allocation request which triggered this shrinker wants to allocate > from only node-A" > would be confused by this change, for the pagecache pages for allocating > thread's interested node are already depleted but the balloon cannot shrink > when it should because the pagecache pages for allocating thread's > uninterested nodes are not yet depleted. The existing balloon isn't numa aware. "but the balloon canno...

Hi, I was wondering about the ARC and its interaction with the VM pagecache... When a file on a ZFS filesystem is mmaped, does the ARC cache get mapped to the process'' virtual memory? Or is there another copy? -Manoj

...olks > here will know more about this problem... > > We have noticed when running some HPC applications on our OpenStack > (libvirt+KVM) cloud that the same application occasionally performs much > worse (4-5x slowdown) than normal. We can reproduce this quite easily by > filling pagecache (i.e. dd-ing a single large file to /dev/null) before > running the application. The problem seems to be that the kernel is not > freeing (or has some trouble freeing) the non-dirty and (presumably > immediately) reclaimable pagecache in order to allocate THPs for the > application. &gt...

...of which are complete black-boxes) that explicit hugepage allocation for application memory is generally not viable, so we are reliant on THP to avoid TLB thrashing. > The kernel can't predict the future usage pattern of processes so it is not at > all clear cut that evicting the entire pagecache in order to allocate more > huge pages is going to be beneficial for system performance as a whole. Yet the default behaviour seems to be to stall on fault, then directly reclaim and defrag in order to allocate a hugepage if at all possible. In my test-case there is almost no free memory, so so...

...not 0, but allocation request which triggered this > shrinker wants to allocate from only node-B" happen? No, it's a global counter. >Can some thread keep > this shrinker defunctional by keep increasing NR_FILE_PAGES? Yes. Actually it's our intention - as long as there are pagecache pages, balloon pages are avoided to be reclaimed. > > Is this patch from "Re: Balloon pressuring page cache" thread? I hope that > the guest could start reclaiming memory based on host's request (like OOM > notifier chain) which is issued when host thinks that host is g...

...not 0, but allocation request which triggered this > shrinker wants to allocate from only node-B" happen? No, it's a global counter. >Can some thread keep > this shrinker defunctional by keep increasing NR_FILE_PAGES? Yes. Actually it's our intention - as long as there are pagecache pages, balloon pages are avoided to be reclaimed. > > Is this patch from "Re: Balloon pressuring page cache" thread? I hope that > the guest could start reclaiming memory based on host's request (like OOM > notifier chain) which is issued when host thinks that host is g...

Add a new field, VIRTIO_BALLOON_S_AVAIL, to virtio_balloon memory statistics protocol, corresponding to 'Available' in /proc/meminfo. It indicates to the hypervisor how big the balloon can be inflated without pushing the guest system to swap. This metric would be very useful in VM orchestration software to improve memory management of different VMs under overcommit. Signed-off-by: Igor

Add a new field, VIRTIO_BALLOON_S_AVAIL, to virtio_balloon memory statistics protocol, corresponding to 'Available' in /proc/meminfo. It indicates to the hypervisor how big the balloon can be inflated without pushing the guest system to swap. This metric would be very useful in VM orchestration software to improve memory management of different VMs under overcommit. Signed-off-by: Igor

...d the module in an intel quad, and got the same error). Follow some outputs: The error stack: Oct 28 23:40:35 goku kernel: [ 4472.129718] BUG: unable to handle kernel paging request at 000003d600000004 Oct 28 23:40:35 goku kernel: [ 4472.129933] IP: [<ffffffffa08d118c>] get_files+0x6c/0x220 [pagecache] Oct 28 23:40:35 goku kernel: [ 4472.130098] PGD 0 Oct 28 23:40:35 goku kernel: [ 4472.130250] Oops: 0000 [#4] SMP Oct 28 23:40:35 goku kernel: [ 4472.130452] last sysfs file: /sys/devices/system/cpu/cpu7/online Oct 28 23:40:35 goku kernel: [ 4472.130562] CPU 0 Oct 28 23:40:35 goku kernel: [ 4472.1...

...d the module in an intel quad, and got the same error). Follow some outputs: The error stack: Oct 28 23:40:35 goku kernel: [ 4472.129718] BUG: unable to handle kernel paging request at 000003d600000004 Oct 28 23:40:35 goku kernel: [ 4472.129933] IP: [<ffffffffa08d118c>] get_files+0x6c/0x220 [pagecache] Oct 28 23:40:35 goku kernel: [ 4472.130098] PGD 0 Oct 28 23:40:35 goku kernel: [ 4472.130250] Oops: 0000 [#4] SMP Oct 28 23:40:35 goku kernel: [ 4472.130452] last sysfs file: /sys/devices/system/cpu/cpu7/online Oct 28 23:40:35 goku kernel: [ 4472.130562] CPU 0 Oct 28 23:40:35 goku kernel: [ 4472.1...

Greetings Has libvirt dedicated page cache area for guest ? If not - what is the difference between cache='none' and cache='directsync' ? >The optional cache attribute controls the cache mechanism, possible >values are "default", "none", "writethrough", "writeback", "directsync" >(like "writethrough", but it

On Tue, Nov 14, 2017 at 10:52:03AM -0700, Blair Bethwaite wrote: > Thanks for the reply Daniel, > > However I think you slightly misunderstood the scenario... > > On 14 November 2017 at 10:32, Daniel P. Berrange <berrange@redhat.com> wrote: > > IOW, if your application has a certain expectation of performance that can only > > be satisfied by having the KVM guest

On 14 November 2017 at 10:56, Daniel P. Berrange <berrange@redhat.com> wrote: > Oh well THP usage inside the guest is then not really anything todo with > virt, just a regular Linux questions, so not sure libvirt is the best > place to ask. True, I just hoped you or one of the other devs might have some insight on reclaim behaviour that would provide a clue. I guess I'll try a

...nd instead starts swapping...which i have noticed in my little install here. What i did to fix it as a workaround(noted in the buzilla report here: https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=89226 is to send the following command at the command line: (echo 1 10 10 >/proc/sys/vm/pagecache) I have noticed this is another RH9 server i ahve access to and will be trying it on that box too as well as my linux firewall.> -- My "Foundation" verse: Isa 54:17 No weapon that is formed against thee shall prosper; and every tongue that shall rise against thee in judgment...

On Thu, Feb 06, 2020 at 04:01:47PM +0800, Wei Wang wrote: > There are cases that users want to shrink balloon pages after the > pagecache depleted. The conservative_shrinker lets the shrinker > shrink balloon pages when all the pagecache has been reclaimed. > > Signed-off-by: Wei Wang <wei.w.wang at intel.com> I'd rather avoid module parameters, but otherwise looks like a reasonable idea. Tyler, what do you thin...