search for: madv_dontneed

On 07/31/2017 02:55 PM, Michal Hocko wrote: > On Mon 31-07-17 12:13:33, Wei Wang wrote: >> Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and >> shouldn't be given to the host ksmd to scan. > Could you point me where this MADV_DONTNEED is done, please? Sure. It's done in the hypervisor when the balloon pages are received. Please see line 40 at https://github.com/qemu/qemu/blob/master/hw/virtio/vir...

On 07/31/2017 02:55 PM, Michal Hocko wrote: > On Mon 31-07-17 12:13:33, Wei Wang wrote: >> Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and >> shouldn't be given to the host ksmd to scan. > Could you point me where this MADV_DONTNEED is done, please? Sure. It's done in the hypervisor when the balloon pages are received. Please see line 40 at https://github.com/qemu/qemu/blob/master/hw/virtio/vir...

This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: enqueue zero page to balloon device")' Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and shouldn't be given to the host ksmd to scan. Therefore, it is not necessary to zero ballooned pages, which is very time consuming when the page amount is large. The ongoing fast balloon tests show that the time to balloon 7G pages is increased from ~491ms to 2.8 seconds wi...

This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: enqueue zero page to balloon device")' Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and shouldn't be given to the host ksmd to scan. Therefore, it is not necessary to zero ballooned pages, which is very time consuming when the page amount is large. The ongoing fast balloon tests show that the time to balloon 7G pages is increased from ~491ms to 2.8 seconds wi...

Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and shouldn't be given to the host ksmd to scan. Therefore, it is not necessary to zero ballooned pages, which is very time consuming when the page amount is large. The ongoing fast balloon tests show that the time to balloon 7G pages is increased from ~491ms to 2.8 seconds wi...

Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and shouldn't be given to the host ksmd to scan. Therefore, it is not necessary to zero ballooned pages, which is very time consuming when the page amount is large. The ongoing fast balloon tests show that the time to balloon 7G pages is increased from ~491ms to 2.8 seconds wi...

On Mon 07-08-17 10:44:50, David Hildenbrand wrote: > On 03.08.2017 13:59, Wei Wang wrote: > > This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: > > enqueue zero page to balloon device")' > > > > Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > > shouldn't be given to the host ksmd to scan. Therefore, it is not > > necessary to zero ballooned pages, which is very time consuming when > > the page amount is large. The ongoing fast balloon tests show that the > > time to balloon 7G pages is...

On Mon 07-08-17 10:44:50, David Hildenbrand wrote: > On 03.08.2017 13:59, Wei Wang wrote: > > This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: > > enqueue zero page to balloon device")' > > > > Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > > shouldn't be given to the host ksmd to scan. Therefore, it is not > > necessary to zero ballooned pages, which is very time consuming when > > the page amount is large. The ongoing fast balloon tests show that the > > time to balloon 7G pages is...

...; On 07/31/2017 03:51 PM, Michal Hocko wrote: > > >On Mon 31-07-17 15:41:49, Wei Wang wrote: > >>>On 07/31/2017 02:55 PM, Michal Hocko wrote: > >>>> >On Mon 31-07-17 12:13:33, Wei Wang wrote: > >>>>> >>Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > >>>>> >>shouldn't be given to the host ksmd to scan. > >>>> >Could you point me where this MADV_DONTNEED is done, please? > >>> > >>>Sure. It's done in the hypervisor when the balloon pages are receive...

On 03.08.2017 13:59, Wei Wang wrote: > This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: > enqueue zero page to balloon device")' > > Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > shouldn't be given to the host ksmd to scan. Therefore, it is not > necessary to zero ballooned pages, which is very time consuming when > the page amount is large. The ongoing fast balloon tests show that the > time to balloon 7G pages is increased from ~491...

On Mon 31-07-17 12:13:33, Wei Wang wrote: > Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > shouldn't be given to the host ksmd to scan. Could you point me where this MADV_DONTNEED is done, please? > Therefore, it is not > necessary to zero ballooned pages, which is very time consuming when > the page amount is large. The ongoing fast balloon test...

On Mon 31-07-17 15:41:49, Wei Wang wrote: > On 07/31/2017 02:55 PM, Michal Hocko wrote: > >On Mon 31-07-17 12:13:33, Wei Wang wrote: > >>Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > >>shouldn't be given to the host ksmd to scan. > >Could you point me where this MADV_DONTNEED is done, please? > > Sure. It's done in the hypervisor when the balloon pages are received. > > Please see line 40 at > https://github.com/qe...

On Thu 03-08-17 19:59:17, Wei Wang wrote: > This patch is a revert of 'commit bb01b64cfab7 ("mm/balloon_compaction.c: > enqueue zero page to balloon device")' > > Ballooned pages will be marked as MADV_DONTNEED by the hypervisor and > shouldn't be given to the host ksmd to scan. I find MADV_DONTNEED reference still quite confusing. What do you think about the following wording instead: " Zeroying ballon pages is rather time consuming, especially when a lot of pages are in flight. E.g. 7GB wor...

...ot assume implementation details of the host/hypervisor. The PAGE_POISON feature is being used to indicate that we expect the page to contain a certain value when it is returned to us. With the current implementation in QEMU if that value is zero we can work with it because that is the effect that MADV_DONTNEED has. However if there were some other effect being used by QEMU we would want to know that the guest is expecting us to write a 0 page. So I believe it makes sense to inform the hypervisor that we are doing something that expects us to fill the page with zeros in the case of want_init_on_free rathe...

...ot assume implementation details of the host/hypervisor. The PAGE_POISON feature is being used to indicate that we expect the page to contain a certain value when it is returned to us. With the current implementation in QEMU if that value is zero we can work with it because that is the effect that MADV_DONTNEED has. However if there were some other effect being used by QEMU we would want to know that the guest is expecting us to write a 0 page. So I believe it makes sense to inform the hypervisor that we are doing something that expects us to fill the page with zeros in the case of want_init_on_free rathe...

...t; > Because it can't. sbrk() can only increase/decrease memory usage at the > end (like a stack), you can't release something in the middle. > Thats one of the reasons why we wrote a pool-based memory allocator for > ClamAV. Another thing malloc could do is to use madvise with MADV_DONTNEED to free the pages in the middle of t heap, but malloc can't read your mind, so it doesn't know that you aren't about to reallocate that region of the heap. >> >> > With malloc() no, it takes just 1 allocation at the end of the heap >> > to keep all the rest allo...

On 12.05.20 11:41, Hui Zhu wrote: This description needs an overhaul, it's hard to parse. > If the guest kernel has many fragmentation pages, use virtio_balloon > will split THP of QEMU when it calls MADV_DONTNEED madvise to release > the balloon pages. This is very unclear and confusing. You will *always* split THPs when inflating 4k pages and there are THPs around. This is completely independent of any fragmentation in the guest. The thing you are trying to achieve here is trying to *minimize* the numb...

This implementation of a virtio balloon driver uses the page cache to "store" pages that have been released to the host. The communication (outside of target counts) is one way--the guest notifies the host when it adds a page to the page cache, allowing the host to madvise(2) with MADV_DONTNEED. Reclaim in the guest is therefore automatic and implicit (via the regular page reclaim). This means that inflating the balloon is similar to the existing balloon mechanism, but the deflate is different--it re-uses existing Linux kernel functionality to automatically reclaim. Signed-off-by: Fran...

This implementation of a virtio balloon driver uses the page cache to "store" pages that have been released to the host. The communication (outside of target counts) is one way--the guest notifies the host when it adds a page to the page cache, allowing the host to madvise(2) with MADV_DONTNEED. Reclaim in the guest is therefore automatic and implicit (via the regular page reclaim). This means that inflating the balloon is similar to the existing balloon mechanism, but the deflate is different--it re-uses existing Linux kernel functionality to automatically reclaim. Signed-off-by: Fran...

...+#include <sys/mman.h> #define MIN_FINALIZE_SIZE (200 << 10) #define MAX_ITERATIONS 30 @@ -765,6 +766,9 @@ static void migrate_incoming_homogeneous for (i=0; i<n; i++) p[i] = v; + + if (v == 0) + madvise(phys_ram_base + addr, TARGET_PAGE_SIZE, MADV_DONTNEED); } static int migrate_incoming_page(QEMUFile *f, uint32_t addr) Index: kvm-userspace/qemu/monitor.c =================================================================== --- kvm-userspace.orig/qemu/monitor.c +++ kvm-userspace/qemu/monitor.c @@ -1339,6 +1339,8 @@ static term_cmd_t term_cmds[] = {...