search for: alloc_contig_rang

...14:49:34, David Hildenbrand wrote: > A virtio-mem device wants to allocate memory from the memory region it > manages in order to unplug it in the hypervisor - similar to > a balloon driver. Also, it might want to plug previously unplugged > (allocated) memory and give it back to Linux. alloc_contig_range() / > free_contig_range() seem to be the perfect interface for this task. > > In contrast to existing balloon devices, a virtio-mem device operates > on bigger chunks (e.g., 4MB) and only on physical memory it manages. It > tracks which chunks (subblocks) are still plugged, so it c...

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task. In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to al...

A virtio-mem device wants to allocate memory from the memory region it manages in order to unplug it in the hypervisor - similar to a balloon driver. Also, it might want to plug previously unplugged (allocated) memory and give it back to Linux. alloc_contig_range() / free_contig_range() seem to be the perfect interface for this task. In contrast to existing balloon devices, a virtio-mem device operates on bigger chunks (e.g., 4MB) and only on physical memory it manages. It tracks which chunks (subblocks) are still plugged, so it can go ahead and try to al...

...16:22:23, David Hildenbrand wrote: > A virtio-mem device wants to allocate memory from the memory region it > manages in order to unplug it in the hypervisor - similar to > a balloon driver. Also, it might want to plug previously unplugged > (allocated) memory and give it back to Linux. alloc_contig_range() / > free_contig_range() seem to be the perfect interface for this task. > > In contrast to existing balloon devices, a virtio-mem device operates > on bigger chunks (e.g., 4MB) and only on physical memory it manages. It > tracks which chunks (subblocks) are still plugged, so it c...

...l for testing, but also paves the way for virtio-mem optimizations, allowing more memory to get reliably unplugged. Cleanup has_unmovable_pages() and set_migratetype_isolate(), providing better documentation of how ZONE_MOVABLE interacts with different kind of unmovable pages (memory offlining vs. alloc_contig_range()). David Hildenbrand (6): mm/page_alloc: tweak comments in has_unmovable_pages() mm/page_isolation: don't dump_page(NULL) in set_migratetype_isolate() mm/page_isolation: drop WARN_ON_ONCE() in set_migratetype_isolate() mm/page_isolation: cleanup set_migratetype_isolate() mm/page_al...

...l for testing, but also paves the way for virtio-mem optimizations, allowing more memory to get reliably unplugged. Cleanup has_unmovable_pages() and set_migratetype_isolate(), providing better documentation of how ZONE_MOVABLE interacts with different kind of unmovable pages (memory offlining vs. alloc_contig_range()). David Hildenbrand (6): mm/page_alloc: tweak comments in has_unmovable_pages() mm/page_isolation: don't dump_page(NULL) in set_migratetype_isolate() mm/page_isolation: drop WARN_ON_ONCE() in set_migratetype_isolate() mm/page_isolation: cleanup set_migratetype_isolate() mm/page_al...

...already done for > has_unmovable_pages(). Use a new flag to indicate the > requirement of memory offlining to skip over these special pages. > > In has_unmovable_pages(), make sure the pages won't be detected as > movable. This is not strictly necessary, however makes e.g., > alloc_contig_range() stop early, trying to isolate such page blocks - > compared to failing later when testing if all pages were isolated. > > Also, make sure that when a reference to a PageOffline() page is > dropped, that the page will not be returned to the buddy. > > memory devices (like virt...

...already done for > has_unmovable_pages(). Use a new flag to indicate the > requirement of memory offlining to skip over these special pages. > > In has_unmovable_pages(), make sure the pages won't be detected as > movable. This is not strictly necessary, however makes e.g., > alloc_contig_range() stop early, trying to isolate such page blocks - > compared to failing later when testing if all pages were isolated. > > Also, make sure that when a reference to a PageOffline() page is > dropped, that the page will not be returned to the buddy. > > memory devices (like virt...

On Thu 19-09-19 16:22:27, David Hildenbrand wrote: > virtio-mem wants to offline and remove a memory block once it unplugged > all subblocks (e.g., using alloc_contig_range()). Let's provide > an interface to do that from a driver. virtio-mem already supports to > offline partially unplugged memory blocks. Offlining a fully unplugged > memory block will not require to migrate any pages. All unplugged > subblocks are PageOffline() and have a reference...

On 11.03.20 18:14, David Hildenbrand wrote: > virtio-mem wants to offline and remove a memory block once it unplugged > all subblocks (e.g., using alloc_contig_range()). Let's provide > an interface to do that from a driver. virtio-mem already supports to > offline partially unplugged memory blocks. Offlining a fully unplugged > memory block will not require to migrate any pages. All unplugged > subblocks are PageOffline() and have a reference...

...similar to other balloon drivers like virtio-balloon and Hyper-V. PG_offline + reference count of 0 [new] is now also used to mark pages as a "skip" when offlining memory blocks. This allows to offline memory blocks that have partially unplugged subblocks - or are completely unplugged. alloc_contig_range()/free_contig_range() [now exposed] is used to unplug/plug subblocks of memory blocks the are already exposed to Linux. offline_and_remove_memory() [new] is used to offline a fully unplugged memory block and remove it from Linux. A lot of additional information can be found in the separate patc...

...per-V. > > Memory offlining code is extended to allow drivers to drop their reference > to PG_offline pages when MEM_GOING_OFFLINE, so these pages can be skipped > when offlining memory blocks. This allows to offline memory blocks that > have partially unplugged (allocated e.g., via alloc_contig_range()) > subblocks - or are completely unplugged. > > alloc_contig_range()/free_contig_range() [now exposed] is used to > unplug/plug subblocks of memory blocks the are already exposed to Linux. > > offline_and_remove_memory() [new] is used to offline a fully unplugged > memory...

...new flag to indicate the > > > requirement of memory offlining to skip over these special pages. > > > > > > In has_unmovable_pages(), make sure the pages won't be detected as > > > movable. This is not strictly necessary, however makes e.g., > > > alloc_contig_range() stop early, trying to isolate such page blocks - > > > compared to failing later when testing if all pages were isolated. > > > > > > Also, make sure that when a reference to a PageOffline() page is > > > dropped, that the page will not be returned to the bud...

...new flag to indicate the > > > requirement of memory offlining to skip over these special pages. > > > > > > In has_unmovable_pages(), make sure the pages won't be detected as > > > movable. This is not strictly necessary, however makes e.g., > > > alloc_contig_range() stop early, trying to isolate such page blocks - > > > compared to failing later when testing if all pages were isolated. > > > > > > Also, make sure that when a reference to a PageOffline() page is > > > dropped, that the page will not be returned to the bud...

...ft - in roughly MAX_ORDER - 1 granularity. In theory, the movable ZONE part would only shrink when unplugging memory from ZONE_MOVABLE. Let's perform the ZONE_MOVABLE optimization only for memory offlining, such that we reduce the number of false positives from has_unmovable_pages() in case of alloc_contig_range() on ZONE_MOVABLE. Note: We currently don't seem to have any user of alloc_contig_range() that actually uses ZONE_MOVABLE. This change is mostly valuable for the documentation. Cc: Andrew Morton <akpm at linux-foundation.org> Cc: Michal Hocko <mhocko at suse.com> Cc: Michael S. T...

...m. I'll try to find my testing mechanism. My concern is reintroducing > this abandoning of pageblocks. I have not looked further in your series > to see if this potentially addressed later. If not, then we should not > remove the return code. > Memory offlining could race with alloc_contig_range(), e.g., called when allocating gigantic pages, or when virtio-mem tries to unplug memory. The latter two could also race. We are getting more alloc_contig_range() users, which is why these races will become more relevant. I have no clue what you mean with "reintroducing this abandoning of...

...t; has_unmovable_pages(). Use a new flag to indicate the >> requirement of memory offlining to skip over these special pages. >> >> In has_unmovable_pages(), make sure the pages won't be detected as >> movable. This is not strictly necessary, however makes e.g., >> alloc_contig_range() stop early, trying to isolate such page blocks - >> compared to failing later when testing if all pages were isolated. >> >> Also, make sure that when a reference to a PageOffline() page is >> dropped, that the page will not be returned to the buddy. >> >> mem...

...l for testing, but also paves the way for virtio-mem optimizations, allowing more memory to get reliably unplugged. Cleanup has_unmovable_pages() and set_migratetype_isolate(), providing better documentation of how ZONE_MOVABLE interacts with different kind of unmovable pages (memory offlining vs. alloc_contig_range()). v4 -> v5: - Rename "mm/page_isolation: don't dump_page(NULL) in set_migratetype_isolate()" to "mm/page_isolation: exit early when pageblock is isolated in set_migratetype_isolate()" as I was messing up things while reshuffling patches (dump_page(NULL) could ne...

...l for testing, but also paves the way for virtio-mem optimizations, allowing more memory to get reliably unplugged. Cleanup has_unmovable_pages() and set_migratetype_isolate(), providing better documentation of how ZONE_MOVABLE interacts with different kind of unmovable pages (memory offlining vs. alloc_contig_range()). v4 -> v5: - Rename "mm/page_isolation: don't dump_page(NULL) in set_migratetype_isolate()" to "mm/page_isolation: exit early when pageblock is isolated in set_migratetype_isolate()" as I was messing up things while reshuffling patches (dump_page(NULL) could ne...

...ike virtio-balloon and Hyper-V. Memory offlining code is extended to allow drivers to drop their reference to PG_offline pages when MEM_GOING_OFFLINE, so these pages can be skipped when offlining memory blocks. This allows to offline memory blocks that have partially unplugged (allocated e.g., via alloc_contig_range()) subblocks - or are completely unplugged. alloc_contig_range()/free_contig_range() [now exposed] is used to unplug/plug subblocks of memory blocks the are already exposed to Linux. offline_and_remove_memory() [new] is used to offline a fully unplugged memory block and remove it from Linux. --...