search for: pg_offline

...ree it to the > > > > system by clearing the offline state and put_page. > > > > > > I think I explained how the reference count of 1 is problematic when wanting > > > to offline the memory. After all that's the problem I try to solve: Keep > > > PG_offline set until the memory is offline and make sure nobody will touch > > > the page. > > > > Please bear with me but I still believe that elevated reference count > > has some merits. I do understand that you maintain your metadata to > > recognize that the memory hande...

...ree it to the > > > > system by clearing the offline state and put_page. > > > > > > I think I explained how the reference count of 1 is problematic when wanting > > > to offline the memory. After all that's the problem I try to solve: Keep > > > PG_offline set until the memory is offline and make sure nobody will touch > > > the page. > > > > Please bear with me but I still believe that elevated reference count > > has some merits. I do understand that you maintain your metadata to > > recognize that the memory hande...

...ence count of 1 (unless the pages are movable, like with balloon > compaction). I am pretty sure that has_unmovable_pages can special these pages regardless of the reference count for the memory hotplug. We already do that for HWPoison pages. > Please note that we have other users that use PG_offline + refcount >= 1 > (HyperV balloon, XEN). We should not affect these users (IOW, > has_unmovable_pages() has to stop right there if we see one of these pages). OK, this is exactly what I was worried about. I can see why you might want to go an easier way and rule those users out but wouldn...

...ence count of 1 (unless the pages are movable, like with balloon > compaction). I am pretty sure that has_unmovable_pages can special these pages regardless of the reference count for the memory hotplug. We already do that for HWPoison pages. > Please note that we have other users that use PG_offline + refcount >= 1 > (HyperV balloon, XEN). We should not affect these users (IOW, > has_unmovable_pages() has to stop right there if we see one of these pages). OK, this is exactly what I was worried about. I can see why you might want to go an easier way and rule those users out but wouldn...

>> Please note that we have other users that use PG_offline + refcount >= 1 >> (HyperV balloon, XEN). We should not affect these users (IOW, >> has_unmovable_pages() has to stop right there if we see one of these pages). > > OK, this is exactly what I was worried about. I can see why you might > want to go an easier way and rule tho...

...est for whatever reason it can free it to the >>> system by clearing the offline state and put_page. >> >> I think I explained how the reference count of 1 is problematic when wanting >> to offline the memory. After all that's the problem I try to solve: Keep >> PG_offline set until the memory is offline and make sure nobody will touch >> the page. > > Please bear with me but I still believe that elevated reference count > has some merits. I do understand that you maintain your metadata to > recognize that the memory handed over to the hypervisor w...

...o the >>>>> system by clearing the offline state and put_page. >>>> >>>> I think I explained how the reference count of 1 is problematic when wanting >>>> to offline the memory. After all that's the problem I try to solve: Keep >>>> PG_offline set until the memory is offline and make sure nobody will touch >>>> the page. >>> >>> Please bear with me but I still believe that elevated reference count >>> has some merits. I do understand that you maintain your metadata to >>> recognize that the...

...returned to the guest for whatever reason it can free it to the > > system by clearing the offline state and put_page. > > I think I explained how the reference count of 1 is problematic when wanting > to offline the memory. After all that's the problem I try to solve: Keep > PG_offline set until the memory is offline and make sure nobody will touch > the page. Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear...

...returned to the guest for whatever reason it can free it to the > > system by clearing the offline state and put_page. > > I think I explained how the reference count of 1 is problematic when wanting > to offline the memory. After all that's the problem I try to solve: Keep > PG_offline set until the memory is offline and make sure nobody will touch > the page. Please bear with me but I still believe that elevated reference count has some merits. I do understand that you maintain your metadata to recognize that the memory handed over to the hypervisor will not magically appear...

...21,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count, > if (!page_ref_count(page)) { > if (PageBuddy(page)) > iter += (1 << page_order(page)) - 1; > + /* > + * Memory devices allow to offline a page if it is > + * marked PG_offline and has a reference count of 0. > + * However, the pages are not movable as it would be > + * required e.g., for alloc_contig_range(). > + */ > + if (PageOffline(page) && !(flags & SKIP_OFFLINE)) > + if (++found > count) > + goto unmovable; > c...

...21,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count, > if (!page_ref_count(page)) { > if (PageBuddy(page)) > iter += (1 << page_order(page)) - 1; > + /* > + * Memory devices allow to offline a page if it is > + * marked PG_offline and has a reference count of 0. > + * However, the pages are not movable as it would be > + * required e.g., for alloc_contig_range(). > + */ > + if (PageOffline(page) && !(flags & SKIP_OFFLINE)) > + if (++found > count) > + goto unmovable; > c...

...t to answer this questions. The __put_page() change is necessary > for the following race I identified: > > Page has a refcount of 1 (e.g., allocated by virtio-mem using > alloc_contig_range()). > > a) kernel: get_page_unless_zero(page): refcount = 2 > b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1 > c) kernel: put_page(page): refcount = 0 > > The page would suddenly be given to the buddy. which is bad. But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been...

...t to answer this questions. The __put_page() change is necessary > for the following race I identified: > > Page has a refcount of 1 (e.g., allocated by virtio-mem using > alloc_contig_range()). > > a) kernel: get_page_unless_zero(page): refcount = 2 > b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1 > c) kernel: put_page(page): refcount = 0 > > The page would suddenly be given to the buddy. which is bad. But why cannot you keep the reference count at 1 (do get_page when offlining the page)? In other words as long as the driver knows the page has been...

.... The __put_page() change is necessary >> for the following race I identified: >> >> Page has a refcount of 1 (e.g., allocated by virtio-mem using >> alloc_contig_range()). >> >> a) kernel: get_page_unless_zero(page): refcount = 2 >> b) virtio-mem: set page PG_offline, reduce refcount): refocunt = 1 >> c) kernel: put_page(page): refcount = 0 >> >> The page would suddenly be given to the buddy. which is bad. > > But why cannot you keep the reference count at 1 (do get_page when > offlining the page)? In other words as long as the drive...

...tart_isolate_page_range() > -> offline pages with a reference count of one will be detected as unmovable > -> BAD, we abort right away. We could hack around that. > > 2) memory_notify(MEM_GOING_OFFLINE, &arg); > -> Here, we could release all pages to the buddy, clearing PG_offline > -> BAD, PF_offline must not be cleared so dumping tools will not touch > these pages. I don't see a way to hack around that. > > 3) scan_movable_pages() ... > > 4a) memory_notify(MEM_OFFLINE, &arg); > > Perfect, it worked. Sections are offline. > >...

...tart_isolate_page_range() > -> offline pages with a reference count of one will be detected as unmovable > -> BAD, we abort right away. We could hack around that. > > 2) memory_notify(MEM_GOING_OFFLINE, &arg); > -> Here, we could release all pages to the buddy, clearing PG_offline > -> BAD, PF_offline must not be cleared so dumping tools will not touch > these pages. I don't see a way to hack around that. > > 3) scan_movable_pages() ... > > 4a) memory_notify(MEM_OFFLINE, &arg); > > Perfect, it worked. Sections are offline. > >...

...__offline_pages(): 1) start_isolate_page_range() -> offline pages with a reference count of one will be detected as unmovable -> BAD, we abort right away. We could hack around that. 2) memory_notify(MEM_GOING_OFFLINE, &arg); -> Here, we could release all pages to the buddy, clearing PG_offline -> BAD, PF_offline must not be cleared so dumping tools will not touch these pages. I don't see a way to hack around that. 3) scan_movable_pages() ... 4a) memory_notify(MEM_OFFLINE, &arg); Perfect, it worked. Sections are offline. 4b) undo_isolate_page_range(start_pfn, end_pfn, M...

...c +++ b/mm/page_alloc.c @@ -8221,6 +8221,15 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count, if (!page_ref_count(page)) { if (PageBuddy(page)) iter += (1 << page_order(page)) - 1; + /* + * Memory devices allow to offline a page if it is + * marked PG_offline and has a reference count of 0. + * However, the pages are not movable as it would be + * required e.g., for alloc_contig_range(). + */ + if (PageOffline(page) && !(flags & SKIP_OFFLINE)) + if (++found > count) + goto unmovable; continue; } @@ -8444,7 +8453,7...

...right now). >> >> So it's effectively "re-initializing the memmap using the driver knowledge" >> when onlining. > > I am not sure I follow. So you exclude those pages when onlining? Exactly, using the online_page callback. The pages will - again - be marked PG_offline with a refcount of 0. They will not be put to the buddy. > >>> Should we allow to try_remove_memory to succeed with these pages? >> >> I think we should first properly offline them (mark sections offline and >> memory blocks, fixup numbers, shrink zones ...). The we...

...is done similar to the PPC CMA implementation. The set_online_page() callback is used to keep unplugged subblocks of a memory block fake-offline when onlining the memory block. generic_online_page() is used to fake-online plugged subblocks. This handling is similar to the Hyper-V balloon driver. PG_offline is used to mark unplugged subblocks as offline, so e.g., dumping tools (makedumpfile) will skip these pages. This is similar to other balloon drivers like virtio-balloon and Hyper-V. PG_offline + reference count of 0 [new] is now also used to mark pages as a "skip" when offlining memory...