search for: __offline_isolated_pages

...> > > Yeah, set_page_count(page, 0) if you do not want to release that page > > from the notifier context to reflect that the page is ok to be offlined > > with the rest. > > > > I neither see how you deal with __test_page_isolated_in_pageblock() nor with > __offline_isolated_pages(). Sorry, but what I read is incomplete and you > probably have a full proposal in your head. Please read below how I think > you want to solve it. Yeah, sorry that I am throwing incomplete ideas at you. I am just trying to really nail down how to deal with reference counting here because it...

...> > > Yeah, set_page_count(page, 0) if you do not want to release that page > > from the notifier context to reflect that the page is ok to be offlined > > with the rest. > > > > I neither see how you deal with __test_page_isolated_in_pageblock() nor with > __offline_isolated_pages(). Sorry, but what I read is incomplete and you > probably have a full proposal in your head. Please read below how I think > you want to solve it. Yeah, sorry that I am throwing incomplete ideas at you. I am just trying to really nail down how to deal with reference counting here because it...

...o avoid putting pages to the buddy. > > Yeah, set_page_count(page, 0) if you do not want to release that page > from the notifier context to reflect that the page is ok to be offlined > with the rest. > I neither see how you deal with __test_page_isolated_in_pageblock() nor with __offline_isolated_pages(). Sorry, but what I read is incomplete and you probably have a full proposal in your head. Please read below how I think you want to solve it. > >>> explicit control via the reference count which is the standard way to >>> control the struct page life cycle. >>> &gt...

On Fri 18-10-19 14:35:06, David Hildenbrand wrote: > On 18.10.19 13:20, Michal Hocko wrote: > > On Fri 18-10-19 10:50:24, David Hildenbrand wrote: > > > On 18.10.19 10:15, Michal Hocko wrote: [...] > > > > for that - MEM_GOING_OFFLINE notification. This sounds like a good place > > > > for the driver to decide whether it is safe to let the page go or not.

On Fri 18-10-19 14:35:06, David Hildenbrand wrote: > On 18.10.19 13:20, Michal Hocko wrote: > > On Fri 18-10-19 10:50:24, David Hildenbrand wrote: > > > On 18.10.19 10:15, Michal Hocko wrote: [...] > > > > for that - MEM_GOING_OFFLINE notification. This sounds like a good place > > > > for the driver to decide whether it is safe to let the page go or not.

..., set_page_count(page, 0) if you do not want to release that page >>> from the notifier context to reflect that the page is ok to be offlined >>> with the rest. >>> >> >> I neither see how you deal with __test_page_isolated_in_pageblock() nor with >> __offline_isolated_pages(). Sorry, but what I read is incomplete and you >> probably have a full proposal in your head. Please read below how I think >> you want to solve it. > > Yeah, sorry that I am throwing incomplete ideas at you. I am just trying > to really nail down how to deal with reference c...

...ing to actually + * move these pages that still have a reference count > 0. + * (false negatives in this function only) + */ + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + continue; + if (__PageMovable(page) || PageLRU(page)) continue; @@ -8786,6 +8799,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) offlined_pages++; continue; } + /* + * At this point all remaining PageOffline() pages have a + * reference count of 0 and can simply be skipped. + */ + if (PageOffline(page)) { + BUG_ON(page_count(page)); + BUG_ON(PageBuddy(pag...

...ing to actually + * move these pages that still have a reference count > 0. + * (false negatives in this function only) + */ + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + continue; + if (__PageMovable(page) || PageLRU(page)) continue; @@ -8786,6 +8799,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) offlined_pages++; continue; } + /* + * At this point all remaining PageOffline() pages have a + * reference count of 0 and can simply be skipped. + */ + if (PageOffline(page)) { + BUG_ON(page_count(page)); + BUG_ON(PageBuddy(pag...

...ing to actually + * move these pages that still have a reference count > 0. + * (false negatives in this function only) + */ + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) + continue; + if (__PageMovable(page) || PageLRU(page)) continue; @@ -8650,6 +8663,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) offlined_pages++; continue; } + /* + * At this point all remaining PageOffline() pages have a + * reference count of 0 and can simply be skipped. + */ + if (PageOffline(page)) { + BUG_ON(page_count(page)); + BUG_ON(PageBuddy(pag...

...till have a reference count > 0. > + * (false negatives in this function only) > + */ > + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) > + continue; > + > if (__PageMovable(page) || PageLRU(page)) > continue; > > @@ -8786,6 +8799,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) > offlined_pages++; > continue; > } > + /* > + * At this point all remaining PageOffline() pages have a > + * reference count of 0 and can simply be skipped. > + */ > + if (PageOffline(page)) { > + BUG_O...

...end, } /* Make sure the range is really isolated. */ - if (test_pages_isolated(outer_start, end, false)) { + if (test_pages_isolated(outer_start, end, 0)) { pr_info_ratelimited("%s: [%lx, %lx) PFNs busy\n", __func__, outer_start, end); ret = -EBUSY; @@ -8563,6 +8572,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) offlined_pages++; continue; } + /* + * Memory devices allow to offline a page if it is marked + * PG_offline and has a reference count of 0. + */ + if (PageOffline(page) && !page_count(page)) { + BUG_ON(PageBuddy(page)); +...

On Wed 23-10-19 12:03:51, David Hildenbrand wrote: > >Do you see any downsides? > > The only downside I see is that we get more false negatives on > has_unmovable_pages(), eventually resulting in the offlining stage after > isolation to loop forever (as some PageOffline() pages are not movable > (especially, XEN balloon, HyperV balloon), there won't be progress). >

On Wed 23-10-19 12:03:51, David Hildenbrand wrote: > >Do you see any downsides? > > The only downside I see is that we get more false negatives on > has_unmovable_pages(), eventually resulting in the offlining stage after > isolation to loop forever (as some PageOffline() pages are not movable > (especially, XEN balloon, HyperV balloon), there won't be progress). >

...till have a reference count > 0. > + * (false negatives in this function only) > + */ > + if ((flags & MEMORY_OFFLINE) && PageOffline(page)) > + continue; > + > if (__PageMovable(page) || PageLRU(page)) > continue; > > @@ -8786,6 +8799,17 @@ __offline_isolated_pages(unsigned long start_pfn, unsigned long end_pfn) > offlined_pages++; > continue; > } > + /* > + * At this point all remaining PageOffline() pages have a > + * reference count of 0 and can simply be skipped. > + */ > + if (PageOffline(page)) { > + BUG_O...

This series is based on latest linux-next. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-v2 I now have acks for all !virtio-mem changes. I'll be happy to get review feedback, testing reports, etc. for the virtio-mem changes. If there are no further comments, I guess this is good to go as a v1 soon. The basic idea of virtio-mem is to provide a

This series is based on latest linux-next. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-v3 Patch #1 - #10 where contained in v2 and only contain minor modifications (mostly smaller fixes). The remaining patches are new and contain smaller optimizations. Details about virtio-mem can be found in the cover letter of v2 [1]. A basic QEMU implementation was

This series is based on latest linux-next. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-v3 Patch #1 - #10 where contained in v2 and only contain minor modifications (mostly smaller fixes). The remaining patches are new and contain smaller optimizations. Details about virtio-mem can be found in the cover letter of v2 [1]. A basic QEMU implementation was

This series is based on latest linux-next. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-v1 The basic idea of virtio-mem is to provide a flexible, cross-architecture memory hot(un)plug solution that avoids many limitations imposed by existing technologies, architectures, and interfaces. More details can be found below and in linked material. It's

This series is based on latest linux-next. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-v1 The basic idea of virtio-mem is to provide a flexible, cross-architecture memory hot(un)plug solution that avoids many limitations imposed by existing technologies, architectures, and interfaces. More details can be found below and in linked material. It's

This series is based on v5.7-rc4. The patches are located at: https://github.com/davidhildenbrand/linux.git virtio-mem-v4 This is basically a resend of v3 [1], now based on v5.7-rc4 and restested. One patch was reshuffled and two ACKs I missed to add were added. The rebase did not require any modifications to patches. Details about virtio-mem can be found in the cover letter of v2 [2]. A