Linux Virtualization - Jul 2019 - [PATCH v4 1/2] mm/balloon_compaction: avoid duplicate page removal

From: Wei Wang <wei.w.wang at intel.com>

A #GP is reported in the guest when requesting balloon inflation via
virtio-balloon. The reason is that the virtio-balloon driver has
removed the page from its internal page list (via balloon_page_pop),
but balloon_page_enqueue_one also calls "list_del"  to do the removal.
This is necessary when it's used from balloon_page_enqueue_list, but
not from balloon_page_enqueue.

Move list_del to balloon_page_enqueue, and update comments accordingly.

Fixes: 418a3ab1e778 (mm/balloon_compaction: List interfaces)
Signed-off-by: Wei Wang <wei.w.wang at intel.com>
Signed-off-by: Michael S. Tsirkin <mst at redhat.com>
---

This patch is same as v3.

 mm/balloon_compaction.c | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index 83a7b614061f..d25664e1857b 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -21,7 +21,6 @@ static void balloon_page_enqueue_one(struct balloon_dev_info
*b_dev_info,
 	 * memory corruption is possible and we should stop execution.
 	 */
 	BUG_ON(!trylock_page(page));
-	list_del(&page->lru);
 	balloon_page_insert(b_dev_info, page);
 	unlock_page(page);
 	__count_vm_event(BALLOON_INFLATE);
@@ -47,6 +46,7 @@ size_t balloon_page_list_enqueue(struct balloon_dev_info
*b_dev_info,
 
 	spin_lock_irqsave(&b_dev_info->pages_lock, flags);
 	list_for_each_entry_safe(page, tmp, pages, lru) {
+		list_del(&page->lru);
 		balloon_page_enqueue_one(b_dev_info, page);
 		n_pages++;
 	}
@@ -128,13 +128,19 @@ struct page *balloon_page_alloc(void)
 EXPORT_SYMBOL_GPL(balloon_page_alloc);
 
 /*
- * balloon_page_enqueue - allocates a new page and inserts it into the balloon
- *			  page list.
+ * balloon_page_enqueue - inserts a new page into the balloon page list.
+ *
  * @b_dev_info: balloon device descriptor where we will insert a new page to
  * @page: new page to enqueue - allocated using balloon_page_alloc.
  *
  * Driver must call it to properly enqueue a new allocated balloon page
  * before definitively removing it from the guest system.
+ *
+ * Drivers must not call balloon_page_enqueue on pages that have been
+ * pushed to a list with balloon_page_push before removing them with
+ * balloon_page_pop. To all pages on a list, use balloon_page_list_enqueue
+ * instead.
+ *
  * This function returns the page address for the recently enqueued page or
  * NULL in the case we fail to allocate a new page this turn.
  */
-- 
MST

Lots of comments bitrotted. Fix them up.

Fixes: 418a3ab1e778 (mm/balloon_compaction: List interfaces)
Reviewed-by: Wei Wang <wei.w.wang at intel.com>
Signed-off-by: Michael S. Tsirkin <mst at redhat.com>
---

fixes since v3:
	teaks suggested by Wei

 mm/balloon_compaction.c | 71 ++++++++++++++++++++++-------------------
 1 file changed, 39 insertions(+), 32 deletions(-)

diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
index d25664e1857b..7e95d2cd185a 100644
--- a/mm/balloon_compaction.c
+++ b/mm/balloon_compaction.c
@@ -32,10 +32,10 @@ static void balloon_page_enqueue_one(struct balloon_dev_info
*b_dev_info,
  * @b_dev_info: balloon device descriptor where we will insert a new page to
  * @pages: pages to enqueue - allocated using balloon_page_alloc.
  *
- * Driver must call it to properly enqueue a balloon pages before definitively
- * removing it from the guest system.
+ * Driver must call this function to properly enqueue balloon pages before
+ * definitively removing them from the guest system.
  *
- * Return: number of pages that were enqueued.
+ * Returns: number of pages that were enqueued.
  */
 size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info,
 				 struct list_head *pages)
@@ -63,14 +63,15 @@ EXPORT_SYMBOL_GPL(balloon_page_list_enqueue);
  * @n_req_pages: number of requested pages.
  *
  * Driver must call this function to properly de-allocate a previous enlisted
- * balloon pages before definetively releasing it back to the guest system.
+ * balloon pages before definitively releasing it back to the guest system.
  * This function tries to remove @n_req_pages from the ballooned pages and
  * return them to the caller in the @pages list.
  *
- * Note that this function may fail to dequeue some pages temporarily empty due
- * to compaction isolated pages.
+ * Note that this function may fail to dequeue some pages even if the balloon
+ * isn't empty - since the page list can be temporarily empty due to
compaction
+ * of isolated pages.
  *
- * Return: number of pages that were added to the @pages list.
+ * Returns: number of pages that were added to the @pages list.
  */
 size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,
 				 struct list_head *pages, size_t n_req_pages)
@@ -112,12 +113,13 @@ EXPORT_SYMBOL_GPL(balloon_page_list_dequeue);
 
 /*
  * balloon_page_alloc - allocates a new page for insertion into the balloon
- *			  page list.
+ *			page list.
  *
- * Driver must call it to properly allocate a new enlisted balloon page.
- * Driver must call balloon_page_enqueue before definitively removing it from
- * the guest system.  This function returns the page address for the recently
- * allocated page or NULL in the case we fail to allocate a new page this turn.
+ * Driver must call this function to properly allocate a new balloon page.
+ * Driver must call balloon_page_enqueue before definitively removing the page
+ * from the guest system.
+ *
+ * Returns: struct page for the allocated page or NULL on allocation failure.
  */
 struct page *balloon_page_alloc(void)
 {
@@ -130,19 +132,15 @@ EXPORT_SYMBOL_GPL(balloon_page_alloc);
 /*
  * balloon_page_enqueue - inserts a new page into the balloon page list.
  *
- * @b_dev_info: balloon device descriptor where we will insert a new page to
+ * @b_dev_info: balloon device descriptor where we will insert a new page
  * @page: new page to enqueue - allocated using balloon_page_alloc.
  *
- * Driver must call it to properly enqueue a new allocated balloon page
- * before definitively removing it from the guest system.
+ * Drivers must call this function to properly enqueue a new allocated balloon
+ * page before definitively removing the page from the guest system.
  *
- * Drivers must not call balloon_page_enqueue on pages that have been
- * pushed to a list with balloon_page_push before removing them with
- * balloon_page_pop. To all pages on a list, use balloon_page_list_enqueue
- * instead.
- *
- * This function returns the page address for the recently enqueued page or
- * NULL in the case we fail to allocate a new page this turn.
+ * Drivers must not call balloon_page_enqueue on pages that have been pushed to
+ * a list with balloon_page_push before removing them with balloon_page_pop. To
+ * enqueue a list of pages, use balloon_page_list_enqueue instead.
  */
 void balloon_page_enqueue(struct balloon_dev_info *b_dev_info,
 			  struct page *page)
@@ -157,14 +155,23 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);
 
 /*
  * balloon_page_dequeue - removes a page from balloon's page list and
returns
- *			  the its address to allow the driver release the page.
+ *			  its address to allow the driver to release the page.
  * @b_dev_info: balloon device decriptor where we will grab a page from.
  *
- * Driver must call it to properly de-allocate a previous enlisted balloon page
- * before definetively releasing it back to the guest system.
- * This function returns the page address for the recently dequeued page or
- * NULL in the case we find balloon's page list temporarily empty due to
- * compaction isolated pages.
+ * Driver must call this function to properly dequeue a previously enqueued
page
+ * before definitively releasing it back to the guest system.
+ *
+ * Caller must perform its own accounting to ensure that this
+ * function is called only if some pages are actually enqueued.
+ *
+ * Note that this function may fail to dequeue some pages even if there are
+ * some enqueued pages - since the page list can be temporarily empty due to
+ * the compaction of isolated pages.
+ *
+ * TODO: remove the caller accounting requirements, and allow caller to wait
+ * until all pages can be dequeued.
+ *
+ * Returns: struct page for the dequeued page, or NULL if no page was dequeued.
  */
 struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
 {
@@ -177,9 +184,9 @@ struct page *balloon_page_dequeue(struct balloon_dev_info
*b_dev_info)
 	if (n_pages != 1) {
 		/*
 		 * If we are unable to dequeue a balloon page because the page
-		 * list is empty and there is no isolated pages, then something
+		 * list is empty and there are no isolated pages, then something
 		 * went out of track and some balloon pages are lost.
-		 * BUG() here, otherwise the balloon driver may get stuck into
+		 * BUG() here, otherwise the balloon driver may get stuck in
 		 * an infinite loop while attempting to release all its pages.
 		 */
 		spin_lock_irqsave(&b_dev_info->pages_lock, flags);
@@ -230,8 +237,8 @@ int balloon_page_migrate(struct address_space *mapping,
 
 	/*
 	 * We can not easily support the no copy case here so ignore it as it
-	 * is unlikely to be use with ballon pages. See include/linux/hmm.h for
-	 * user of the MIGRATE_SYNC_NO_COPY mode.
+	 * is unlikely to be used with balloon pages. See include/linux/hmm.h
+	 * for a user of the MIGRATE_SYNC_NO_COPY mode.
 	 */
 	if (mode == MIGRATE_SYNC_NO_COPY)
 		return -EINVAL;
-- 
MST

On 7/18/19 7:01 AM, Michael S. Tsirkin wrote:
> Lots of comments bitrotted. Fix them up.
> 
> Fixes: 418a3ab1e778 (mm/balloon_compaction: List interfaces)
> Reviewed-by: Wei Wang <wei.w.wang at intel.com>
> Signed-off-by: Michael S. Tsirkin <mst at redhat.com>
> ---
> 
> fixes since v3:
> 	teaks suggested by Wei
> 
>   mm/balloon_compaction.c | 71 ++++++++++++++++++++++-------------------
>   1 file changed, 39 insertions(+), 32 deletions(-)
> 
> diff --git a/mm/balloon_compaction.c b/mm/balloon_compaction.c
> index d25664e1857b..7e95d2cd185a 100644
> --- a/mm/balloon_compaction.c
> +++ b/mm/balloon_compaction.c
> @@ -32,10 +32,10 @@ static void balloon_page_enqueue_one(struct
balloon_dev_info *b_dev_info,
>    * @b_dev_info: balloon device descriptor where we will insert a new page
to
>    * @pages: pages to enqueue - allocated using balloon_page_alloc.
>    *
> - * Driver must call it to properly enqueue a balloon pages before
definitively
> - * removing it from the guest system.
> + * Driver must call this function to properly enqueue balloon pages before
> + * definitively removing them from the guest system.
>    *
> - * Return: number of pages that were enqueued.
> + * Returns: number of pages that were enqueued.
According to Documentation/doc-guide/kernel-doc.rst,
this is going in the wrong direction and "Return:" is correct.
Ditto for other occurrences below.
>    */
>   size_t balloon_page_list_enqueue(struct balloon_dev_info *b_dev_info,
>   				 struct list_head *pages)
> @@ -63,14 +63,15 @@ EXPORT_SYMBOL_GPL(balloon_page_list_enqueue);
>    * @n_req_pages: number of requested pages.
>    *
>    * Driver must call this function to properly de-allocate a previous
enlisted
> - * balloon pages before definetively releasing it back to the guest
system.
> + * balloon pages before definitively releasing it back to the guest
system.
>    * This function tries to remove @n_req_pages from the ballooned pages
and
>    * return them to the caller in the @pages list.
>    *
> - * Note that this function may fail to dequeue some pages temporarily
empty due
> - * to compaction isolated pages.
> + * Note that this function may fail to dequeue some pages even if the
balloon
> + * isn't empty - since the page list can be temporarily empty due to
compaction
> + * of isolated pages.
>    *
> - * Return: number of pages that were added to the @pages list.
> + * Returns: number of pages that were added to the @pages list.
>    */
>   size_t balloon_page_list_dequeue(struct balloon_dev_info *b_dev_info,
>   				 struct list_head *pages, size_t n_req_pages)
> @@ -112,12 +113,13 @@ EXPORT_SYMBOL_GPL(balloon_page_list_dequeue);
>   
>   /*
>    * balloon_page_alloc - allocates a new page for insertion into the
balloon
> - *			  page list.
> + *			page list.
>    *
> - * Driver must call it to properly allocate a new enlisted balloon page.
> - * Driver must call balloon_page_enqueue before definitively removing it
from
> - * the guest system.  This function returns the page address for the
recently
> - * allocated page or NULL in the case we fail to allocate a new page this
turn.
> + * Driver must call this function to properly allocate a new balloon page.
> + * Driver must call balloon_page_enqueue before definitively removing the
page
> + * from the guest system.
> + *
> + * Returns: struct page for the allocated page or NULL on allocation
failure.
>    */
>   struct page *balloon_page_alloc(void)
>   {
> @@ -130,19 +132,15 @@ EXPORT_SYMBOL_GPL(balloon_page_alloc);
>   /*
>    * balloon_page_enqueue - inserts a new page into the balloon page list.
>    *
> - * @b_dev_info: balloon device descriptor where we will insert a new page
to
> + * @b_dev_info: balloon device descriptor where we will insert a new page
>    * @page: new page to enqueue - allocated using balloon_page_alloc.
>    *
> - * Driver must call it to properly enqueue a new allocated balloon page
> - * before definitively removing it from the guest system.
> + * Drivers must call this function to properly enqueue a new allocated
balloon
> + * page before definitively removing the page from the guest system.
>    *
> - * Drivers must not call balloon_page_enqueue on pages that have been
> - * pushed to a list with balloon_page_push before removing them with
> - * balloon_page_pop. To all pages on a list, use balloon_page_list_enqueue
> - * instead.
> - *
> - * This function returns the page address for the recently enqueued page
or
> - * NULL in the case we fail to allocate a new page this turn.
> + * Drivers must not call balloon_page_enqueue on pages that have been
pushed to
> + * a list with balloon_page_push before removing them with
balloon_page_pop. To
> + * enqueue a list of pages, use balloon_page_list_enqueue instead.
>    */
>   void balloon_page_enqueue(struct balloon_dev_info *b_dev_info,
>   			  struct page *page)
> @@ -157,14 +155,23 @@ EXPORT_SYMBOL_GPL(balloon_page_enqueue);
>   
>   /*
>    * balloon_page_dequeue - removes a page from balloon's page list and
returns
> - *			  the its address to allow the driver release the page.
> + *			  its address to allow the driver to release the page.
>    * @b_dev_info: balloon device decriptor where we will grab a page from.
>    *
> - * Driver must call it to properly de-allocate a previous enlisted balloon
page
> - * before definetively releasing it back to the guest system.
> - * This function returns the page address for the recently dequeued page
or
> - * NULL in the case we find balloon's page list temporarily empty due
to
> - * compaction isolated pages.
> + * Driver must call this function to properly dequeue a previously
enqueued page
> + * before definitively releasing it back to the guest system.
> + *
> + * Caller must perform its own accounting to ensure that this
> + * function is called only if some pages are actually enqueued.
> + *
> + * Note that this function may fail to dequeue some pages even if there
are
> + * some enqueued pages - since the page list can be temporarily empty due
to
> + * the compaction of isolated pages.
> + *
> + * TODO: remove the caller accounting requirements, and allow caller to
wait
> + * until all pages can be dequeued.
> + *
> + * Returns: struct page for the dequeued page, or NULL if no page was
dequeued.
>    */
>   struct page *balloon_page_dequeue(struct balloon_dev_info *b_dev_info)
>   {
> @@ -177,9 +184,9 @@ struct page *balloon_page_dequeue(struct
balloon_dev_info *b_dev_info)
>   	if (n_pages != 1) {
>   		/*
>   		 * If we are unable to dequeue a balloon page because the page
> -		 * list is empty and there is no isolated pages, then something
> +		 * list is empty and there are no isolated pages, then something
>   		 * went out of track and some balloon pages are lost.
> -		 * BUG() here, otherwise the balloon driver may get stuck into
> +		 * BUG() here, otherwise the balloon driver may get stuck in
>   		 * an infinite loop while attempting to release all its pages.
>   		 */
>   		spin_lock_irqsave(&b_dev_info->pages_lock, flags);
> @@ -230,8 +237,8 @@ int balloon_page_migrate(struct address_space *mapping,
>   
>   	/*
>   	 * We can not easily support the no copy case here so ignore it as it
> -	 * is unlikely to be use with ballon pages. See include/linux/hmm.h for
> -	 * user of the MIGRATE_SYNC_NO_COPY mode.
> +	 * is unlikely to be used with balloon pages. See include/linux/hmm.h
> +	 * for a user of the MIGRATE_SYNC_NO_COPY mode.
>   	 */
>   	if (mode == MIGRATE_SYNC_NO_COPY)
>   		return -EINVAL;
> 
Reviewed-by: Ralph Campbell <rcampbell at nvidia.com>