search for: get_free_pag

...ually what Matthew suggested doesn't make a difference here. That method always steal the first free page blocks, and sure can be changed to take more. But all these can be achieved via kmalloc > > I'd do get_user_pages really. You don't want pages split, etc. Oops sorry. I meant get_free_pages . > > > > by the caller which is more prudent and makes the code more straightforward. I think we don't need to take that risk unless the MM folks strongly endorse that approach. > > > > > > The max size of the kmalloc-ed memory is 4MB, which gives us the limi...

...have even been able to cause a panic on a system which does NOT have /proc/sys/kernel/file-max > 1024. As a work-around, the following patch will cause the garbage collection routine to calculate the exact _maximum_ stack depth it must allocate for, as well as using kmalloc()/kfree() instead of get_free_page()/free_page(). - --CUT HERE- *** net/unix/garbage.c.orig Wed Dec 3 14:55:10 1997 - --- net/unix/garbage.c Wed Dec 3 15:04:57 1997 *************** *** 5,10 **** - --- 5,20 ---- * Copyright (C) Barak A. Pearlmutter. * Released under the GPL version 2 or later. * + * 12/3/97 -- Flood +...

The current QEMU live migration implementation mark the all the guest's RAM pages as dirtied in the ram bulk stage, all these pages will be processed and that takes quit a lot of CPU cycles. >From guest's point of view, it doesn't care about the content in free pages. We can make use of this fact and skip processing the free pages in the ram bulk stage, it can save a lot CPU cycles

The current QEMU live migration implementation mark the all the guest's RAM pages as dirtied in the ram bulk stage, all these pages will be processed and that takes quit a lot of CPU cycles. >From guest's point of view, it doesn't care about the content in free pages. We can make use of this fact and skip processing the free pages in the ram bulk stage, it can save a lot CPU cycles

...> --- a/drivers/virtio/virtio_balloon.c > +++ b/drivers/virtio/virtio_balloon.c > @@ -62,10 +62,13 @@ module_param(oom_pages, int, S_IRUSR | S_IWUSR); > MODULE_PARM_DESC(oom_pages, "pages to free on OOM"); > > extern unsigned long get_max_pfn(void); > +extern int get_free_pages(unsigned long start_pfn, unsigned long end_pfn, > + unsigned long *bitmap, unsigned long len); > + > > struct virtio_balloon { > struct virtio_device *vdev; > - struct virtqueue *inflate_vq, *deflate_vq, *stats_vq; > + struct virtqueue *inflate_vq, *deflate_vq, *stats_v...

...> --- a/drivers/virtio/virtio_balloon.c > +++ b/drivers/virtio/virtio_balloon.c > @@ -62,10 +62,13 @@ module_param(oom_pages, int, S_IRUSR | S_IWUSR); > MODULE_PARM_DESC(oom_pages, "pages to free on OOM"); > > extern unsigned long get_max_pfn(void); > +extern int get_free_pages(unsigned long start_pfn, unsigned long end_pfn, > + unsigned long *bitmap, unsigned long len); > + > > struct virtio_balloon { > struct virtio_device *vdev; > - struct virtqueue *inflate_vq, *deflate_vq, *stats_vq; > + struct virtqueue *inflate_vq, *deflate_vq, *stats_v...

...; + if (pfn >= start_pfn && pfn <= end_pfn) { + page_num = 1UL << order; + if (pfn + page_num > end_pfn) + page_num = end_pfn - pfn; + bitmap_set(bitmap, pfn - start_pfn, page_num); + } + } + } + + spin_unlock_irqrestore(&zone->lock, flags); +} + +int get_free_pages(unsigned long start_pfn, unsigned long end_pfn, + unsigned long *bitmap, unsigned long len) +{ + struct zone *zone; + int ret = 0; + + if (bitmap == NULL || start_pfn > end_pfn || start_pfn >= max_pfn) + return 0; + if (end_pfn < max_pfn) + ret = 1; + if (end_pfn >= max_pfn) + ret...

...nce here. > > > > That method always steal the first free page blocks, and sure can > > > > be changed to take more. But all these can be achieved via kmalloc > > > I'd do get_user_pages really. You don't want pages split, etc. > > Oops sorry. I meant get_free_pages . Yes, we can use __get_free_pages, and the max allocation is MAX_ORDER - 1, which can report up to 2TB free memory. "getting two pages isn't harder", do you mean passing two arrays (two allocations by get_free_pages(,MAX_ORDER -1)) to the mm API? Please see if the following log...

...{ > + page_num = 1UL << order; > + if (pfn + page_num > end_pfn) > + page_num = end_pfn - pfn; > + bitmap_set(bitmap, pfn - start_pfn, page_num); > + } > + } > + } > + > + spin_unlock_irqrestore(&zone->lock, flags); > +} > + > +int get_free_pages(unsigned long start_pfn, unsigned long end_pfn, > + unsigned long *bitmap, unsigned long len) > +{ > + struct zone *zone; > + int ret = 0; > + > + if (bitmap == NULL || start_pfn > end_pfn || start_pfn >= max_pfn) > + return 0; > + if (end_pfn < max_pfn) > +...

...{ > + page_num = 1UL << order; > + if (pfn + page_num > end_pfn) > + page_num = end_pfn - pfn; > + bitmap_set(bitmap, pfn - start_pfn, page_num); > + } > + } > + } > + > + spin_unlock_irqrestore(&zone->lock, flags); > +} > + > +int get_free_pages(unsigned long start_pfn, unsigned long end_pfn, > + unsigned long *bitmap, unsigned long len) > +{ > + struct zone *zone; > + int ret = 0; > + > + if (bitmap == NULL || start_pfn > end_pfn || start_pfn >= max_pfn) > + return 0; > + if (end_pfn < max_pfn) > +...

...oon.c index 2d18ff6..5ca4ad3 100644 --- a/drivers/virtio/virtio_balloon.c +++ b/drivers/virtio/virtio_balloon.c @@ -62,10 +62,13 @@ module_param(oom_pages, int, S_IRUSR | S_IWUSR); MODULE_PARM_DESC(oom_pages, "pages to free on OOM"); extern unsigned long get_max_pfn(void); +extern int get_free_pages(unsigned long start_pfn, unsigned long end_pfn, + unsigned long *bitmap, unsigned long len); + struct virtio_balloon { struct virtio_device *vdev; - struct virtqueue *inflate_vq, *deflate_vq, *stats_vq; + struct virtqueue *inflate_vq, *deflate_vq, *stats_vq, *misc_vq; /* The balloon ser...

This patchset is for kernel and contains two parts of change to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use bitmap to send the page information to host instead of the PFNs, to reduce the overhead of virtio data transmission, address translation and madvise(). This can help to improve the performance by

This patchset is for kernel and contains two parts of change to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use bitmap to send the page information to host instead of the PFNs, to reduce the overhead of virtio data transmission, address translation and madvise(). This can help to improve the performance by

...void update_free_pages_stats(struct virtio_balloon *vb, > > why _stats? Will change. > > + max_pfn = get_max_pfn(); > > + mutex_lock(&vb->balloon_lock); > > + while (pfn < max_pfn) { > > + memset(vb->page_bitmap, 0, vb->bmap_len); > > + ret = get_free_pages(pfn, pfn + vb->pfn_limit, > > + vb->page_bitmap, vb->bmap_len * BITS_PER_BYTE); > > + hdr->cmd = cpu_to_virtio16(vb->vdev, > BALLOON_GET_FREE_PAGES); > > + hdr->page_shift = cpu_to_virtio16(vb->vdev, PAGE_SHIFT); > > + hdr->req_id = cpu_to_v...

On Tue, Jun 19, 2018 at 01:06:48AM +0000, Wang, Wei W wrote: > On Monday, June 18, 2018 10:29 AM, Michael S. Tsirkin wrote: > > On Sat, Jun 16, 2018 at 01:09:44AM +0000, Wang, Wei W wrote: > > > Not necessarily, I think. We have min(4m_page_blocks / 512, 1024) above, > > so the maximum memory that can be reported is 2TB. For larger guests, e.g. > > 4TB, the

...g/Documentation/fault-injection/fault-injection.txt 2018-04-16 21:08:34.000000000 +0200 > +++ linux-2.6/Documentation/fault-injection/fault-injection.txt 2018-04-25 21:36:36.000000000 +0200 > @@ -15,6 +15,12 @@ o fail_page_alloc > > injects page allocation failures. (alloc_pages(), get_free_pages(), ...) > > +o kvmalloc_faillback kvmalloc_fallback > + > + makes the function kvmalloc randonly fall back to vmalloc. This could be used randomly > + to detects bugs such as using DMA-API on the result of kvmalloc or freeing > + th...

...g/Documentation/fault-injection/fault-injection.txt 2018-04-16 21:08:34.000000000 +0200 > +++ linux-2.6/Documentation/fault-injection/fault-injection.txt 2018-04-25 21:36:36.000000000 +0200 > @@ -15,6 +15,12 @@ o fail_page_alloc > > injects page allocation failures. (alloc_pages(), get_free_pages(), ...) > > +o kvmalloc_faillback kvmalloc_fallback > + > + makes the function kvmalloc randonly fall back to vmalloc. This could be used randomly > + to detects bugs such as using DMA-API on the result of kvmalloc or freeing > + th...

This patch set contains two parts of changes to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use bitmap to send the page information to host instead of the PFNs, to reduce the overhead of virtio data transmission, address translation and madvise(). This can help to improve the performance by about 85%.

This patch set contains two parts of changes to the virtio-balloon. One is the change for speeding up the inflating & deflating process, the main idea of this optimization is to use bitmap to send the page information to host instead of the PFNs, to reduce the overhead of virtio data transmission, address translation and madvise(). This can help to improve the performance by about 85%.

On Tue 24-04-18 13:28:49, Mikulas Patocka wrote: > > > On Tue, 24 Apr 2018, Michal Hocko wrote: > > > On Tue 24-04-18 13:00:11, Mikulas Patocka wrote: > > > > > > > > > On Tue, 24 Apr 2018, Michal Hocko wrote: > > > > > > > On Tue 24-04-18 11:50:30, Mikulas Patocka wrote: > > > > > > > > > >