search for: gfp_nowait

...ilures on my production supermicro servers resulted in some interesting findings: Jul 9 05:02:47 xen kernel: [3040088.089379] gzip: page allocation failure: order:0, mode:0x2080020(GFP_ATOMIC) Jul 10 12:18:01 xen kernel: [3152495.802565] 2.xvda5-0: page allocation failure: order:0, mode:0x2200000(GFP_NOWAIT|__GFP_NOTRACK) Jul 10 12:18:01 xen kernel: [3152495.815871] SLUB: Unable to allocate memory on node -1, gfp=0x2000000(GFP_NOWAIT) Jul 10 12:18:01 xen kernel: [3152495.816826] 2.xvda5-0: page allocation failure: order:0, mode:0x2200000(GFP_NOWAIT|__GFP_NOTRACK) Jul 10 12:18:01 xen kernel: [3152495.8...

...43312] CPU: 0 PID: 28450 Comm: > 12.xvda3-0 Tainted: G O 4.9.34-29.el6.x86_64 #1 > Jul 16 06:01:03 guest kernel: end_request: I/O error, dev xvda3, sector > 9200640 > Jul 16 06:01:03 dom0 kernel: [452360.758931] SLUB: Unable to allocate > memory on node -1, gfp=0x2000000(GFP_NOWAIT) > Jul 16 06:01:03 guest kernel: Buffer I/O error on device xvda3, logical > block 1150080 > Jul 16 06:01:03 guest kernel: lost page write due to I/O error on xvda3 > Jul 16 06:01:03 guest kernel: Buffer I/O error on device xvda3, logical > block 1150081 > Jul 16 06:01:03 guest ke...

...03 dom0 kernel: [452360.743312] CPU: 0 PID: 28450 Comm: 12.xvda3-0 Tainted: G O 4.9.34-29.el6.x86_64 #1 Jul 16 06:01:03 guest kernel: end_request: I/O error, dev xvda3, sector 9200640 Jul 16 06:01:03 dom0 kernel: [452360.758931] SLUB: Unable to allocate memory on node -1, gfp=0x2000000(GFP_NOWAIT) Jul 16 06:01:03 guest kernel: Buffer I/O error on device xvda3, logical block 1150080 Jul 16 06:01:03 guest kernel: lost page write due to I/O error on xvda3 Jul 16 06:01:03 guest kernel: Buffer I/O error on device xvda3, logical block 1150081 Jul 16 06:01:03 guest kernel: lost page write due to I...

...tthew Wilcox wrote: > > On Sat, Dec 23, 2017 at 11:59:54AM +0900, Tetsuo Handa wrote: > > > Matthew Wilcox wrote: > > > > + bit %= IDA_BITMAP_BITS; > > > > + radix_tree_iter_init(&iter, index); > > > > + slot = idr_get_free_cmn(root, &iter, GFP_NOWAIT | __GFP_NOWARN, index); > > > > + if (IS_ERR(slot)) { > > > > + if (slot == ERR_PTR(-ENOSPC)) > > > > + return 0; /* Already set */ > > > > > > Why already set? I guess something is there, but is it guaranteed that > > > there is a...

...e added (i.e. num > > 1) and the implementation inside virtqueue_add_inbuf > need allocation of indirect descriptor table. > > We could also add some comments above the function to explain a little > about this if necessary. Yes, please do so. Or maybe replace GFP_KERNEL with GFP_NOWAIT or 0. Though Michael might remove that GFP argument ( http://lkml.kernel.org/r/201710022344.JII17368.HQtLOMJOOSFFVF at I-love.SAKURA.ne.jp ). > > If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not > > use __GFP_DIRECT_RECLAIM allocation, for leak_ball...

...e added (i.e. num > > 1) and the implementation inside virtqueue_add_inbuf > need allocation of indirect descriptor table. > > We could also add some comments above the function to explain a little > about this if necessary. Yes, please do so. Or maybe replace GFP_KERNEL with GFP_NOWAIT or 0. Though Michael might remove that GFP argument ( http://lkml.kernel.org/r/201710022344.JII17368.HQtLOMJOOSFFVF at I-love.SAKURA.ne.jp ). > > If this is inside vb->balloon_lock mutex (isn't this?), xb_set_page() must not > > use __GFP_DIRECT_RECLAIM allocation, for leak_ball...

On 07/19/2017 04:27 AM, Piotr Gackiewicz wrote: > On Mon, 17 Jul 2017, Johnny Hughes wrote: > >> Are the testing kernels (kernel-4.9.37-29.el7 and kernel-4.9.37-29.el6, >> with the one config file change) working for everyone: >> >> (turn off: CONFIG_IO_STRICT_DEVMEM) > > Hello. > Maybe it's not the most appropriate thread or time, but I have been >

On Tue, Dec 12, 2017 at 07:55:55PM +0800, Wei Wang wrote: > +int xb_preload_and_set_bit(struct xb *xb, unsigned long bit, gfp_t gfp); I'm struggling to understand when one would use this. The xb_ API requires you to handle your own locking. But specifying GFP flags here implies you can sleep. So ... um ... there's no locking? > +void xb_clear_bit_range(struct xb *xb, unsigned

On Tue, Dec 12, 2017 at 07:55:55PM +0800, Wei Wang wrote: > +int xb_preload_and_set_bit(struct xb *xb, unsigned long bit, gfp_t gfp); I'm struggling to understand when one would use this. The xb_ API requires you to handle your own locking. But specifying GFP flags here implies you can sleep. So ... um ... there's no locking? > +void xb_clear_bit_range(struct xb *xb, unsigned

...t/clear within the lock. >> >> In the virtio-balloon usage, we have a large number of bits to set with >> the balloon_lock being held (we're not unlocking for each bit), so we >> used the above wrapper to do preload and set within the balloon_lock, >> and passed in GFP_NOWAIT to avoid sleeping. Probably we can change to >> put this wrapper implementation to virtio-balloon, since it would not be >> useful for the regular cases. > GFP_NOWAIT is chosen in order not to try to OOM-kill something, isn't it? Yes, I think that's right the issue we are di...

On Sat, Dec 23, 2017 at 11:59:54AM +0900, Tetsuo Handa wrote: > Matthew Wilcox wrote: > > + bit %= IDA_BITMAP_BITS; > > + radix_tree_iter_init(&iter, index); > > + slot = idr_get_free_cmn(root, &iter, GFP_NOWAIT | __GFP_NOWARN, index); > > + if (IS_ERR(slot)) { > > + if (slot == ERR_PTR(-ENOSPC)) > > + return 0; /* Already set */ > > Why already set? I guess something is there, but is it guaranteed that > there is a bitmap with the "bit" set? Yes. For radix trees...

Wei Wang wrote: > @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) > break; > } > > + if (use_sg && xb_set_page(vb, page, &pfn_min, &pfn_max) < 0) Isn't this leaking "page" ? > + break; > balloon_page_push(&pages, page); > } > > @@ -184,8 +307,12 @@ static unsigned

Wei Wang wrote: > @@ -164,6 +284,8 @@ static unsigned fill_balloon(struct virtio_balloon *vb, size_t num) > break; > } > > + if (use_sg && xb_set_page(vb, page, &pfn_min, &pfn_max) < 0) Isn't this leaking "page" ? > + break; > balloon_page_push(&pages, page); > } > > @@ -184,8 +307,12 @@ static unsigned

...unsigned long max); unsigned long xbit_find_set(struct xbitmap *, unsigned long start, unsigned long max); > static bool set_ulong(struct ulong_list_head *head, const unsigned long value) > { > if (!ptr) { > ptr = kzalloc(sizeof(*ptr), GFP_NOWAIT | __GFP_NOWARN); > if (!ptr) > goto out1; > ptr->bitmap = kzalloc(BITMAP_LEN / 8, > GFP_NOWAIT | __GFP_NOWARN); > if (!ptr->bitmap) > goto out2; > if (btree_insertl(&head->btree, ~segment, ptr, > GFP_NOWAIT | __GFP_NOWARN)) >...

..., Tetsuo Handa wrote: > Matthew Wilcox wrote: >>> + unsigned long pfn = page_to_pfn(page); >>> + int ret; >>> + >>> + *pfn_min = min(pfn, *pfn_min); >>> + *pfn_max = max(pfn, *pfn_max); >>> + >>> + do { >>> + if (xb_preload(GFP_NOWAIT | __GFP_NOWARN) < 0) >>> + return -ENOMEM; >>> + >>> + ret = xb_set_bit(&vb->page_xb, pfn); >>> + xb_preload_end(); >>> + } while (unlikely(ret == -EAGAIN)); >> OK, so you don't need a spinlock because you're under a mutex? B...

...because oom_lock mutex is already held before calling out_of_memory(). > > > > > > I guess we should just do > > > > > > GFP_KERNEL & ~__GFP_DIRECT_RECLAIM there then? > > > > Yes, but GFP_KERNEL & ~__GFP_DIRECT_RECLAIM will effectively be GFP_NOWAIT, for > > __GFP_IO and __GFP_FS won't make sense without __GFP_DIRECT_RECLAIM. It might > > significantly increases possibility of memory allocation failure. > > > > > > > > > > > > > > > > OOM notifier callback should not (directly...

Hello. I noticed that virtio_balloon is using register_oom_notifier() and leak_balloon() from virtballoon_oom_notify() might depend on __GFP_DIRECT_RECLAIM memory allocation. In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to serialize against fill_balloon(). But in fill_balloon(), alloc_page(GFP_HIGHUSER[_MOVABLE] | __GFP_NOMEMALLOC | __GFP_NORETRY) is called with

Hello. I noticed that virtio_balloon is using register_oom_notifier() and leak_balloon() from virtballoon_oom_notify() might depend on __GFP_DIRECT_RECLAIM memory allocation. In leak_balloon(), mutex_lock(&vb->balloon_lock) is called in order to serialize against fill_balloon(). But in fill_balloon(), alloc_page(GFP_HIGHUSER[_MOVABLE] | __GFP_NOMEMALLOC | __GFP_NORETRY) is called with

On Tue, Jul 10, 2018 at 6:24 PM Wei Wang <wei.w.wang at intel.com> wrote: > > We only get addresses of the "MAX_ORDER-1" blocks into the array. The > max size of the array that could be allocated by kmalloc is > KMALLOC_MAX_SIZE (i.e. 4MB on x86). With that max array, we could load > "4MB / sizeof(u64)" addresses of "MAX_ORDER-1" blocks, that is,

On Tue, Jul 10, 2018 at 6:24 PM Wei Wang <wei.w.wang at intel.com> wrote: > > We only get addresses of the "MAX_ORDER-1" blocks into the array. The > max size of the array that could be allocated by kmalloc is > KMALLOC_MAX_SIZE (i.e. 4MB on x86). With that max array, we could load > "4MB / sizeof(u64)" addresses of "MAX_ORDER-1" blocks, that is,