On Wed, 28 Aug 2013 12:53:17 -0700 Kent Overstreet <kmo at daterainc.com> wrote:> > > + while (1) { > > > + spin_lock(&pool->lock); > > > + > > > + /* > > > + * prepare_to_wait() must come before steal_tags(), in case > > > + * percpu_ida_free() on another cpu flips a bit in > > > + * cpus_have_tags > > > + * > > > + * global lock held and irqs disabled, don't need percpu lock > > > + */ > > > + prepare_to_wait(&pool->wait, &wait, TASK_UNINTERRUPTIBLE); > > > + > > > + if (!tags->nr_free) > > > + alloc_global_tags(pool, tags); > > > + if (!tags->nr_free) > > > + steal_tags(pool, tags); > > > + > > > + if (tags->nr_free) { > > > + tag = tags->freelist[--tags->nr_free]; > > > + if (tags->nr_free) > > > + set_bit(smp_processor_id(), > > > + pool->cpus_have_tags); > > > + } > > > + > > > + spin_unlock(&pool->lock); > > > + local_irq_restore(flags); > > > + > > > + if (tag >= 0 || !(gfp & __GFP_WAIT)) > > > + break; > > > + > > > + schedule(); > > > + > > > + local_irq_save(flags); > > > + tags = this_cpu_ptr(pool->tag_cpu); > > > + } > > > > What guarantees that this wait will terminate? > > It seems fairly clear to me from the break statement a couple lines up; > if we were passed __GFP_WAIT we terminate iff we succesfully allocated a > tag. If we weren't passed __GFP_WAIT we never actually sleep.OK ;) Let me rephrase. What guarantees that a tag will become available? If what we have here is an open-coded __GFP_NOFAIL then that is potentially problematic.
On Wed, Aug 28, 2013 at 01:23:32PM -0700, Andrew Morton wrote:> On Wed, 28 Aug 2013 12:53:17 -0700 Kent Overstreet <kmo at daterainc.com> wrote: > > > > > + while (1) { > > > > + spin_lock(&pool->lock); > > > > + > > > > + /* > > > > + * prepare_to_wait() must come before steal_tags(), in case > > > > + * percpu_ida_free() on another cpu flips a bit in > > > > + * cpus_have_tags > > > > + * > > > > + * global lock held and irqs disabled, don't need percpu lock > > > > + */ > > > > + prepare_to_wait(&pool->wait, &wait, TASK_UNINTERRUPTIBLE); > > > > + > > > > + if (!tags->nr_free) > > > > + alloc_global_tags(pool, tags); > > > > + if (!tags->nr_free) > > > > + steal_tags(pool, tags); > > > > + > > > > + if (tags->nr_free) { > > > > + tag = tags->freelist[--tags->nr_free]; > > > > + if (tags->nr_free) > > > > + set_bit(smp_processor_id(), > > > > + pool->cpus_have_tags); > > > > + } > > > > + > > > > + spin_unlock(&pool->lock); > > > > + local_irq_restore(flags); > > > > + > > > > + if (tag >= 0 || !(gfp & __GFP_WAIT)) > > > > + break; > > > > + > > > > + schedule(); > > > > + > > > > + local_irq_save(flags); > > > > + tags = this_cpu_ptr(pool->tag_cpu); > > > > + } > > > > > > What guarantees that this wait will terminate? > > > > It seems fairly clear to me from the break statement a couple lines up; > > if we were passed __GFP_WAIT we terminate iff we succesfully allocated a > > tag. If we weren't passed __GFP_WAIT we never actually sleep. > > OK ;) Let me rephrase. What guarantees that a tag will become available? > > If what we have here is an open-coded __GFP_NOFAIL then that is > potentially problematic.It's the same semantics as a mempool, really - it'll succeed when a tag gets freed. If we are sleeping then there isn't really anything else we can do, there isn't anything we're trying in the __GFP_WAIT case that we're not trying in the GFP_NOWAIT case.
On Wed, 28 Aug 2013 13:44:54 -0700 Kent Overstreet <kmo at daterainc.com> wrote:> > > > What guarantees that this wait will terminate? > > > > > > It seems fairly clear to me from the break statement a couple lines up; > > > if we were passed __GFP_WAIT we terminate iff we succesfully allocated a > > > tag. If we weren't passed __GFP_WAIT we never actually sleep. > > > > OK ;) Let me rephrase. What guarantees that a tag will become available? > > > > If what we have here is an open-coded __GFP_NOFAIL then that is > > potentially problematic. > > It's the same semantics as a mempool, really - it'll succeed when a tag > gets freed.OK, that's reasonable if the code is being used to generate IO tags - we expect the in-flight tags to eventually be returned. But if a client of this code is using the allocator for something totally different, there is no guarantee that the act of waiting will result in any tags being returned. (These are core design principles/constraints which should be explicitly documented in a place where future readers will see them!)