Dan Williams
2019-Apr-11 14:51 UTC
[PATCH v5 1/6] libnvdimm: nd_region flush callback support
On Tue, Apr 9, 2019 at 9:09 PM Pankaj Gupta <pagupta at redhat.com> wrote:> > This patch adds functionality to perform flush from guest > to host over VIRTIO. We are registering a callback based > on 'nd_region' type. virtio_pmem driver requires this special > flush function. For rest of the region types we are registering > existing flush function. Report error returned by host fsync > failure to userspace. > > This also handles asynchronous flush requests from the block layer > by creating a child bio and chaining it with parent bio. > > Signed-off-by: Pankaj Gupta <pagupta at redhat.com> > ---[..]> diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c > index b4ef7d9ff22e..fb1041ab32a6 100644 > --- a/drivers/nvdimm/region_devs.c > +++ b/drivers/nvdimm/region_devs.c > @@ -295,7 +295,9 @@ static ssize_t deep_flush_store(struct device *dev, struct device_attribute *att > return rc; > if (!flush) > return -EINVAL; > - nvdimm_flush(nd_region); > + rc = nvdimm_flush(nd_region, NULL, false); > + if (rc) > + return rc; > > return len; > } > @@ -1085,6 +1087,11 @@ static struct nd_region *nd_region_create(struct nvdimm_bus *nvdimm_bus, > dev->of_node = ndr_desc->of_node; > nd_region->ndr_size = resource_size(ndr_desc->res); > nd_region->ndr_start = ndr_desc->res->start; > + if (ndr_desc->flush) > + nd_region->flush = ndr_desc->flush; > + else > + nd_region->flush = generic_nvdimm_flush; > + > nd_device_register(dev); > > return nd_region; > @@ -1125,11 +1132,36 @@ struct nd_region *nvdimm_volatile_region_create(struct nvdimm_bus *nvdimm_bus, > } > EXPORT_SYMBOL_GPL(nvdimm_volatile_region_create); > > +int nvdimm_flush(struct nd_region *nd_region, struct bio *bio, bool async) > +{I don't quite see the point of the 'async' argument. All the usages of this routine are either nvdimm_flush(nd_region, bio, true) ...or: nvdimm_flush(nd_region, NULL, false) ...so why not gate async behavior on the presence of the 'bio' argument?> + int rc = 0; > + > + /* Create child bio for asynchronous flush and chain with > + * parent bio. Otherwise directly call nd_region flush. > + */ > + if (async && bio->bi_iter.bi_sector != -1) { > + > + struct bio *child = bio_alloc(GFP_ATOMIC, 0); > + > + if (!child) > + return -ENOMEM; > + bio_copy_dev(child, bio); > + child->bi_opf = REQ_PREFLUSH; > + child->bi_iter.bi_sector = -1; > + bio_chain(child, bio); > + submit_bio(child);I understand how this works, but it's a bit too "magical" for my taste. I would prefer that all flush implementations take an optional 'bio' argument rather than rely on the make_request implementation to stash the bio away on a driver specific list.> + } else { > + if (nd_region->flush(nd_region)) > + rc = -EIO;Given the common case wants to be fast and synchronous I think we should try to avoid retpoline overhead by default. So something like this: if (nd_region->flush == generic_nvdimm_flush) rc = generic_nvdimm_flush(...);
Pankaj Gupta
2019-Apr-11 15:57 UTC
[Qemu-devel] [PATCH v5 1/6] libnvdimm: nd_region flush callback support
> > > > This patch adds functionality to perform flush from guest > > to host over VIRTIO. We are registering a callback based > > on 'nd_region' type. virtio_pmem driver requires this special > > flush function. For rest of the region types we are registering > > existing flush function. Report error returned by host fsync > > failure to userspace. > > > > This also handles asynchronous flush requests from the block layer > > by creating a child bio and chaining it with parent bio. > > > > Signed-off-by: Pankaj Gupta <pagupta at redhat.com> > > ---bio_chain Dan williams > [..] > > diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c > > index b4ef7d9ff22e..fb1041ab32a6 100644 > > --- a/drivers/nvdimm/region_devs.c > > +++ b/drivers/nvdimm/region_devs.c > > @@ -295,7 +295,9 @@ static ssize_t deep_flush_store(struct device *dev, > > struct device_attribute *att > > return rc; > > if (!flush) > > return -EINVAL; > > - nvdimm_flush(nd_region); > > + rc = nvdimm_flush(nd_region, NULL, false); > > + if (rc) > > + return rc; > > > > return len; > > } > > @@ -1085,6 +1087,11 @@ static struct nd_region *nd_region_create(struct > > nvdimm_bus *nvdimm_bus, > > dev->of_node = ndr_desc->of_node; > > nd_region->ndr_size = resource_size(ndr_desc->res); > > nd_region->ndr_start = ndr_desc->res->start; > > + if (ndr_desc->flush) > > + nd_region->flush = ndr_desc->flush; > > + else > > + nd_region->flush = generic_nvdimm_flush; > > + > > nd_device_register(dev); > > > > return nd_region; > > @@ -1125,11 +1132,36 @@ struct nd_region > > *nvdimm_volatile_region_create(struct nvdimm_bus *nvdimm_bus, > > } > > EXPORT_SYMBOL_GPL(nvdimm_volatile_region_create); > > > > +int nvdimm_flush(struct nd_region *nd_region, struct bio *bio, bool async) > > +{ > > I don't quite see the point of the 'async' argument. All the usages of > this routine are either > > nvdimm_flush(nd_region, bio, true) > ...or: > nvdimm_flush(nd_region, NULL, false)Agree.> > ...so why not gate async behavior on the presence of the 'bio' argument?Sure.> > > > + int rc = 0; > > + > > + /* Create child bio for asynchronous flush and chain with > > + * parent bio. Otherwise directly call nd_region flush. > > + */ > > + if (async && bio->bi_iter.bi_sector != -1) { > > + > > + struct bio *child = bio_alloc(GFP_ATOMIC, 0); > > + > > + if (!child) > > + return -ENOMEM; > > + bio_copy_dev(child, bio); > > + child->bi_opf = REQ_PREFLUSH; > > + child->bi_iter.bi_sector = -1; > > + bio_chain(child, bio); > > + submit_bio(child); > > I understand how this works, but it's a bit too "magical" for my > taste. I would prefer that all flush implementations take an optional > 'bio' argument rather than rely on the make_request implementation to > stash the bio away on a driver specific list.I did this to make use of "bio_chain" for chaining child bio for async flush suggested [1]. Are you saying to remove this and just call "flush" based on bio argument? Or I implemented the 'bio_chain' request entirely wrong? [1] https://lkml.org/lkml/2018/9/27/1028> > > + } else { > > + if (nd_region->flush(nd_region)) > > + rc = -EIO; > > Given the common case wants to be fast and synchronous I think we > should try to avoid retpoline overhead by default. So something like > this: > > if (nd_region->flush == generic_nvdimm_flush) > rc = generic_nvdimm_flush(...);Sure. Thanks, Pankaj> >
Dan Williams
2019-Apr-11 16:09 UTC
[Qemu-devel] [PATCH v5 1/6] libnvdimm: nd_region flush callback support
On Thu, Apr 11, 2019 at 9:02 AM Pankaj Gupta <pagupta at redhat.com> wrote:> > > > > > > > > This patch adds functionality to perform flush from guest > > > to host over VIRTIO. We are registering a callback based > > > on 'nd_region' type. virtio_pmem driver requires this special > > > flush function. For rest of the region types we are registering > > > existing flush function. Report error returned by host fsync > > > failure to userspace. > > > > > > This also handles asynchronous flush requests from the block layer > > > by creating a child bio and chaining it with parent bio. > > > > > > Signed-off-by: Pankaj Gupta <pagupta at redhat.com> > > > ---bio_chain Dan williams > > [..] > > > diff --git a/drivers/nvdimm/region_devs.c b/drivers/nvdimm/region_devs.c > > > index b4ef7d9ff22e..fb1041ab32a6 100644 > > > --- a/drivers/nvdimm/region_devs.c > > > +++ b/drivers/nvdimm/region_devs.c > > > @@ -295,7 +295,9 @@ static ssize_t deep_flush_store(struct device *dev, > > > struct device_attribute *att > > > return rc; > > > if (!flush) > > > return -EINVAL; > > > - nvdimm_flush(nd_region); > > > + rc = nvdimm_flush(nd_region, NULL, false); > > > + if (rc) > > > + return rc; > > > > > > return len; > > > } > > > @@ -1085,6 +1087,11 @@ static struct nd_region *nd_region_create(struct > > > nvdimm_bus *nvdimm_bus, > > > dev->of_node = ndr_desc->of_node; > > > nd_region->ndr_size = resource_size(ndr_desc->res); > > > nd_region->ndr_start = ndr_desc->res->start; > > > + if (ndr_desc->flush) > > > + nd_region->flush = ndr_desc->flush; > > > + else > > > + nd_region->flush = generic_nvdimm_flush; > > > + > > > nd_device_register(dev); > > > > > > return nd_region; > > > @@ -1125,11 +1132,36 @@ struct nd_region > > > *nvdimm_volatile_region_create(struct nvdimm_bus *nvdimm_bus, > > > } > > > EXPORT_SYMBOL_GPL(nvdimm_volatile_region_create); > > > > > > +int nvdimm_flush(struct nd_region *nd_region, struct bio *bio, bool async) > > > +{ > > > > I don't quite see the point of the 'async' argument. All the usages of > > this routine are either > > > > nvdimm_flush(nd_region, bio, true) > > ...or: > > nvdimm_flush(nd_region, NULL, false) > > Agree. > > > > > ...so why not gate async behavior on the presence of the 'bio' argument? > > Sure. > > > > > > > > + int rc = 0; > > > + > > > + /* Create child bio for asynchronous flush and chain with > > > + * parent bio. Otherwise directly call nd_region flush. > > > + */ > > > + if (async && bio->bi_iter.bi_sector != -1) { > > > + > > > + struct bio *child = bio_alloc(GFP_ATOMIC, 0); > > > + > > > + if (!child) > > > + return -ENOMEM; > > > + bio_copy_dev(child, bio); > > > + child->bi_opf = REQ_PREFLUSH; > > > + child->bi_iter.bi_sector = -1; > > > + bio_chain(child, bio); > > > + submit_bio(child); > > > > I understand how this works, but it's a bit too "magical" for my > > taste. I would prefer that all flush implementations take an optional > > 'bio' argument rather than rely on the make_request implementation to > > stash the bio away on a driver specific list. > > I did this to make use of "bio_chain" for chaining child bio for async flush > suggested [1]. Are you saying to remove this and just call "flush" based on > bio argument? Or I implemented the 'bio_chain' request entirely wrong?No, I think you implemented it correctly. I'm just asking for the chaining to be performed internal to the ->flush() callback rather than in the common nvdimm_flush() front-end.
On Thu 11-04-19 07:51:48, Dan Williams wrote:> On Tue, Apr 9, 2019 at 9:09 PM Pankaj Gupta <pagupta at redhat.com> wrote: > > + } else { > > + if (nd_region->flush(nd_region)) > > + rc = -EIO; > > Given the common case wants to be fast and synchronous I think we > should try to avoid retpoline overhead by default. So something like > this: > > if (nd_region->flush == generic_nvdimm_flush) > rc = generic_nvdimm_flush(...);I'd either add a comment about avoiding retpoline overhead here or just make ->flush == NULL mean generic_nvdimm_flush(). Just so that people don't get confused by the code. Honza -- Jan Kara <jack at suse.com> SUSE Labs, CR
Jeff Moyer
2019-Apr-12 13:12 UTC
[PATCH v5 1/6] libnvdimm: nd_region flush callback support
Jan Kara <jack at suse.cz> writes:> On Thu 11-04-19 07:51:48, Dan Williams wrote: >> On Tue, Apr 9, 2019 at 9:09 PM Pankaj Gupta <pagupta at redhat.com> wrote: >> > + } else { >> > + if (nd_region->flush(nd_region)) >> > + rc = -EIO; >> >> Given the common case wants to be fast and synchronous I think we >> should try to avoid retpoline overhead by default. So something like >> this: >> >> if (nd_region->flush == generic_nvdimm_flush) >> rc = generic_nvdimm_flush(...); > > I'd either add a comment about avoiding retpoline overhead here or just > make ->flush == NULL mean generic_nvdimm_flush(). Just so that people don't > get confused by the code.Isn't this premature optimization? I really don't like adding things like this without some numbers to show it's worth it. -Jeff