similar to: [RFC nbdkit PATCH] utils: Revamp nbdkit_parse_size

The existing implementation admitted that it was not very robust in the face of garbage input (including, but not limited to, the the fact that the scanf() family has undefined behavior on integer overflow). Tighten things up by reimplementing the function. The old comment mentioned the 'human*' interface from gnulib; we can't use that for licensing reasons; so my rewrite is done

From: Nikolay Ivanets <stenavin@gmail.com> nbdkit_parse_size() uses strtoumax() function to parse input strings which states: "if there was a leading minus sign, the negation of the result of the conversion represented as an unsigned value, unless the original (nonnegated) value would overflow." Later validation doesn't catch the situation when parsed value appeared within

From: Nikolay Ivanets <stenavin@gmail.com> nbdkit_parse_size() uses strtoumax() function to parse input strings which states: "if there was a leading minus sign, the negation of the result of the conversion represented as an unsigned value, unless the original (nonnegated) value would overflow." Later validation doesn't catch the situation when parsed value is within the

Take two, this time split into two patches. I liked Rich's suggestion of unit-testing src/ files directly in src/, and automake is a lot happier with this than with my v1 attempt that tried to compile .c files across directories. It's still enough of a change that I'm not pushing it right away. Eric Blake (2): build: Add unit-testing of internal files utils: Revamp

--- lib/Makefile.am | 27 ++ server/Makefile.am | 24 -- server/nbdkit.syms | 28 +-- server/public.c | 459 +--------------------------------- {server => lib}/extents.c | 4 +- lib/parse.c | 341 +++++++++++++++++++++++++ lib/password.c | 152 +++++++++++ lib/path.c | 97 +++++++

On Mon, Sep 23, 2019 at 01:23:28PM -0500, Eric Blake wrote: > > Hopefully it will warn us if uid_t stops being int. (Also > > we're assuming pid_t == int). > > 64-bit mingw has had a long-standing bug that pid_t was declared as > 64-bit, even though getpid() returns a 32-bit 'int'; I wish they'd fix > their headers, but it violates the assumption of pid_t

This is the first part of a pair of patch series which aim to let us use nbdkit_parse_size (or rather, an equivalent common function) in nbdcopy, so we can write: nbdcopy --request-size=32M ... We can't do that now which was annoying me earlier in the week. This commit creates a new function called human_size_parse which is basically nbdkit_parse_size, and turns nbdkit_parse_size into a

sscanf is sadly not safe (because it doesn't handle integer overflow correctly), and strto*l functions are a pain to use correctly. Therefore add some functions to hide the pain of parsing integers from the command line. The simpler uses of sscanf and strto*l are replaced. There are still a few where we are using advanced features of sscanf. --- docs/nbdkit-plugin.pod | 48

On 9/23/19 3:06 PM, Richard W.M. Jones wrote: > On Mon, Sep 23, 2019 at 01:23:28PM -0500, Eric Blake wrote: >>> Hopefully it will warn us if uid_t stops being int. (Also >>> we're assuming pid_t == int). >> >> 64-bit mingw has had a long-standing bug that pid_t was declared as >> 64-bit, even though getpid() returns a 32-bit 'int'; I wish

On 9/21/19 6:56 AM, Richard W.M. Jones wrote: > sscanf is sadly not safe (because it doesn't handle integer overflow > correctly), and strto*l functions are a pain to use correctly. > Therefore add some functions to hide the pain of parsing integers from > the command line. > > The simpler uses of sscanf and strto*l are replaced. There are still > a few where we are

On Mon, Sep 23, 2019 at 12:05:11PM -0500, Eric Blake wrote: > > + int nbdkit_parse_long (const char *what, const char *str, long *r); > > + int nbdkit_parse_unsigned_long (const char *what, > > + const char *str, unsigned long *r); > > Do we really want to encourage the use of parse_long and > parse_unsigned_long? Those differ between

Trying to read a password or inline script from stdin when the -s option was used to connect to our client over stdin is not going to work. It would be nice to fail such usage as soon as possible, by giving plugins a means to decide if it is safe to use stdio during .config. Update nbdkit_read_password and the sh plugin to take advantage of the new entry point. Signed-off-by: Eric Blake

Trying to read a password or inline script from stdin when the -s option was used to connect to our client over stdin is not going to work. It would be nice to fail such usage as soon as possible, by giving plugins a means to decide if it is safe to use stdio during .config. Update nbdkit_read_password and the sh plugin to take advantage of the new entry point. Also, treat 'nbdkit -s

We no longer have to track a linked list of in-flight transactions that are pending resolution, but rely instead on libnbd 0.1.6+ doing it on our behalf. Normally, we will get to call nbdplug_register() prior to the notify callback being reached, but under a heavily-loaded system, it is conceivable that the libnbd state machine can manage to fire off our request and receive a server reply all

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

This conversion should be feature compatible with the standalone nbd code. Note that the use of libnbd makes the binary for this particular plugin fall under an LGPLv2+ license rather than BSD; but the source code in nbd.c remains BSD. A lot of code simply disappears, now that I'm no longer directly utilizing the NBD protocol files but relying on libnbd. Coordination between threads from

The upstream NBD protocol recently clarified that servers can advertise block size limitations to clients that ask with NBD_OPT_GO (although we're still a ways off from implementing that in nbdkit); and that in the absence of that, then clients should agree on limits using out-of-band information or stick to sane defaults (everything 512-byte-aligned, no reads or writes larger than 32M). But

On Tue, Sep 05, 2023 at 11:09:02AM +0100, Richard W.M. Jones wrote: > > > +static inline int64_t > > > +human_size_parse (const char *str, > > > + const char **error, const char **pstr) > > > +{ > > > + int64_t size; > > > + char *end; > > > + uint64_t scale = 1; > > > + > > > + /* XXX Should we

This can truncate, extend, or round up/down to a multiple. --- common-rules.mk | 3 +- configure.ac | 1 + filters/offset/nbdkit-offset-filter.pod | 7 +- filters/partition/nbdkit-partition-filter.pod | 1 + filters/truncate/Makefile.am | 60 ++++ filters/truncate/nbdkit-truncate-filter.pod | 87

This can truncate, extend, or round up/down to a multiple. --- common-rules.mk | 3 +- configure.ac | 1 + filters/offset/nbdkit-offset-filter.pod | 7 +- filters/partition/nbdkit-partition-filter.pod | 1 + filters/truncate/Makefile.am | 61 ++++ filters/truncate/nbdkit-truncate-filter.pod | 88