search for: max_request_size

Hi, For virtio block device, actually there is no a hard limit for max request size, and virtio_blk driver set -1 to blk_queue_max_hw_sectors(q, -1U);. But it doesn't work, because there is a default upper limitation BLK_DEF_MAX_SECTORS (1280 sectors). So this series want to add a new helper blk_queue_max_hw_sectors_no_limit to set a proper max reqeust size. Weiping Zhang (2): blk-setting:

Hi, For virtio block device, actually there is no a hard limit for max request size, and virtio_blk driver set -1 to blk_queue_max_hw_sectors(q, -1U);. But it doesn't work, because there is a default upper limitation BLK_DEF_MAX_SECTORS (1280 sectors). So this series want to add a new helper blk_queue_max_hw_sectors_no_limit to set a proper max reqeust size. Weiping Zhang (2): blk-setting:

...ile changed, 30 insertions(+), 2 deletions(-) diff --git a/drivers/block/virtio_blk.c b/drivers/block/virtio_blk.c index 4a07593c..5ac6d59 100644 --- a/drivers/block/virtio_blk.c +++ b/drivers/block/virtio_blk.c @@ -64,6 +64,34 @@ struct virtblk_req { struct scatterlist sg[]; }; + +static int max_request_size_set(const char *val, const struct kernel_param *kp); + +static const struct kernel_param_ops max_request_size_ops = { + .set = max_request_size_set, + .get = param_get_uint, +}; + +static unsigned long max_request_size = 4096; /* in unit of KiB */ +module_param_cb(max_request_size, &max_request...

...ns.c +++ b/server/plugins.c @@ -594,7 +594,7 @@ plugin_zero (struct backend *b, struct connection *conn, uint32_t count, uint64_t offset, uint32_t flags, int *err) { struct backend_plugin *p = container_of (b, struct backend_plugin, backend); - char *buf; + static const char buf[MAX_REQUEST_SIZE]; uint32_t limit; int r = -1; bool may_trim = flags & NBDKIT_FLAG_MAY_TRIM; @@ -639,12 +639,7 @@ plugin_zero (struct backend *b, struct connection *conn, assert (p->plugin.pwrite || p->plugin._pwrite_old); flags &= ~NBDKIT_FLAG_MAY_TRIM; threadlocal_set_error (0); -...

On 4/5/18 4:09 AM, Weiping Zhang wrote: > Hi, > > For virtio block device, actually there is no a hard limit for max request > size, and virtio_blk driver set -1 to blk_queue_max_hw_sectors(q, -1U);. > But it doesn't work, because there is a default upper limitation > BLK_DEF_MAX_SECTORS (1280 sectors). So this series want to add a new helper >

...r/states-newstyle.c +++ b/generator/states-newstyle.c @@ -64,9 +64,12 @@ prepare_for_reply_payload (struct nbd_handle *h, uint32_t opt) /* Read the following payload if it is short enough to fit in the * static buffer. If it's too long, skip it. */ - /* XXX Move to DEAD if len > MAX_REQUEST_SIZE */ len = be32toh (h->sbuf.or.option_reply.replylen); - if (len <= maxpayload) + if (len > MAX_REQUEST_SIZE) { + set_error (0, "handshake: invalid option reply length"); + return -1; + } + else if (len <= maxpayload) h->rbuf = &h->sbuf.or.payload;...

A suggested readahead filter. I've only lightly tested this, but it seems to work fine with qemu-img convert. The commit needs proper tests. Rich.

...be done later in backend) */ switch (cmd) { - case NBD_CMD_READ: case NBD_CMD_CACHE: - case NBD_CMD_WRITE: case NBD_CMD_TRIM: case NBD_CMD_WRITE_ZEROES: + break; + + case NBD_CMD_READ: + case NBD_CMD_WRITE: + /* Refuse over-large read and write requests. */ + if (count > MAX_REQUEST_SIZE) { + nbdkit_error ("invalid request: %s: data request is too large (%" PRIu32 + " > %d)", + name_of_nbd_cmd (cmd), count, MAX_REQUEST_SIZE); + *error = ENOMEM; + return false; + } + break; + case NBD_CMD_BLOCK_STAT...

We have logic to send an ENOMEM error to a client that tries NBD_CMD_WRITE with a payload larger than MAX_REQUEST_SIZE (64M), but still end up trying to skip over the client's payload to stay in sync for receiving the next command. If the bad client request is only partially larger than our maximum, this is still nice behavior; but a worst-case client could cause us to waste time on read()ing nearly 4G of data...

Simpler, and including tests. Rich.

The upstream protocol recently promoted NBD_CMD_WRITE_ZEROES from experimental to a documented extension. Exposing support for this allows plugin writers to create sparse files when driven by a client that knows how to use the extension; meanwhile, even if a plugin does not support this extension, the server benefits from less network traffic from the client. Eric Blake (5): protocol: Support

When facing a malicious client that is sending bogus NBD_CMD_READ, we should make sure that we never end up in a situation where we could try to treat the tail from a command that we diagnosed as bad as being further commands. Eric Blake (2): connections: Report mid-message EOF as fatal connections: Hang up early on insanely large WRITE requests src/connections.c | 35

See companion patch: Subject: [PATCH nbdkit] server: Move size parsing code (nbdkit_parse_size) to common/include This is the second part of the patch. It adds the new human_size_parse function to libnbd and then uses it for parsing --queue-size, --request-size and --sparse. The main complication here is that there was already a common/utils/human-size.h header which ends up (eventually)

...4364cd2..ce4700a3 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; bool emulate = false; bool need_flush = false; + void *zero_buffer = NULL; + int buffer_size = MIN (MAX_REQUEST_SIZE, count); if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { flags &= ~NBDKIT_FLAG_FUA; @@ -669,19 +671,25 @@ plugin_zero (struct backend *b, void *handle, threadlocal_set_error (0); *err = 0; + *err = posix_memalign(&zero_buffer, 4096, buffer_size); + if (*er...

...uint16_t flags, uint16_t type, + uint64_t offset, uint64_t count, void *data, + int64_t id, extent_fn extent) { struct command_in_flight *cmd; @@ -255,7 +256,7 @@ command_common (struct nbd_connection *conn, if (count > MAX_REQUEST_SIZE) { set_error (ERANGE, "request too large: maximum request size is %d", MAX_REQUEST_SIZE); - return NULL; + return -1; } break; @@ -268,7 +269,7 @@ command_common (struct nbd_connection *conn, if (count > UINT32_MAX) { set_error...

...= EINVAL; - return 0; + return false; } /* Refuse over-large read and write requests. */ @@ -732,33 +739,24 @@ validate_request (struct connection *conn, nbdkit_error ("invalid request: data request is too large (%" PRIu32 " > %d)", count, MAX_REQUEST_SIZE); *error = ENOMEM; - return 0; - } - - /* Readonly connection? */ - if (conn->readonly && - (cmd == NBD_CMD_WRITE || cmd == NBD_CMD_FLUSH || - cmd == NBD_CMD_TRIM || cmd == NBD_CMD_WRITE_ZEROES)) { - nbdkit_error ("invalid request: write request on readonly...

...n (struct nbd_handle *h, { struct command_in_flight *cmd, *prev_cmd; + if (h->disconnect_request) { + set_error (EINVAL, "cannot request more commands after NBD_CMD_DISC"); + return -1; + } + switch (type) { /* Commands which send or receive data are limited to MAX_REQUEST_SIZE. */ case NBD_CMD_READ: diff --git a/tests/errors.c b/tests/errors.c index 415c378..faa1488 100644 --- a/tests/errors.c +++ b/tests/errors.c @@ -168,7 +168,7 @@ main (int argc, char *argv[]) check (ERANGE, "nbd_aio_pwrite: "); /* Queue up a write command so large that we block on...

...may_trim); - if (result == 0 || errno != EOPNOTSUPP) + if (result == -1) { + err = tls_get_error (); + if (!err) + err = errno; + } + if (result == 0 || err != EOPNOTSUPP) return result; } assert (plugin.pwrite); + tls_set_error (0); limit = count < MAX_REQUEST_SIZE ? count : MAX_REQUEST_SIZE; buf = calloc (limit, 1); if (!buf) { diff --git a/src/tls.c b/src/tls.c index 390b03e..e80e925 100644 --- a/src/tls.c +++ b/src/tls.c @@ -44,8 +44,10 @@ #include "nbdkit-plugin.h" #include "internal.h" -/* Note currently all thread-local stor...

...4364cd2..ce4700a3 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; bool emulate = false; bool need_flush = false; + void *zero_buffer = NULL; + int buffer_size = MIN (MAX_REQUEST_SIZE, count); if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { flags &= ~NBDKIT_FLAG_FUA; @@ -669,19 +671,25 @@ plugin_zero (struct backend *b, void *handle, threadlocal_set_error (0); *err = 0; + *err = posix_memalign(&zero_buffer, 4096, buffer_size); + if (*er...

...er/plugins.c > +++ b/server/plugins.c > @@ -631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, > bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; > bool emulate = false; > bool need_flush = false; > + void *zero_buffer = NULL; > + int buffer_size = MIN (MAX_REQUEST_SIZE, count); > > if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { > flags &= ~NBDKIT_FLAG_FUA; > @@ -669,19 +671,25 @@ plugin_zero (struct backend *b, void *handle, > threadlocal_set_error (0); > *err = 0; > > + *err = posix_memalign(&zer...