search for: nbdkit_fua_native

...-MT test_layers_plugin_la-test-layers-plugin.lo -MD -MP -MF .deps/test_layers_plugin_la-test-layers-plugin.Tpo -c test-layers-plugin.c -fPIC -DPIC -o .libs/test_layers_plugin_la-test-layers-plugin.o test-layers-plugin.c: In function ‘test_layers_plugin_can_fua’: test-layers-plugin.c:137:10: error: ‘NBDKIT_FUA_NATIVE’ undeclared (first use in this function); did you mean ‘NBDKIT_CXX_LANG_C’? return NBDKIT_FUA_NATIVE; ^~~~~~~~~~~~~~~~~ NBDKIT_CXX_LANG_C test-layers-plugin.c:137:10: note: each undeclared identifier is reported only once for each function it appears in test-layers-plugin.c:...

...ensuring that plugins compiled to the old API still work. The new API adds .can_fua, then adds a flags parameter to all five data callbacks, even though only three of them will use a flag at the moment. A plugin client has to opt in to both the version 2 API and provide .can_fua with a return of NBDKIT_FUA_NATIVE before nbdkit will pass the NBDKIT_FLAG_FUA to the plugin. Signed-off-by: Eric Blake <eblake@redhat.com> --- docs/nbdkit-filter.pod | 12 ++-- docs/nbdkit-plugin.pod | 151 +++++++++++++++++++++++++++++++++++++++++++----- docs/nbdkit.pod | 7 ++- include/nbdkit-plugin.h | 30 +...

...f --git a/server/plugins.c b/server/plugins.c index d1654f8d..c8f4af90 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -509,7 +509,7 @@ plugin_pwrite (struct backend *b, struct connection *conn, assert (connection_get_handle (conn, 0)); - if (fua && plugin_can_fua (b, conn) != NBDKIT_FUA_NATIVE) { + if (fua && backend_can_fua (b, conn) != NBDKIT_FUA_NATIVE) { flags &= ~NBDKIT_FLAG_FUA; need_flush = true; } @@ -541,7 +541,7 @@ plugin_trim (struct backend *b, struct connection *conn, assert (connection_get_handle (conn, 0)); - if (fua && plugin_can_fu...

On Wed, Apr 11, 2018 at 12:03:41AM -0500, Eric Blake wrote: > +=item C<can_fua> > + > +(Optional) > + > + def can_fua(h): > + # return a boolean > + > +Unlike the C counterpart, the Python callback does not need a > +tri-state return value, because Python introspection is sufficient to > +learn whether callbacks support FUA. Thus, this function only returns

...stant (m, "FLAG_FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); > + > + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); > + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); > + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); > + > + PyModule_AddIntConstant (m, "CACHE_NONE", NBDKIT_CACHE_NONE); > + PyModule_AddIntConstant (m, "CACHE_EMULATE", NBDKIT_CACHE_EMULATE); > + PyModule_AddIntConstant (m, "CACHE_NATIVE", NBDKIT_CACHE_NATIVE); > + > + PyModule_AddIntCon...

...DKIT_FLAG_FAST_ZERO); > > > + > > > + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); > > > + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); > > > + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); > > > + > > > + PyModule_AddIntConstant (m, "CACHE_NONE", NBDKIT_CACHE_NONE); > > > + PyModule_AddIntConstant (m, "CACHE_EMULATE", NBDKIT_CACHE_EMULATE); > > > + PyModule_AddIntConstant (m, "CACHE_NATIVE", NBDKIT_CACHE_NA...

...BUTE_FORMAT_PRINTF(fmtpos, argpos) +#endif + #define NBDKIT_THREAD_MODEL_SERIALIZE_CONNECTIONS 0 #define NBDKIT_THREAD_MODEL_SERIALIZE_ALL_REQUESTS 1 #define NBDKIT_THREAD_MODEL_SERIALIZE_REQUESTS 2 @@ -57,11 +64,9 @@ extern "C" { #define NBDKIT_FUA_EMULATE 1 #define NBDKIT_FUA_NATIVE 2 -extern void nbdkit_error (const char *msg, ...) - __attribute__((__format__ (__printf__, 1, 2))); +extern void nbdkit_error (const char *msg, ...) ATTRIBUTE_FORMAT_PRINTF (1, 2); extern void nbdkit_verror (const char *msg, va_list args); -extern void nbdkit_debug (const char *msg, ...) -...

...PyModule_AddIntConstant (m, "FLAG_FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); + + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); + + PyModule_AddIntConstant (m, "CACHE_NONE", NBDKIT_CACHE_NONE); + PyModule_AddIntConstant (m, "CACHE_EMULATE", NBDKIT_CACHE_EMULATE); + PyModule_AddIntConstant (m, "CACHE_NATIVE", NBDKIT_CACHE_NATIVE); + + PyModule_AddIntConstant (m, "EXTENT_HOLE&qu...

..._FAST_ZERO", NBDKIT_FLAG_FAST_ZERO); > > + > > + PyModule_AddIntConstant (m, "FUA_NONE", NBDKIT_FUA_NONE); > > + PyModule_AddIntConstant (m, "FUA_EMULATE", NBDKIT_FUA_EMULATE); > > + PyModule_AddIntConstant (m, "FUA_NATIVE", NBDKIT_FUA_NATIVE); > > + > > + PyModule_AddIntConstant (m, "CACHE_NONE", NBDKIT_CACHE_NONE); > > + PyModule_AddIntConstant (m, "CACHE_EMULATE", NBDKIT_CACHE_EMULATE); > > + PyModule_AddIntConstant (m, "CACHE_NATIVE", NBDKIT_CACHE_NATIVE); > > + &...

...PyErr_Clear (); + + r = PyObject_CallFunctionObjArgs (fn, obj, NULL); + Py_DECREF (fn); + if (check_python_failure ("can_fua") == -1) + return -1; + if (r == Py_False) + fua = NBDKIT_FUA_NONE; + else if (pwrite_has_fua || zero_has_fua || trim_has_fua) + fua = NBDKIT_FUA_NATIVE; + else + fua = NBDKIT_FUA_EMULATE; + Py_DECREF (r); + } + else if (pwrite_has_fua || zero_has_fua || trim_has_fua) + fua = NBDKIT_FUA_NATIVE; + else if (callback_defined ("flush", NULL)) + fua = NBDKIT_FUA_EMULATE; + else + fua = NBDKIT_FUA_NONE; + + return fua;...

...all fallocate modes, it would be nice if we had a way + * from fpathconf() to definitively learn what will work on a given + * fd for a more precise answer; oh well. */ +#ifdef FALLOC_FL_PUNCH_HOLE + return 1; +#else + return 0; +#endif +} + +int +fileops_can_fua (void *handle) +{ + return NBDKIT_FUA_NATIVE; +} + +int +fileops_can_cache (void *handle) +{ + /* Prefer posix_fadvise(), but letting nbdkit call .pread on our + * behalf also tends to work well for the local file system + * cache. + */ +#if HAVE_POSIX_FADVISE + return NBDKIT_FUA_NATIVE; +#else + return NBDKIT_FUA_EMULATE; +#endif +}...

There's a lot of code in nbdkit-file-plugin which it would be nice to reuse elsewhere. One possible approach (as outlined here) is simply to move the file callbacks (like file.pread, file.pwrite, file.zero etc) to a new mini-library. They can then be consumed by other plugins fairly easily by doing: static void * foo_open (int readonly) { struct fileops *fops; int fd, flags; /*

...-52,6 +52,8 @@ #include <linux/fs.h> /* For BLKZEROOUT */ #endif +#define NBDKIT_API_VERSION 2 + #include <nbdkit-plugin.h> #include "isaligned.h" @@ -263,9 +265,30 @@ file_can_trim (void *handle) #endif } +static int +file_can_fua (void *handle) +{ + return NBDKIT_FUA_NATIVE; +} + +/* Flush the file to disk. */ +static int +file_flush (void *handle, uint32_t flags) +{ + struct handle *h = handle; + + if (fdatasync (h->fd) == -1) { + nbdkit_error ("fdatasync: %m"); + return -1; + } + + return 0; +} + /* Read data from the file. */ static int -fi...

On 3/18/20 8:21 PM, Eric Blake wrote: > In commit c306fa93ab and neighbors (v1.15.1), a concerted effort went > into caching the results of .can_FOO callbacks, with commit messages > demonstrating that a plugin with a slow callback should not have that > delay magnified multiple times. But nothing was added to the > testsuite at the time, and with the sh and eval plugins, we still

On 11/22/19 3:14 PM, Richard W.M. Jones wrote: >>> @@ -54,20 +61,20 @@ def get_size(h): >>> return len(disk) >>> -def pread(h, count, offset): >>> +def pread(h, count, offset, flags): >>> global disk >>> return disk[offset:offset+count] >> >> Do we really want to be passing 'flags' as an integer that the

First thing to say is that I need to do a *lot* more testing on this, so this is just an early peek. In particular, although it passed ‘make check && make check-valgrind’ I have *not* tested it against a multi-conn-aware client such as the Linux kernel >= 4.9. This implements NBD_FLAG_CAN_MULTI_CONN, described in the protocol doc as: "NBD_FLAG_CAN_MULTI_CONN: Indicates that

Needs some work still, see in particular the commit message for patch 3. Rich.

...-631,6 +631,8 @@ plugin_zero (struct backend *b, void *handle, bool fast_zero = flags & NBDKIT_FLAG_FAST_ZERO; bool emulate = false; bool need_flush = false; + void *zero_buffer = NULL; + int buffer_size = MIN (MAX_REQUEST_SIZE, count); if (fua && backend_can_fua (b) != NBDKIT_FUA_NATIVE) { flags &= ~NBDKIT_FLAG_FUA; @@ -669,19 +671,25 @@ plugin_zero (struct backend *b, void *handle, threadlocal_set_error (0); *err = 0; + *err = posix_memalign(&zero_buffer, 4096, buffer_size); + if (*err != 0) { + r = -1; + goto done; + } + + memset(zero_buffer, 0...

After more than a month since v2 [1], I've finally got my FUA support series polished. This is all of my outstanding patches, even though some of them were originally posted in separate threads from the original FUA post [2], [3] [1] https://www.redhat.com/archives/libguestfs/2018-January/msg00113.html [2] https://www.redhat.com/archives/libguestfs/2018-January/msg00219.html [3]

For existing commits, this is almost identical to v1, except that I updated some commit messages and reordered the commits in a somewhat more logical sequence. The main changes are the extra commits: [06/11] plugins: Return NBD_FLAG_CAN_MULTI_CONN from some readonly plugins. - Readonly plugins that can set the flag unconditionally. [09/11] partitioning: Return NBD_FLAG_CAN_MULTI_CONN. [10/11]