search for: total_chunk

...unt; + uint32_t flags; + size_t chunks; + struct range *remaining; +}; + +#define MAX_BUF (2 * 1024 * 1024) +static char buf[MAX_BUF]; + +/* Various statistics */ +static int total_data_chunks; +static int total_data_bytes; +static int total_hole_chunks; +static int total_hole_bytes; +static int total_chunks; +static int total_df_reads; +static int total_reads; +static int64_t total_bytes; +static int total_success; + +static int +read_chunk (void *opaque, const void *bufv, size_t count, uint64_t offset, + int *error, int status) +{ + struct data *data = opaque; + struct range *r, **prev;...

Hi Chris, this serie of patches updated the command "btrfs filesystem df". I update this command because it is not so easy to get the information about the disk usage from the command "fi df" and "fi show". This patch was the result of some discussions on the btrfs mailing list. Many thanks to all the contributors. From the man page (see 2nd patch): [...] The

...g, void *opaque, int *error) > +read_verify (void *opaque, int *error) > { > int ret = 0; > + struct data *data = opaque; > > - if (valid_flag & LIBNBD_CALLBACK_VALID) { > - struct data *data = opaque; > - > - ret = -1; > - total_reads++; > - total_chunks += data->chunks; > - if (*error) > - goto cleanup; > - assert (data->chunks > 0); > - if (data->flags & LIBNBD_CMD_FLAG_DF) { > - total_df_reads++; > - if (data->chunks > 1) { > - fprintf (stderr, "buggy server: too man...

...wrapped cluster objects to unchanged code, but requires knowing the precise number of chunks that the workload will be split into. Could it be feasible to add an optional .progress argument after the ellipsis to parLapply() and its friends? We can require it to be a function accepting (done_chunk, total_chunks, ...). If not a new argument, what other interfaces could be used to get accurate progress information from staticClusterApply and dynamicClusterApply? I understand that the default parLapply() behaviour is not very amenable to progress tracking, but when running clusterMap(.scheduling = 'dyn...

...to unchanged code, but requires knowing the precise > number of chunks that the workload will be split into. > > Could it be feasible to add an optional .progress argument after the > ellipsis to parLapply() and its friends? We can require it to be a > function accepting (done_chunk, total_chunks, ...). If not a new > argument, what other interfaces could be used to get accurate progress > information from staticClusterApply and dynamicClusterApply? > > I understand that the default parLapply() behaviour is not very > amenable to progress tracking, but when running clusterMa...

...aque, } static int -read_verify (unsigned valid_flag, void *opaque, int *error) +read_verify (void *opaque, int *error) { int ret = 0; - if (valid_flag & LIBNBD_CALLBACK_VALID) { - struct data *data = opaque; + struct data *data = opaque; - ret = -1; - total_reads++; - total_chunks += data->chunks; - if (*error) - goto cleanup; - assert (data->chunks > 0); - if (data->flags & LIBNBD_CMD_FLAG_DF) { - total_df_reads++; - if (data->chunks > 1) { - fprintf (stderr, "buggy server: too many chunks for DF flag\n"); -...

...aque, } static int -read_verify (unsigned valid_flag, void *opaque, int *error) +read_verify (void *opaque, int *error) { int ret = 0; + struct data *data = opaque; - if (valid_flag & LIBNBD_CALLBACK_VALID) { - struct data *data = opaque; - - ret = -1; - total_reads++; - total_chunks += data->chunks; - if (*error) - goto cleanup; - assert (data->chunks > 0); - if (data->flags & LIBNBD_CMD_FLAG_DF) { - total_df_reads++; - if (data->chunks > 1) { - fprintf (stderr, "buggy server: too many chunks for DF flag\n"); -...

...d *opaque, const void *bufv, size_t count, uint64_t offset, } static int -read_verify (void *opaque, int64_t cookie, int *error) +read_verify (unsigned valid_flag, void *opaque, int64_t cookie, int *error) { - struct data *data = opaque; - int ret = -1; + int ret = 0; - total_reads++; - total_chunks += data->chunks; - if (*error) - goto cleanup; - assert (data->chunks > 0); - if (data->flags & LIBNBD_CMD_FLAG_DF) { - total_df_reads++; - if (data->chunks > 1) { - fprintf (stderr, "buggy server: too many chunks for DF flag\n"); + if (valid_flag...

Patches 1 & 2 are rather complex, but the end result is that we pass closures + user_data + free function in single struct parameters as I described previously in this email: https://www.redhat.com/archives/libguestfs/2019-August/msg00210.html Patch 3 adds a convenient FREE_CALLBACK macro which seems a worthwhile simplification if you buy into 1 & 2. Patch 4 adds another macro which is

...ret = 0; > + > + if (valid_flag & LIBNBD_CALLBACK_VALID) { > struct data *data = opaque; > - int ret = -1; > > + ret = -1; > total_reads++; Was this patch posted with whitespace changes ignored? Otherwise, it looks like you missed reindenting things. > total_chunks += data->chunks; > if (*error) > @@ -160,7 +167,11 @@ read_verify (void *opaque, int64_t cookie, int *error) > data->remaining = r->next; > free (r); > } > - free (data); > + } > + > + if (valid_flag & LIBNBD_CALLBACK_FREE) > + free...

I think I've addressed everything that was raised in review. Some of the highlights: - Callbacks should be freed reliably along all exit paths. - There's a simple test of closure lifetimes. - I've tried to use VALID|FREE in all the places where I'm confident that it's safe and correct to do. There may be more places. Note this is an optimization and shouldn't

I still need to wire in the use of *_notify functions into nbdkit to prove whether it makes the code any faster or easier to maintain, but at least the added example shows one good use case for the new API. Eric Blake (6): api: Add nbd_aio_in_flight generator: Allow DEAD state actions to run generator: Allow Int64 in callbacks states: Prepare for aio notify callback api: Add new

...t64_t offset, } static int -read_verify (void *opaque, int64_t cookie, int *error) +read_verify (int valid_flag, void *opaque, int64_t cookie, int *error) { struct data *data = opaque; int ret = -1; + if (!(valid_flag & LIBNBD_CALLBACK_VALID)) + return 0; + total_reads++; total_chunks += data->chunks; if (*error) diff --git a/generator/generator b/generator/generator index 3b57713..5d5c267 100755 --- a/generator/generator +++ b/generator/generator @@ -849,10 +849,7 @@ and arg = written by the function *) | BytesPersistIn of string * string...

As proposed here: https://www.redhat.com/archives/libguestfs/2019-August/msg00130.html I didn't actually read Eric's replies to that yet because I've been concentrating on writing these patches all day. Anyway here they are and I'll look at what Eric said about the proposal next. Rich.

...static int -read_verify (void *opaque, int64_t cookie, int *error) +read_verify (unsigned valid_flag, void *opaque, int64_t cookie, int *error) { + int ret = 0; + + if (valid_flag & LIBNBD_CALLBACK_VALID) { struct data *data = opaque; - int ret = -1; + ret = -1; total_reads++; total_chunks += data->chunks; if (*error) @@ -160,7 +167,11 @@ read_verify (void *opaque, int64_t cookie, int *error) data->remaining = r->next; free (r); } - free (data); + } + + if (valid_flag & LIBNBD_CALLBACK_FREE) + free (opaque); + return ret; } diff --git a/gener...

v1 was here: https://www.redhat.com/archives/libguestfs/2019-July/thread.html#00231 The changes address everything that Eric picked up in his review of the first two patches. I have also added two more patches (4 and 5) which respectively fix docs and change int status -> unsigned status, as discussed. Passes make, check, check-valgrind. Rich.

This implements most of what I wrote here: https://www.redhat.com/archives/libguestfs/2019-July/msg00213.html