search for: guestfs_readdir

On Wed, Sep 06, 2017 at 03:55:05PM +0300, Maxim Kozover wrote: > Hi Richard! > Tell me, please, is there any chance you have an (unofficial) patch ready > or almost ready that removes a protocol limit from guestfs_readdir (returns > a large answer in chunks in a loop etc)? Yes this is one of the very few remaining APIs which has a protocol limit. > I happen to have a need to use libguestfs fuse accessing the directory with > a million of files in it, the individual files are accessed ok, but the > dire...

...= NULL, *version = NULL, *desc = NULL; + CLEANUP_FREE char *arch = NULL, *url = NULL; + char *release = NULL; + char **key = NULL; + const size_t path_len = strlen ("/var/lib/pacman/local/") + strlen ("/desc"); + struct guestfs_dirent_list *local_db = NULL; + + local_db = guestfs_readdir (g, "/var/lib/pacman/local"); + if (local_db == NULL) + return NULL; + + /* Allocate 'apps' list. */ + apps = safe_malloc (g, sizeof *apps); + apps->len = 0; + apps->val = NULL; + + + for (i = 0; i < local_db->len; i++) { + curr = &local_db->val[i];...

...u.com> --- TODO | 6 ------ tests/bigdirs/test-big-dirs.pl | 18 ++++-------------- 2 files changed, 4 insertions(+), 20 deletions(-) diff --git a/TODO b/TODO index 1ff661f..1cde423 100644 --- a/TODO +++ b/TODO @@ -565,12 +565,6 @@ the p.o.v of the API and ABI. - guestfs_readdir -mke2fs ------- - -Add a mke2fs API call allowing full configuration of filesystems. -Then fix tests/bigdirs/test-big-dirs.pl to use it. - hivex ----- diff --git a/tests/bigdirs/test-big-dirs.pl b/tests/bigdirs/test-big-dirs.pl index 052163b..35fbc90 100755 --- a/tests/bigdirs/test-big-dirs.p...

The existing mechanism was clunky, slow and used ~ 10 MB of local disk. Rich.

...CLEANUP_FREE char *name = NULL, *version = NULL, *desc = NULL; + CLEANUP_FREE char *arch = NULL, *url = NULL; + char **key = NULL, *rel = NULL, *ver = NULL, *p; + int32_t epoch; + const size_t path_len = strlen ("/var/lib/pacman/local/") + strlen ("/desc"); + + local_db = guestfs_readdir (g, "/var/lib/pacman/local"); + if (local_db == NULL) + return NULL; + + /* Allocate 'apps' list. */ + apps = safe_malloc (g, sizeof *apps); + apps->len = 0; + apps->val = NULL; + + for (i = 0; i < local_db->len; i++) { + curr = &local_db->val[i]; +...

...ULL; > + CLEANUP_FREE char *arch = NULL, *url = NULL; > + char *release = NULL; > + char **key = NULL; > + const size_t path_len = strlen ("/var/lib/pacman/local/") + strlen ("/desc"); > + struct guestfs_dirent_list *local_db = NULL; > + > + local_db = guestfs_readdir (g, "/var/lib/pacman/local"); > + if (local_db == NULL) > + return NULL; > + > + /* Allocate 'apps' list. */ > + apps = safe_malloc (g, sizeof *apps); > + apps->len = 0; > + apps->val = NULL; > + > + Extra blank line here. > + for (i...

...E char *arch = NULL, *url = NULL; >> + char *release = NULL; >> + char **key = NULL; >> + const size_t path_len = strlen ("/var/lib/pacman/local/") + strlen ("/desc"); >> + struct guestfs_dirent_list *local_db = NULL; >> + >> + local_db = guestfs_readdir (g, "/var/lib/pacman/local"); >> + if (local_db == NULL) >> + return NULL; >> + >> + /* Allocate 'apps' list. */ >> + apps = safe_malloc (g, sizeof *apps); >> + apps->len = 0; >> + apps->val = NULL; >> + >> + >...

...uot;/p"); +guestfs_touch ($g, "/q"); + +function dir_cmp ($a, $b) +{ + return strcmp ($a["name"], $b["name"]); +} +function dir_extract ($n) +{ + return array ("name" => $n["name"], "ftyp" => $n["ftyp"]); +} +$dirs = guestfs_readdir ($g, "/"); +usort ($dirs, "dir_cmp"); +$dirs = array_map ("dir_extract", $dirs); +var_dump ($dirs); + +guestfs_shutdown ($g); +echo ("OK\n"); +?> +--EXPECT-- +array(2) { + [0]=> + string(11) "/dev/VG/LV1" + [1]=> + string(11) "/dev...

This version doesn't crash or cause hung processes or stuck mountpoints, so that's an improvement. Rich.

This patch is just for review. It enables FUSE support in the API via two new calls, 'guestfs_mount_local' and 'guestfs_umount_local'. FUSE turns out to be very easy to deadlock (necessitating that the machine be rebooted). Running the test from the third patch is usually an effective way to demonstrate this. However I have not yet managed to produce a simple reproducer that

This changes the proposed API slightly. Previously 'mount-local' generating a 'mounted' event when the filesystem was ready, and from the 'mounted' event you had to effectively do a fork. Now, 'mount-local' just initializes the mountpoint and you have to call 'mount-local-run' to enter the FUSE main loop. Between these calls you can do a fork or whatever

Tested with 'make check-release'. tests/parallel (in check-slow) failed, although it does regularly and that seems to be because of libvirt. Rich.

...\ } \ } \ } while (0) @@ -157,46 +157,46 @@ complete_dest_paths_generator (const char *text, int state) p = strrchr (text, '/'); dir = p && p > text ? strndup (text, p - text) : strdup ("/"); if (dir) { - dirents = guestfs_readdir (g, dir); - - /* Prepend directory to names before adding them to the list - * of words. - */ - if (dirents) { - size_t i; - - for (i = 0; i < dirents->len; ++i) { - int err; - - if (strcmp (dirents->val[i].name, ".") != 0 && - strcmp (dirents->val[i].na...

In the kernel and qemu communities it is routine for patches that will be backported to stable branches to be posted for review. I'm proposing we do the same for libguestfs stable branches. All of the attached have been tested with 'make check-release'. Rich.

...;guestfs_sh" has no tests warning: "guestfs_sh_lines" has no tests warning: "guestfs_scrub_freespace" has no tests warning: "guestfs_df" has no tests warning: "guestfs_df_h" has no tests warning: "guestfs_mount_loop" has no tests warning: "guestfs_readdir" has no tests warning: "guestfs_sfdiskM" has no tests warning: "guestfs_zfile" has no tests warning: "guestfs_getxattrs" has no tests warning: "guestfs_lgetxattrs" has no tests warning: "guestfs_setxattr" has no tests warning: "guestfs_lse...

...;guestfs_sh" has no tests warning: "guestfs_sh_lines" has no tests warning: "guestfs_scrub_freespace" has no tests warning: "guestfs_df" has no tests warning: "guestfs_df_h" has no tests warning: "guestfs_mount_loop" has no tests warning: "guestfs_readdir" has no tests warning: "guestfs_sfdiskM" has no tests warning: "guestfs_zfile" has no tests warning: "guestfs_getxattrs" has no tests warning: "guestfs_lgetxattrs" has no tests warning: "guestfs_setxattr" has no tests warning: "guestfs_lse...

...;guestfs_sh" has no tests warning: "guestfs_sh_lines" has no tests warning: "guestfs_scrub_freespace" has no tests warning: "guestfs_df" has no tests warning: "guestfs_df_h" has no tests warning: "guestfs_mount_loop" has no tests warning: "guestfs_readdir" has no tests warning: "guestfs_sfdiskM" has no tests warning: "guestfs_zfile" has no tests warning: "guestfs_getxattrs" has no tests warning: "guestfs_lgetxattrs" has no tests warning: "guestfs_setxattr" has no tests warning: "guestfs_lse...

...;guestfs_sh" has no tests warning: "guestfs_sh_lines" has no tests warning: "guestfs_scrub_freespace" has no tests warning: "guestfs_df" has no tests warning: "guestfs_df_h" has no tests warning: "guestfs_mount_loop" has no tests warning: "guestfs_readdir" has no tests warning: "guestfs_sfdiskM" has no tests warning: "guestfs_zfile" has no tests warning: "guestfs_getxattrs" has no tests warning: "guestfs_lgetxattrs" has no tests warning: "guestfs_setxattr" has no tests warning: "guestfs_lse...

...;guestfs_sh" has no tests warning: "guestfs_sh_lines" has no tests warning: "guestfs_scrub_freespace" has no tests warning: "guestfs_df" has no tests warning: "guestfs_df_h" has no tests warning: "guestfs_mount_loop" has no tests warning: "guestfs_readdir" has no tests warning: "guestfs_sfdiskM" has no tests warning: "guestfs_zfile" has no tests warning: "guestfs_getxattrs" has no tests warning: "guestfs_lgetxattrs" has no tests warning: "guestfs_setxattr" has no tests warning: "guestfs_lse...

...;guestfs_sh" has no tests warning: "guestfs_sh_lines" has no tests warning: "guestfs_scrub_freespace" has no tests warning: "guestfs_df" has no tests warning: "guestfs_df_h" has no tests warning: "guestfs_mount_loop" has no tests warning: "guestfs_readdir" has no tests warning: "guestfs_sfdiskM" has no tests warning: "guestfs_zfile" has no tests warning: "guestfs_getxattrs" has no tests warning: "guestfs_lgetxattrs" has no tests warning: "guestfs_setxattr" has no tests warning: "guestfs_lse...