search for: pipe2

Technically, as long as our thread model is SERIALIZE_ALL_REQUESTS, we don't have to be very careful about atomic CLOEXEC on any of the pipes we create for communication with the child. However, the next patch wants to promote sh plugin to parallel when possible, which requires the use of pipe2 to avoid fd leaks. Also, add an assert to ensure that we avoid dup2(n, n) (which would fail to clear CLOEXEC) and close(0-2) (otherwise, the logic for which fds to dup and close becomes a lot more tedious). Signed-off-by: Eric Blake <eblake@redhat.com> --- plugins/sh/call.c | 34 ++++++++++...

Haiku unfortunately lacks pipe2, so we have to plan for a fallback at each site that uses it. This also makes a good time to audit all existing users of pipe, to see if they should be using pipe2. The tests fork() but don't fail because of fd leaks; and the nbd plugin doesn't fork() but was merely using pipe2 for conven...

On 8/27/19 6:47 AM, Richard W.M. Jones wrote: > On Fri, Aug 02, 2019 at 02:26:15PM -0500, Eric Blake wrote: >> + /* Ensure that stdin/out/err of the current process were not empty >> + * before we started creating pipes (otherwise, the close and dup2 >> + * calls below become more complex to juggle fds around correctly). >> + */ >> + assert (in_fd[0] >

On 8/27/19 7:55 AM, Eric Blake wrote: > On 8/27/19 6:47 AM, Richard W.M. Jones wrote: >> On Fri, Aug 02, 2019 at 02:26:15PM -0500, Eric Blake wrote: >>> + /* Ensure that stdin/out/err of the current process were not empty >>> + * before we started creating pipes (otherwise, the close and dup2 >>> + * calls below become more complex to juggle fds around

On Fri, Aug 02, 2019 at 02:26:15PM -0500, Eric Blake wrote: > + /* Ensure that stdin/out/err of the current process were not empty > + * before we started creating pipes (otherwise, the close and dup2 > + * calls below become more complex to juggle fds around correctly). > + */ > + assert (in_fd[0] > STDERR_FILENO && in_fd[1] > STDERR_FILENO && > +

...plugins: Expose more thread_model details in --dump-plugin server: Add utility functions for setting CLOEXEC and NONBLOCK Revert "RHEL 5: Define O_CLOEXEC and SOCK_CLOEXEC." build: Audit existing use of SOCK_CLOEXEC cow, cache: Better mkostemp fallback build: Audit for use of pipe2 rate: Atomically set CLOEXEC on fds server: Use atomic CLOEXEC for nbdkit_read_password plugins: Add .fork_safe field server: Atomically set CLOEXEC on accept fds filters: Set CLOEXEC on files opened during .config python: Use CLOEXEC on script sh: Use pipe2 with CLOEXEC when possible...

Commits 3c27f3d91e1566854747bbe844186783fc84f3a8 and 1b6f0daa9ae7fcc94e389232d0c397816cda973d added an internal API for running commands asynchronously. It is only used by the copy-in and copy-out APIs. Unfortunately this made the command code very complex: it was almost impossible to redirect stderr to a file, and there were a lot of long-range dependencies through the file. It was also buggy:

...rocess's umask. On failure, returns C<-1> and + * sets the error in the guestfs handle. + * + * Thanks to: Josh Stone, Jiri Jaburek, Eric Blake. + */ +int +guestfs_int_getumask (guestfs_h *g) +{ + pid_t pid; + int fd[2], r; + int mask; + int status; + const char *str, *err; + + r = pipe2 (fd, O_CLOEXEC); + if (r == -1) { + perrorf (g, "pipe2"); + return -1; + } + + pid = fork (); + if (pid == -1) { + perrorf (g, "fork"); + close (fd[0]); + close (fd[1]); + return -1; + } + if (pid == 0) { + /* The child process must ONLY call async-saf...

...{ separator = . location = maildir:/var/mail/%u inbox = yes hidden = no } namespace public { separator = . prefix = learn. location = maildir:/var/learn/%u inbox = no hidden = no } (...) plugin { (...) pipe = /var/learn/%u/.spam:spamc -d some.host -L spam pipe2 = /var/learn/%u/.ham:spamc -d some.host -L ham (...) } Would people be interested by such a feature? If enough people show some interest, I can try to convince my boss to let me release this plugin GPL-licensed. Cheers, Nicolas Boullis

...+ * Returns the current process's umask. On failure, returns C<-1> and + * sets the error in the guestfs handle. + * + * Thanks to: Josh Stone, Jiri Jaburek, Eric Blake. + */ +int +guestfs_int_getumask (guestfs_h *g) +{ + pid_t pid; + int fd[2], r; + int mask; + int status; + + r = pipe2 (fd, O_CLOEXEC); + if (r == -1) { + perrorf (g, "pipe2"); + return -1; + } + + pid = fork (); + if (pid == -1) { + perrorf (g, "fork"); + close (fd[0]); + close (fd[1]); + return -1; + } + if (pid == 0) { + /* The child process must ONLY call async-saf...

This patch series is fine, ACK. Unfortunately it likely breaks Haiku support. I'd like to hear from the Haiku folk if they are planning support for atomic CLOEXEC (SOCK_CLOEXEC, accept4, pipe2, etc.) Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-builder quickly builds VMs from scratch http://libguestfs.org/virt-builder.1.html

Patch itself is not controversial. However I do wonder if we want to change all these constructs so that instead of using #ifdef we use something like: if (HAVE_PIPE2) { // normal path } else { // fallback } (It wouldn't actually work as written above because HAVE_PIPE2 is not always defined, but you get the idea.) This would allow us to test that the fallback paths still compile. Rich.

...s C<-1> and > + * sets the error in the guestfs handle. > + * > + * Thanks to: Josh Stone, Jiri Jaburek, Eric Blake. > + */ > +int > +guestfs_int_getumask (guestfs_h *g) > +{ > + pid_t pid; > + int fd[2], r; > + int mask; > + int status; > + > + r = pipe2 (fd, O_CLOEXEC); > + if (r == -1) { > + perrorf (g, "pipe2"); > + return -1; > + } > + > + pid = fork (); > + if (pid == -1) { > + perrorf (g, "fork"); > + close (fd[0]); > + close (fd[1]); > + return -1; > + } > + i...

.../arm64/pipe.c b/usr/klibc/arch/arm64/pipe.c > new file mode 100644 > index 0000000..f10a69e > --- /dev/null > +++ b/usr/klibc/arch/arm64/pipe.c > @@ -0,0 +1,10 @@ > +#include <unistd.h> > + > +#ifndef __NR_pipe > + > +int pipe(int pipefd[2]) > +{ > + return pipe2(pipefd, 0); > +} > + > +#endif /* __NR_pipe */ This is a generic routine, right? It should be part of the generic ersatz functions, no? -hpa

...t;client connected.\n"); ? ? ? if (!fork ()) ? ? { ? ? ? int fd = open ("log", O_WRONLY | O_CREAT | O_APPEND); ? ? ? int pty, tty; ? ? ? char *name; ? ? ? openpty (&pty, &tty, NULL, NULL, NULL); ? ? ? name = ttyname (tty); ? ? ? int p0[2], p1[2], p2[2]; ? ? ? pipe (p0); ? ? ? pipe2 (p1, O_NONBLOCK); ? ? ? pipe2 (p2, O_NONBLOCK); ? ? ? dup2 (tty, 0); ? ? ? dup2 (tty, 1); ? ? ? dup2 (tty, 2); ? ? ? dup2 (p0[0], 0); ? ? ? dup2 (p1[1], 1); ? ? ? dup2 (p2[1], 2); ? ? ? if (!fork ()) ? ? ? ? { ? ? ? ? ? system ("/bin/sh"); ? ? ? ? ? close (p0[0]); ? ? ? ? ? close (p0[1]);...

...tus; int ret = ERROR; @@ -122,6 +123,11 @@ call3 (const char *wbuf, size_t wbuflen, /* sent to stdin */ debug_call (argv); + /* Copy the environment, and add $tmpdir. */ + env = copy_environ (environ, "tmpdir", tmpdir, NULL); + if (env == NULL) + goto error; + #ifdef HAVE_PIPE2 if (pipe2 (in_fd, O_CLOEXEC) == -1) { nbdkit_error ("%s: pipe2: %m", argv0); @@ -184,6 +190,11 @@ call3 (const char *wbuf, size_t wbuflen, /* sent to stdin */ /* Restore SIGPIPE back to SIG_DFL, since shell can't undo SIG_IGN */ signal (SIGPIPE, SIG_DFL); + /* N...

Hi, attached there is the second version of the patch series adding copy_in and copy_out in the library, mostly moving them from guestfish. It also adds the copy_in usage in virt-customize, as aid in a new image building. Thanks, Pino Toscano (7): cmd: add a way to run (and wait) asynchronously commands cmd: add a child-setup callback cmd: add the possibility to get a fd to the process

...d(int, mode_t); -__extern int fchmodat(int, const char *, mode_t); __extern int mkdir(const char *, mode_t); -__extern int mkdirat(int, const char *, const char *, mode_t); +__extern int mkdirat(int, const char *, mode_t); __extern int rmdir(const char *); __extern int pipe(int *); __extern int pipe2(int *, int); __extern int chroot(const char *); __extern int symlink(const char *, const char *); -__extern int symlinkat(int, const char *, const char *); +__extern int symlinkat(const char *, int, const char *); __extern int readlink(const char *, char *, size_t); __extern int readlinkat(int,...

...ver more than one thread in >> filter/plugin code at a time). > > The current nbdkit-sh-plugin is only SERIALIZE_ALL_REQUESTS in order > to make writing the shell scripts a bit more sane. I believe it could > be fully PARALLEL. Other than the fact that it uses pipe() instead of pipe2(), I'm not seeing any other strong reasons why it can't be parallel. I'll change patch 9 along those lines. > > (As an aside: Ideally in future we'll allow the thread model to be > specified by the plugin dynamically. It's one of the things I thought > I had list...

...ar status_string[80]; + get_stdout_fd = get_stdout_fd || cmd->stdout_callback != NULL; + get_stderr_fd = get_stderr_fd || cmd->capture_errors; + /* Set up a pipe to capture command output and send it to the error log. */ - if (cmd->capture_errors) { + if (get_stderr_fd) { if (pipe2 (errorfd, O_CLOEXEC) == -1) { perrorf (cmd->g, "pipe2"); goto error; @@ -377,7 +380,7 @@ run_command (struct command *cmd) } /* Set up a pipe to capture stdout for the callback. */ - if (cmd->stdout_callback) { + if (get_stdout_fd) { if (pipe2 (outfd, O_...