search for: quit_fd

This is an incomplete port to Windows. Currently the server compiles and starts up successfully, but goes into an infinite loop when you connect to it. Nevertheless I think the approach is ready for feedback. This being Windows the changes go quite deep. Rich.

...t. This also makes a good time to audit all existing users of pipe, to see if they should be using pipe2. The tests fork() but don't fail because of fd leaks; and the nbd plugin doesn't fork() but was merely using pipe2 for convenience over multiple fcntl calls. However, the server's quit_fd definitely needs to be marked CLOEXEC (it's easy to use the sh plugin to show that we are currently leaking it to children), although doing so can occur without worrying about atomicity since it is called before threads begin. Finally, it's also worth updating our set_cloexec helper functi...

I've posted some of these patches or ideas before; but now I'm confident enough with the series that it should be ready to push; at any rate, I can now run test-socket-activation in a tight loop without triggering any crashes or hangs. With this in place, I'm going back to work on making the nbd forwarder wort with the parallel thread model. Eric Blake (8): sockets: Use

...bdkit_nanosleep (unsigned sec, unsigned nsec) * NBD_CMD_DISC or a problem with the connection * - the input socket detects POLLRDHUP/POLLHUP/POLLERR */ - struct connection *conn = threadlocal_get_conn (); + struct connection *conn = GET_CONN; struct pollfd fds[] = { [0].fd = quit_fd, [0].events = POLLIN, @@ -595,7 +595,7 @@ nbdkit_nanosleep (unsigned sec, unsigned nsec) const char * nbdkit_export_name (void) { - struct connection *conn = threadlocal_get_conn (); + struct connection *conn = GET_CONN; if (!conn) { nbdkit_error ("no connection in this thr...

...t > or ppoll to break our wait as soon as we know there is no reason to > continue on with the transaction. > > +++ b/server/public.c > +#else > + struct timespec ts; > + struct connection *conn = threadlocal_get_conn (); > + struct pollfd fds[2] = { > + [0].fd = quit_fd, > + [0].events = POLLIN, > + [1].fd = conn ? conn->status_pipe[0] : -1, > + [1].events = POLLIN, In testing this, the code is responsive to a multi-threaded connection detecting client death on any other thread, but not responsive to a single-threaded connection detecting cli...

...+extern int threads; extern int tls; extern const char *tls_certificates_dir; extern const char *tls_psk; -extern int tls_verify_peer; +extern bool tls_verify_peer; extern char *unixsocket; -extern int verbose; -extern int threads; +extern bool verbose; extern volatile int quit; extern int quit_fd; -extern int forked_into_background; +extern bool forked_into_background; extern struct backend *backend; #define for_each_backend(b) for (b = backend; b != NULL; b = b->next) @@ -140,7 +140,7 @@ extern void connection_set_close (struct connection *, connection_close_function /* crypto....

...11,7 +321,6 @@ nbdkit_nanosleep (unsigned sec, unsigned nsec) * NBD_CMD_DISC or a problem with the connection * - the input socket detects POLLRDHUP/POLLHUP/POLLERR */ - struct timespec ts; struct connection *conn = threadlocal_get_conn (); struct pollfd fds[] = { [0].fd = quit_fd, @@ -323,14 +332,6 @@ nbdkit_nanosleep (unsigned sec, unsigned nsec) }; sigset_t all; - if (sec >= INT_MAX - nsec / 1000000000) { - nbdkit_error ("sleep request is too long"); - errno = EINVAL; - return -1; - } - ts.tv_sec = sec + nsec / 1000000000; - ts.tv_nsec =...

v1 was here: https://www.redhat.com/archives/libguestfs/2019-November/msg00003.html This version depends on: https://www.redhat.com/archives/libguestfs/2019-November/msg00004.html and this series: https://www.redhat.com/archives/libguestfs/2019-November/msg00009.html The delta has been reduced slightly because of changes made possible by cleaning up and fixing the quit path in nbdkit. It's

Somewhat invasive but mostly mechanical change to how plugins are called. This patch is in preparation for adding a second backend subtype for filters. Rich.

...18-2020 Red Hat Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are @@ -53,6 +53,8 @@ nbdkit_error (const char *fs, ...) error_flagged = true; } +bool listen_stdin; + volatile int quit; int quit_fd = -1; @@ -427,7 +429,24 @@ test_nbdkit_read_password (void) pass = false; } - /* XXX Testing reading from stdin would require setting up a pty */ + /* XXX Testing reading from stdin would require setting up a pty. But + * we can test that it is forbidden with -s. + */ + listen_stdi...

...18-2020 Red Hat Inc. * * Redistribution and use in source and binary forms, with or without * modification, are permitted provided that the following conditions are @@ -53,6 +53,8 @@ nbdkit_error (const char *fs, ...) error_flagged = true; } +bool listen_stdin; + volatile int quit; int quit_fd = -1; @@ -427,7 +429,24 @@ test_nbdkit_read_password (void) pass = false; } - /* XXX Testing reading from stdin would require setting up a pty */ + /* XXX Testing reading from stdin would require setting up a pty. But + * we can test that it is forbidden with -s. + */ + listen_stdi...

We noticed while writing various libnbd tests that when the delay filter is in use, there are scenarios where we had to resort to SIGKILL to get rid of nbdkit, because it was non-responsive to SIGINT. I'm still trying to figure out the best way to add testsuite coverage of this, but already proved to myself that it works from the command line, under two scenarios that both used to cause long

Also available here: https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw This is the port to Windows using native Windows APIs (not MSYS or Cygwin). This patch series is at the point where it basically now works. I can run the server with the memory plugin, and access it remotely using guestfish, creating filesystems and so on without any apparent problems. Nevertheless there are many

...b/lib/Makefile.am @@ -48,6 +48,7 @@ libnbdkit_la_SOURCES = \ parse.c \ password.c \ path.c \ + slow.c \ vfprintf.c \ $(NULL) diff --git a/server/internal.h b/server/internal.h index a3f4d1f1..9ef0b066 100644 --- a/server/internal.h +++ b/server/internal.h @@ -153,6 +153,7 @@ extern int quit_fd; extern void set_up_quit_pipe (void); extern void close_quit_pipe (void); extern void handle_quit (int sig); +extern void do_nbdkit_shutdown (void); /* signals.c */ extern void set_up_signals (void); @@ -473,6 +474,7 @@ extern int backend_cache (struct backend *b, extern struct backend *plu...

...on of the timeout to still reach the end time + */ + nbdkit_error ("nbdkit_nanosleep not yet ported to systems without ppoll"); + errno = ENOSYS; + return -1; +#else + struct timespec ts; + struct connection *conn = threadlocal_get_conn (); + struct pollfd fds[2] = { + [0].fd = quit_fd, + [0].events = POLLIN, + [1].fd = conn ? conn->status_pipe[0] : -1, + [1].events = POLLIN, + }; + sigset_t all; + + if (sec >= INT_MAX - nsec / 1000000000) { + nbdkit_error ("sleep request is too long"); + errno = EINVAL; + return -1; + } + ts.tv_sec = sec +...

Also available here: https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw-nolib After a lot of work I have made the port to Windows work without using a separate library. Instead, on Windows only, we build an "import library" (library of stubs) which resolves references to nbdkit_* functions in the main program and fixes up the plugin, basically the first technique outlined in

...ommand && backend->flush (backend, conn) == -1) return get_error (conn); return 0; diff --git a/src/internal.h b/src/internal.h index 068204b..d0c60f2 100644 --- a/src/internal.h +++ b/src/internal.h @@ -108,6 +108,8 @@ extern int threads; extern volatile int quit; extern int quit_fd; +struct backend *backend; + /* cleanup.c */ extern void cleanup_free (void *ptr); #define CLEANUP_FREE __attribute__((cleanup (cleanup_free))) @@ -142,28 +144,31 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou #define debug nbdkit_debug /* plugins.c */...

...const typeof (((type *) 0)->member) *__mptr = (ptr); \ + (type *) ((char *) __mptr - offsetof(type, member)); \ + }) + /* main.c */ extern const char *exportname; extern const char *ipaddr; @@ -108,6 +114,8 @@ extern int threads; extern volatile int quit; extern int quit_fd; +extern struct backend *backend; + /* cleanup.c */ extern void cleanup_free (void *ptr); #define CLEANUP_FREE __attribute__((cleanup (cleanup_free))) @@ -142,28 +150,31 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou #define debug nbdkit_debug /* plugin...

...const typeof (((type *) 0)->member) *__mptr = (ptr); \ + (type *) ((char *) __mptr - offsetof(type, member)); \ + }) + /* main.c */ extern const char *exportname; extern const char *ipaddr; @@ -108,6 +114,8 @@ extern int threads; extern volatile int quit; extern int quit_fd; +extern struct backend *backend; + /* cleanup.c */ extern void cleanup_free (void *ptr); #define CLEANUP_FREE __attribute__((cleanup (cleanup_free))) @@ -142,28 +150,31 @@ extern int crypto_negotiate_tls (struct connection *conn, int sockin, int sockou #define debug nbdkit_debug /* plugin...

The patches following do indeed allow you to compile nbdkit.exe, but it does not actually work yet. I'm posting this experimental series more as a work in progress and to get feedback. Note this does not require Windows itself to build or test. You can cross-compile it using mingw64-* packages on Fedora or Debian, and test it [spoiler alert: it fails] using Wine. Rich.