Richard W.M. Jones
2020-Aug-15 21:16 UTC
[Libguestfs] [PATCH EXPERIMENTAL nbdkit 0/2] Port to Windows using mingw.
The patches following do indeed allow you to compile nbdkit.exe, but it does not actually work yet. I'm posting this experimental series more as a work in progress and to get feedback. Note this does not require Windows itself to build or test. You can cross-compile it using mingw64-* packages on Fedora or Debian, and test it [spoiler alert: it fails] using Wine. Rich.
Richard W.M. Jones
2020-Aug-15 21:16 UTC
[Libguestfs] [PATCH EXPERIMENTAL nbdkit 1/2] Add libnbdkit.so.
This essentially turns the whole of nbdkit into a library. There is a rump nbdkit program left which simply contains a main() function that forwards to the nbdkit_main() function in the library. The reason for this is to allow nbdkit to be compiled on Windows, because Windows does not allow shared libraries to contain any undefined symbols. nbdkit previously relied on undefined symbols in all plugins and filters (eg. nbdkit_parse_int) which get resolved at run time by the nbdkit program. In order to make this work on Windows we need to have a new library (libnbdkit.so on Linux, but probably called something like LIBNBDKIT.DLL on Windows) which will contain these symbols, and plugins can then be compiled with -lnbdkit so they will link with this library, hence no undefined symbols. Note this change is backwards compatible. Because the rump nbdkit binary links with -lnbdkit it still contains and reexports all the symbols, so plugins and filters which don't explicitly link with -lnbdkit will still be loadable (on existing platforms). --- server/Makefile.am | 20 +++++++------ server/main.c | 2 +- server/nbdkit.c | 46 ++++++++++++++++++++++++++++++ server/nbdkit.syms | 1 + tests/test-nbdkit-backend-debug.sh | 24 ++++++++-------- wrapper.c | 10 +++++-- 6 files changed, 79 insertions(+), 24 deletions(-) diff --git a/server/Makefile.am b/server/Makefile.am index 58b22341..79e51294 100644 --- a/server/Makefile.am +++ b/server/Makefile.am @@ -34,8 +34,12 @@ include $(top_srcdir)/common-rules.mk EXTRA_DIST = nbdkit.syms sbin_PROGRAMS = nbdkit +nbdkit_SOURCES = nbdkit.c +nbdkit_CFLAGS = $(WARNINGS_CFLAGS) +nbdkit_LDADD = libnbdkit.la -nbdkit_SOURCES = \ +lib_LTLIBRARIES = libnbdkit.la +libnbdkit_la_SOURCES = \ backend.c \ background.c \ captive.c \ @@ -70,10 +74,10 @@ nbdkit_SOURCES = \ $(top_srcdir)/include/nbdkit-filter.h \ $(NULL) if ENABLE_LIBFUZZER -nbdkit_SOURCES += fuzzer.c +libnbdkit_la_SOURCES += fuzzer.c endif -nbdkit_CPPFLAGS = \ +libnbdkit_la_CPPFLAGS = \ -Dbindir=\"$(bindir)\" \ -Dlibdir=\"$(libdir)\" \ -Dmandir=\"$(mandir)\" \ @@ -86,21 +90,21 @@ nbdkit_CPPFLAGS = \ -I$(top_srcdir)/common/protocol \ -I$(top_srcdir)/common/utils \ $(NULL) -nbdkit_CFLAGS = \ +libnbdkit_la_CFLAGS = \ $(PTHREAD_CFLAGS) \ $(WARNINGS_CFLAGS) \ $(GNUTLS_CFLAGS) \ $(LIBSELINUX_CFLAGS) \ $(VALGRIND_CFLAGS) \ $(NULL) -nbdkit_LDADD = \ +libnbdkit_la_LIBADD = \ $(GNUTLS_LIBS) \ $(LIBSELINUX_LIBS) \ $(DL_LIBS) \ $(top_builddir)/common/protocol/libprotocol.la \ $(top_builddir)/common/utils/libutils.la \ $(NULL) -nbdkit_LDFLAGS = \ +libnbdkit_la_LDFLAGS = \ $(PTHREAD_LIBS) \ $(DL_LDFLAGS) \ $(NULL) @@ -109,7 +113,7 @@ if USE_LINKER_SCRIPT_FOR_SERVER # adds loads of fuzzer and ASAN-related symbols that are required by # the plugins but which our linker script tries to hide. if !ENABLE_LIBFUZZER -nbdkit_LDFLAGS += -Wl,--version-script=$(srcdir)/nbdkit.syms +libnbdkit_la_LDFLAGS += -Wl,--version-script=$(srcdir)/nbdkit.syms endif endif @@ -118,7 +122,7 @@ endif BUILT_SOURCES = synopsis.c EXTRA_DIST += synopsis.c -EXTRA_nbdkit_DEPENDENCIES = synopsis.c +EXTRA_libnbdkit_la_DEPENDENCIES = synopsis.c CLEANFILES += synopsis.c main.c: synopsis.c synopsis.c: $(top_srcdir)/docs/synopsis.txt diff --git a/server/main.c b/server/main.c index eaa052ce..1f806824 100644 --- a/server/main.c +++ b/server/main.c @@ -169,7 +169,7 @@ dump_config (void) } int -main (int argc, char *argv[]) +nbdkit_main (int argc, char *argv[]) { int c; bool help = false, version = false, dump_plugin = false; diff --git a/server/nbdkit.c b/server/nbdkit.c new file mode 100644 index 00000000..e2e55cbb --- /dev/null +++ b/server/nbdkit.c @@ -0,0 +1,46 @@ +/* nbdkit + * Copyright (C) 2013-2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> + +extern int nbdkit_main (int argc, char *argv[]); + +int +main (int argc, char *argv[]) +{ + /* This does nothing except call into the real main function in + * libnbdkit.so. + */ + return nbdkit_main (argc, argv); +} diff --git a/server/nbdkit.syms b/server/nbdkit.syms index a67669b7..a516cc0f 100644 --- a/server/nbdkit.syms +++ b/server/nbdkit.syms @@ -54,6 +54,7 @@ nbdkit_extents_new; nbdkit_get_extent; nbdkit_is_tls; + nbdkit_main; nbdkit_nanosleep; nbdkit_parse_bool; nbdkit_parse_int8_t; diff --git a/tests/test-nbdkit-backend-debug.sh b/tests/test-nbdkit-backend-debug.sh index 3a28b756..1533ba44 100755 --- a/tests/test-nbdkit-backend-debug.sh +++ b/tests/test-nbdkit-backend-debug.sh @@ -49,10 +49,10 @@ nbdkit -U - \ --run "qemu-img convert \$nbd $out" |& tee $debug # Should contain all debugging messages. -grep '^nbdkit:.*debug: nofilter: open' $debug -grep '^nbdkit:.*debug: memory: open' $debug -grep '^nbdkit:.*debug: nofilter: pread' $debug -grep '^nbdkit:.*debug: memory: pread' $debug +grep 'nbdkit:.*debug: nofilter: open' $debug +grep 'nbdkit:.*debug: memory: open' $debug +grep 'nbdkit:.*debug: nofilter: pread' $debug +grep 'nbdkit:.*debug: memory: pread' $debug nbdkit -U - \ -v -D nbdkit.backend.controlpath=0 \ @@ -61,10 +61,10 @@ nbdkit -U - \ --run "qemu-img convert \$nbd $out" |& tee $debug # Should contain only datapath messages. -grep -v '^nbdkit:.*debug: nofilter: open' $debug -grep -v '^nbdkit:.*debug: memory: open' $debug -grep '^nbdkit:.*debug: nofilter: pread' $debug -grep '^nbdkit:.*debug: memory: pread' $debug +grep -v 'nbdkit:.*debug: nofilter: open' $debug +grep -v 'nbdkit:.*debug: memory: open' $debug +grep 'nbdkit:.*debug: nofilter: pread' $debug +grep 'nbdkit:.*debug: memory: pread' $debug nbdkit -U - \ -v -D nbdkit.backend.datapath=0 \ @@ -73,7 +73,7 @@ nbdkit -U - \ --run "qemu-img convert \$nbd $out" |& tee $debug # Should contain only controlpath messages. -grep '^nbdkit:.*debug: nofilter: open' $debug -grep '^nbdkit:.*debug: memory: open' $debug -grep -v '^nbdkit:.*debug: nofilter: pread' $debug -grep -v '^nbdkit:.*debug: memory: pread' $debug +grep 'nbdkit:.*debug: nofilter: open' $debug +grep 'nbdkit:.*debug: memory: open' $debug +grep -v 'nbdkit:.*debug: nofilter: pread' $debug +grep -v 'nbdkit:.*debug: memory: pread' $debug diff --git a/wrapper.c b/wrapper.c index c27afae0..7029c2cd 100644 --- a/wrapper.c +++ b/wrapper.c @@ -168,12 +168,16 @@ main (int argc, char *argv[]) } } - /* Needed for plugins written in OCaml. */ + /* As well as being required by nbdkit, this is also needed for + * plugins written in OCaml. + */ s = getenv ("LD_LIBRARY_PATH"); if (s) - r = asprintf (&s, "%s/plugins/ocaml/.libs:%s", builddir, s); + r = asprintf (&s, "%s/server/.libs:%s/plugins/ocaml/.libs:%s", + builddir, builddir, s); else - r = asprintf (&s, "%s/plugins/ocaml/.libs", builddir); + r = asprintf (&s, "%s/server/.libs:%s/plugins/ocaml/.libs", + builddir, builddir); if (r < 0) { perror ("asprintf"); exit (EXIT_FAILURE); -- 2.27.0
Richard W.M. Jones
2020-Aug-15 21:16 UTC
[Libguestfs] [PATCH EXPERIMENTAL nbdkit 2/2] Port to Windows using mingw.
--- include/nbdkit-common.h | 6 ++ configure.ac | 41 +++++++-- common/utils/Makefile.am | 2 + server/internal.h | 6 +- common/utils/windows-compat.h | 99 ++++++++++++++++++++ server/background.c | 20 +++++ server/captive.c | 20 ++++- server/connections.c | 9 +- server/crypto.c | 7 ++ server/debug.c | 8 ++ server/log-stderr.c | 4 + server/log-syslog.c | 19 ++++ server/main.c | 28 +++++- server/plugins.c | 3 + server/protocol.c | 4 + server/public.c | 48 +++++++++- server/quit.c | 4 + server/signals.c | 2 + server/socket-activation.c | 12 +++ server/sockets.c | 32 ++++++- server/usergroup.c | 25 +++++- common/utils/utils.c | 44 ++++++++- common/utils/windows-compat.c | 165 ++++++++++++++++++++++++++++++++++ plugins/cdi/cdi.c | 7 -- README | 50 +++++++++++ 25 files changed, 637 insertions(+), 28 deletions(-) diff --git a/include/nbdkit-common.h b/include/nbdkit-common.h index d38b37d2..c7d91512 100644 --- a/include/nbdkit-common.h +++ b/include/nbdkit-common.h @@ -40,7 +40,13 @@ #include <stdarg.h> #include <stdint.h> #include <errno.h> + +#if !defined(_WIN32) && !defined(__MINGW32__) && \ + !defined(__CYGWIN__) && !defined(_MSC_VER) #include <sys/socket.h> +#else +#include <ws2tcpip.h> +#endif #include <nbdkit-version.h> diff --git a/configure.ac b/configure.ac index a4d7fa88..df42594e 100644 --- a/configure.ac +++ b/configure.ac @@ -306,11 +306,22 @@ AC_CHECK_HEADERS([\ alloca.h \ byteswap.h \ endian.h \ + grp.h \ + poll.h \ + netdb.h \ + netinet/in.h \ + netinet/tcp.h \ + pwd.h \ + termios.h \ stdatomic.h \ + syslog.h \ sys/endian.h \ sys/mman.h \ sys/prctl.h \ - sys/procctl.h]) + sys/procctl.h \ + sys/socket.h \ + sys/un.h \ + sys/wait.h]) AC_CHECK_HEADERS([linux/vm_sockets.h], [], [], [#include <sys/socket.h>]) @@ -318,13 +329,21 @@ dnl Check for functions in libc, all optional. AC_CHECK_FUNCS([\ accept4 \ fdatasync \ + flockfile \ + funlockfile \ get_current_dir_name \ + getpeername \ mkostemp \ mlockall \ openlog \ + pipe \ pipe2 \ + poll \ ppoll \ - posix_fadvise]) + posix_fadvise \ + realpath \ + sigaction \ + strndup]) dnl Check whether printf("%m") works AC_CACHE_CHECK([whether the printf family supports %m], @@ -440,11 +459,21 @@ LIBS="$DL_LIBS $LIBS" AC_CHECK_FUNCS([dladdr]) LIBS="$old_LIBS" -dnl On mingw we should enable the -no-undefined flag. +dnl Is the platform Windows? +AC_MSG_CHECKING([if we are on the Windows platform]) +AS_CASE([$host_os], + [mingw*|msys*|cygwin*], [ + is_windows=yes + AC_DEFINE([IS_WINDOWS],[1],[this is the Windows platform.]) + LIBS="$LIBS -lwsock32 -lws2_32" + ], + [is_windows=no] +) +AC_MSG_RESULT([$is_windows]) + AC_MSG_CHECKING([SHARED_LDFLAGS]) -AS_CASE([$host_os], - [mingw*|msys*|cygwin*], [SHARED_LDFLAGS="$SHARED_LDFLAGS -no-undefined"], - [] +AS_IF([test "x$is_windows" = "xyes"], + [SHARED_LDFLAGS="$SHARED_LDFLAGS -no-undefined"] ) AC_MSG_RESULT([$SHARED_LDFLAGS]) AC_SUBST([SHARED_LDFLAGS]) diff --git a/common/utils/Makefile.am b/common/utils/Makefile.am index a621790a..46d9de98 100644 --- a/common/utils/Makefile.am +++ b/common/utils/Makefile.am @@ -43,6 +43,8 @@ libutils_la_SOURCES = \ utils.h \ vector.c \ vector.h \ + windows-compat.c \ + windows-compat.h \ $(NULL) libutils_la_CPPFLAGS = \ -I$(top_srcdir)/include \ diff --git a/server/internal.h b/server/internal.h index d043225a..3b52ad00 100644 --- a/server/internal.h +++ b/server/internal.h @@ -36,9 +36,12 @@ #include <stdbool.h> #include <stddef.h> #include <stdarg.h> -#include <sys/socket.h> #include <pthread.h> +#ifdef HAVE_SYS_SOCKET_H +#include <sys/socket.h> +#endif + #define NBDKIT_API_VERSION 2 #define NBDKIT_INTERNAL #include "nbdkit-plugin.h" @@ -47,6 +50,7 @@ #include "nbd-protocol.h" #include "unix-path-max.h" #include "vector.h" +#include "windows-compat.h" /* Define unlikely macro, but only for GCC. These are used to move * debug and error handling code out of hot paths. diff --git a/common/utils/windows-compat.h b/common/utils/windows-compat.h new file mode 100644 index 00000000..1f9a8d85 --- /dev/null +++ b/common/utils/windows-compat.h @@ -0,0 +1,99 @@ +/* nbdkit + * Copyright (C) 2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#ifndef NBDKIT_WINDOWS_COMPAT_H +#define NBDKIT_WINDOWS_COMPAT_H + +#include <config.h> + +#include <ws2tcpip.h> +#include <windows.h> + +/* Windows doesn't have O_CLOEXEC, but it also doesn't have file + * descriptors that can be inherited across exec. Similarly for + * O_NOCTTY. + */ +#ifndef O_CLOEXEC +#define O_CLOEXEC 0 +#endif +#ifndef O_NOCTTY +#define O_NOCTTY 0 +#endif + +/* AI_ADDRCONFIG is not available on Windows. It enables a rather + * obscure feature of getaddrinfo to do with IPv6. + */ +#ifndef AI_ADDRCONFIG +#define AI_ADDRCONFIG 0 +#endif + +/* Unfortunately quite commonly used at the moment. Make it a common + * macro so we can easily find places which need porting. + * + * Note: Don't use this for things which can never work on Windows + * (eg. Unix socket support). Those should just give regular errors. + */ +#define NOT_IMPLEMENTED_ON_WINDOWS(feature) \ + do { \ + fprintf (stderr, "nbdkit: %s is not implemented for Windows.\n", feature); \ + fprintf (stderr, "You can help by contributing to the Windows port, see\n"); \ + fprintf (stderr, "nbdkit README in the source for how to contribute.\n"); \ + exit (EXIT_FAILURE); \ + } while (0) + +/* On Windows we can replace certain functions if they appear to be + * missing. Implementations of these are present in windows-compat.c. + * We don't bother using the LIBOBJS mechanism. + */ +#ifdef IS_WINDOWS +#ifndef HAVE_POLL +struct pollfd { + int fd; + short events; + short revents; +}; +#define POLLIN 0x0001 +#define POLLOUT 0x0002 +#define POLLERR 0x0008 +#define POLLHUP 0x0010 +#define POLLRDHUP 0x2000 +extern int poll (struct pollfd *fds, int n, int timeout); +#endif +#ifndef HAVE_REALPATH +extern char *realpath (const char *path, char *resolved_path); +#endif +#ifndef HAVE_STRNDUP +extern char *strndup (const char *s, size_t n); +#endif +#endif + +#endif /* NBDKIT_WINDOWS_COMPAT_H */ diff --git a/server/background.c b/server/background.c index 72ab1ef6..371ee4ce 100644 --- a/server/background.c +++ b/server/background.c @@ -44,6 +44,8 @@ /* True if we forked into the background (used to control log messages). */ bool forked_into_background; +#ifndef IS_WINDOWS + /* Run as a background process. If foreground is set (ie. -f or * equivalent) then this does nothing. Otherwise it forks into the * background and sets forked_into_background. @@ -79,3 +81,21 @@ fork_into_background (void) forked_into_background = true; debug ("forked into background (new pid = %d)", getpid ()); } + +#else /* IS_WINDOWS */ + +/* Note if you implement this, you must also implement (or change) + * server/log-syslog.c + */ + +void +fork_into_background (void) +{ + if (foreground) + return; + + fprintf (stderr, "nbdkit: You must use the -f option on Windows.\n"); + NOT_IMPLEMENTED_ON_WINDOWS ("daemonizing"); +} + +#endif /* IS_WINDOWS */ diff --git a/server/captive.c b/server/captive.c index a8947d7c..06aea714 100644 --- a/server/captive.c +++ b/server/captive.c @@ -38,14 +38,19 @@ #include <string.h> #include <unistd.h> #include <sys/types.h> -#include <sys/wait.h> #include <signal.h> #include <assert.h> +#ifdef HAVE_SYS_WAIT_H +#include <sys/wait.h> +#endif + #include "utils.h" #include "internal.h" +#ifndef IS_WINDOWS + /* Handle the --run option. If run is NULL, does nothing. If run is * not NULL then run nbdkit as a captive subprocess of the command. */ @@ -208,3 +213,16 @@ run_command (void) debug ("forked into background (new pid = %d)", getpid ()); } + +#else /* IS_WINDOWS */ + +void +run_command (void) +{ + if (!run) + return; + + NOT_IMPLEMENTED_ON_WINDOWS ("--run"); +} + +#endif /* IS_WINDOWS */ diff --git a/server/connections.c b/server/connections.c index a3dd4ca7..318186ef 100644 --- a/server/connections.c +++ b/server/connections.c @@ -38,10 +38,13 @@ #include <inttypes.h> #include <string.h> #include <unistd.h> -#include <sys/socket.h> #include <fcntl.h> #include <assert.h> +#ifdef HAVE_SYS_SOCKET_H +#include <sys/socket.h> +#endif + #include "internal.h" #include "utils.h" @@ -267,7 +270,7 @@ new_connection (int sockin, int sockout, int nworkers) perror ("pipe2"); goto error2; } -#else +#elifdef HAVE_PIPE /* If we were fully parallel, then this function could be * accepting connections in one thread while another thread could * be in a plugin trying to fork. But plugins.c forced @@ -296,6 +299,8 @@ new_connection (int sockin, int sockout, int nworkers) goto error2; } unlock_request (); +#else + /* Windows has neither pipe2 nor pipe. XXX */ #endif } diff --git a/server/crypto.c b/server/crypto.c index d291f4e7..ba8b558d 100644 --- a/server/crypto.c +++ b/server/crypto.c @@ -156,6 +156,7 @@ load_certificates (const char *path) static int start_certificates (void) { +#ifndef IS_WINDOWS /* Try to locate the certificates directory and load them. */ if (tls_certificates_dir == NULL) { const char *home; @@ -190,6 +191,12 @@ start_certificates (void) } return -1; +#else /* IS_WINDOWS */ + if (load_certificates (root_tls_certificates_dir)) + goto found_certificates; + return -1; +#endif /* IS_WINDOWS */ + found_certificates: #ifdef HAVE_GNUTLS_CERTIFICATE_SET_KNOWN_DH_PARAMS gnutls_certificate_set_known_dh_params (x509_creds, GNUTLS_SEC_PARAM_MEDIUM); diff --git a/server/debug.c b/server/debug.c index 96c59798..956e61ad 100644 --- a/server/debug.c +++ b/server/debug.c @@ -67,13 +67,17 @@ nbdkit_vdebug (const char *fs, va_list args) if (!verbose) return; +#ifdef HAVE_FLOCKFILE flockfile (stderr); +#endif prologue (); vfprintf (stderr, fs, args); fprintf (stderr, "\n"); +#ifdef HAVE_FUNLOCKFILE funlockfile (stderr); +#endif errno = err; } @@ -88,7 +92,9 @@ nbdkit_debug (const char *fs, ...) if (!verbose) return; +#ifdef HAVE_FLOCKFILE flockfile (stderr); +#endif prologue (); va_start (args, fs); @@ -96,7 +102,9 @@ nbdkit_debug (const char *fs, ...) va_end (args); fprintf (stderr, "\n"); +#ifdef HAVE_FUNLOCKFILE funlockfile (stderr); +#endif errno = err; } diff --git a/server/log-stderr.c b/server/log-stderr.c index 4a8cec00..7a33a043 100644 --- a/server/log-stderr.c +++ b/server/log-stderr.c @@ -51,7 +51,9 @@ log_stderr_verror (const char *fs, va_list args) size_t instance_num = threadlocal_get_instance_num (); int tty; +#ifdef HAVE_FLOCKFILE flockfile (stderr); +#endif tty = isatty (fileno (stderr)); if (tty) fputs ("\033[1;31m", stderr); @@ -71,7 +73,9 @@ log_stderr_verror (const char *fs, va_list args) if (tty) fputs ("\033[0m", stderr); +#ifdef HAVE_FUNLOCKFILE funlockfile (stderr); +#endif errno = err; /* must be last line of function */ } diff --git a/server/log-syslog.c b/server/log-syslog.c index 7f4a6cbb..6297e377 100644 --- a/server/log-syslog.c +++ b/server/log-syslog.c @@ -37,10 +37,15 @@ #include <stdarg.h> #include <string.h> #include <errno.h> + +#ifdef HAVE_SYSLOG_H #include <syslog.h> +#endif #include "internal.h" +#ifndef IS_WINDOWS + /* Tempted to use LOG_FTP instead of LOG_DAEMON! */ static const int PRIORITY = LOG_DAEMON|LOG_ERR; @@ -79,3 +84,17 @@ log_syslog_verror (const char *fs, va_list args) out: errno = err; } + +#else /* IS_WINDOWS */ + +/* Note that because daemonization is not implemented, this should + * never be called unless the user explicitly added --log=syslog. + */ + +void +log_syslog_verror (const char *fs, va_list args) +{ + NOT_IMPLEMENTED_ON_WINDOWS ("--log=syslog"); +} + +#endif diff --git a/server/main.c b/server/main.c index 1f806824..9451bc72 100644 --- a/server/main.c +++ b/server/main.c @@ -42,15 +42,21 @@ #include <limits.h> #include <errno.h> #include <assert.h> -#include <syslog.h> #include <sys/types.h> #include <sys/stat.h> -#include <sys/socket.h> + +#ifdef HAVE_SYSLOG_H +#include <syslog.h> +#endif #ifdef HAVE_SYS_MMAN_H #include <sys/mman.h> #endif +#ifdef HAVE_SYS_SOCKET_H +#include <sys/socket.h> +#endif + #ifdef HAVE_LINUX_VM_SOCKETS_H #include <linux/vm_sockets.h> #endif @@ -729,6 +735,8 @@ nbdkit_main (int argc, char *argv[]) return EXIT_SUCCESS; } +#ifndef IS_WINDOWS + /* Implementation of '-U -' */ static char * make_random_fifo (void) @@ -761,6 +769,16 @@ make_random_fifo (void) return sock; } +#else /* IS_WINDOWS */ + +static char * +make_random_fifo (void) +{ + NOT_IMPLEMENTED_ON_WINDOWS ("-U -"); +} + +#endif /* IS_WINDOWS */ + static struct backend * open_plugin_so (size_t i, const char *name, int short_name) { @@ -1002,6 +1020,7 @@ is_config_key (const char *key, size_t len) static void error_if_stdio_closed (void) { +#ifdef F_GETFL if (fcntl (STDERR_FILENO, F_GETFL) == -1) { /* Nowhere we can report the error. Oh well. */ exit (EXIT_FAILURE); @@ -1011,6 +1030,7 @@ error_if_stdio_closed (void) perror ("expecting stdin/stdout to be opened"); exit (EXIT_FAILURE); } +#endif } /* Sanitize stdin/stdout to /dev/null, after saving the originals @@ -1025,6 +1045,7 @@ error_if_stdio_closed (void) static void switch_stdio (void) { +#if defined(F_DUPFD_CLOEXEC) || defined(F_DUPFD) fflush (stdin); fflush (NULL); if (listen_stdin || run) { @@ -1042,6 +1063,8 @@ switch_stdio (void) exit (EXIT_FAILURE); } } +#endif +#ifndef IS_WINDOWS close (STDIN_FILENO); close (STDOUT_FILENO); if (open ("/dev/null", O_RDONLY) != STDIN_FILENO || @@ -1049,4 +1072,5 @@ switch_stdio (void) perror ("open"); exit (EXIT_FAILURE); } +#endif } diff --git a/server/plugins.c b/server/plugins.c index 218764da..736154b8 100644 --- a/server/plugins.c +++ b/server/plugins.c @@ -39,7 +39,10 @@ #include <inttypes.h> #include <assert.h> #include <errno.h> + +#ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> +#endif #include "internal.h" #include "minmax.h" diff --git a/server/protocol.c b/server/protocol.c index e7ba4bf1..7f995baa 100644 --- a/server/protocol.c +++ b/server/protocol.c @@ -706,7 +706,11 @@ protocol_recv_request_send_reply (void) /* Perform the request. Only this part happens inside the request lock. */ if (quit || !connection_get_status ()) { +#ifdef ESHUTDOWN error = ESHUTDOWN; +#else + error = EIO; +#endif } else { lock_request (); diff --git a/server/public.c b/server/public.c index f682d732..fbc27f5b 100644 --- a/server/public.c +++ b/server/public.c @@ -45,11 +45,20 @@ #include <string.h> #include <unistd.h> #include <limits.h> -#include <termios.h> #include <errno.h> -#include <poll.h> #include <signal.h> + +#ifdef HAVE_POLL_H +#include <poll.h> +#endif + +#ifdef HAVE_TERMIOS_H +#include <termios.h> +#endif + +#ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> +#endif #include "ascii-ctype.h" #include "ascii-string.h" @@ -466,6 +475,8 @@ nbdkit_read_password (const char *value, char **password) return 0; } +#ifndef IS_WINDOWS + static int read_password_interactive (char **password) { @@ -528,6 +539,18 @@ read_password_interactive (char **password) return 0; } +#else /* IS_WINDOWS */ + +static int +read_password_interactive (char **password) +{ + NOT_IMPLEMENTED_ON_WINDOWS ("password=-"); +} + +#endif + +#ifndef IS_WINDOWS + static int read_password_from_fd (const char *what, int fd, char **password) { @@ -575,6 +598,21 @@ read_password_from_fd (const char *what, int fd, char **password) return 0; } +#else /* IS_WINDOWS */ + +/* As far as I know this will never be possible on Windows, so it's a + * simple error. + */ +static int +read_password_from_fd (const char *what, int fd, char **password) +{ + nbdkit_error ("not possible to read passwords from file descriptors " + "under Windows"); + return -1; +} + +#endif /* IS_WINDOWS */ + int nbdkit_nanosleep (unsigned sec, unsigned nsec) { @@ -703,10 +741,14 @@ nbdkit_peer_name (struct sockaddr *addr, socklen_t *addrlen) return -1; } +#ifdef HAVE_GETPEERNAME if (getpeername (s, addr, addrlen) == -1) { nbdkit_error ("peername: %m"); return -1; } - return 0; +#else + nbdkit_error ("getpeername not available on this platform"); + return -1; +#endif } diff --git a/server/quit.c b/server/quit.c index 13fef437..9d189a90 100644 --- a/server/quit.c +++ b/server/quit.c @@ -54,6 +54,7 @@ static int write_quit_fd; void set_up_quit_pipe (void) { +#ifndef IS_WINDOWS int fds[2]; #ifdef HAVE_PIPE2 @@ -78,13 +79,16 @@ set_up_quit_pipe (void) #endif quit_fd = fds[0]; write_quit_fd = fds[1]; +#endif } void close_quit_pipe (void) { +#ifndef IS_WINDOWS close (quit_fd); close (write_quit_fd); +#endif } static void diff --git a/server/signals.c b/server/signals.c index d7dc17d0..6354b630 100644 --- a/server/signals.c +++ b/server/signals.c @@ -44,6 +44,7 @@ void set_up_signals (void) { +#ifdef HAVE_SIGACTION struct sigaction sa; memset (&sa, 0, sizeof sa); @@ -58,4 +59,5 @@ set_up_signals (void) sa.sa_flags = SA_RESTART; sa.sa_handler = SIG_IGN; sigaction (SIGPIPE, &sa, NULL); +#endif /* HAVE_SIGACTION */ } diff --git a/server/socket-activation.c b/server/socket-activation.c index f273f8cc..609c8efc 100644 --- a/server/socket-activation.c +++ b/server/socket-activation.c @@ -42,6 +42,8 @@ #include "internal.h" +#ifndef IS_WINDOWS + /* Handle socket activation. This is controlled through special * environment variables inherited by nbdkit. Returns 0 if no socket * activation. Otherwise returns the number of FDs. See also @@ -105,3 +107,13 @@ get_socket_activation (void) return nr_fds; } + +#else /* IS_WINDOWS */ + +unsigned int +get_socket_activation (void) +{ + return 0; +} + +#endif /* IS_WINDOWS */ diff --git a/server/sockets.c b/server/sockets.c index f6c9643a..1af88f37 100644 --- a/server/sockets.c +++ b/server/sockets.c @@ -38,15 +38,33 @@ #include <inttypes.h> #include <string.h> #include <unistd.h> -#include <poll.h> #include <errno.h> #include <assert.h> #include <sys/types.h> + +#ifdef HAVE_POLL_H +#include <poll.h> +#endif + +#ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> +#endif + +#ifdef HAVE_SYS_UN_H #include <sys/un.h> +#endif + +#ifdef HAVE_NETINET_IN_H #include <netinet/in.h> +#endif + +#ifdef HAVE_NETINET_TCP_H #include <netinet/tcp.h> +#endif + +#ifdef HAVE_NETDB_H #include <netdb.h> +#endif #ifdef HAVE_LINUX_VM_SOCKETS_H #include <linux/vm_sockets.h> @@ -94,6 +112,8 @@ clear_selinux_label (void) #endif } +#ifndef IS_WINDOWS + void bind_unix_socket (sockets *socks) { @@ -149,6 +169,16 @@ bind_unix_socket (sockets *socks) debug ("bound to unix socket %s", unixsocket); } +#else /* IS_WINDOWS */ + +void +bind_unix_socket (sockets *socks) +{ + NOT_IMPLEMENTED_ON_WINDOWS ("-U"); +} + +#endif /* IS_WINDOWS */ + void bind_tcpip_socket (sockets *socks) { diff --git a/server/usergroup.c b/server/usergroup.c index 11bafceb..0875e660 100644 --- a/server/usergroup.c +++ b/server/usergroup.c @@ -37,13 +37,21 @@ #include <stdarg.h> #include <string.h> #include <unistd.h> -#include <pwd.h> -#include <grp.h> #include <errno.h> #include <sys/types.h> +#ifdef HAVE_PWD_H +#include <pwd.h> +#endif + +#ifdef HAVE_GRP_H +#include <grp.h> +#endif + #include "internal.h" +#ifndef IS_WINDOWS + static uid_t parseuser (const char *); static gid_t parsegroup (const char *); @@ -138,3 +146,16 @@ parsegroup (const char *id) return grp->gr_gid; } + +#else /* IS_WINDOWS */ + +void +change_user (void) +{ + if (!user && !group) + return; + + NOT_IMPLEMENTED_ON_WINDOWS ("--user/--group"); +} + +#endif /* IS_WINDOWS */ diff --git a/common/utils/utils.c b/common/utils/utils.c index 275835e0..4c0a0fc3 100644 --- a/common/utils/utils.c +++ b/common/utils/utils.c @@ -36,12 +36,22 @@ #include <stdlib.h> #include <fcntl.h> #include <unistd.h> -#include <sys/socket.h> #include <sys/types.h> + +#ifdef HAVE_SYS_SOCKET_H +#include <sys/socket.h> +#endif + +#ifdef HAVE_SYS_WAIT_H #include <sys/wait.h> +#endif #include <nbdkit-plugin.h> +#include "windows-compat.h" + +#ifndef IS_WINDOWS + /* Convert exit status to nbd_error. If the exit status was nonzero * or another failure then -1 is returned. */ @@ -67,6 +77,10 @@ exit_status_to_nbd_error (int status, const char *cmd) return 0; } +#endif /* !IS_WINDOWS */ + +#ifndef IS_WINDOWS + /* Set the FD_CLOEXEC flag on the given fd, if it is non-negative. * On failure, close fd and return -1; on success, return fd. * @@ -76,7 +90,8 @@ exit_status_to_nbd_error (int status, const char *cmd) * prevent fd leaks to plugins that want to fork(). */ int -set_cloexec (int fd) { +set_cloexec (int fd) +{ #if (defined SOCK_CLOEXEC && defined HAVE_MKOSTEMP && defined HAVE_PIPE2 && \ defined HAVE_ACCEPT4) nbdkit_error ("prefer creating fds with CLOEXEC atomically set"); @@ -106,11 +121,24 @@ set_cloexec (int fd) { #endif } +#else /* IS_WINDOWS */ + +int +set_cloexec (int fd) +{ + return fd; +} + +#endif /* IS_WINDOWS */ + +#ifndef IS_WINDOWS + /* Set the O_NONBLOCK flag on the given fd, if it is non-negative. * On failure, close fd and return -1; on success, return fd. */ int -set_nonblock (int fd) { +set_nonblock (int fd) +{ int f; int err; @@ -127,3 +155,13 @@ set_nonblock (int fd) { } return fd; } + +#else /* IS_WINDOWS */ + +int +set_nonblock (int fd) +{ + return fd; +} + +#endif /* IS_WINDOWS */ diff --git a/common/utils/windows-compat.c b/common/utils/windows-compat.c new file mode 100644 index 00000000..6722c110 --- /dev/null +++ b/common/utils/windows-compat.c @@ -0,0 +1,165 @@ +/* nbdkit + * Copyright (C) 2020 Red Hat Inc. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions are + * met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in the + * documentation and/or other materials provided with the distribution. + * + * * Neither the name of Red Hat nor the names of its contributors may be + * used to endorse or promote products derived from this software without + * specific prior written permission. + * + * THIS SOFTWARE IS PROVIDED BY RED HAT AND CONTRIBUTORS ''AS IS'' AND + * ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, + * THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A + * PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL RED HAT OR + * CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF + * USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND + * ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, + * OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT + * OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF + * SUCH DAMAGE. + */ + +#include <config.h> + +#include <stdio.h> +#include <stdlib.h> + +#include <nbdkit-plugin.h> + +#include "windows-compat.h" + +#ifdef IS_WINDOWS + +/* Replacement functions. */ + +#ifndef HAVE_POLL +/* Windows doesn't have poll. It has something called WSAPoll in + * Winsock, but even MSFT admit it is broken. Gnulib contains an + * elaborate emulation of poll written by Paolo, but it's distributed + * under an incompatible license. However Winsock has select so we + * can write a simple (but slow) emulation of poll using select. + */ +int +poll (struct pollfd *fds, int n, int timeout) +{ + int i, nfds = 0, r; + fd_set readfds, writefds; + struct timeval tv; + + FD_ZERO (&readfds); + FD_ZERO (&writefds); + + for (i = 0; i < n; ++i) { + if (fds[i].events & POLLIN) + FD_SET (fds[i].fd, &readfds); + if (fds[i].events & POLLOUT) + FD_SET (fds[i].fd, &writefds); + if (fds[i].fd > nfds) + nfds = fds[i].fd; + fds[i].revents = 0; + } + nfds++; + + tv.tv_sec = timeout / 1000; + tv.tv_usec = timeout % 1000; + + r = select (nfds, &readfds, &writefds, NULL, &tv); + if (r == -1) + return -1; + + r = 0; + for (i = 0; i < n; ++i) { + if (FD_ISSET (fds[i].fd, &readfds)) + fds[i].revents |= POLLIN; + if (FD_ISSET (fds[i].fd, &writefds)) + fds[i].revents |= POLLOUT; + if (fds[i].revents != 0) + r++; + } + + return r; +} +#endif /* HAVE_POLL */ + +#ifndef HAVE_REALPATH +char * +realpath (const char *path, char *resolved_path) +{ + DWORD retval = 0; + char c, *r; + + /* Note that GetFullPathNameA is not thread-safe on Windows(!) so + * this will probably fail if called from a multi-threaded part of + * nbdkit. Also this function doesn't check that the file exists, + * so it's more like nbdkit_absolute_path rather than + * nbdkit_realpath. To see how we might do this properly (it's + * complicated) see https://github.com/JuliaLang/julia/issues/30588 + * XXX + */ + + /* Discover the length of the path. */ + retval = GetFullPathNameA (path, sizeof c, &c, NULL); + if (retval == 0) { + error: + nbdkit_error ("GetFullPathNameA: %s: %d", path, (int) GetLastError ()); + return NULL; + } + r = malloc (retval); + if (r == NULL) { + nbdkit_error ("malloc: %d", (int) GetLastError ()); + return NULL; + } + + /* Return the path. */ + retval = GetFullPathNameA (path, retval, r, NULL); + if (retval == 0) goto error; + return r; /* caller frees */ +} +#endif /* HAVE_REALPATH */ + +#ifndef HAVE_STRNDUP +/* From + * https://github.com/freebsd/freebsd/blob/master/lib/libc/string/strnlen.c + */ +size_t +strnlen (const char *s, size_t maxlen) +{ + size_t len; + + for (len = 0; len < maxlen; len++, s++) { + if (!*s) + break; + } + return len; +} + +/* From + * https://github.com/freebsd/freebsd/blob/master/lib/libc/string/strndup.c + */ +char * +strndup (const char *str, size_t maxlen) +{ + char *copy; + size_t len = strnlen (str, maxlen); + + copy = malloc (len+1); + if (!copy) + return NULL; + memcpy (copy, str, len); + copy[len] = '\0'; + return copy; +} +#endif /* HAVE_STRNDUP */ + +#endif /* IS_WINDOWS */ diff --git a/plugins/cdi/cdi.c b/plugins/cdi/cdi.c index 70738da6..371e7f9e 100644 --- a/plugins/cdi/cdi.c +++ b/plugins/cdi/cdi.c @@ -45,13 +45,6 @@ #include "cleanup.h" #include "utils.h" -/* This plugin doesn't do anything tricky with threads so we don't - * need O_CLOEXEC on platforms where it is not defined. - */ -#ifndef O_CLOEXEC -#define O_CLOEXEC 0 -#endif - /* Parameters. */ static const char *name; /* Name or URI of container image. */ static int layer = 0; /* Layer (may be negative to count from end). */ diff --git a/README b/README index 9054d12f..03b735bf 100644 --- a/README +++ b/README @@ -300,3 +300,53 @@ Test coverage Open your browser and examine the coverage/ directory. At the time of writing (2020-04) test coverage of the server is reasonable, but things are much worse for certain plugins and filters. + +WINDOWS +======+ +Experimentally, the server can be compiled on Windows or +cross-compiled from Linux using mingw-w64. Currently many features +are DISABLED, including: + +* Daemonization. This is not really applicable for Windows where you + would instead want to run nbdkit as a service using something like + SRVANY. Anyway at the moment you must use the -f option or one of + the other options that implies -f. + +* These options are all unimplemented: + --group, --log=syslog, --pidfile, --run, --selinux-label, --single + --swap, --unix, --user, --vsock + +* Plugins cannot read passwords interactively. + +* The file plugin. The current file plugin is essentially POSIX-only. + We would like to eventually write an alternate file plugin which + uses Windows APIs. + +* Many other plugins and filters. + +* Most tests will fail because of the missing features above. + +For the rest of this section we talk about cross-compiling for Windows +using Linux and mingw-w64. At a minimum you will need: + + mingw-w64 GCC + mingw-w64 dlfcn + mingw-w64 winpthreads + mingw-w64 gnutls (optional, but highly recommended) + mingw-w64 libxml2 (optional, but highly recommended) + +Other mingw-w64 libraries may be installed which will add +functionality (see full list of requirements above), but you may end +up hitting areas we have not compiled or tested before. + +To cross compile do: + + mingw64-configure + make + +The server will be compiled as server/nbdkit.exe. The plugins which +are available will be compiled to .dll files. You can run them +without installing using for example: + + server/nbdkit.exe plugins/memory/.libs/nbdkit-memory-plugin.dll 1G -- 2.27.0
Richard W.M. Jones
2020-Aug-17 09:55 UTC
Re: [Libguestfs] [PATCH EXPERIMENTAL nbdkit 0/2] Port to Windows using mingw.
Because these patches are rapidly changing all the time, I pushed them to a public git repo here: https://github.com/rwmjones/nbdkit/tree/2020-windows-mingw And guess what! In the process of doing that I discovered my previous partially working port to Windows. (I knew I'd done this, but I couldn't find it anywhere until just now.) There's some good stuff in this tree which I will need to integrate with the above: https://github.com/rwmjones/nbdkit/tree/windows For completeness since we're talking about Windows ports of nbdkit, here is Frank Gu's attempt. It's a bit different from the above since it uses MSYS2 (a cut down Cygwin) so it's not a port to the pure Win32 API. I don't think this is the approach that we want to take since it will add extra dependencies and change the license of the resulting binary: https://github.com/gyf304/nbdkit/commits/master Rich. -- Richard Jones, Virtualization Group, Red Hat http://people.redhat.com/~rjones Read my programming and virtualization blog: http://rwmj.wordpress.com virt-top is 'top' for virtual machines. Tiny program with many powerful monitoring features, net stats, disk stats, logging, etc. http://people.redhat.com/~rjones/virt-top
Maybe Matching Threads
- [PATCH nbdkit 0/9] Port to Windows.
- [PATCH nbdkit 0/13] Port to Windows without using a separate library.
- [PATCH nbdkit 0/9] Create libnbdkit.so
- [PATCH nbdkit incomplete 0/5] Port to Windows.
- [nbdkit PATCH v4 0/4] vddk: Drive library loading from libdir parameter.