similar to: Re: [nbdkit] leak? (#1)

This reverts commit 25206df20275aeff346d9b86adf5e9be99cc9e43. An upcoming patch wants to ensure no leaked fds from the server to a child process. POSIX has required O_CLOEXEC since 2008, and although current POSIX doesn't yet specify full atomic interfaces everywhere such as SOCK_CLOEXEC, it does have an open bug since 2014 [1] recommending the full set of interfaces that will be mandatory

This reverts commit 25206df20275aeff346d9b86adf5e9be99cc9e43, and temporarily breaks compilation on Haiku which has O_CLOEXEC but lacks SOCK_CLOEXEC [1]. An upcoming patch wants to ensure no leaked fds from the server to a child process. POSIX has required O_CLOEXEC since 2008, and that is long enough that we should start blindly relying on it. Meanwhile, POSIX doesn't yet specify full

Dear guangchuang, I was stopped by the following error when using Package: clusterProfiler on R 2.15.3 on windows 7. Error in xzfile(file, “wb”, compression = 9) : Cannot open the connection In addition: warning message: cannot initialize lzma encoder, error 5 The too functions: Gff2GeneTable() and buildGOmap() are not working because of this error. Could you please let me know how to fix these

On Wed, Jul 31, 2019 at 04:31:32PM -0500, Eric Blake wrote: > This reverts commit 25206df20275aeff346d9b86adf5e9be99cc9e43. > > An upcoming patch wants to ensure no leaked fds from the server to a > child process. POSIX has required O_CLOEXEC since 2008, and although > current POSIX doesn't yet specify full atomic interfaces everywhere > such as SOCK_CLOEXEC, it does have

Matt asked if xz should really be a filter rather than a plugin. The answer is yes, of course it should be! That's been something in the todo file for a while. The commit converts the xz plugin code into a filter (leaving the plugin around, but deprecating it). plugin: nbdkit xz file.xz filter: nbdkit --filter=xz file file.xz plugin: # can't be done filter: nbdkit

When extracting an obvious subset of a larger container (ext2, gzip, partition, tar, xz), it's fairly easy to add a nice updated description for what the client is seeing. Not all clients request the description, but if you are worried about this leaking too much information, it can be silenced with nbdkit-exportname-filter. Signed-off-by: Eric Blake <eblake@redhat.com> --- And maybe

On Thu, Aug 27, 2020 at 05:03:45PM -0500, Eric Blake wrote: > When extracting an obvious subset of a larger container (ext2, gzip, > partition, tar, xz), it's fairly easy to add a nice updated > description for what the client is seeing. Not all clients request > the description, but if you are worried about this leaking too much > information, it can be silenced with

There's enough here to need a review; some of it probably needs backporting to stable-1.12. This probably breaks tests on Haiku or other platforms that have not been as on-the-ball about atomic CLOEXEC; feel free to report issues that arise, and I'll help come up with workarounds (even if we end up leaving a rare fd leak on less-capable systems). Meanwhile, I'm still working on my

By default valgrind suppresses many leaks. I'm not even sure exactly how it decides which ones to suppress, but certainly global variables pointing to malloc’d data are suppressed, which is not useful behaviour. Tell valgrind to show all leaks. It won't give an error on them. However to do this we also need a much more comprehensive list of suppressions so that we don't constantly

v2: - Fixes a number of bugs in corner cases. - Uses a 1M block size to fetch from the underlying plugin. This improves performance considerably. I also tested this much more thoroughly and can't find any more bugs. Rich.

May I ask for self-education, how did you conclude memory leak, based on that, I guess, 'ps' output? -- Mykola Ivanets пн, 10 серп. 2020, 10:48 користувач Richard W.M. Jones <rjones@redhat.com> пише: > I'm using nbdkit + the file plugin to serve NBD root filesystems for > some machines, so I get to observe how it behaves for very long runs. > I think this indicates

As you may have seen if you've been following discussions on the mailing list, we discovered a low priority security problem with nbdkit's handling of TLS connections. If TLS is enabled without either client certificate validation or PSK, untrusted clients can connect, negotiate the TLS handshake, disconnect and leak about 14K of memory each time. So after tens of thousands to millions

I'm using nbdkit + the file plugin to serve NBD root filesystems for some machines, so I get to observe how it behaves for very long runs. I think this indicates a memory leak? PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND 5038 root 20 0 4577220 16000 796 S 0.0 0.1 30:52.40 nbdkit 5592 root 20 0 4159224 11708 784 S 0.0 0.1 22:21.67

Introduce a new helper function to resolve a path name, calling nbdkit_error on failure: other than doing what nbdkit_absolute_path does, it also checks that the file exists (and thus avoids errors later on). To help distinguish it from nbdkit_absolute_path, improve the documentation of the latter. Apply it where an existing path is required, both in nbdkit itself and in plugins. Related to:

Introduce a new helper function to resolve a path name, calling nbdkit_error on failure: other than doing what nbdkit_absolute_path does, it also checks that the file exist (and thus avoid errors later on). Apply it where an existing path is required, both in nbdkit itself and in plugins. Related to: https://bugzilla.redhat.com/show_bug.cgi?id=1527334 --- docs/nbdkit-plugin.pod | 13

On Wed, Nov 21, 2018 at 09:59:51AM -0600, Eric Blake wrote: > On 11/21/18 9:46 AM, Richard W.M. Jones wrote: > >Matt asked if xz should really be a filter rather than a plugin. The > >answer is yes, of course it should be! That's been something in the > >todo file for a while. > > > >The commit converts the xz plugin code into a filter (leaving the >

I’m pleased to announce the next stable release of nbdkit. This release concentrates on performance and fuzzing, along with numerous other enhancements (full list below). NBD — Network Block Device — is a protocol for accessing Block Devices (hard disks and disk-like things) over a Network. nbdkit is a toolkit for creating NBD servers. The key features are: * Multithreaded NBD server written

I found a core dump: term1$ nbdkit -fv --filter=xz null xz-max-depth=$((0x4000000000000000)) term1... term2$ qemu-nbd --list term1... nbdkit: debug: null: open readonly=1 nbdkit: error: calloc: Cannot allocate memory nbdkit: debug: xz: close Segmentation fault (core dumped) (Note that the use of --run or daemonizing nbdkit makes it a bit harder to see the core dump, but it is still happening.)

I've just uploaded the development version 1.11.12: http://download.libguestfs.org/nbdkit/1.11-development/ Please try it out because it's hopefully close to the next stable release, 1.12. Below are some preliminary release notes. If there's anything else that I've missed and should go into this version then let me know. Rich.

I’m pleased to announce the next stable release of nbdkit. This release concentrates on numerous feature enhancements - see the release notes below. NBD — Network Block Device — is a protocol for accessing Block Devices (hard disks and disk-like things) over a Network. nbdkit is a toolkit for creating NBD servers. The key features are: * Multithreaded NBD server written in C with good