similar to: Package compression benchmarks for zstd vs gzip

zstd is accessible within R using the archive package [1]. I use it all the time when saving large objects, using code I adapted from [2]. Is your suggestion to import the libraries/source code into base? [1] https://CRAN.R-project.org/package=archive [2] https://coolbutuseless.github.io/2018/10/02/using-lz4-and-zstandard-to-compress-files-with-saverds/ On Fri, Jan 10, 2025 at 6:17?PM Jeroen

I think the first step would have to be to add zstd support to R. zstd is a bit controversial (as shown by the community blowback of the changes you mentioned) and their build system (calling it that is being very generous) is mess so it would require a bit of testing, but it is doable. That said, assuming the above is solved, we have been debating the change of compression at CRAN in general for

With the changes to add zstd support yesterday, the build of R-devel is failing when zstd is not present, even though the docs say that zstd is optional. The error comes in building the datasets package, see e.g. https://github.com/r-devel/r-svn/actions/runs/12760693086/job/35566530112. Best wishes, Heather On Mon, Jan 13, 2025, at 1:26 AM, Simon Urbanek wrote: > I think the first step

Can't speak for Jeroen, but it sounds like it's worth adding support for tar.zstd package files, just like how tar.gz, tar.xz, and tar.bzip2 are currently supported. I'd also argue for support zstd compression throughout R, including adding zstdfile(), support for saveRDS(..., compress = "zstd"), and so on. Then it could be discussed later what the default(s) should be.

On 2020-09-07 11:21:13 [+1000], Darren Tucker wrote: > The zstd part would be a larger discussion because we would need to > either carry it as a Portable patch or have zstd added to OpenBSD > base, and I don't know if that would be accepted. Do you have any > performance numbers for zstd in this application? A key stroke is here 10 bytes of raw data which zstd compresses usually

https://bugzilla.mindrot.org/show_bug.cgi?id=2888 Bug ID: 2888 Summary: Consider adding other compression schemes (lz4, zstd) Product: Portable OpenSSH Version: 7.7p1 Hardware: All OS: All Status: NEW Severity: enhancement Priority: P5 Component: Miscellaneous Assignee:

I added ZSTD support to OpenSSH roughly three years ago and I've been playing with it ever since. The nice part is that ZSTD achieves reasonable compression (like zlib) but consumes little CPU so it is unlikely that compression becomes the bottle neck of a transfer. The compression overhead (CPU) is negligible even when uncompressed data is tunneled over the SSH connection (SOCKS proxy, port

On 9. Sep 2020, at 11.14, Robert Nowotny <rnowotny1966 at gmail.com> wrote: Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: read(zlib(/home/vmail/virtualmailboxes/mpaul/storage/m.119)) failed: read(/home/vmail/virtualmailboxes/mpaul/storage/m.119) failed: Broken pipe (FETCH BODY[]) Also this way you can see if the broken

From: Sebastian Andrzej Siewior <sebastian at breakpoint.cc> The "zstd at breakpoint.cc" compression algorithm enables ZSTD based compression as defined in RFC8478. The compression is delayed until the server sends the SSH_MSG_USERAUTH_SUCCESS which is the same time as with the "zlib at openssh.com" method. Signed-off-by: Sebastian Andrzej Siewior <sebastian at

https://bugzilla.samba.org/show_bug.cgi?id=14338 Bug ID: 14338 Summary: ZSTD support Product: rsync Version: 3.1.3 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wayne at opencoder.net Reporter: Rsync at ml.breakpoint.cc

I hacked zstd support into OpenSSH a while ago and just started to clean it up in the recent days. The cleanup includes configuration support among other things that I did not have. During testing I noticed the following differences compared to zlib: - highly interactive shell output (as in refreshed at a _very_ high rate) may result in higher bandwidth compared to zlib. Since zstd is quicker

I added ZSTD support to OpenSSH roughly over a year and I've been playing with it ever since. The nice part is that ZSTD achieves reasonable compression (like zlib) but consumes little CPU so it is unlikely that compression becomes the bottle neck of a transfer. The compression overhead (CPU) is negligible even when uncompressed data is tunneled over the SSH connection (SOCKS proxy, port

From: Sebastian Andrzej Siewior <sebastian at breakpoint.cc> zstd compression was announced as "good compression with high throughput" so I gave it a try. With zlib, on high speed links the CPU is usually the bottle neck. With zstd I'm able to fill a 200Mbit link :) zstd detection happens automatically via pkg-config. No zstd header means no error about missing zstd. So that

Allow an alternate libc, such as dietlibc, to be used for the init binary in the supermin appliance. Rich.

ok, I did install zstd (zstd command line interface 64-bits v1.4.4, by Yann Collet) reconfigured & compiled dovecot 2.3.10.1 again with option : with-zstd changed the configuration file to : zlib_save = zstd and restarted dovecot - now I have following log file errors : Aug 28 16:17:39 lxc-imap dovecot: imap(rnowotny)<237930><H1kft/CtENRUclcc>: Error: zlib_save: Unknown

v1 -> v2: - If we split out the init program into a separate init/ directory, that makes it much easier to build against an alternate libc. I tried to build against uClibc, but uClibc requires an entire build chain, which looked like it was going to be a massive ballache. Rich.

On 8. Sep 2020, at 12.35, Robert Nowotny <rnowotny1966 at gmail.com> wrote: Dear Aki, I switched to "gz" now, since "zstd" also gave some errors on writing to files. What kind of errors? Probably this? : Panic: file ostream.c: line 287 (o_stream_sendv_int): assertion failed: (!stream->blocking) can be - unfortunately I do not have the log files anymore... Sep

> > On 8. Sep 2020, at 12.35, Robert Nowotny <rnowotny1966 at gmail.com <mailto:rnowotny1966 at gmail.com>> wrote: >> >> >> Dear Aki, >> I switched to "gz" now, since "zstd" also gave some errors on writing to files. > > What kind of errors? Probably this? : Panic: file ostream.c: line 287 (o_stream_sendv_int): assertion

Is there a mechanism by which you can perform a zfs send | zfs receive and not have the data uncompressed and recompressed at the other end? I have a gzip-9 compressed filesystem that I want to backup to a remote system and would prefer not to have to recompress everything again at such great computation expense. If this doesn''t exist, how would one go about creating an RFE for

On 10. Sep 2020, at 14.07, Robert Nowotny <rnowotny1966 at gmail.com> wrote: > > > On 9. Sep 2020, at 11.14, Robert Nowotny <rnowotny1966 at gmail.com <mailto:rnowotny1966 at gmail.com>> wrote: >> >>>> Sep 3 08:33:25 lxc-imap dovecot: imap(mpaul)<48684><2/9E5mKuAezAqKjk>: Error: Mailbox Sent: UID=2171: