search for: zstd_compressstream2

https://bugzilla.samba.org/show_bug.cgi?id=14338 Bug ID: 14338 Summary: ZSTD support Product: rsync Version: 3.1.3 Hardware: All OS: All Status: NEW Severity: normal Priority: P5 Component: core Assignee: wayne at opencoder.net Reporter: Rsync at ml.breakpoint.cc

...+ ssh->state->compress_zstd_out_raw += in_buff.size; + out_buff.dst = buf; + out_buff.size = sizeof(buf); + + /* + * Consume input and immediatelly flush compressed data. It will loop + * multiple times if the output does not fit into the buffer + */ + do { + out_buff.pos = 0; + + comp = ZSTD_compressStream2(ssh->state->compression_zstd_out_stream, + &out_buff, &in_buff, ZSTD_e_flush); + if (ZSTD_isError(comp)) + return SSH_ERR_ALLOC_FAIL; + /* Append compressed data to output_buffer. */ + r = sshbuf_put(out, buf, out_buff.pos); + if (r != 0) + return r; + ssh->state->...

...n_buff.src = map_ptr(buf, offset, nb); + zstd_in_buff.size = nb; + zstd_in_buff.pos = 0; + + do { + if (zstd_out_buff.size == 0) { + zstd_out_buff.size = MAX_DATA_COUNT; + zstd_out_buff.pos = 0; + } + + /* File ended, flush */ + if (token != -2) + flush = ZSTD_e_flush; + + r = ZSTD_compressStream2(zstd_cctx, &zstd_out_buff, &zstd_in_buff, flush); + if (ZSTD_isError(r)) { + rprintf(FERROR, "ZSTD_compressStream returned %d\n", r); + exit_cleanup(RERR_STREAMIO); + } + + /* + * Nothing is sent if the buffer isn't full so avoid smaller + * transfers. If a f...

I added ZSTD support to OpenSSH roughly three years ago and I've been playing with it ever since. The nice part is that ZSTD achieves reasonable compression (like zlib) but consumes little CPU so it is unlikely that compression becomes the bottle neck of a transfer. The compression overhead (CPU) is negligible even when uncompressed data is tunneled over the SSH connection (SOCKS proxy, port

I hacked zstd support into OpenSSH a while ago and just started to clean it up in the recent days. The cleanup includes configuration support among other things that I did not have. During testing I noticed the following differences compared to zlib: - highly interactive shell output (as in refreshed at a _very_ high rate) may result in higher bandwidth compared to zlib. Since zstd is quicker

I added ZSTD support to OpenSSH roughly over a year and I've been playing with it ever since. The nice part is that ZSTD achieves reasonable compression (like zlib) but consumes little CPU so it is unlikely that compression becomes the bottle neck of a transfer. The compression overhead (CPU) is negligible even when uncompressed data is tunneled over the SSH connection (SOCKS proxy, port