search for: 9213826

...e nature of the gzip format which is not blocked based > and thus not seekable, this filter caches the whole uncompressed file > in a hidden temporary file. This is required in order to implement > .get_size. See this link for a more detailed explanation: > https://stackoverflow.com/a/9213826 > > This commit deprecates nbdkit-gzip-plugin and suggests removal in > nbdkit 1.26. > --- Nice - this one seems like a fairly straight-forward conversion. > +++ b/filters/tar/nbdkit-tar-filter.pod > @@ -42,11 +42,13 @@ server use: > nbdkit -r curl https://example.com/fi...

...sk.gz Because of the nature of the gzip format which is not blocked based and thus not seekable, this filter caches the whole uncompressed file in a hidden temporary file. This is required in order to implement .get_size. See this link for a more detailed explanation: https://stackoverflow.com/a/9213826 This commit deprecates nbdkit-gzip-plugin and suggests removal in nbdkit 1.26. --- filters/gzip/nbdkit-gzip-filter.pod | 85 +++++++ filters/tar/nbdkit-tar-filter.pod | 7 +- plugins/gzip/nbdkit-gzip-plugin.pod | 9 + configure.ac | 10 +- filters/gzip/Makefile.am...

...D client could query the list of files [ie exports] in the tarball and choose one to download. (3) gzip & tar require full downloads - why not “docker/podman save/export”? Stepping back to get the bigger picture: Because the OCI standard uses gzip for compression (https://stackoverflow.com/a/9213826), and because the tar index is interspersed with the tar data, you always need to download the whole container layer before you can access the disk image inside. Currently nbdkit-gzip-filter hides this from the end user, but it's still downloading the whole thing to a temporary file. There...

...hing on the host by docker/podman, so we > have to pull the image from the registry only once on every host. Then we can > access the local cache. > > > Stepping back to get the bigger picture: Because the OCI standard uses > > gzip for compression (https://stackoverflow.com/a/9213826), and > > because the tar index is interspersed with the tar data, you always > > need to download the whole container layer before you can access the > > disk image inside. > > You need to download most of the tar, but you don't need to keep the tar > in a temporary...

...er, we have automatic caching on the host by docker/podman, so we have to pull the image from the registry only once on every host. Then we can access the local cache. > Stepping back to get the bigger picture: Because the OCI standard uses > gzip for compression (https://stackoverflow.com/a/9213826), and > because the tar index is interspersed with the tar data, you always > need to download the whole container layer before you can access the > disk image inside. You need to download most of the tar, but you don't need to keep the tar in a temporary file. For example in python y...