GNU bug report logs - #79794
how to programmatically decompress a stream of gzipped data

Previous Next

To reply to this bug, email your comments to 79794 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gzip <at> gnu.org
Subject: how to programmatically decompress a stream of gzipped data
Date: Sat, 08 Nov 2025 22:42:32 +0100

Hi,

For various compression formats there is a C library for compressing
and decompressing:
  - libbz2
  - liblz4
  - liblzma
  - libzstd

But there is no 'libgz'. It appears that in order to decompress a stream
of gzipped data:
  1) one needs the 'libz' library from zlib.net,
  2) there are two APIs [1]:
     - gzopen() and gzread() for input from a file descriptor,
     - inflateInit2 (..., 16 + MAX_WBITS) and inflate() for input from a
       region of memory.

It's unclear to me whether it can alternatively be achieved with
libzip from libzip.org.

How about documenting this in the GNU gzip manual? I find it strange
to have to either read the source code of libarchive or some other
package, or ask some LLM about it. gzipped data streams frequently
occur as part of HTTP responses and surely also in many other areas.

The GNU gzip manual references the two RFCs, that define the data
format. But these references don't have the answer either.

Bruno

[1] https://zlib.net/manual.html

     

Message #8 received at 79794 <at> debbugs.gnu.org (full text, mbox):

From: Collin Funk <collin.funk1 <at> gmail.com>
To: Bruno Haible via "GNU gzip discussion and bug reports." <bug-gzip <at> gnu.org>
Cc: 79794 <at> debbugs.gnu.org, Bruno Haible <bruno <at> clisp.org>
Subject: Re: bug#79794: how to programmatically decompress a stream of
 gzipped data
Date: Sat, 08 Nov 2025 14:19:48 -0800

Hi Bruno,

Bruno Haible via "GNU gzip discussion and bug reports."
<bug-gzip <at> gnu.org> writes:

> Hi,
>
> For various compression formats there is a C library for compressing
> and decompressing:
>   - libbz2
>   - liblz4
>   - liblzma
>   - libzstd
>
> But there is no 'libgz'. It appears that in order to decompress a stream
> of gzipped data:
>   1) one needs the 'libz' library from zlib.net,
>   2) there are two APIs [1]:
>      - gzopen() and gzread() for input from a file descriptor,
>      - inflateInit2 (..., 16 + MAX_WBITS) and inflate() for input from a
>        region of memory.
>
> It's unclear to me whether it can alternatively be achieved with
> libzip from libzip.org.
>
> How about documenting this in the GNU gzip manual? I find it strange
> to have to either read the source code of libarchive or some other
> package, or ask some LLM about it. gzipped data streams frequently
> occur as part of HTTP responses and surely also in many other areas.
>
> The GNU gzip manual references the two RFCs, that define the data
> format. But these references don't have the answer either.

Seems reasonable to mention it in the documentation.

zlib is the right one. Some distributions like Fedora default to
zlib-ng, a fork with optimizations at the expense of portability, when
installing it [1]. Not sure if that matters to you.

libarchive is nice and has man pages [2], but based on your message is
probably excessive for your purpose. It would make more sense if you
were writing something like a package manager that supports different
archive formats (e.g. tar, zip, cpio) with multiple compression
algorithms.

I don't know about libzip.

Collin

[1] https://github.com/zlib-ng/zlib-ng
[2] https://github.com/libarchive/libarchive/wiki/ManualPages

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.