GNU bug report logs - #56247
inflate fails to reject invalid distance

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 56247 in the body.
You can then email your comments to 56247 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Young Mo Kang <kym327 <at> gmail.com>
To: bug-gzip <at> gnu.org
Subject: inflate fails to reject invalid distance
Date: Sun, 26 Jun 2022 20:21:09 -0700

Hello gzip devs and maintainers,

I noticed that GNU gzip does not check for error during decompression when the distance value exceeds the current accumulative position, i.e., back reference to position beyond the starting point.

The following shows an example .gz file with invalid distance

```
cat << EOF > file.gz.hex
1f8b0800000000000003738aca3034748c540b8ebc297129888141e1ffff
1f0c131842041818172b5c6169686068706438c0aad0c0c8c0e0c0c0d8e0
e8c0b1802124d94d81218571e1ab8c6f2b4c4dd9579830c82d7cc5c0ae72
e18bffefcab6244e0e4536861486f5e612abda4c336cf6dc5d5da4bd6c4d
ed49a13bffe37813164d39d8aafdb15c25b895937762afaa02c205deff85
c2181818a3241f30303031303231340830b0303032702e60306476b82035
6ff92b379ff625574ca6594d5a9cf7464ebf7ba7d21c0d0fd5ff9dda107f
000040136103cd000000
EOF

xxd -r -ps file.gz.hex | gzip -d > file
```

GNU gzip decompresses it w/o an error, while both macOS’s gzip and zlib reject the file and outputs an error:

macOS gzip: gzip: data stream error
zlib: <fd:0>: invalid distance too far back

I believe GNU gzip also needs to reject this file, since the file is not a valid deflate format.

Thanks,
Young

Message #8 received at 56247 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Young Mo Kang <kym327 <at> gmail.com>
Cc: 56247 <at> debbugs.gnu.org, Mark Adler <madler <at> alumni.caltech.edu>
Subject: Re: bug#56247: inflate fails to reject invalid distance
Date: Mon, 27 Jun 2022 14:10:11 -0500

[Message part 1 (text/plain, inline)]

On 6/26/22 22:21, Young Mo Kang wrote:
> I believe GNU gzip also needs to reject this file, since the file is not a valid deflate format.

gzip is compatible with pigz here. It's not clear to me that gzip should 
be pedantic and reject input that does not strictly conform to RFC 1952.

I'll cc this to Mark Adler in hopes that he has an opinion. Mark, if I 
understand things correctly, the complaint is that the attached 
"compressed" file does not conform to RFC 1952, but gzip and pigz do not 
complain about it. You can see the original gzip bug report here:

https://bugs.gnu.org/56247

[bad-gzip-data (application/octet-stream, attachment)]

Message #11 received at 56247 <at> debbugs.gnu.org (full text, mbox):

From: "Adler, Mark" <madler <at> alumni.caltech.edu>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: "56247 <at> debbugs.gnu.org" <56247 <at> debbugs.gnu.org>,
 Young Mo Kang <kym327 <at> gmail.com>
Subject: Re: bug#56247: inflate fails to reject invalid distance
Date: Mon, 27 Jun 2022 21:29:37 +0000

[Message part 1 (text/plain, inline)]

Paul,

gzip should reject invalid inflate input. The patch below does that.

The fact that pigz also doesn’t reject it is an independent bug in zlib’s uncommonly-used inflateBack functions, fixed with this commit: https://github.com/madler/zlib/commit/2333419cd76cb9ae5f15c9b240b16a2052b27691

Mark


On Jun 27, 2022, at 12:10 PM, Paul Eggert <eggert <at> cs.ucla.edu<mailto:eggert <at> cs.ucla.edu>> wrote:

On 6/26/22 22:21, Young Mo Kang wrote:
I believe GNU gzip also needs to reject this file, since the file is not a valid deflate format.

gzip is compatible with pigz here. It's not clear to me that gzip should be pedantic and reject input that does not strictly conform to RFC 1952.

I'll cc this to Mark Adler in hopes that he has an opinion. Mark, if I understand things correctly, the complaint is that the attached "compressed" file does not conform to RFC 1952, but gzip and pigz do not complain about it. You can see the original gzip bug report here:

https://bugs.gnu.org/56247
<bad-gzip-data>


--- inflate-orig.c 2022-01-03 10:16:30.000000000 -0800
+++ inflate.c 2022-06-27 14:01:42.000000000 -0700
@@ -153,8 +153,9 @@
    "uch *slide;" and then malloc'ed in the latter case.  The definition
    must be in unzip.h, included above. */
 /* unsigned wp;             current position in slide */
+int fresh;
 #define wp outcnt
-#define flush_output(w) (wp=(w),flush_window())
+#define flush_output(w) (fresh=0,wp=(w),flush_window())



 /* Tables for deflate from PKZIP's appnote.txt. */
 static unsigned border[] = {    /* Order of the bit length code lengths */
@@ -572,6 +573,8 @@
       NEEDBITS(e)
       d = w - t->v.n - ((unsigned)b & mask_bits[e]);
       DUMPBITS(e)
+      if (fresh && d >= w)
+        return 1;
       Tracevv((stderr,"\\[%d,%d]", w-d, n));



       /* do the copy */
@@ -954,6 +957,7 @@
   wp = 0;
   bk = 0;
   bb = 0;
+  fresh = 1;





   /* decompress until the last block */

[Message part 2 (text/html, inline)]

Message #16 received at 56247-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: "Adler, Mark" <madler <at> alumni.caltech.edu>
Cc: Young Mo Kang <kym327 <at> gmail.com>,
 GNU bug 56247 <56247-done <at> debbugs.gnu.org>
Subject: Re: bug#56247: inflate fails to reject invalid distance
Date: Tue, 28 Jun 2022 22:44:57 -0500

[Message part 1 (text/plain, inline)]

On 6/27/22 16:29, Adler, Mark wrote:
> Paul,
>
> gzip should reject invalid inflate input. The patch below does that.
>
Thanks for the quick patch. I installed that, along with Young Mo Kang's 
test case, as per the attached.

[0001-gzip-detect-invalid-input.patch (text/x-patch, attachment)]

[0002-gzip-test-invalid-input-bug.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.