GNU bug report logs - #49741
basenc --base64url decoding bug

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 49741 in the body.
You can then email your comments to 49741 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Emil Lundberg <lundberg.emil <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: basenc --base64url decoding bug
Date: Mon, 26 Jul 2021 13:24:01 +0200

[Message part 1 (text/plain, inline)]

Hi! I seem to have encountered a bug in basenc. While decoding a large
base64url-encoded JSON blob, the decoder drops some characters,
rendering the output invalid JSON. I've verified against Python's
built-in base64url decoder, which correctly produces the expected result
while basenc does not.

I've attached the test case, which I've tried to minimize as much as I
can. All my attempts to remove more of the JSON values have made the bug
not trigger, or at least not as easily detectable.

Reproduction instructions:

$ uname -a
Linux HOST 5.13.4-arch1-1 #1 SMP PREEMPT Tue, 20 Jul 2021 16:58:51 +0000
x86_64 GNU/Linux

$ basenc --version
basenc (GNU coreutils) 8.32

$ cat expected-output.txt | sha256sum
fdb9a77c44e9cd612ad3a3cc210e03ea9782e342bb8293b49530e032b2e4ed0e  -

$ cat actual-output.txt | sha256sum
86bce7aa1d0c2da8432cfbb6da4ad2e559012dadbd1abde711e96b2c518d2b11  -

$ basenc -d --base64 input.txt | sha256sum
86bce7aa1d0c2da8432cfbb6da4ad2e559012dadbd1abde711e96b2c518d2b11  -

$ diff actual-output.txt expected-output.txt
160c160
<             "minor: 0
---
>             "minor": 0

Installed from Arch Linux official repos, package version coreutils 8.32-1.

Thanks for making basenc, and please let me know if I can do anything
more to help!

/Emil

[input.txt (text/plain, attachment)]

[expected-output.txt (text/plain, attachment)]

[actual-output.txt (text/plain, attachment)]

Message #8 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Simon Josefsson <simon <at> josefsson.org>
Cc: Emil Lundberg <lundberg.emil <at> gmail.com>, 49741 <at> debbugs.gnu.org
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Thu, 12 Aug 2021 18:37:08 -0700

Simon, this looks like some sort of minor buffering problem in 'basenc 
--base64', since plain 'base64' works correctly. Is this something you 
have time to look into?

https://bugs.gnu.org/49741

Message #11 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 49741 <at> debbugs.gnu.org
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Fri, 13 Aug 2021 10:12:15 -0600

Hi,

I will also work on it this weekend.

 -assaf


On 2021-08-12 7:37 p.m., Paul Eggert wrote:
> Simon, this looks like some sort of minor buffering problem in 'basenc 
> --base64', since plain 'base64' works correctly. Is this something you 
> have time to look into?
> 
> https://bugs.gnu.org/49741
> 
> 
> 

Message #14 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Emil Lundberg <lundberg.emil <at> gmail.com>
Cc: 49741 <at> debbugs.gnu.org
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Mon, 16 Aug 2021 15:17:14 -0600

[Message part 1 (text/plain, inline)]

Hello Emil and all,

Thanks for the clear and easily reproducible bug report.

Attached a suggested fix.
Comments very welcomed,

- Assaf

[0001-basenc-fix-bug49741-using-wrong-decoding-buffer-leng.patch (text/x-patch, attachment)]

Message #17 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Assaf Gordon <assafgordon <at> gmail.com>, Paul Eggert <eggert <at> cs.ucla.edu>,
 Emil Lundberg <lundberg.emil <at> gmail.com>
Cc: 49741 <at> debbugs.gnu.org
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Tue, 17 Aug 2021 01:01:01 +0100

On 16/08/2021 22:17, Assaf Gordon wrote:
> Hello Emil and all,
> 
> Thanks for the clear and easily reproducible bug report.
> 
> Attached a suggested fix.
> Comments very welcomed,

minor nit in NEWS:
s/silently discard/silently discards/

Otherwise it looks good.

thanks!
Pádraig

Message #20 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Emil Lundberg <lundberg.emil <at> gmail.com>, 49741 <at> debbugs.gnu.org,
 Assaf Gordon <assafgordon <at> gmail.com>, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Tue, 17 Aug 2021 11:37:41 +0200

On Tue, Aug 17, 2021 at 2:02 AM Pádraig Brady <P <at> draigbrady.com> wrote:
> On 16/08/2021 22:17, Assaf Gordon wrote:
> > Hello Emil and all,
> >
> > Thanks for the clear and easily reproducible bug report.
> >
> > Attached a suggested fix.
> > Comments very welcomed,
>
> minor nit in NEWS:
> s/silently discard/silently discards/
>
> Otherwise it looks good!

Nice, indeed!
a nit in the commit log:

-The input buffer was not divisible by 3
+The input buffer length was not divisible by 3

Message #23 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>, Pádraig Brady
 <P <at> draigbrady.com>
Cc: Emil Lundberg <lundberg.emil <at> gmail.com>, 49741 <at> debbugs.gnu.org,
 Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Sun, 22 Aug 2021 16:15:00 -0600

[Message part 1 (text/plain, inline)]

On 2021-08-17 3:37 a.m., Jim Meyering wrote:
> On Tue, Aug 17, 2021 at 2:02 AM Pádraig Brady <P <at> draigbrady.com> wrote:
>> On 16/08/2021 22:17, Assaf Gordon wrote:
>>>
>>> Attached a suggested fix.
>>
>> minor nit in NEWS:
> a nit in the commit log:

Thanks, attached updated patch.
Will push this week if there are no other comments.

-assaf

[0001-basenc-fix-bug49741-using-wrong-decoding-buffer-leng.patch (text/x-patch, attachment)]

Message #26 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Assaf Gordon <assafgordon <at> gmail.com>
To: Jim Meyering <jim <at> meyering.net>, Pádraig Brady
 <P <at> draigbrady.com>
Cc: Emil Lundberg <lundberg.emil <at> gmail.com>, 49741 <at> debbugs.gnu.org,
 Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Sun, 29 Aug 2021 22:14:05 -0600

tag 49741 fixed
close 49741
stop

On 2021-08-22 4:15 p.m., Assaf Gordon wrote:
>>>> Attached a suggested fix.

pushed in:

https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=709d1f8253072804cc27189a6f2b873d8d563399

Message #33 received at 49741 <at> debbugs.gnu.org (full text, mbox):

From: Emil Lundberg <lundberg.emil <at> gmail.com>
To: Assaf Gordon <assafgordon <at> gmail.com>, Jim Meyering <jim <at> meyering.net>,
 Pádraig Brady <P <at> draigbrady.com>
Cc: 49741 <at> debbugs.gnu.org, Paul Eggert <eggert <at> cs.ucla.edu>
Subject: Re: bug#49741: basenc --base64url decoding bug
Date: Fri, 3 Sep 2021 23:41:38 +0200

Thanks a lot everyone!

/Emil

On 2021-08-30 06:14, Assaf Gordon wrote:
> tag 49741 fixed
> close 49741
> stop
>
> On 2021-08-22 4:15 p.m., Assaf Gordon wrote:
>>>>> Attached a suggested fix.
>
> pushed in:
>
> https://git.savannah.gnu.org/cgit/coreutils.git/commit/?id=709d1f8253072804cc27189a6f2b873d8d563399
>
>

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.