GNU logs - #53400, boring messages


Message sent to bug-coreutils@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#53400: base64: decode Base64URL
Resent-From: Sergey Ponomarev <stokito@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Thu, 20 Jan 2022 21:33:01 +0000
Resent-Message-ID: <handler.53400.B.16427143593682 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 53400
X-GNU-PR-Package: coreutils
X-GNU-PR-Keywords: 
To: 53400 <at> debbugs.gnu.org
X-Debbugs-Original-To: submit <at> debbugs.gnu.org
Received: via spool by submit <at> debbugs.gnu.org id=B.16427143593682
          (code B ref -1); Thu, 20 Jan 2022 21:33:01 +0000
Received: (at submit) by debbugs.gnu.org; 20 Jan 2022 21:32:39 +0000
Received: from localhost ([127.0.0.1]:59342 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nAf2s-0000xK-Rz
	for submit <at> debbugs.gnu.org; Thu, 20 Jan 2022 16:32:39 -0500
Received: from mail-yb1-f173.google.com ([209.85.219.173]:44567)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stokito@HIDDEN>) id 1nAf2r-0000x5-5W
 for submit <at> debbugs.gnu.org; Thu, 20 Jan 2022 16:32:37 -0500
Received: by mail-yb1-f173.google.com with SMTP id r65so18131508ybc.11
 for <submit <at> debbugs.gnu.org>; Thu, 20 Jan 2022 13:32:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:from:date:message-id:subject:to;
 bh=LN22aUgDgoPJYctTayiGgAEqKqm6U+wmIigMtwVcaf0=;
 b=Mrt3/JP5v5wvoHiy1SBOjc9wfdFnLj5ha1EFX4PPJM+dhOxZwNQCjRnW3gbTdaS+wG
 VrlyjQtZadHHm/y9Pv8nCCE40Lc/cmO38pKu1xYqXqNTVvyAkw40IJ8+FlIVh3reEP0d
 qpUgJ8Uf+AivzJ0tfOibHTClFD6ej/YZociJPzs1n8cxvB8WFBbB0b36iLiKp1kdcASs
 38Zy30YYQ7KjIjzTSZ5AhXAsF2EyBu0nm68cM+TL/TYQuKmbojURK/wSapaoQ07oJjG4
 FqGNYZWmJoN5eZ5TC1VMSclyHaRiNYZOxWWn5UMdXmIsBwAi76iHuerlxXXV5kVkzT/r
 9X7A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=LN22aUgDgoPJYctTayiGgAEqKqm6U+wmIigMtwVcaf0=;
 b=dS3ztzY3tRt15RckSonKGk05jxEk/oYfKLuuj4xPWQ0QSYOPL400mnGLBkTv8EMrFj
 zyjS5Tkp74SNV7GCY91SKcaioMV4OJ8HPJmf6FiEMU7Iqxy+eQ6DD6nKol+czVio35Zt
 5KWGCs6dUjFINJE0okVBnbB+10piNctXrsi9o3d/SoXNOgiQVAh70mb/Au5Lrzd83t3H
 hlfIrCa/RISluNdSV9Vbu1u40D5+AzF1BzLK6PZtUC/cUNNWa+bak3mhpGTHVAJ7fDI8
 1wSMRlkj93lktMw8cycUHlDzUMXWedpS8K0QdEvern/8KA/cqLu5vUJqK3E9jtsgmDM7
 L/oA==
X-Gm-Message-State: AOAM532vlGJ4cN5K7ono/2povX/tVItJlTikMKixK9MwB6uJ7LUwSxgH
 18dXEUMIW3PXkfnnuG7OR66B4MHl7KtEorORn69bzHqie90=
X-Google-Smtp-Source: ABdhPJyvKsvRGTGSGe1HoSKDeDx9q098z3yvY1MP51koxtjHsfT7z04ug72gGueb0WwC+JA3ilPzm2K4WGYp77lB1rI=
X-Received: by 2002:a25:6f82:: with SMTP id k124mr1808435ybc.80.1642714351049; 
 Thu, 20 Jan 2022 13:32:31 -0800 (PST)
MIME-Version: 1.0
From: Sergey Ponomarev <stokito@HIDDEN>
Date: Thu, 20 Jan 2022 23:31:55 +0200
Message-ID: <CADR0UcVx6EbHpehk1RxZu5gB8KCYXm7peK0B5GU24VATdrHzGA@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Package: coreutils

Currently the base64 util can't  parse base64 url safe.
RFC 3548 Base 64 Encoding with URL and Filename Safe Alphabet
https://datatracker.ietf.org/doc/html/rfc3548#page-6

But basically this is the same encoding: just two symbols differ and
no padding. So it's easy and safe just to parse both forms.
The coreutils have a separate command `basenc --base64url` to parse
the b64u but actually there is no any sense to make it as a separate
command.
Having this as a separate command makes it unusable for small Linux
distros like OpenWrt where the device's disk is very small.

Currently **many** shell scripts uses additional transformation like:

```sh
base64_padding()
{
  local len=$(( ${#1} % 4 ))
  local padded_b64=''
  if [ ${len} = 2 ]; then
    padded_b64="${1}=="
  elif [ ${len} = 3 ]; then
    padded_b64="${1}="
  else
    padded_b64="${1}"
  fi
  echo -n "$padded_b64"
}

base64url_to_b64()
{
  base64_padding "${1}" | tr -- '-_' '+/'
}
```

But this over complicates everything, reduces performance and is error prone.

I created a similar ticked for OpenSSL base64
https://github.com/openssl/openssl/issues/17559


Also as an additional request: can we add the new param `-D` in upper
case which will be an alias to `-d`.
The issue is that on MacOS the small -d was used for debug and they
used the -D in upper case. This creates a lot of problems for scripts
that should work on both Linux and MacOS. So most people use the long
--decode. But many users just don't know about this interop problem
and once I faced a problem when my college gave me a script that
didn't work for me.
This will prevent unexpected compatibility problems.

Regards,
Sergey Ponomarev




Message sent:


Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Sergey Ponomarev <stokito@HIDDEN>
Subject: bug#53400: Acknowledgement (base64: decode Base64URL)
Message-ID: <handler.53400.B.16427143593682.ack <at> debbugs.gnu.org>
References: <CADR0UcVx6EbHpehk1RxZu5gB8KCYXm7peK0B5GU24VATdrHzGA@HIDDEN>
X-Gnu-PR-Message: ack 53400
X-Gnu-PR-Package: coreutils
Reply-To: 53400 <at> debbugs.gnu.org
Date: Thu, 20 Jan 2022 21:33:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-coreutils@HIDDEN

If you wish to submit further information on this problem, please
send it to 53400 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
53400: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D53400
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems


Message sent to bug-coreutils@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#53400: base64: decode Base64URL
Resent-From: =?UTF-8?Q?P=C3=A1draig?= Brady <P@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Fri, 21 Jan 2022 15:20:02 +0000
Resent-Message-ID: <handler.53400.B53400.16427783643592 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 53400
X-GNU-PR-Package: coreutils
X-GNU-PR-Keywords: 
To: Sergey Ponomarev <stokito@HIDDEN>, 53400 <at> debbugs.gnu.org
Received: via spool by 53400-submit <at> debbugs.gnu.org id=B53400.16427783643592
          (code B ref 53400); Fri, 21 Jan 2022 15:20:02 +0000
Received: (at 53400) by debbugs.gnu.org; 21 Jan 2022 15:19:24 +0000
Received: from localhost ([127.0.0.1]:34903 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nAvhD-0000vs-K3
	for submit <at> debbugs.gnu.org; Fri, 21 Jan 2022 10:19:23 -0500
Received: from mail-wr1-f45.google.com ([209.85.221.45]:33291)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pixelbeat@HIDDEN>) id 1nAvhA-0000vd-FL
 for 53400 <at> debbugs.gnu.org; Fri, 21 Jan 2022 10:19:22 -0500
Received: by mail-wr1-f45.google.com with SMTP id d19so270512wrb.0
 for <53400 <at> debbugs.gnu.org>; Fri, 21 Jan 2022 07:19:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=sender:message-id:date:mime-version:user-agent:subject
 :content-language:to:references:from:in-reply-to
 :content-transfer-encoding;
 bh=jNdEdYZQ+aHg9Sb8PqPljdvDr10NLA/XHRa4m86dUpE=;
 b=R3K9IWyEjdQl+UJJKjS2towRi0I/fevsU6gpE3HJuxuCZBRSOWywyDfwrOm9lW2/Vc
 FB78VHmWbZm986VoIj6+HB0bXsmchNOC/25KG7M6OQkpnes+6ngZq5jFzuZR6a/Vcg3T
 fyBKGk/VsqmE2YXjxtGWeUDL2YEIC2U+p976m/KBhFZ4GqnMu7Hf43VkYNClr2uaUPdq
 WyXcSJ6/50wGZ56GakV9f33EaoBiSI0L8swnrHgqKBqrVKAOdrvZ00EUU+SRJ3kyuCm8
 RDwwWitLkMLoehnO1lKH8pws3TpoIy8oNIEeqvrDyY+/GrCZD2lDEGhG2RCKad19v1B2
 G4ww==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:sender:message-id:date:mime-version:user-agent
 :subject:content-language:to:references:from:in-reply-to
 :content-transfer-encoding;
 bh=jNdEdYZQ+aHg9Sb8PqPljdvDr10NLA/XHRa4m86dUpE=;
 b=Virj2W81DxmKbIsJ/6TqcBiIZzSTnuWs6UWvwWiiYATukWVa8FtRIfeQiTfiBuaK3R
 v4C23fHAMDp8nldCNSYuD2FZydhkA2q/HatNkysw3nATcZtkjr578qYD3eB4LK6DPpSm
 yt27eSXfR1s4GZzqASdXNPkjqG/XYnD8yNcGhwTRFGHii7Jo6+UmKH+96Ifu4Llvi/oS
 DM/tG37V2rl9hRw5JmHVhuy70hd3JWoUFO8g55cpYgs7ux9JuAnz0DeELVhhdfMAsbxO
 V2OzLleRVlDtAhmlLKR6q6+3EiMzws0YELNVFXkuPVBhBD6lIOJwXahsAHHMV0FUAomJ
 lVaA==
X-Gm-Message-State: AOAM532I1q1A32EFKyT05WPnd14gMLNUP7q5Z9FETZT4em3e5t5UdBLs
 ARCKK7L1OiGSXZ3Dw0hxkYw=
X-Google-Smtp-Source: ABdhPJxRcyFAtKsO2m0DHOPsDo9LU8+XOp88hPvqW2b2ayObY73B2RjFTTRjPuERhwRlzrMC4QaHFQ==
X-Received: by 2002:a5d:53d1:: with SMTP id a17mr4144784wrw.209.1642778354363; 
 Fri, 21 Jan 2022 07:19:14 -0800 (PST)
Received: from [192.168.1.9]
 (86-40-129-104-dynamic.agg2.lod.rsl-rtd.eircom.net. [86.40.129.104])
 by smtp.googlemail.com with ESMTPSA id r15sm5721490wmq.3.2022.01.21.07.19.13
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Fri, 21 Jan 2022 07:19:13 -0800 (PST)
Message-ID: <3cc3ba54-d491-7a7c-ee5e-38251080dfcd@HIDDEN>
Date: Fri, 21 Jan 2022 15:18:13 +0000
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:95.0) Gecko/20100101
 Thunderbird/95.0
Content-Language: en-US
References: <CADR0UcVx6EbHpehk1RxZu5gB8KCYXm7peK0B5GU24VATdrHzGA@HIDDEN>
From: =?UTF-8?Q?P=C3=A1draig?= Brady <P@HIDDEN>
In-Reply-To: <CADR0UcVx6EbHpehk1RxZu5gB8KCYXm7peK0B5GU24VATdrHzGA@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.6 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.6 (-)

On 20/01/2022 21:31, Sergey Ponomarev wrote:
> Package: coreutils
> 
> Currently the base64 util can't  parse base64 url safe.
> RFC 3548 Base 64 Encoding with URL and Filename Safe Alphabet
> https://datatracker.ietf.org/doc/html/rfc3548#page-6
> 
> But basically this is the same encoding: just two symbols differ and
> no padding. So it's easy and safe just to parse both forms.
> The coreutils have a separate command `basenc --base64url` to parse
> the b64u but actually there is no any sense to make it as a separate
> command.
> Having this as a separate command makes it unusable for small Linux
> distros like OpenWrt where the device's disk is very small.

Note coreutils can be built like busybox to include multiple commands in a single binary

> 
> Currently **many** shell scripts uses additional transformation like:
> 
> ```sh
> base64_padding()
> {
>    local len=$(( ${#1} % 4 ))
>    local padded_b64=''
>    if [ ${len} = 2 ]; then
>      padded_b64="${1}=="
>    elif [ ${len} = 3 ]; then
>      padded_b64="${1}="
>    else
>      padded_b64="${1}"
>    fi
>    echo -n "$padded_b64"
> }
> 
> base64url_to_b64()
> {
>    base64_padding "${1}" | tr -- '-_' '+/'
> }
> ```

I'm not sure about base64 accepting _either_ encoding without options.
There would be robustness issues with that.

Also blinding assuming padding may introduce robustness issues with truncated data.

It might be useful to have a "--decode-relaxed" option or something
to support decoding any b64 variant and assume no padding,
though preprocessing isn't that onerous, so this is marginal.

> But this over complicates everything, reduces performance and is error prone.
> 
> I created a similar ticked for OpenSSL base64
> https://github.com/openssl/openssl/issues/17559
> 
> 
> Also as an additional request: can we add the new param `-D` in upper
> case which will be an alias to `-d`.
> The issue is that on MacOS the small -d was used for debug and they
> used the -D in upper case. This creates a lot of problems for scripts
> that should work on both Linux and MacOS. So most people use the long
> --decode. But many users just don't know about this interop problem
> and once I faced a problem when my college gave me a script that
> didn't work for me.
> This will prevent unexpected compatibility problems.

macos using -D is unfortunate.
It's surprising they diverged from FreeBSD here.
The fact there is a cross platform --decode option,
suggests we don't need to add -D support,
as it doesn't help folks using -d on other systems.

cheers,
Pádraig




Message sent to bug-coreutils@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#53400: base64: decode Base64URL
Resent-From: Sergey Ponomarev <stokito@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Fri, 21 Jan 2022 16:02:02 +0000
Resent-Message-ID: <handler.53400.B53400.16427809018043 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 53400
X-GNU-PR-Package: coreutils
X-GNU-PR-Keywords: 
To: =?UTF-8?Q?P=C3=A1draig?= Brady <P@HIDDEN>
Cc: 53400 <at> debbugs.gnu.org
Received: via spool by 53400-submit <at> debbugs.gnu.org id=B53400.16427809018043
          (code B ref 53400); Fri, 21 Jan 2022 16:02:02 +0000
Received: (at 53400) by debbugs.gnu.org; 21 Jan 2022 16:01:41 +0000
Received: from localhost ([127.0.0.1]:34928 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nAwM8-00025f-UL
	for submit <at> debbugs.gnu.org; Fri, 21 Jan 2022 11:01:41 -0500
Received: from mail-yb1-f169.google.com ([209.85.219.169]:45900)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stokito@HIDDEN>) id 1nAwM5-00025O-Pt
 for 53400 <at> debbugs.gnu.org; Fri, 21 Jan 2022 11:01:38 -0500
Received: by mail-yb1-f169.google.com with SMTP id h14so28777376ybe.12
 for <53400 <at> debbugs.gnu.org>; Fri, 21 Jan 2022 08:01:37 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc:content-transfer-encoding;
 bh=tpZ3PbsSBpQHeee8eciTNYHiN06ggdyYRkqeG9rILlg=;
 b=B9MhPFxH4Rw7v2eCLzmWx2nDDtdqhCwsEqqauBtj7HGiVz3/uRVIgBCFR5umJzj+vt
 EeigsSQZVxzW5GrsZgeKq+Bm8+0qAWcGCEO5vku/GHnM4Waoz9K7S7FkeTrlIMSgOxOT
 QtFS/imDbjGmq5CT04sU5/UUSOvq5x6ogKV2Z26QWQWQsr0+InZUfF9Fpm3R6VoS2F+e
 206OiPaip8posy4zaS+i8rtL6DUg62Xa3z1+UlM+QFqmPn6lEIE1+8V7EYLWgA92lt6f
 KiFRdC+BJufeo3tvzpayf2Yz8kCgImpY5k7YZIhAe6qiex2Cwsi0MhyAv+zSfiNd3BSW
 tdmg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc:content-transfer-encoding;
 bh=tpZ3PbsSBpQHeee8eciTNYHiN06ggdyYRkqeG9rILlg=;
 b=0GVI5STpg4KEsekKOmMguN/bLarxeKolqYYi7CeAfES+pQbNFKmBEvyfaAQTvYzBxW
 D0comdNAre9VhFa5Dl723Jb2uibIllxz7nYpjWvovSLSHQ4IYmv1Dem/yEifsYgE5uoM
 /5MuhHl9AWBCAAQ0yjGAca9o3K8/gchp8MlSmOjYRHn0HfA6FTk7s3vp76zYCO7eBdC2
 UuvmdN6hm6rcd+U23IE5NcKzf6VMEmrBBeFdboPWVvAgWHq664Ks2L26A/xAX9rL0Wx+
 HI20AOubxBnvdMzAILQWKKZ+yofqn6rHwZclYSnlb6WZhNmAbghFFMHqUlKV/FPWECxd
 GNqw==
X-Gm-Message-State: AOAM530YYjobvRTdnC607nVS5p4bIlynbNGFjT9upuA6NQj4KC8Mojjj
 ZMsg5Tw0aGHSbNfWup4hKQEJnEajSsEYqdsZyA/gfVksKZU=
X-Google-Smtp-Source: ABdhPJwkMTv9xi56dKtsDxnobY1rCSOh3GOn1qfhoKi2tyFcG5a8D7H4w9CFuv6JXM1useFIaNZE5JNsrCSTylHteio=
X-Received: by 2002:a25:2649:: with SMTP id m70mr6857811ybm.86.1642780891648; 
 Fri, 21 Jan 2022 08:01:31 -0800 (PST)
MIME-Version: 1.0
References: <CADR0UcVx6EbHpehk1RxZu5gB8KCYXm7peK0B5GU24VATdrHzGA@HIDDEN>
 <3cc3ba54-d491-7a7c-ee5e-38251080dfcd@HIDDEN>
In-Reply-To: <3cc3ba54-d491-7a7c-ee5e-38251080dfcd@HIDDEN>
From: Sergey Ponomarev <stokito@HIDDEN>
Date: Fri, 21 Jan 2022 18:00:55 +0200
Message-ID: <CADR0UcWOgb6+4yy=vN1drHNt8YphBaHptZpUQFzDHNJnZvhAbA@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Thank you, P=C3=A1draig,

Your are absolutely right that for a better correctness it must be a
way to catch padding errors. And I just decided to check how it
behaves currently:

$ echo "Aladin" | base64
QWxhZGluCg=3D=3D

Now let's remove the padding:
$ echo "QWxhZGluCg" | base64 -d
Aladin
base64: invalid input
$ echo $?
1

So base64 decoded as much as it can but wrote a message into stderr
and returned an error code.
I think this is a good compromise: for those who are really want to
know about the padding they can check an error code.
We may add an additional error code to show that the base64 was
decoded as an Base64URL.

Also we may introduce an additional flag `-u` to say that we do expect
a Base64URL input and don't set the error code. The same flag may be
also used for base64 encoding.

Speaking about an additional flag the same idea was proposed in the
ticket for OpenSSL and I answered:

> Having it as a separate flag is a good thing to do for a better correctne=
ss. Still many unexpirienced users will fail just because they copied a fir=
st answer on SO. This makes the feature much less useful. Especially the bi=
g problem is that some base64 data may be decoded successfully during a dev=
elopment and testing and the code with bugs may be delivered to end users. =
Again, this is a very widely used tool.

So maybe the idea with decode both but set the exit code + additional
flag will be sufficient?

On Fri, 21 Jan 2022 at 17:19, P=C3=A1draig Brady <P@HIDDEN> wrote:
>
> On 20/01/2022 21:31, Sergey Ponomarev wrote:
> > Package: coreutils
> >
> > Currently the base64 util can't  parse base64 url safe.
> > RFC 3548 Base 64 Encoding with URL and Filename Safe Alphabet
> > https://datatracker.ietf.org/doc/html/rfc3548#page-6
> >
> > But basically this is the same encoding: just two symbols differ and
> > no padding. So it's easy and safe just to parse both forms.
> > The coreutils have a separate command `basenc --base64url` to parse
> > the b64u but actually there is no any sense to make it as a separate
> > command.
> > Having this as a separate command makes it unusable for small Linux
> > distros like OpenWrt where the device's disk is very small.
>
> Note coreutils can be built like busybox to include multiple commands in =
a single binary
>
> >
> > Currently **many** shell scripts uses additional transformation like:
> >
> > ```sh
> > base64_padding()
> > {
> >    local len=3D$(( ${#1} % 4 ))
> >    local padded_b64=3D''
> >    if [ ${len} =3D 2 ]; then
> >      padded_b64=3D"${1}=3D=3D"
> >    elif [ ${len} =3D 3 ]; then
> >      padded_b64=3D"${1}=3D"
> >    else
> >      padded_b64=3D"${1}"
> >    fi
> >    echo -n "$padded_b64"
> > }
> >
> > base64url_to_b64()
> > {
> >    base64_padding "${1}" | tr -- '-_' '+/'
> > }
> > ```
>
> I'm not sure about base64 accepting _either_ encoding without options.
> There would be robustness issues with that.
>
> Also blinding assuming padding may introduce robustness issues with trunc=
ated data.
>
> It might be useful to have a "--decode-relaxed" option or something
> to support decoding any b64 variant and assume no padding,
> though preprocessing isn't that onerous, so this is marginal.
>
> > But this over complicates everything, reduces performance and is error =
prone.
> >
> > I created a similar ticked for OpenSSL base64
> > https://github.com/openssl/openssl/issues/17559
> >
> >
> > Also as an additional request: can we add the new param `-D` in upper
> > case which will be an alias to `-d`.
> > The issue is that on MacOS the small -d was used for debug and they
> > used the -D in upper case. This creates a lot of problems for scripts
> > that should work on both Linux and MacOS. So most people use the long
> > --decode. But many users just don't know about this interop problem
> > and once I faced a problem when my college gave me a script that
> > didn't work for me.
> > This will prevent unexpected compatibility problems.
>
> macos using -D is unfortunate.
> It's surprising they diverged from FreeBSD here.
> The fact there is a cross platform --decode option,
> suggests we don't need to add -D support,
> as it doesn't help folks using -d on other systems.
>
> cheers,
> P=C3=A1draig



--=20
Sergey Ponomarev, skype:stokito




Message received at control <at> debbugs.gnu.org:


Received: (at control) by debbugs.gnu.org; 28 Jan 2022 01:58:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 27 20:58:04 2022
Received: from localhost ([127.0.0.1]:57026 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nDGWa-0005r8-8G
	for submit <at> debbugs.gnu.org; Thu, 27 Jan 2022 20:58:04 -0500
Received: from zimbra.cs.ucla.edu ([131.179.128.68]:59486)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eggert@HIDDEN>) id 1nDGWY-0005qe-Pv
 for control <at> debbugs.gnu.org; Thu, 27 Jan 2022 20:58:03 -0500
Received: from localhost (localhost [127.0.0.1])
 by zimbra.cs.ucla.edu (Postfix) with ESMTP id 43C70160126
 for <control <at> debbugs.gnu.org>; Thu, 27 Jan 2022 17:57:57 -0800 (PST)
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
 by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10032)
 with ESMTP id 6G2tV34_eODM for <control <at> debbugs.gnu.org>;
 Thu, 27 Jan 2022 17:57:56 -0800 (PST)
Received: from localhost (localhost [127.0.0.1])
 by zimbra.cs.ucla.edu (Postfix) with ESMTP id A3920160133
 for <control <at> debbugs.gnu.org>; Thu, 27 Jan 2022 17:57:56 -0800 (PST)
X-Virus-Scanned: amavisd-new at zimbra.cs.ucla.edu
Received: from zimbra.cs.ucla.edu ([127.0.0.1])
 by localhost (zimbra.cs.ucla.edu [127.0.0.1]) (amavisd-new, port 10026)
 with ESMTP id ean83Se0DdVU for <control <at> debbugs.gnu.org>;
 Thu, 27 Jan 2022 17:57:56 -0800 (PST)
Received: from [192.168.1.9] (cpe-172-91-119-151.socal.res.rr.com
 [172.91.119.151])
 by zimbra.cs.ucla.edu (Postfix) with ESMTPSA id 80AA3160126
 for <control <at> debbugs.gnu.org>; Thu, 27 Jan 2022 17:57:56 -0800 (PST)
Message-ID: <9cd128fd-ef6d-2d10-db37-d4fc978ab51b@HIDDEN>
Date: Thu, 27 Jan 2022 17:57:56 -0800
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101
 Thunderbird/91.5.0
Content-Language: en-US
To: control <at> debbugs.gnu.org
From: Paul Eggert <eggert@HIDDEN>
Subject: bug report maintenance
Organization: UCLA Computer Science Department
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

severity 53400 wishlist
severity 52525 wishlist
tags 42766 wontfix





Last modified: Fri, 28 Jan 2022 02:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.