GNU bug report logs - #34133
Huge memory usage and output size when using "H" and "G"

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: sed; Reported by: Hongxu Chen <leftcopy.chx@HIDDEN>; Keywords: notabug; Done: Assaf Gordon <assafgordon@HIDDEN>; Maintainer for sed is bug-sed@HIDDEN.
bug closed, send any further explanations to 34133 <at> debbugs.gnu.org and Hongxu Chen <leftcopy.chx@HIDDEN> Request was from Assaf Gordon <assafgordon@HIDDEN> to control <at> debbugs.gnu.org. Full text available.
Added tag(s) notabug. Request was from Assaf Gordon <assafgordon@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 34133 <at> debbugs.gnu.org:


Received: (at 34133) by debbugs.gnu.org; 19 Jan 2019 21:27:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 19 16:27:42 2019
Received: from localhost ([127.0.0.1]:38096 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gky9V-00028b-PW
	for submit <at> debbugs.gnu.org; Sat, 19 Jan 2019 16:27:42 -0500
Received: from mail-pg1-f173.google.com ([209.85.215.173]:40689)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <assafgordon@HIDDEN>)
 id 1gky9U-00028H-1N; Sat, 19 Jan 2019 16:27:40 -0500
Received: by mail-pg1-f173.google.com with SMTP id z10so7697797pgp.7;
 Sat, 19 Jan 2019 13:27:39 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=subject:to:references:from:message-id:date:user-agent:mime-version
 :in-reply-to:content-language:content-transfer-encoding;
 bh=YYYitr2rM+iekmYRU11pvONcXqjxUizuFczPwObyLf0=;
 b=WMZeaQabEb4XL2wDB9OtIFImVpDjjbh5pfaGIhpnrduuDIu/b4p77jdEvH4o2oKW9m
 6hKnUWbxRDLuIurvdelMlpIkR2PlhEkJ7DvqSFpkOii1tBwmZuOW4XvD3i9ExbwTKk4C
 61vWWCge3gSxThOrBeKd+qiNWmy4woVDPPiMy84kej8qfXUIynwFQwjHiksnNK02NudY
 R1QOkFGoVwmpqDiKBNkjtGiAdKEzOfDvqVPtzz1J5fCyPNnvNm0i5ug2HlQq1WhPBkHD
 6tI6ebvXP593/IxbmEl56b6Ak0n/fv/0FNNdEY69cYI5BhSFuNlOYljooHyFpZRNLtu3
 /FCg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:subject:to:references:from:message-id:date
 :user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=YYYitr2rM+iekmYRU11pvONcXqjxUizuFczPwObyLf0=;
 b=SARHcq0IyLsQkS3siHckUTslUILvA2iL1zQVbqcwpTWL3sB17/P0omJ5IolzmfaZtO
 sGuwM85UKQzRQOMaY3jwcHY4QfzEPrI4pVKs9n7At3CPGxO5PMAQ1iozYbavijrpC0kb
 gXPIpsKtrwK2HkWcyoyGJ+H9+1BTkbS54biDBsgp/ai+nvQrB/TfBb+rbyQx5yT/+/CD
 XX2KGWYubvwET3UnZdXAhI8AGl6Jh5Jx+iLPW8xfY2HTc1/DtDpzOYFTrSy+tVVnIIyH
 1lces3t8HE8VPtAGvo1r3PTwo19aoWsihi0DciUNlMxJXVI1QZBUN95r1Xqe4JhN1FA/
 /3qA==
X-Gm-Message-State: AJcUukekl3brfhQZrKVAnJChePIy1xkpgvAS0WHCq5cIflXWRDXMQz3e
 xVvnOVWVJViEu9V6XJhd6V/Epc9L
X-Google-Smtp-Source: ALg8bN4i02x4NeKSJ9K3hbnnmJ+8NawqM2zQTI9aL+FhLsaSitPd/cSYHlIlYA0DX/SCR7d7hJR4Uw==
X-Received: by 2002:a63:451a:: with SMTP id s26mr9994514pga.150.1547933253323; 
 Sat, 19 Jan 2019 13:27:33 -0800 (PST)
Received: from tomato.housegordon.com (moose.housegordon.com. [184.68.105.38])
 by smtp.googlemail.com with ESMTPSA id
 t90sm12283121pfj.23.2019.01.19.13.27.31
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 19 Jan 2019 13:27:31 -0800 (PST)
Subject: Re: bug#34133: Huge memory usage and output size when using "H" and
 "G"
To: Hongxu Chen <leftcopy.chx@HIDDEN>, 34133 <at> debbugs.gnu.org
References: <CAJPBKOHpVQ-S_nT2agaB+M4i3Z-DtTTG_PKa=AeHpmh3AaCdeQ@HIDDEN>
From: Assaf Gordon <assafgordon@HIDDEN>
Message-ID: <16b6994c-7224-8869-baf5-6df68a2ded79@HIDDEN>
Date: Sat, 19 Jan 2019 14:27:30 -0700
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.4.0
MIME-Version: 1.0
In-Reply-To: <CAJPBKOHpVQ-S_nT2agaB+M4i3Z-DtTTG_PKa=AeHpmh3AaCdeQ@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 34133
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

tags 34133 notabug
close 34133
stop

Hello,

On 2019-01-19 2:53 a.m., Hongxu Chen wrote:
>      We found an issue that are relevant to use of "H" and "G" for appending
> hold space and pattern space.

It is an "issue" in the sense that your example does consume large
amounts of memory, but it is not a bug - this is how sed works.

>      The input file is attached which is a file of 30 lines and 80 columns
> filled with 'a'. And my memory is 64G with equivalent swap.
> 
>        # these two may eat up the memory
>      sed 's/a/d/; G; H;' input
>      sed '/b/d; G; H;' input


Let's simplify:
The "s/a/d/" does not change anything related to memory
(it changes a single letter "a" to "d" in the input), so I'll omit it.

The '/b/d' command is a no-op, because your input does not contain
the letter "b".

We're left with:
    sed 'G;H'
The length of each line also doesn't matter, so I'll use shorter lines.

Now observe the following:

$ printf "%s\n" 0 | sed 'G;H' | wc -l
2
$ printf "%s\n" 0 1 | sed 'G;H' | wc -l
6
$ printf "%s\n" 0 1 2 | sed 'G;H' | wc -l
14
$ printf "%s\n" 0 1 2 3 | sed 'G;H' | wc -l
30
$ printf "%s\n" 0 1 2 3 4 | sed 'G;H' | wc -l
62
$ printf "%s\n" 0 1 2 3 4 5 | sed 'G;H' | wc -l
126
$ printf "%s\n" 0 1 2 3 4 5 6 | sed 'G;H' | wc -l
254
$ printf "%s\n" 0 1 2 3 4 5 6 7 | sed 'G;H' | wc -l
510
$ printf "%s\n" 0 1 2 3 4 5 6 7 8 | sed 'G;H' | wc -l
1022
$ printf "%s\n" 0 1 2 3 4 5 6 7 8 9 | sed 'G;H' | wc -l
2046
$ printf "%s\n" 0 1 2 3 4 5 6 7 8 9 10 | sed 'G;H' | wc -l
4094
$ printf "%s\n" 0 1 2 3 4 5 6 7 8 9 10 11 | sed 'G;H' | wc -l
8190
$ printf "%s\n" 0 1 2 3 4 5 6 7 8 9 10 11 12 | sed 'G;H' | wc -l
16382

Notice the trend?
The number of lines (and by proxy: size of buffer and memory usage)
is exponential.

With 20 lines, you'll need O(2^20) = 1M memory (plus size of each line,
and size of pointers overhead, etc.). Still doable.

With 30 lines, you'll need O(2^30) = 1G of lines.
If each of your lines is 80 characters, you'll need 80GB (before
counting overhead of pointers).


>       # this is fine
>      sed '/a/d; G; H;' input

This is "fine" because the "/a/d" command deletes all lines of your
input, hence nothing is stored in the pattern/hold buffers.

>      I learned from http://www.grymoire.com/Unix/Sed.html that 'G' appends
> hold space to pattern space, and 'H' does the inverse.
>      In the first two examples, the buffer of hold space will be appended to
> pattern space, and subsequently content of pattern space will be appended
> to hold space once more. With one more input line, the two buffers will be
> doubled; and as long as the input file is big enough, sed may finally eat
> up the memory and populate the output.

Yes, that how it works.

>      We think this is vulnerable since it may eat up the memory in a few
> seconds.

Any program that keeps the input in memory is vulnerable
to unbounded input size. That is not a bug.

As such, I'm closing this as "not a bug", but discussion can continue
by replying to this thread.

regards,
  - assaf





Information forwarded to bug-sed@HIDDEN:
bug#34133; Package sed. Full text available.

Message received at 34133 <at> debbugs.gnu.org:


Received: (at 34133) by debbugs.gnu.org; 19 Jan 2019 09:57:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 19 04:57:58 2019
Received: from localhost ([127.0.0.1]:37306 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gknO2-0004NU-O3
	for submit <at> debbugs.gnu.org; Sat, 19 Jan 2019 04:57:58 -0500
Received: from mail-it1-f172.google.com ([209.85.166.172]:55195)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <leftcopy.chx@HIDDEN>) id 1gknO0-0004NG-Q6
 for 34133 <at> debbugs.gnu.org; Sat, 19 Jan 2019 04:57:57 -0500
Received: by mail-it1-f172.google.com with SMTP id i145so10283311ita.4
 for <34133 <at> debbugs.gnu.org>; Sat, 19 Jan 2019 01:57:56 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=RmL7mkYw23bqqnWYuwJiAoTUg0R8THG8GiipJ+Qroa4=;
 b=ZuuGyZRecqa40KJdI0R6gryM9AGug8/G4ymxoItvatW+E3zlhMcip3I5UUIiPTbecA
 rJtpcvHTUKanVmmlVDTGYWZAcyXjV40hPWEaNFCxnCWqMJoldjqoXcdx953v1gPsRcb1
 m3ZtY6X3UyYVYgHUCbddW05h7tdY5xKzgNWJc+hULRu1gvMHHZDy5f4aoDskmnfkdROU
 YXdqItjVnhhlHwigDiMNOFgs5pugC/4gjHMOLz/4JfwwilOlBv30c9CfVAnJl7F3lCg/
 sB+SFPh0QRX9aj4EMMz+MtuxVBZ9WVZdqK/Vl300i8Ys3UrMThgqWYL9MJMACOv6oZXj
 JvDQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=RmL7mkYw23bqqnWYuwJiAoTUg0R8THG8GiipJ+Qroa4=;
 b=ZE6dOBeOswBWS+obFIbj+Ka0H0BdROBkIKyfNaBIJREIBn+jOTHxDrDRQxrh9qngDF
 Gonq+RHP9wdC+61TpQjeGXvyLLykqBtoFLzlAtqpDyy3h8oKzaUR3ZH+0k2XvzFhgisY
 fNmspbhEmz7gk5M68rnUPz/pYZ68mzdED9BxNw6fPwpkKBH2CJYa6Zt7192Rg0uJUdj2
 UFs8guAA8QQcel0Ec+S2UGxDEVVyRMDiy7GEFFm1nSbm2Clfb2FMkDuZbQC6kL80/DSZ
 d/gAJxQbDB31JW8khWBx0uLiqKA7dR391WJnMliHqGkeMibOBlalgD1i/EtsBUuFGNcZ
 Bb7w==
X-Gm-Message-State: AJcUukeAjL3rtWSl9r12lEQPpRF0zo+TlBKgK2wsJjMgA2YDJaxV+OvC
 5pZinEZOAvXfEFkTG9C2kROP/kkNzDC1f0Zs25K4ABk1
X-Google-Smtp-Source: ALg8bN4JNqCzLTG13u5h6j5yVde4KYzHVyoXFo2zEIRLNHLsufnmjIaUMxznv5P1jVWPb1u56yzkXuJeTLPwzoRah0g=
X-Received: by 2002:a02:9d4b:: with SMTP id m11mr13104889jal.121.1547891870610; 
 Sat, 19 Jan 2019 01:57:50 -0800 (PST)
MIME-Version: 1.0
From: Hongxu Chen <leftcopy.chx@HIDDEN>
Date: Sat, 19 Jan 2019 17:57:39 +0800
Message-ID: <CAJPBKOHENoHoryzfWPiG_Mq5FYUi1Vou58N2PREOU1BwjeCkGw@HIDDEN>
Subject: Duplicate of 34133
To: 34133 <at> debbugs.gnu.org
Content-Type: multipart/alternative; boundary="000000000000193af5057fcca9d0"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 34133
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--000000000000193af5057fcca9d0
Content-Type: text/plain; charset="UTF-8"

Hi GNU sed maintainers,

    Sorry I mistakenly sent an email with empty content.
    Please close this issue and track #34133 instead.
    Thank you!

Best Regards,
Hongxu

--000000000000193af5057fcca9d0
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr">Hi GNU sed maintainers,<div><br></div><di=
v>=C2=A0 =C2=A0 Sorry I mistakenly sent an email with empty content.</div><=
div>=C2=A0 =C2=A0 Please close this issue and track #34133 instead.</div><d=
iv>=C2=A0 =C2=A0 Thank you!</div><div><br clear=3D"all"><div><div dir=3D"lt=
r" class=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><font c=
olor=3D"#313131" face=3D"Arial, sans-serif"><span style=3D"font-size:12px">=
Best Regards,</span></font><div><font color=3D"#313131" face=3D"Arial, sans=
-serif"><span style=3D"font-size:12px">Hongxu</span></font></div></div></di=
v></div></div></div></div></div></div>

--000000000000193af5057fcca9d0--




Information forwarded to bug-sed@HIDDEN:
bug#34133; Package sed. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 19 Jan 2019 09:53:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 19 04:53:41 2019
Received: from localhost ([127.0.0.1]:37301 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1gknJt-0004H2-0r
	for submit <at> debbugs.gnu.org; Sat, 19 Jan 2019 04:53:41 -0500
Received: from eggs.gnu.org ([209.51.188.92]:42437)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <leftcopy.chx@HIDDEN>) id 1gknJr-0004Gp-CF
 for submit <at> debbugs.gnu.org; Sat, 19 Jan 2019 04:53:39 -0500
Received: from lists.gnu.org ([209.51.188.17]:39163)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <leftcopy.chx@HIDDEN>)
 id 1gknJi-00042h-DJ
 for submit <at> debbugs.gnu.org; Sat, 19 Jan 2019 04:53:31 -0500
Received: from eggs.gnu.org ([209.51.188.92]:52620)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <leftcopy.chx@HIDDEN>) id 1gknJe-0007LU-Kd
 for bug-sed@HIDDEN; Sat, 19 Jan 2019 04:53:30 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM,
 HTML_MESSAGE autolearn=disabled version=3.3.2
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <leftcopy.chx@HIDDEN>) id 1gknJX-0003vC-4h
 for bug-sed@HIDDEN; Sat, 19 Jan 2019 04:53:23 -0500
Received: from mail-io1-xd31.google.com ([2607:f8b0:4864:20::d31]:46624)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <leftcopy.chx@HIDDEN>)
 id 1gknJW-0003u9-Qh
 for bug-sed@HIDDEN; Sat, 19 Jan 2019 04:53:19 -0500
Received: by mail-io1-xd31.google.com with SMTP id s8so547919iob.13
 for <bug-sed@HIDDEN>; Sat, 19 Jan 2019 01:53:18 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to;
 bh=IjJbCwr7cTJ7HCx+v0iYXL668qO3Omi7a5KcV8k8kn8=;
 b=mX8yq6AbcZ8x9yQW9m+KpVMiZl/vdMWovY1GVNTId822dQXSrlRPSq+acIAqFtQDlt
 Q5CaBGVOyT6C9+IwH//QKnYUFjH1T0+yWl6rbi8VrU5CoW26UwFg2OMKOrJn2HhIIhda
 MEh+UPe8ZMT5G8qPvWJ0OrICBrJC10wemwGlEv90EeR5FfnuM/nXDIyPaLowhUC5AYs2
 HAahUSX9WIfSKzUAMwswKs2WLnuhK+6d64IhyVmA6e8qkLqE5RahcL2R1YQQcieTVtrs
 ePGIUm11mUOr/oPgxhkFsEvSJfkycjVl2EzMiPu2SlTtp8b5FTq5vxpI2hQKtQqUBBsY
 T7dw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to;
 bh=IjJbCwr7cTJ7HCx+v0iYXL668qO3Omi7a5KcV8k8kn8=;
 b=IdyHxp+HVIFED9lui0w6wyTI+WPRxoej52bH0bYkr4qTgGCx6zcurqq21fr74Gug0N
 c4ZzfbzpLlFHrZQIB6xiYUvCl55UJrVnUh3p7qeIJWCvxbkxQp72mVn9Chc/CS1HuGAG
 8mcpk1CmZd1Zqqs8KBhJwF4vjY2g6jYvoIjf7ctmkNre8wDcofxSbCq1ZsgwgQiPbMdU
 1KxdV+uQ9fwSJ4FyQnwTokc1CLQIdjSdfyfe+K9eGQb13KeO09CPcnPsB54rmimN3Ygd
 XuBifJ6XxdrBqJVfLAgfiYtGSZZykJSafyMtXFK30ygRtBqiJKeTgtevy0Yw+BqhULBh
 OULw==
X-Gm-Message-State: AJcUukd1lX/IkkFpDWPTTRDA0HzmxsxkgmWLQOI85x42WCOw8mniasLk
 znt2uwqeFujPL+HQPHGQU6raiYck8+Q5zoUpHuXtYMD0
X-Google-Smtp-Source: ALg8bN7mLO7g6c4eIMrgwBrsfBVbCUMG/ic9otyzS5Bp9jcjjQxMUzPVKJxAZ6By0wAzEliihVw9DnQg387ZNeN0V8M=
X-Received: by 2002:a5e:d808:: with SMTP id l8mr11976586iok.299.1547891596697; 
 Sat, 19 Jan 2019 01:53:16 -0800 (PST)
MIME-Version: 1.0
From: Hongxu Chen <leftcopy.chx@HIDDEN>
Date: Sat, 19 Jan 2019 17:53:05 +0800
Message-ID: <CAJPBKOHpVQ-S_nT2agaB+M4i3Z-DtTTG_PKa=AeHpmh3AaCdeQ@HIDDEN>
Subject: Huge memory usage and output size when using "H" and "G"
To: bug-sed@HIDDEN
Content-Type: multipart/mixed; boundary="000000000000c5d98e057fcc98bc"
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-Received-From: 2607:f8b0:4864:20::d31
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)

--000000000000c5d98e057fcc98bc
Content-Type: multipart/alternative; boundary="000000000000c5d98a057fcc98ba"

--000000000000c5d98a057fcc98ba
Content-Type: text/plain; charset="UTF-8"

Hi,

    We found an issue that are relevant to use of "H" and "G" for appending
hold space and pattern space.

    The input file is attached which is a file of 30 lines and 80 columns
filled with 'a'. And my memory is 64G with equivalent swap.

      # these two may eat up the memory
    sed 's/a/d/; G; H;' input
    sed '/b/d; G; H;' input

     # this is fine
    sed '/a/d; G; H;' input

    I learned from http://www.grymoire.com/Unix/Sed.html that 'G' appends
hold space to pattern space, and 'H' does the inverse.
    In the first two examples, the buffer of hold space will be appended to
pattern space, and subsequently content of pattern space will be appended
to hold space once more. With one more input line, the two buffers will be
doubled; and as long as the input file is big enough, sed may finally eat
up the memory and populate the output.
    We think this is vulnerable since it may eat up the memory in a few
seconds.

Best Regards,
Hongxu

--000000000000c5d98a057fcc98ba
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div dir=3D"ltr"><div di=
r=3D"ltr"><div>Hi,</div><div><br></div><div>=C2=A0 =C2=A0 We found an issue=
 that are relevant to use of &quot;H&quot; and &quot;G&quot; for appending =
hold space and pattern space.</div><div><br></div><div>=C2=A0 =C2=A0 The in=
put file is attached which is a file of 30 lines and 80 columns filled with=
 &#39;a&#39;. And my memory is 64G with equivalent swap.</div><div><br></di=
v><div>=C2=A0 =C2=A0 =C2=A0 # these two may eat up the memory</div><div>=C2=
=A0 =C2=A0=C2=A0sed &#39;s/a/d/; G; H;&#39; input</div><div>=C2=A0 =C2=A0=
=C2=A0sed &#39;/b/d; G; H;&#39; input</div><div><br></div><div>=C2=A0 =C2=
=A0 =C2=A0# this is fine</div><div>=C2=A0 =C2=A0 sed &#39;/a/d; G; H;&#39; =
input</div><div><br></div><div>=C2=A0 =C2=A0 I learned from=C2=A0<a href=3D=
"http://www.grymoire.com/Unix/Sed.html">http://www.grymoire.com/Unix/Sed.ht=
ml</a> that &#39;G&#39; appends hold space to pattern space, and &#39;H&#39=
; does the inverse.</div><div>=C2=A0 =C2=A0 In the first two examples, the =
buffer of hold space will be appended to pattern space, and subsequently co=
ntent of pattern space will be appended to hold space once more. With one m=
ore input line, the two buffers will be doubled; and as long as the input f=
ile is big enough, sed may finally eat up the memory and populate the outpu=
t.</div><div>=C2=A0 =C2=A0 We think this is vulnerable since it may eat up =
the memory in a few seconds.</div><br clear=3D"all"><div><div dir=3D"ltr" c=
lass=3D"gmail_signature"><div dir=3D"ltr"><div><div dir=3D"ltr"><font color=
=3D"#313131" face=3D"Arial, sans-serif"><span style=3D"font-size:12px">Best=
 Regards,</span></font><div><font color=3D"#313131" face=3D"Arial, sans-ser=
if"><span style=3D"font-size:12px">Hongxu</span></font></div></div></div></=
div></div></div></div></div></div></div></div>

--000000000000c5d98a057fcc98ba--

--000000000000c5d98e057fcc98bc
Content-Type: application/octet-stream; name=input
Content-Disposition: attachment; filename=input
Content-Transfer-Encoding: base64
Content-ID: <f_jr39ujpk0>
X-Attachment-Id: f_jr39ujpk0

YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEK
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWEKYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFh
YWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWFhYWEK
--000000000000c5d98e057fcc98bc--




Acknowledgement sent to Hongxu Chen <leftcopy.chx@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-sed@HIDDEN. Full text available.
Report forwarded to bug-sed@HIDDEN:
bug#34133; Package sed. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sat, 19 Jan 2019 21:30:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.