GNU logs - #43999, boring messages

Message sent to bug-sed@HIDDEN:

Subject: bug#43999: sed: couldn't re-allocate memory
Resent-From: Mark Filipak <markfilipak@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-sed@HIDDEN
Resent-Date: Wed, 14 Oct 2020 21:54:02 +0000
Resent-Message-ID: <handler.43999.B.160271238321512 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 43999 <at> debbugs.gnu.org
Reply-To: markfilipak.linux+gnuorg@HIDDEN
From: Mark Filipak <markfilipak@HIDDEN>
Message-ID: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN>
Date: Wed, 14 Oct 2020 16:40:42 -0400
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101
 Firefox/52.0 Thunderbird/52.7.0
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Received-SPF: pass client-ip=47.43.26.135; envelope-from=markfilipak@HIDDEN;
 helo=p-impout004.msg.pkvw.co.charter.net
X-Spam_score_int: 2
X-Spam_score: 0.2
X-Spam_bar: /
X-Spam_report: (0.2 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_REPLYTO=2.095,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.2 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: Kindly expedite this and contact me for any reason. The file
 "1,073,709,056 bytes" provokes an error (& zero output), but only if piped
 from 'tr' and only for a particular pattern: /00000100/. the file "1, 073, 739,
 776 bytes" succeeds with identical para [...] 
 Content analysis details:   (1.2 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 -2.3 RCVD_IN_DNSWL_MED      RBL: Sender listed at https://www.dnswl.org/,
 medium trust [209.51.188.17 listed in list.dnswl.org]
 0.0 RCVD_IN_MSPIKE_H4      RBL: Very Good reputation (+4)
 [209.51.188.17 listed in wl.mailspike.net]
 0.0 RCVD_IN_MSPIKE_WL      Mailspike good senders
 2.5 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From
X-Mailman-Approved-At: Wed, 14 Oct 2020 17:53:02 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Kindly expedite this and contact me for any reason.

The file "1,073,709,056 bytes" provokes an error (& zero output), but only if piped from 'tr' and 
only for a particular pattern: /00000100/.
the file "1,073,739,776 bytes" succeeds with identical parameters.
The pipe through 'tr' appears not to be the problem.

$ sed --version
sed (GNU sed) 4.2.2
...
$ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt
sed: couldn't re-allocate memory
$ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 's/000001/\x0D\x0A&/g' > foo.txt
$ xxd -p -u "1,073,709,056 bytes" | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt
$ xxd -p -u "1,073,739,776 bytes" | tr -d '\n' | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt
$

You probably want the two source files, or perhaps only the source file that provokes the error. 
Kindly let me know how I can send it to you.

Regards,
Mark Filipak.

PS: For your sake, I posted a request for vetting this case
https://forums.linuxmint.com/viewtopic.php?f=47&t=332836
I got no responses, so it has not been replicated by someone else.

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: markfilipak.linux+gnuorg@HIDDEN
Subject: bug#43999: Acknowledgement (sed: couldn't re-allocate memory)
Message-ID: <handler.43999.B.160271238321512.ack <at> debbugs.gnu.org>
References: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN>
X-Gnu-PR-Message: ack 43999
X-Gnu-PR-Package: sed
Reply-To: 43999 <at> debbugs.gnu.org
Date: Wed, 14 Oct 2020 21:54:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-sed@HIDDEN

If you wish to submit further information on this problem, please
send it to 43999 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
43999: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D43999
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-sed@HIDDEN:

Subject: bug#43999: sed: couldn't re-allocate memory
Resent-From: Assaf Gordon <assafgordon@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-sed@HIDDEN
Resent-Date: Thu, 15 Oct 2020 06:36:02 +0000
Resent-Message-ID: <handler.43999.B43999.16027437535652 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: markfilipak.linux+gnuorg@HIDDEN, 43999 <at> debbugs.gnu.org
References: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN>
From: Assaf Gordon <assafgordon@HIDDEN>
Message-ID: <f3c0380d-94cc-af0d-0c91-1eb18da3a7a3@HIDDEN>
Date: Thu, 15 Oct 2020 00:35:42 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.0
MIME-Version: 1.0
In-Reply-To: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

tag 43999 moreinfo
stop

Hello,

On 2020-10-14 2:40 p.m., Mark Filipak wrote:
> The file "1,073,709,056 bytes" provokes an error (& zero output), but 
> only if piped from 'tr' and only for a particular pattern: /00000100/.
> the file "1,073,739,776 bytes" succeeds with identical parameters.
> The pipe through 'tr' appears not to be the problem.
> 
> $ sed --version
> sed (GNU sed) 4.2.2

First,
sed version 4.2.2 is 8 years old.
It is not supported any more.
Please try with the latest version of sed and report back if you still 
get the error.

Second,
please send some information about your environment, namely "uname -a",
are you using a 32bit or 64bit system,
and how much RAM and SWAP does your machine have.

> ...
> $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 
> 's/00000100/\x0D\x0A&/g' > foo.txt
> sed: couldn't re-allocate memory

Third,
You did not describe what your input file contains.
I implicitly assume from the file name that the file is large (~1GB?).
I did not try reproduce this error, but my hunch is that by deleting
input newlines you are asking sed to load a very large input string and
store it in memory - so a memory-related error is not unexpected.

regards,
  - assaf

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 15 Oct 2020 06:35:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 15 02:35:53 2020
Received: from localhost ([127.0.0.1]:53916 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kSwrg-0001Sy-Qb
	for submit <at> debbugs.gnu.org; Thu, 15 Oct 2020 02:35:53 -0400
Received: from mail-pj1-f66.google.com ([209.85.216.66]:38243)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <assafgordon@HIDDEN>)
 id 1kSwrf-0001Sj-G2; Thu, 15 Oct 2020 02:35:51 -0400
Received: by mail-pj1-f66.google.com with SMTP id u3so1347414pjr.3;
 Wed, 14 Oct 2020 23:35:51 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=subject:to:references:from:message-id:date:user-agent:mime-version
 :in-reply-to:content-language:content-transfer-encoding;
 bh=sL/SLZ/zlnVkcWQv/RgDU105ymznSfzvydWTg7J0TCk=;
 b=JEMmSjj8Igw3B6h3fhhx5VBJht6GmxOknQuFswctFw71d0Lf548HeElCWMegqyWjL0
 tw0z0Km3I4ToKa8IFJVdGgACkikD4jCarArEeeMb6n/rDrXM81i+DY60wmj9NnB2DSMu
 7eeKj+jxSrYlgYcDOWz2zhvcbH0FXVcOjvMR6Qde5k3fnHUakf7Xte3y1InF9njGpIyD
 9URMy2ctHH3rBym964oe0O6E0aaVAXZOFFZXBE3H6O2kzUUZJpwrani+zunvJ5poc5JC
 8+7lPfGNevgPrDN/IbkOhFYj4fR8lhiXhoDiyTx6YaFo9LIiOKdEG7dsbaDS0DyuZSHT
 rfIQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:subject:to:references:from:message-id:date
 :user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=sL/SLZ/zlnVkcWQv/RgDU105ymznSfzvydWTg7J0TCk=;
 b=O4wcjuQv3YEUQU9b7ZEl0FmbkOlH26DVUvJTG+jiVEyJTp/kuoSUErOvUXg0Td01Ey
 iWBSK/1krEN72MnEDIC2TTo4kCU6FKJ/7IXkH6KtYp/dHfEWCdayZKqwJVwSAZUQPqYx
 WccyV19RjPqkHM5IXx22YPgoiasjtvmeVCcr56Kwo/ZD4hsZJsqdSygT8oI3WYez8YPz
 5XgU+STeGsQwA7w+u56W3jwnCOcligLns4CPr8ySUSbFvGwKngZ50sBPPsnfpzMKixYX
 vtt7QrJh/HUU1pg0EF5UE/iH3R2okF87TzQBh6BQDGj5KoaO3pdxi1rRk39xUdfjKOKd
 xMpQ==
X-Gm-Message-State: AOAM531UqJ7FFctd4xig49PbvY0L4hoGCfxTFWkM/jc+JHj0IjGTNgv8
 hnbJBecXxxJumH1W/TZ1cRn1DOILKgA=
X-Google-Smtp-Source: ABdhPJwqIhyRu02Kblp84kUK/SWSU2YjGI4KoLdzMXG/fBJ9q+/Z8nn28tUoOnGbswDSvmYNOTkAoA==
X-Received: by 2002:a17:90a:248:: with SMTP id t8mr3146198pje.64.1602743744945; 
 Wed, 14 Oct 2020 23:35:44 -0700 (PDT)
Received: from tomato.moose.housegordon.com (moose.housegordon.com.
 [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id
 c127sm1800842pfc.115.2020.10.14.23.35.43
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Wed, 14 Oct 2020 23:35:43 -0700 (PDT)
Subject: Re: bug#43999: sed: couldn't re-allocate memory
To: markfilipak.linux+gnuorg@HIDDEN, 43999 <at> debbugs.gnu.org
References: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN>
From: Assaf Gordon <assafgordon@HIDDEN>
Message-ID: <f3c0380d-94cc-af0d-0c91-1eb18da3a7a3@HIDDEN>
Date: Thu, 15 Oct 2020 00:35:42 -0600
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101
 Thunderbird/68.12.0
MIME-Version: 1.0
In-Reply-To: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

tag 43999 moreinfo
stop

Hello,

On 2020-10-14 2:40 p.m., Mark Filipak wrote:
> The file "1,073,709,056 bytes" provokes an error (& zero output), but 
> only if piped from 'tr' and only for a particular pattern: /00000100/.
> the file "1,073,739,776 bytes" succeeds with identical parameters.
> The pipe through 'tr' appears not to be the problem.
> 
> $ sed --version
> sed (GNU sed) 4.2.2

First,
sed version 4.2.2 is 8 years old.
It is not supported any more.
Please try with the latest version of sed and report back if you still 
get the error.

Second,
please send some information about your environment, namely "uname -a",
are you using a 32bit or 64bit system,
and how much RAM and SWAP does your machine have.

> ...
> $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 
> 's/00000100/\x0D\x0A&/g' > foo.txt
> sed: couldn't re-allocate memory

Third,
You did not describe what your input file contains.
I implicitly assume from the file name that the file is large (~1GB?).
I did not try reproduce this error, but my hunch is that by deleting
input newlines you are asking sed to load a very large input string and
store it in memory - so a memory-related error is not unexpected.

regards,
  - assaf

Last modified: Thu, 15 Oct 2020 06:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.