Assaf Gordon <assafgordon@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.Received: (at 43999) by debbugs.gnu.org; 15 Oct 2020 06:35:53 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Oct 15 02:35:53 2020 Received: from localhost ([127.0.0.1]:53918 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1kSwrh-0001T0-1n for submit <at> debbugs.gnu.org; Thu, 15 Oct 2020 02:35:53 -0400 Received: from mail-pj1-f66.google.com ([209.85.216.66]:38243) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <assafgordon@HIDDEN>) id 1kSwrf-0001Sj-G2; Thu, 15 Oct 2020 02:35:51 -0400 Received: by mail-pj1-f66.google.com with SMTP id u3so1347414pjr.3; Wed, 14 Oct 2020 23:35:51 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=subject:to:references:from:message-id:date:user-agent:mime-version :in-reply-to:content-language:content-transfer-encoding; bh=sL/SLZ/zlnVkcWQv/RgDU105ymznSfzvydWTg7J0TCk=; b=JEMmSjj8Igw3B6h3fhhx5VBJht6GmxOknQuFswctFw71d0Lf548HeElCWMegqyWjL0 tw0z0Km3I4ToKa8IFJVdGgACkikD4jCarArEeeMb6n/rDrXM81i+DY60wmj9NnB2DSMu 7eeKj+jxSrYlgYcDOWz2zhvcbH0FXVcOjvMR6Qde5k3fnHUakf7Xte3y1InF9njGpIyD 9URMy2ctHH3rBym964oe0O6E0aaVAXZOFFZXBE3H6O2kzUUZJpwrani+zunvJ5poc5JC 8+7lPfGNevgPrDN/IbkOhFYj4fR8lhiXhoDiyTx6YaFo9LIiOKdEG7dsbaDS0DyuZSHT rfIQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:subject:to:references:from:message-id:date :user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=sL/SLZ/zlnVkcWQv/RgDU105ymznSfzvydWTg7J0TCk=; b=O4wcjuQv3YEUQU9b7ZEl0FmbkOlH26DVUvJTG+jiVEyJTp/kuoSUErOvUXg0Td01Ey iWBSK/1krEN72MnEDIC2TTo4kCU6FKJ/7IXkH6KtYp/dHfEWCdayZKqwJVwSAZUQPqYx WccyV19RjPqkHM5IXx22YPgoiasjtvmeVCcr56Kwo/ZD4hsZJsqdSygT8oI3WYez8YPz 5XgU+STeGsQwA7w+u56W3jwnCOcligLns4CPr8ySUSbFvGwKngZ50sBPPsnfpzMKixYX vtt7QrJh/HUU1pg0EF5UE/iH3R2okF87TzQBh6BQDGj5KoaO3pdxi1rRk39xUdfjKOKd xMpQ== X-Gm-Message-State: AOAM531UqJ7FFctd4xig49PbvY0L4hoGCfxTFWkM/jc+JHj0IjGTNgv8 hnbJBecXxxJumH1W/TZ1cRn1DOILKgA= X-Google-Smtp-Source: ABdhPJwqIhyRu02Kblp84kUK/SWSU2YjGI4KoLdzMXG/fBJ9q+/Z8nn28tUoOnGbswDSvmYNOTkAoA== X-Received: by 2002:a17:90a:248:: with SMTP id t8mr3146198pje.64.1602743744945; Wed, 14 Oct 2020 23:35:44 -0700 (PDT) Received: from tomato.moose.housegordon.com (moose.housegordon.com. [184.68.105.38]) by smtp.googlemail.com with ESMTPSA id c127sm1800842pfc.115.2020.10.14.23.35.43 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Wed, 14 Oct 2020 23:35:43 -0700 (PDT) Subject: Re: bug#43999: sed: couldn't re-allocate memory To: markfilipak.linux+gnuorg@HIDDEN, 43999 <at> debbugs.gnu.org References: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN> From: Assaf Gordon <assafgordon@HIDDEN> Message-ID: <f3c0380d-94cc-af0d-0c91-1eb18da3a7a3@HIDDEN> Date: Thu, 15 Oct 2020 00:35:42 -0600 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:68.0) Gecko/20100101 Thunderbird/68.12.0 MIME-Version: 1.0 In-Reply-To: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 43999 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) tag 43999 moreinfo stop Hello, On 2020-10-14 2:40 p.m., Mark Filipak wrote: > The file "1,073,709,056 bytes" provokes an error (& zero output), but > only if piped from 'tr' and only for a particular pattern: /00000100/. > the file "1,073,739,776 bytes" succeeds with identical parameters. > The pipe through 'tr' appears not to be the problem. > > $ sed --version > sed (GNU sed) 4.2.2 First, sed version 4.2.2 is 8 years old. It is not supported any more. Please try with the latest version of sed and report back if you still get the error. Second, please send some information about your environment, namely "uname -a", are you using a 32bit or 64bit system, and how much RAM and SWAP does your machine have. > ... > $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r > 's/00000100/\x0D\x0A&/g' > foo.txt > sed: couldn't re-allocate memory Third, You did not describe what your input file contains. I implicitly assume from the file name that the file is large (~1GB?). I did not try reproduce this error, but my hunch is that by deleting input newlines you are asking sed to load a very large input string and store it in memory - so a memory-related error is not unexpected. regards, - assaf
bug-sed@HIDDEN:bug#43999; Package sed.
Full text available.Received: (at submit) by debbugs.gnu.org; 14 Oct 2020 21:53:03 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 14 17:53:03 2020 Received: from localhost ([127.0.0.1]:53276 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1kSohj-0005au-6Q for submit <at> debbugs.gnu.org; Wed, 14 Oct 2020 17:53:03 -0400 Received: from lists.gnu.org ([209.51.188.17]:59134) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <markfilipak@HIDDEN>) id 1kSnZq-0007ru-KW for submit <at> debbugs.gnu.org; Wed, 14 Oct 2020 16:40:50 -0400 Received: from eggs.gnu.org ([2001:470:142:3::10]:56952) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <markfilipak@HIDDEN>) id 1kSnZq-0007Qh-D6 for bug-sed@HIDDEN; Wed, 14 Oct 2020 16:40:50 -0400 Received: from p-impout004aa.msg.pkvw.co.charter.net ([47.43.26.135]:51613 helo=p-impout004.msg.pkvw.co.charter.net) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <markfilipak@HIDDEN>) id 1kSnZo-00038U-Jd for bug-sed@HIDDEN; Wed, 14 Oct 2020 16:40:50 -0400 Received: from [192.168.0.3] ([76.188.128.133]) by cmsmtp with ESMTPA id SnZiknHMHOboRSnZjkTm3U; Wed, 14 Oct 2020 20:40:44 +0000 X-Authority-Analysis: v=2.3 cv=MM4eZ/Rl c=1 sm=1 tr=0 a=yKNMihaBE5Rt6XcMTKyqHg==:117 a=yKNMihaBE5Rt6XcMTKyqHg==:17 a=4TstAJWtRUUA:10 a=IkcTkHD0fZMA:10 a=x7bEGLp0ZPQA:10 a=UApgNeDKAAAA:8 a=3ItE9Zqe7OjNITQ7FNUA:9 a=ZBJl85-7Spx0tf-K:21 a=QEXdDO2ut3YA:10 a=nmsX9mYhfO4A:10 a=OylmjzD9BNwA:10 a=fasyu4nNmrhVRyehPdzp:22 a=aZ922TYaLRI4xedjMu-5:22 To: bug-sed@HIDDEN From: Mark Filipak <markfilipak@HIDDEN> Subject: sed: couldn't re-allocate memory Message-ID: <29ab0237-d59d-6ab9-efaa-fa01445e5799@HIDDEN> Date: Wed, 14 Oct 2020 16:40:42 -0400 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:52.0) Gecko/20100101 Firefox/52.0 Thunderbird/52.7.0 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-CMAE-Envelope: MS4wfOnPkzp89ZmtlYErYW2wmUoKe/QQho1P8oLKPLeOYc37gq/6QmdRBPrm3DH4Y7YEbeg0HU8/9Yv5R83S8PeVVwRUEjrfifQkU4c9J6ZtxssTqh2GPZcW MDLRdjl2NCpgHYAxIhabAuhAT1N0xKKw7SwpIFSK/3DgC6qq3XDkqZs3 Received-SPF: pass client-ip=47.43.26.135; envelope-from=markfilipak@HIDDEN; helo=p-impout004.msg.pkvw.co.charter.net X-detected-operating-system: by eggs.gnu.org: First seen = 2020/10/14 16:40:45 X-ACL-Warn: Detected OS = ??? X-Spam_score_int: 2 X-Spam_score: 0.2 X-Spam_bar: / X-Spam_report: (0.2 / 5.0 requ) BAYES_00=-1.9, FREEMAIL_FORGED_REPLYTO=2.095, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no X-Spam_action: no action X-Spam-Score: 1.2 (+) X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org", has NOT identified this incoming email as spam. The original message has been attached to this so you can view it or label similar future email. If you have any questions, see the administrator of that system for details. Content preview: Kindly expedite this and contact me for any reason. The file "1,073,709,056 bytes" provokes an error (& zero output), but only if piped from 'tr' and only for a particular pattern: /00000100/. the file "1, 073, 739, 776 bytes" succeeds with identical para [...] Content analysis details: (1.2 points, 10.0 required) pts rule name description ---- ---------------------- -------------------------------------------------- 0.0 SPF_HELO_NONE SPF: HELO does not publish an SPF Record 1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail) -2.3 RCVD_IN_DNSWL_MED RBL: Sender listed at https://www.dnswl.org/, medium trust [209.51.188.17 listed in list.dnswl.org] 0.0 RCVD_IN_MSPIKE_H4 RBL: Very Good reputation (+4) [209.51.188.17 listed in wl.mailspike.net] 0.0 RCVD_IN_MSPIKE_WL Mailspike good senders 2.5 FREEMAIL_FORGED_REPLYTO Freemail in Reply-To, but not From X-Debbugs-Envelope-To: submit X-Mailman-Approved-At: Wed, 14 Oct 2020 17:53:02 -0400 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Reply-To: markfilipak.linux+gnuorg@HIDDEN Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.3 (--) Kindly expedite this and contact me for any reason. The file "1,073,709,056 bytes" provokes an error (& zero output), but only if piped from 'tr' and only for a particular pattern: /00000100/. the file "1,073,739,776 bytes" succeeds with identical parameters. The pipe through 'tr' appears not to be the problem. $ sed --version sed (GNU sed) 4.2.2 ... $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt sed: couldn't re-allocate memory $ xxd -p -u "1,073,709,056 bytes" | tr -d '\n' | sed -r 's/000001/\x0D\x0A&/g' > foo.txt $ xxd -p -u "1,073,709,056 bytes" | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt $ xxd -p -u "1,073,739,776 bytes" | tr -d '\n' | sed -r 's/00000100/\x0D\x0A&/g' > foo.txt $ You probably want the two source files, or perhaps only the source file that provokes the error. Kindly let me know how I can send it to you. Regards, Mark Filipak. PS: For your sake, I posted a request for vetting this case https://forums.linuxmint.com/viewtopic.php?f=47&t=332836 I got no responses, so it has not been replicated by someone else.
markfilipak.linux+gnuorg@HIDDEN:bug-sed@HIDDEN.
Full text available.bug-sed@HIDDEN:bug#43999; Package sed.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.