Package: grep;
Reported by: Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it>
Date: Tue, 25 Aug 2020 15:21:03 UTC
Severity: normal
Done: Paul Eggert <eggert <at> cs.ucla.edu>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 43040 in the body.
You can then email your comments to 43040 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-grep <at> gnu.org:bug#43040; Package grep.
(Tue, 25 Aug 2020 15:21:03 GMT) Full text and rfc822 format available.Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it>:bug-grep <at> gnu.org.
(Tue, 25 Aug 2020 15:21:03 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it> To: bug-grep <at> gnu.org Subject: grep 3.4: memory leak Date: Tue, 25 Aug 2020 17:17:55 +0200
[Message part 1 (text/plain, inline)]
Dear maintainer, grep 3.4 is very slow and uses a lot of memory when executed with the attached inputs: time ./grep-3.4/build/bin/grep -f ./mem_leak ./la_divin.txt [...] real 0m0,442s user 0m0,117s sys 0m0,326s mem usage: ~1.5GB it seems to be a regression, since grep 3.3 runs smoothly on the same inputs: time ./grep-3.3/build/bin/grep -f ./mem_leak ./la_divin.txt [...] real 0m0,016s user 0m0,016s sys 0m0,000s mem usage: ~300KB valgrind detects some memory leaks on grep 3.4, this is its output: ==156624== Memcheck, a memory error detector ==156624== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al. ==156624== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright info ==156624== Command: ./grep-3.4/build/bin/grep -f ./mem_leak ./la_divin.txt ==156624== ==156624== ==156624== HEAP SUMMARY: ==156624== in use at exit: 3,762,378,749 bytes in 281,181 blocks ==156624== total heap usage: 396,668 allocs, 115,487 frees, 3,789,426,889 bytes allocated ==156624== ==156624== 8 bytes in 1 blocks are possibly lost in loss record 2 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x1300D2: analyze (regcomp.c:1169) ==156624== by 0x1300D2: re_compile_internal (regcomp.c:795) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 8 bytes in 1 blocks are possibly lost in loss record 3 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x1255BA: re_node_set_alloc (regex_internal.c:972) ==156624== by 0x1255BA: calc_eclosure_iter (regcomp.c:1700) ==156624== by 0x130316: calc_eclosure (regcomp.c:1677) ==156624== by 0x130316: analyze (regcomp.c:1204) ==156624== by 0x130316: re_compile_internal (regcomp.c:795) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 8 bytes in 1 blocks are possibly lost in loss record 4 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x126009: re_node_set_init_copy (regex_internal.c:1033) ==156624== by 0x1262D1: create_cd_newstate (regex_internal.c:1681) ==156624== by 0x1262D1: re_acquire_state_context (regex_internal.c:1553) ==156624== by 0x13091B: create_initial_state (regcomp.c:1050) ==156624== by 0x13091B: re_compile_internal (regcomp.c:806) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 8 bytes in 1 blocks are possibly lost in loss record 5 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x123AAA: re_node_set_alloc (regex_internal.c:972) ==156624== by 0x123AAA: register_state (regex_internal.c:1574) ==156624== by 0x126430: create_cd_newstate (regex_internal.c:1737) ==156624== by 0x126430: re_acquire_state_context (regex_internal.c:1553) ==156624== by 0x13091B: create_initial_state (regcomp.c:1050) ==156624== by 0x13091B: re_compile_internal (regcomp.c:806) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 16 bytes in 1 blocks are possibly lost in loss record 17 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x12FD2D: init_dfa (regcomp.c:859) ==156624== by 0x12FD2D: re_compile_internal (regcomp.c:758) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 16 bytes in 1 blocks are possibly lost in loss record 18 of 114 ==156624== at 0x483B723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x483E017: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x123B50: register_state (regex_internal.c:1589) ==156624== by 0x126430: create_cd_newstate (regex_internal.c:1737) ==156624== by 0x126430: re_acquire_state_context (regex_internal.c:1553) ==156624== by 0x13091B: create_initial_state (regcomp.c:1050) ==156624== by 0x13091B: re_compile_internal (regcomp.c:806) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 24 bytes in 1 blocks are possibly lost in loss record 34 of 114 ==156624== at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x12FD55: init_dfa (regcomp.c:866) ==156624== by 0x12FD55: re_compile_internal (regcomp.c:758) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 24 bytes in 1 blocks are possibly lost in loss record 35 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x1300F6: analyze (regcomp.c:1171) ==156624== by 0x1300F6: re_compile_internal (regcomp.c:795) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 24 bytes in 1 blocks are possibly lost in loss record 36 of 114 ==156624== at 0x483B7F3: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x130108: analyze (regcomp.c:1172) ==156624== by 0x130108: re_compile_internal (regcomp.c:795) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 112 bytes in 1 blocks are possibly lost in loss record 54 of 114 ==156624== at 0x483DD99: calloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x1262B6: create_cd_newstate (regex_internal.c:1678) ==156624== by 0x1262B6: re_acquire_state_context (regex_internal.c:1553) ==156624== by 0x13091B: create_initial_state (regcomp.c:1050) ==156624== by 0x13091B: re_compile_internal (regcomp.c:806) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 128 bytes in 1 blocks are possibly lost in loss record 64 of 114 ==156624== at 0x483B723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x483E017: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x11FFB9: xrealloc (xmalloc.c:61) ==156624== by 0x114A9B: xpalloc (dfa.c:818) ==156624== by 0x114CD3: realloc_trans_if_necessary (dfa.c:2853) ==156624== by 0x11988B: dfaexec_main (dfa.c:3391) ==156624== by 0x10E082: EGexecute (dfasearch.c:416) ==156624== by 0x10C7C5: main (grep.c:2905) ==156624== ==156624== 232 bytes in 1 blocks are possibly lost in loss record 68 of 114 ==156624== at 0x483B723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x483E017: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x130B24: re_compile_internal (regcomp.c:750) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== 25,222 bytes in 1 blocks are possibly lost in loss record 78 of 114 ==156624== at 0x483DFAF: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x11FFB9: xrealloc (xmalloc.c:61) ==156624== by 0x10C024: main (grep.c:2598) ==156624== ==156624== 10,379,152 (5,013,056 direct, 5,366,096 indirect) bytes in 21,608 blocks are definitely lost in loss record 113 of 114 ==156624== at 0x483B723: malloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x483E017: realloc (in /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so) ==156624== by 0x130B24: re_compile_internal (regcomp.c:750) ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230) ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160) ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247) ==156624== by 0x10C782: main (grep.c:2900) ==156624== ==156624== LEAK SUMMARY: ==156624== definitely lost: 5,013,056 bytes in 21,608 blocks ==156624== indirectly lost: 5,366,096 bytes in 216,158 blocks ==156624== possibly lost: 25,830 bytes in 13 blocks ==156624== still reachable: 3,751,973,767 bytes in 43,402 blocks ==156624== of which reachable via heuristic: ==156624== newarray : 112 bytes in 1 blocks ==156624== suppressed: 0 bytes in 0 blocks ==156624== Reachable blocks (those to which a pointer was found) are not shown. ==156624== To see them, rerun with: --leak-check=full --show-leak-kinds=all ==156624== ==156624== For lists of detected and suppressed errors, rerun with: -s ==156624== ERROR SUMMARY: 14 errors from 14 contexts (suppressed: 0 from 0) --- Regards, Luca Borzacchiello
[Message part 2 (text/html, inline)]
[la_divin.txt (text/plain, attachment)]
[mem_leak (application/octet-stream, attachment)]
bug-grep <at> gnu.org:bug#43040; Package grep.
(Tue, 01 Sep 2020 10:13:02 GMT) Full text and rfc822 format available.Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Shlomi Fish <shlomif <at> shlomifish.org> To: Luca Borzacchiello via Bug reports for GNU grep <bug-grep <at> gnu.org> Cc: Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it>, 43040 <at> debbugs.gnu.org Subject: Re: bug#43040: grep 3.4: memory leak Date: Tue, 1 Sep 2020 13:11:59 +0300
Hi Luca,
On Tue, 25 Aug 2020 17:17:55 +0200
Luca Borzacchiello via Bug reports for GNU grep <bug-grep <at> gnu.org> wrote:
> Dear maintainer,
> grep 3.4 is very slow and uses a lot of memory when executed with the
> attached inputs:
>
> time ./grep-3.4/build/bin/grep -f ./mem_leak ./la_divin.txt
> [...]
> real 0m0,442s
> user 0m0,117s
> sys 0m0,326s
>
> mem usage: ~1.5GB
>
> it seems to be a regression, since grep 3.3 runs smoothly on the same
> inputs:
>
> time ./grep-3.3/build/bin/grep -f ./mem_leak ./la_divin.txt
> [...]
> real 0m0,016s
> user 0m0,016s
> sys 0m0,000s
>
I can reproduce this issue on my mageia v8 x86-64 system (core i3/sandy
bridge). grep-v3.3 built from source is fine.
> mem usage: ~300KB
>
> valgrind detects some memory leaks on grep 3.4, this is its output:
> ==156624== Memcheck, a memory error detector
> ==156624== Copyright (C) 2002-2017, and GNU GPL'd, by Julian Seward et al.
> ==156624== Using Valgrind-3.15.0 and LibVEX; rerun with -h for copyright
> info
> ==156624== Command: ./grep-3.4/build/bin/grep -f ./mem_leak ./la_divin.txt
> ==156624==
> ==156624==
> ==156624== HEAP SUMMARY:
> ==156624== in use at exit: 3,762,378,749 bytes in 281,181 blocks
> ==156624== total heap usage: 396,668 allocs, 115,487 frees, 3,789,426,889
> bytes allocated
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 2 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x1300D2: analyze (regcomp.c:1169)
> ==156624== by 0x1300D2: re_compile_internal (regcomp.c:795)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 3 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x1255BA: re_node_set_alloc (regex_internal.c:972)
> ==156624== by 0x1255BA: calc_eclosure_iter (regcomp.c:1700)
> ==156624== by 0x130316: calc_eclosure (regcomp.c:1677)
> ==156624== by 0x130316: analyze (regcomp.c:1204)
> ==156624== by 0x130316: re_compile_internal (regcomp.c:795)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 4 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x126009: re_node_set_init_copy (regex_internal.c:1033)
> ==156624== by 0x1262D1: create_cd_newstate (regex_internal.c:1681)
> ==156624== by 0x1262D1: re_acquire_state_context (regex_internal.c:1553)
> ==156624== by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624== by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 8 bytes in 1 blocks are possibly lost in loss record 5 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x123AAA: re_node_set_alloc (regex_internal.c:972)
> ==156624== by 0x123AAA: register_state (regex_internal.c:1574)
> ==156624== by 0x126430: create_cd_newstate (regex_internal.c:1737)
> ==156624== by 0x126430: re_acquire_state_context (regex_internal.c:1553)
> ==156624== by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624== by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 16 bytes in 1 blocks are possibly lost in loss record 17 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x12FD2D: init_dfa (regcomp.c:859)
> ==156624== by 0x12FD2D: re_compile_internal (regcomp.c:758)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 16 bytes in 1 blocks are possibly lost in loss record 18 of 114
> ==156624== at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x123B50: register_state (regex_internal.c:1589)
> ==156624== by 0x126430: create_cd_newstate (regex_internal.c:1737)
> ==156624== by 0x126430: re_acquire_state_context (regex_internal.c:1553)
> ==156624== by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624== by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 24 bytes in 1 blocks are possibly lost in loss record 34 of 114
> ==156624== at 0x483DD99: calloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x12FD55: init_dfa (regcomp.c:866)
> ==156624== by 0x12FD55: re_compile_internal (regcomp.c:758)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 24 bytes in 1 blocks are possibly lost in loss record 35 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x1300F6: analyze (regcomp.c:1171)
> ==156624== by 0x1300F6: re_compile_internal (regcomp.c:795)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 24 bytes in 1 blocks are possibly lost in loss record 36 of 114
> ==156624== at 0x483B7F3: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x130108: analyze (regcomp.c:1172)
> ==156624== by 0x130108: re_compile_internal (regcomp.c:795)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 112 bytes in 1 blocks are possibly lost in loss record 54 of 114
> ==156624== at 0x483DD99: calloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x1262B6: create_cd_newstate (regex_internal.c:1678)
> ==156624== by 0x1262B6: re_acquire_state_context (regex_internal.c:1553)
> ==156624== by 0x13091B: create_initial_state (regcomp.c:1050)
> ==156624== by 0x13091B: re_compile_internal (regcomp.c:806)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 128 bytes in 1 blocks are possibly lost in loss record 64 of 114
> ==156624== at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x11FFB9: xrealloc (xmalloc.c:61)
> ==156624== by 0x114A9B: xpalloc (dfa.c:818)
> ==156624== by 0x114CD3: realloc_trans_if_necessary (dfa.c:2853)
> ==156624== by 0x11988B: dfaexec_main (dfa.c:3391)
> ==156624== by 0x10E082: EGexecute (dfasearch.c:416)
> ==156624== by 0x10C7C5: main (grep.c:2905)
> ==156624==
> ==156624== 232 bytes in 1 blocks are possibly lost in loss record 68 of 114
> ==156624== at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x130B24: re_compile_internal (regcomp.c:750)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== 25,222 bytes in 1 blocks are possibly lost in loss record 78 of
> 114
> ==156624== at 0x483DFAF: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x11FFB9: xrealloc (xmalloc.c:61)
> ==156624== by 0x10C024: main (grep.c:2598)
> ==156624==
> ==156624== 10,379,152 (5,013,056 direct, 5,366,096 indirect) bytes in
> 21,608 blocks are definitely lost in loss record 113 of 114
> ==156624== at 0x483B723: malloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x483E017: realloc (in
> /usr/lib/x86_64-linux-gnu/valgrind/vgpreload_memcheck-amd64-linux.so)
> ==156624== by 0x130B24: re_compile_internal (regcomp.c:750)
> ==156624== by 0x130DDB: rpl_re_compile_pattern (regcomp.c:230)
> ==156624== by 0x10D3E6: regex_compile (dfasearch.c:160)
> ==156624== by 0x10D7AC: GEAcompile (dfasearch.c:247)
> ==156624== by 0x10C782: main (grep.c:2900)
> ==156624==
> ==156624== LEAK SUMMARY:
> ==156624== definitely lost: 5,013,056 bytes in 21,608 blocks
> ==156624== indirectly lost: 5,366,096 bytes in 216,158 blocks
> ==156624== possibly lost: 25,830 bytes in 13 blocks
> ==156624== still reachable: 3,751,973,767 bytes in 43,402 blocks
> ==156624== of which reachable via heuristic:
> ==156624== newarray : 112 bytes in 1
> blocks
> ==156624== suppressed: 0 bytes in 0 blocks
> ==156624== Reachable blocks (those to which a pointer was found) are not
> shown.
> ==156624== To see them, rerun with: --leak-check=full --show-leak-kinds=all
> ==156624==
> ==156624== For lists of detected and suppressed errors, rerun with: -s
> ==156624== ERROR SUMMARY: 14 errors from 14 contexts (suppressed: 0 from 0)
>
> ---
> Regards,
> Luca Borzacchiello
--
Shlomi Fish https://www.shlomifish.org/
Apple Inc. is Evil - https://www.shlomifish.org/open-source/anti/apple/
If the mountain does not come to Muhammad, then Chuck Norris will bring the
mountain over.
— https://www.shlomifish.org/humour/bits/facts/Chuck-Norris/
Please reply to list if it's a mailing list post - https://shlom.in/reply .
bug-grep <at> gnu.org:bug#43040; Package grep.
(Tue, 01 Sep 2020 10:13:02 GMT) Full text and rfc822 format available.bug-grep <at> gnu.org:bug#43040; Package grep.
(Tue, 08 Sep 2020 03:00:02 GMT) Full text and rfc822 format available.Message #14 received at 43040 <at> debbugs.gnu.org (full text, mbox):
From: Paul Eggert <eggert <at> cs.ucla.edu> To: Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it> Cc: 43040 <at> debbugs.gnu.org, Shlomi Fish <shlomif <at> shlomifish.org> Subject: Re: bug#43040: grep 3.4: memory leak Date: Mon, 7 Sep 2020 19:59:09 -0700
[Message part 1 (text/plain, inline)]
I don't see the extra memory consumption as necessarily being a bug, if grep is trading space (which is relatively cheap) for time (which is less so). Perhaps someone with some time to spare could look into that in more detail. The excess CPU consumption is clearly problematic, though. I installed the attached patches, which on your example causes 'grep' to be 3x faster than grep 3.3 was. I hope this addresses any practical performance problems uncovered by that artificial test case. The first patch is a bit of a hack but does the real work; the rest are merely cleanups or very minor performance improvements.
[0001-Omit-duplicate-regexps.patch (text/x-patch, attachment)]
[0002-Simplify-regex_compile.patch (text/x-patch, attachment)]
[0003-Simplify-pattern_file_name.patch (text/x-patch, attachment)]
[0004-Prefer-rawmemchr-to-memchr-when-it-s-easy.patch (text/x-patch, attachment)]
bug-grep <at> gnu.org:bug#43040; Package grep.
(Tue, 08 Sep 2020 06:19:01 GMT) Full text and rfc822 format available.Message #17 received at 43040 <at> debbugs.gnu.org (full text, mbox):
From: Shlomi Fish <shlomif <at> shlomifish.org> To: Paul Eggert <eggert <at> cs.ucla.edu> Cc: Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it>, 43040 <at> debbugs.gnu.org Subject: Re: bug#43040: grep 3.4: memory leak Date: Tue, 8 Sep 2020 09:18:11 +0300
On Mon, 7 Sep 2020 19:59:09 -0700
Paul Eggert <eggert <at> cs.ucla.edu> wrote:
> I don't see the extra memory consumption as necessarily being a bug, if grep
> is trading space (which is relatively cheap) for time (which is less so).
> Perhaps someone with some time to spare could look into that in more detail.
>
> The excess CPU consumption is clearly problematic, though. I installed the
> attached patches, which on your example causes 'grep' to be 3x faster than
> grep 3.3 was. I hope this addresses any practical performance problems
> uncovered by that artificial test case. The first patch is a bit of a hack
> but does the real work; the rest are merely cleanups or very minor
> performance improvements.
Thanks, Paul! I can confirm that the latest git master version of gnu grep
(which incorporates the patches) is faster than 3.4.
--
Shlomi Fish https://www.shlomifish.org/
https://youtu.be/n6KAGqjdmsk - “Hurt Me Tomorrow”
And the top story for today: wives live longer than husbands because they are
not married to women.
— Colin Mochrie in Whose Line is it, Anyway?
Please reply to list if it's a mailing list post - https://shlom.in/reply .
Paul Eggert <eggert <at> cs.ucla.edu>:Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it>:Message #22 received at 43040-done <at> debbugs.gnu.org (full text, mbox):
From: Paul Eggert <eggert <at> cs.ucla.edu> To: Shlomi Fish <shlomif <at> shlomifish.org> Cc: Luca Borzacchiello <borzacchiello <at> diag.uniroma1.it>, 43040-done <at> debbugs.gnu.org Subject: Re: bug#43040: grep 3.4: memory leak Date: Fri, 11 Sep 2020 15:53:15 -0700
On 9/7/20 11:18 PM, Shlomi Fish wrote: > Thanks, Paul! I can confirm that the latest git master version of gnu grep > (which incorporates the patches) is faster than 3.4. Thanks for checking; closing the bug report.
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Sat, 10 Oct 2020 11:24:06 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.