GNU bug report logs - #71529
Crash for large directory due to memory outage

Previous Next

Package: diffutils;

Reported by: Evan Greenup <evan_greenup <at> protonmail.com>

Date: Thu, 13 Jun 2024 05:24:03 UTC

Severity: normal

To reply to this bug, email your comments to 71529 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-diffutils <at> gnu.org:
bug#71529; Package diffutils. (Thu, 13 Jun 2024 05:24:04 GMT) Full text and rfc822 format available.

Acknowledgement sent to Evan Greenup <evan_greenup <at> protonmail.com>:
New bug report received and forwarded. Copy sent to bug-diffutils <at> gnu.org. (Thu, 13 Jun 2024 05:24:04 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Evan Greenup <evan_greenup <at> protonmail.com>
To: "bug-diffutils <at> gnu.org" <bug-diffutils <at> gnu.org>
Subject: Crash for large directory due to memory outage
Date: Wed, 12 Jun 2024 22:39:51 +0000
Hi,

The `diff` command in `diffutils` can compare the difference between  two directories. 

However, for large directories (the size for each file is small), when traversal through them, `diff` command will eat up all the system memory. My computer is 16GB RAM. And I believe this issue still exist for system with 32GB or 64GB RAM.

I would suggest that it can use more memory efficient algorithm to compare the difference between directories.

Regards!!!

Evan Greenup.




Information forwarded to bug-diffutils <at> gnu.org:
bug#71529; Package diffutils. (Fri, 14 Jun 2024 00:08:01 GMT) Full text and rfc822 format available.

Message #8 received at 71529 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Evan Greenup <evan_greenup <at> protonmail.com>
Cc: 71529 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#71529: Crash for large directory due to
 memory outage
Date: Thu, 13 Jun 2024 17:07:26 -0700
What do you mean by "crash"? Did you need to reboot the system, or did 
"diff" dump core, or did diff print a diagnostic "memory exhausxted"?

Can you send a recipe for reproducing the problem? I didn't have any 
trouble running diff 3.10 on a Fedora 40 machine with only 8 GiB RAM, in 
directories with 100,000 files. The following shell commands succeeded 
quietly:

  mkdir d e
  (cd d && touch $(seq 100000))
  (cd e && touch $(seq 100000))
  diff -r d e






This bug report was last modified 263 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.