GNU bug report logs - #49038
Speeding up diff on Windows

Previous Next

To reply to this bug, email your comments to 49038 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Gisle Vanem <gisle.vanem <at> gmail.com>
To: bug-diffutils <at> gnu.org
Subject: Speeding up diff on Windows
Date: Tue, 15 Jun 2021 13:57:49 +0200

Hello list.

I feel a 'diff --recursive ..' takes way too much time
to diff 2 trees of a few 1000 files each.

Reading the MSDN docs for '_open()', the under-laying
cache-behaviour and the 'O_SEQUENTIAL' flag, it seems
sensible that the Windows "Cache Manager" [1] works
faster with the 'O_SEQUENTIAL' flag. So with this
little pseudo-patch of 'diff.c':

+#ifndef O_SEQUENTIAL
+#define O_SEQUENTIAL 0
+#endif
       /* Open the files and record their descriptors.  */

       int oflags = O_RDONLY | (binary ? O_BINARY : 0);
+      oflags |= O_SEQUENTIAL;
+
       if (cmp.file[0].desc == UNOPENED)
         if ((cmp.file[0].desc = open (cmp.file[0].name, oflags, 0)) < 0)

I seem to get the job done quicker. I cannot give any numbers on
the speed-increase. But others may have some comments?

[1] https://docs.microsoft.com/en-us/windows/win32/fileio/file-caching

-- 
--gv

Message #8 received at 49038 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Gisle Vanem <gisle.vanem <at> gmail.com>, 49038 <at> debbugs.gnu.org
Subject: Re: [bug-diffutils] bug#49038: Speeding up diff on Windows
Date: Tue, 15 Jun 2021 12:51:16 -0700

On 6/15/21 4:57 AM, Gisle Vanem wrote:

> I seem to get the job done quicker. I cannot give any numbers on
> the speed-increase.

Although GNU/Linux doesn't have O_SEQUENTIAL, it has somewhat similar 
flags (e.g., madvise with MADV_SEQUENTIAL). I've tried using these flags 
in the past and came up empty: they complicate the code and sometimes 
make things faster and sometimes slower and it wasn't worth the 
maintenance trouble to tease things out.

I expect that Microsoft's O_SEQUENTIAL flag is similar. Besides, 
MS-Windows is relatively low priority for us.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.