GNU bug report logs - #53951
[PATCH] m4: speed up filesystem modification checks

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 53951 in the body.
You can then email your comments to 53951 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mike Frysinger <vapier <at> gentoo.org>
To: automake-patches <at> gnu.org
Subject: [PATCH] m4: speed up filesystem modification checks
Date: Sat, 12 Feb 2022 05:06:07 -0500

The current code sleeps at least 1 second to make sure the generated
files are strictly newer than the source files.  It does this for a
few reasons: POSIX only guarantees that `sleep` accept integers, and
filesystems have a history (c.f. Windows) of bad timestamp resolution.

For the first part, we can easily probe sleep to see if it accepts a
decimal number with a fractional part -- just run `sleep 0.001`.

For the second part, we can create two files and then run sleep in a
loop to see when one is considered newer than the other.

For many projects, this 1 second delay is largely amortized by the
rest of the configure script.  Autoconf lends itself to being both
large & slow.  But in projects with many smallish configure scripts
with many cached vars, the time to rerun is dominated by this single
sleep call.  For example, building libgloss against a compiler with
many (60+) multilib configurations, we see:
[Using sleep 1]
$ time ./config.status
real    2m28.164s
user    0m33.651s
sys     0m9.083s
[Using sleep 0.1]
$ time ./config.status
real    0m39.569s
user    0m33.517s
sys     0m8.969s

And in case anyone wonders, going below 0.1s doesn't seem to make a
statistically significant difference, at least in this configuration.
It appears to be within "noise" limits.
[Using sleep 0.001]
$ time ./config.status
real    0m39.760s
user    0m33.342s
sys     0m9.080s

* m4/sanity.m4: Determine whether `sleep` accepts fractional seconds.
Determine (roughly) the filesystem timestamp resolution.  Use this to
sleep less when waiting for generated file timestamps to update.
---
 m4/sanity.m4 | 48 +++++++++++++++++++++++++++++++++++++++++++++---
 1 file changed, 45 insertions(+), 3 deletions(-)

diff --git a/m4/sanity.m4 b/m4/sanity.m4
index 4e44dd9c4096..5aa1e3ad8baf 100644
--- a/m4/sanity.m4
+++ b/m4/sanity.m4
@@ -6,10 +6,52 @@
 # gives unlimited permission to copy and/or distribute it,
 # with or without modifications, as long as this notice is preserved.
 
+# _AM_SLEEP_FRACTIONAL_SECONDS
+# ----------------------------
+AC_DEFUN([_AM_SLEEP_FRACTIONAL_SECONDS], [dnl
+AC_CACHE_CHECK([whether sleep supports fractional seconds], am_cv_sleep_fractional_seconds, [dnl
+AS_IF([sleep 0.001 2>/dev/null], [am_cv_sleep_fractional_seconds=true], [am_cv_sleep_fractional_seconds=false])
+])])
+
+# _AM_FILESYSTEM_TIMESTAMP_RESOLUTION
+# -----------------------------------
+# Determine the filesystem timestamp resolution.  Modern systems are nanosecond
+# capable, but historical systems could be millisecond, second, or even 2-second
+# resolution.
+AC_DEFUN([_AM_FILESYSTEM_TIMESTAMP_RESOLUTION], [dnl
+AC_REQUIRE([_AM_SLEEP_FRACTIONAL_SECONDS])
+AC_CACHE_CHECK([the filesystem timestamp resolution], am_cv_filesystem_timestamp_resolution, [dnl
+# Use names that lexically sort older-first when the timestamps are equal.
+rm -f conftest.file.a conftest.file.b
+: > conftest.file.a
+AS_IF([$am_cv_sleep_fractional_seconds], [dnl
+  am_try_sleep=0.1 am_try_loops=20
+], [dnl
+  am_try_sleep=1   am_try_loops=2
+])
+am_try=0
+while :; do
+  AS_VAR_ARITH([am_try], [$am_try + 1])
+  echo "timestamp $am_try" > conftest.file.b
+  set X `ls -t conftest.file.a conftest.file.b`
+  if test "$[2]" = conftest.file.b || test $am_try -eq $am_try_loops; then
+    break
+  fi
+  sleep $am_try_sleep
+done
+rm -f conftest.file.a conftest.file.b
+am_cv_filesystem_timestamp_resolution=$am_try
+AS_IF([$am_cv_sleep_fractional_seconds], [dnl
+  AS_VAR_ARITH([am_cv_filesystem_timestamp_resolution], [$am_try / 10])
+  AS_VAR_ARITH([am_fraction], [$am_try % 10])
+  AS_VAR_APPEND([am_cv_filesystem_timestamp_resolution], [.$am_fraction])
+])
+])])
+
 # AM_SANITY_CHECK
 # ---------------
 AC_DEFUN([AM_SANITY_CHECK],
-[dnl
+[AC_REQUIRE([_AM_FILESYSTEM_TIMESTAMP_RESOLUTION])
 rm -f conftest.file
 AC_CACHE_CHECK([whether build environment is sane], am_cv_build_env_is_sane, [dnl
 # Reject unsafe characters in $srcdir or the absolute working directory
@@ -53,7 +95,7 @@ if (
        break
      fi
      # Just in case.
-     sleep 1
+     sleep $am_cv_filesystem_timestamp_resolution
      am_has_slept=yes
    done
    test "$[2]" = conftest.file
@@ -69,7 +111,7 @@ fi
 # generated files are strictly newer.
 am_sleep_pid=
 if ! test -e conftest.file || grep 'slept: no' conftest.file >/dev/null 2>&1; then
-  ( sleep 1 ) &
+  ( sleep $am_cv_filesystem_timestamp_resolution ) &
   am_sleep_pid=$!
 fi
 AC_CONFIG_COMMANDS_PRE(
-- 
2.34.1

Message #8 received at 53951 <at> debbugs.gnu.org (full text, mbox):

From: Jim Meyering <jim <at> meyering.net>
To: Mike Frysinger <vapier <at> gentoo.org>
Cc: 53951 <at> debbugs.gnu.org
Subject: Re: [bug#53951] [PATCH] m4: speed up filesystem modification checks
Date: Sat, 12 Feb 2022 13:51:01 -0800

On Sat, Feb 12, 2022 at 2:07 AM Mike Frysinger <vapier <at> gentoo.org> wrote:
> The current code sleeps at least 1 second to make sure the generated
> files are strictly newer than the source files.  It does this for a
> few reasons: POSIX only guarantees that `sleep` accept integers, and
> filesystems have a history (c.f. Windows) of bad timestamp resolution.
>
> For the first part, we can easily probe sleep to see if it accepts a
> decimal number with a fractional part -- just run `sleep 0.001`.
>
> For the second part, we can create two files and then run sleep in a
> loop to see when one is considered newer than the other.
>
> For many projects, this 1 second delay is largely amortized by the
> rest of the configure script.  Autoconf lends itself to being both
> large & slow.  But in projects with many smallish configure scripts
> with many cached vars, the time to rerun is dominated by this single
> sleep call.  For example, building libgloss against a compiler with
> many (60+) multilib configurations, we see:
> [Using sleep 1]
> $ time ./config.status
> real    2m28.164s
> user    0m33.651s
> sys     0m9.083s
> [Using sleep 0.1]
> $ time ./config.status
> real    0m39.569s
> user    0m33.517s
> sys     0m8.969s
>
> And in case anyone wonders, going below 0.1s doesn't seem to make a
> statistically significant difference, at least in this configuration.
> It appears to be within "noise" limits.
> [Using sleep 0.001]
> $ time ./config.status
> real    0m39.760s
> user    0m33.342s
> sys     0m9.080s
>
> * m4/sanity.m4: Determine whether `sleep` accepts fractional seconds.
> Determine (roughly) the filesystem timestamp resolution.  Use this to
> sleep less when waiting for generated file timestamps to update.

Nice work. I looked through the patch and didn't see any issue.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.