GNU bug report logs - #20831
`missing` script handling of broken timestamps

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 20831 in the body.
You can then email your comments to 20831 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Gang Chen <gchengordon <at> gmail.com>,
 "m4-discuss <at> gnu.org" <m4-discuss <at> gnu.org>, bug-automake <at> gnu.org
Subject: Re: Can m4 be built without automake?
Date: Tue, 16 Jun 2015 15:39:52 -0600

[Message part 1 (text/plain, inline)]

[re-adding the list, with permission]
[adding bug-automake]
[please don't top-post on technical lists]

On 06/16/2015 02:45 PM, Gang Chen wrote:
> Hi Eric,
> 
> Thanks for the quick answer.
> 
> What I'm trying to do is investigating if m4 can be built on the linux like
> os on IBM mainframe. I don't have m4, autoconf or automake on the system. I
> assumed m4 source code can be built without autoconf and or automake just
> as you suggested. But when I check the Makfile generated by ./configure, I
> see aclocal-1.14 is required, which is part of automake. You may check the
> Makefile attached.

It should indeed be possible to build m4 1.4.17 from a tarball without
having m4, autoconf, or automake already on the system.  Although I've
stripped the makefile, I see that it contains the following:

# Makefile.in generated by automake 1.14 from Makefile.am.
...
PACKAGE_VERSION = 1.4.17

which is the case when you are building from the 1.4.17 tarball.  Good -
you are not trying to build a development version from .git.  That
tarball should not require the autotools to be pre-installed, and even
if you botched timestamps on files when unpacking the tarball, the
package includes the 'missing' script which is supposed to work around
that situation (that is, the 'missing' script is what is supposed to let
the build proceed normally even when automake-1.14 is not present).

> 
> FSUM7351 is an error code, which means " You attempted to execute a command
> that could not be found". (
> http://www-01.ibm.com/support/docview.wss?uid=swg21396470). So the error
> actually tells us automake is not available on my system, which is true. So
> I suspect it's not a automake problem but a problem in the configure
> script, which uses automake for some purpose.

All I can conclude, then, is that you somehow botched file timestamps
when unpacking the tarball, and that you have triggered some problem in
the 'missing' script that is unable to recover gracefully, even though
it is supposed to work just fine.  Since 'missing' is a shell script
shipped by automake, you may have better luck debugging why it failed on
your system, and/or asking for help on the automake list to figure out
why the build is failing even though 'missing' is being properly invoked.

>>> make  all-recursive
>>> Making all in .
>>> CDPATH="${ZSH_VERSION+.}:" && cd . && /bin/sh
>>> /plex/gordonc/work/m4-1.4.17/build-aux/missing aclocal-1.14 -I m4
>>> aclocal-1.14: /plex/gordonc/work/m4-1.4.17/build-aux/missing 81: FSUM7351
>>> not found

This says that 'make' ran 'missing', and missing properly determined
that 'aclocal-1.14' is not present...

>>
>> This says that the 'missing' script was trying to run a program named
>> FSUM7351, which doesn't exist.  Based on later transcript lines...
>>
>>> WARNING: 'aclocal-1.14' is missing on your system.
>>>          You should only need it if you modified 'acinclude.m4' or
>>>          'configure.ac' or m4 files included by 'configure.ac'.
>>>          The 'aclocal' program is part of the GNU Automake package:
>>>          <http://www.gnu.org/software/automake>
>>>          It also requires GNU Autoconf, GNU m4 and Perl in order to run:
>>>          <http://www.gnu.org/software/autoconf>
>>>          <http://www.gnu.org/software/m4/>
>>>          <http://www.perl.org/>
>>> FSUM8226 make: Error code 127

...but that missing exited with status 127 instead of status 0.  Hmm.
Reading automake's NEWS, I see this:

> New in 1.13:
> ...
> * Automatic remake rules and 'missing' script:
> 
>   - The 'missing' script no longer tries to update the timestamp of
>     out-of-date files that require a maintainer-specific tool to be
>     remade, in case the user lacks such a tool (or has a too-old version
>     of it).  It just gives a useful warning, and in some cases also a
>     tip about how to obtain such a tool.
> 
>   - The missing script has thus become useless as a (poor) way to work
>     around the sketched-timestamps issues that can happen for projects
>     that keep generated files committed in their VCS repository.  Such
>     projects are now encouraged to write a custom "fix-timestamps.sh"
>     script to avoid such issues; a simple example is provided in the
>     "CVS and generated files" chapter of the automake manual.

Yuck.  This is a regression in behavior in upstream automake.  It means
that any automake-generated file that is unpacked with timestamp skew
will now fail to build, instead of the old behavior of faking things and
exiting with status 0.  This in turn grinds the 'make' run to a halt,
where it used to succeed.

Oh, and it points out an automake doc bug:
https://www.gnu.org/software/automake/manual/automake.html#CVS has a
mention of 'fix-timestamp.sh', but not 'fix-timestamps.sh' as called out
in NEWS.

I'd still be interested in figuring out how you managed to botch
timestamps when unpacking your tarball; can you show a fine-grained
listing of the timestamps in your working directory that is resulting in
make thinking aclocal has to be re-run?  Does your file system have
sub-second granularity?  Does your version of 'make' consider a file as
up-to-date or out-of-date if it is the same timestamp as its
prerequisite?  Once you have diagnosed where the timestamp skew is
coming from, does the advice on the automake page of running these steps
allow you to then build?

> #!/bin/sh
> # fix-timestamp.sh: prevents useless rebuilds after "cvs update"
> sleep 1
> # aclocal-generated aclocal.m4 depends on locally-installed
> # '.m4' macro files, as well as on 'configure.ac'
> touch aclocal.m4
> sleep 1
> # autoconf-generated configure depends on aclocal.m4 and on
> # configure.ac
> touch configure
> # so does autoheader-generated config.h.in
> touch config.h.in
> # and all the automake-generated Makefile.in files
> touch `find . -name Makefile.in -print`
> # finally, the makeinfo-generated '.info' files depend on the
> # corresponding '.texi' files
> touch doc/*.info

Another thing to try: can you unpack an older GNU m4 tarball, one that
was generated with automake 1.12 or newer (1.4.16 should fit the bill),
and see if that builds just fine, because it used the older semantics of
'missing' that don't kill the build?  If so, another thing to try to
pinpoint the 'missing' script as the culprit would be to reproduce the
unpacking timestamp skews in both 1.4.16 and 1.4.17 working directories,
but then swap the 'missing' script between the two trees, before trying
to build, and see if the swap causes the build failure to migrate to the
build tree with the newer version of 'missing'.

-- 
Eric Blake   eblake redhat com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #8 received at 20831 <at> debbugs.gnu.org (full text, mbox):

From: Mike Frysinger <vapier <at> gentoo.org>
To: 20831 <at> debbugs.gnu.org
Subject: [PATCH/committed] NEWS: fix typo in fix-timestamp.sh script name
Date: Sun, 20 Feb 2022 19:56:27 -0500

As pointed out by Eric Blake.

* NEWS: Fix typo.
---
 NEWS | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/NEWS b/NEWS
index 74ad6d612560..c5ccd9387c0a 100644
--- a/NEWS
+++ b/NEWS
@@ -821,7 +821,7 @@ New in 1.13:
   - The missing script has thus become useless as a (poor) way to work
     around the sketched-timestamps issues that can happen for projects
     that keep generated files committed in their VCS repository.  Such
-    projects are now encouraged to write a custom "fix-timestamps.sh"
+    projects are now encouraged to write a custom "fix-timestamp.sh"
     script to avoid such issues; a simple example is provided in the
     "CVS and generated files" chapter of the automake manual.
 
-- 
2.34.1

Message #11 received at 20831 <at> debbugs.gnu.org (full text, mbox):

From: Mike Frysinger <vapier <at> gentoo.org>
To: 20831 <at> debbugs.gnu.org
Subject: Re: Can m4 be built without automake?
Date: Mon, 21 Feb 2022 00:57:23 +0000 (UTC)

retitle 20831 `missing` script handling of broken timestamps
tag 20831 = confirmed
thankyou

Message #20 received at 20831 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: eblake <at> redhat.com
Cc: m4-discuss <at> gnu.org, 20831 <at> debbugs.gnu.org, gchengordon <at> gmail.com
Subject: Re: bug#20831: Can m4 be built without automake?
Date: Tue, 25 Feb 2025 10:14:25 -0700

Hi Eric and all - back on this report from 2015 (sorry):

    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=20831

    This is a regression in behavior in upstream automake.  It means
    that any automake-generated file that is unpacked with timestamp skew
    will now fail to build

Yes. I'm afraid this is a wontfix. Stefano L wrote a long description of
why he changed the behavior of missing to no longer touch files (copied
below).

I'm not sure I entirely buy it, but it wasn't just a capricious change,
and I'm not inclined to reverse it and go back to the problems that the
"touch version" of missing had. Especially since this report is 10 years
old, not followed up on, and no other reports have surfaced, as far as
far as I can remmber.

Best,
karl


2012-06-21  Stefano Lattarini  <stefano.lattarini <at> gmail.com>

	missing: do not touch timestamps; only warn for out-of-date files

	Before this change, the missing script had a twofold role:

	  - it warned the user if some required maintainer tools was missing,
	    or too old;

	  - in such a case, it tried to "fix" the timestamp of the files that
	    should have been rebuilt by that tool (without actually updating
	    the file contents, of course), to allow the build to continue.

	The second capability used to be quite useful in the days when most
	projects committed files generated by maintainer-only tools in their
	VCS repository (today the trend is not to keep such generated files
	VCS-committed anymore).  In such a setup, the "timestamp-fixing"
	capability of 'missing' was quite useful, in that it allowed users
	lacking some required maintainer tool to build from a VCS checkout
	in the face of skewed timestamps (as could have been caused by
	"cvs update" or "git checkout").

	But then, when the automatic remake rules kicked in due to the
	generated files being *actually out-of-date* (e.g., because the user
	had modified 'configure.ac' but lacked a modern-enough autoconf to
	rebuild it), that behaviour of 'missing' caused the same problem that
	plagued AM_MAINTAINER_MODE; i.e., the user would get non-dependable
	builds and inconsistent statuses of the build tree -- changes to
	source files don't reflect on generated files, and this can be very
	confusing and cause hard-to-spot errors).

	So we now believe that the best approach to deal with timestamp-related
	issues is not to have 'missing' to "automagically" try to resolve
	them (with all the risk and brittleness entailed), but rather to
	suggest those projects still keeping generated files committed in their
	VCS to provide a proper (say) 'fix-timestamp.sh' script that touches
	the timestamp of the checked-out files, to ensure no spurious rebuild
	will be triggered.  As a bonus, such a script can be more aware of the
	particularities, nooks and corner cases of a project, and thus more
	reliable than the old 'missing' script.

	An example of this approach is offered by GNU awk (release 4.0.1, Git
	tag 'gawk-4.0.1', commit b85b04e8).  The GNU awk maintainers commit
	the Autotools-generated files (configure, Makefile.in, etc.) and other
	generated in the project's Git repository, but offer a useful script
	'bootstrap.sh' that fixes the timestamps of those files, to ensure no
	useless remake is triggered in a freshly cloned repository:

	    #! /bin/sh
	    # bootstrap.sh --- touch relevant files to avoid out-of-date issues
	    #                  in Git sandboxes
	    touch aclocal.m4
	    find awklib -type f -print | xargs touch
	    sleep 1
	    touch configure
	    sleep 2
	    touch configh.in
	    sleep 1
	    touch test/Maketests
	    find . -name Makefile.in -print | xargs touch
	    touch doc/*.info
	    touch po/*.gmo
	    touch po/stamp-po
	    touch awkgram.c
	    touch command.c
	    touch version.c

	A similar, simplified script is also reported as an example in the
	manual.

	* NEWS: Update.
	* doc/automake.texi (Auxiliary Programs, maintainer-mod): Update.
	(CVS): Do not suggest that 'missing' can "automagically" fix botched
	timestamp due to a "cvs update"; this isn't true anymore.  Give an
	example of a custom 'fix-timestamp.sh' script that can be used to
	obtain the same effect (in a slightly more laborious but also more
	explicit and less brittle way).  Fix some minor typos and improper
	wordings while we are at it.
	* lib/missing: Basically rewritten to implement the new semantics.
	As a side effect (one of the several), the '--run' option is no more
	required nor recognized.
	* m4/missing.m4 (AM_MISSING_HAS_RUN): Enhance the test on '$MISSING'
	to ensure it actually provides the new semantics (by trying the new
	"witness" option '--is-lightweight').
	* lib/am/distdir.am (distdir): No need anymore to check for "bad"
	distributed man pages that were actually dummy stubs generated by
	the 'missing' script.
	* t/missing4.sh: Rename ...
	* t/remake-aclocal-version-mismatch.sh: ... like this.
	* t/missing2.sh: Rename ...
	* t/missing-version-mismatch.sh: ... like this, and adjust to the
	new semantics.
	* t/missing3.sh: Adjust to the new semantics.
	* t/man4.sh: Remove as obsolete.
	* t/missing.sh: Likewise.
	* t/missing5.sh: Likewise.
	* t/txinfo30.sh: Likewise.
	* t/man6.sh: Adjust grepping checks.
	* t/remake6.sh: Likewise.
	* t/list-of-tests.mk: Adjust.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.