GNU bug report logs - #78850
improve debuggability of installcheck failures

Previous Next

To reply to this bug, email your comments to 78850 AT debbugs.gnu.org.
There is no need to reopen the bug first.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: automake-patches <at> gnu.org
Subject: improve debuggability of installcheck failures
Date: Fri, 20 Jun 2025 16:44:52 +0200

[Message part 1 (text/plain, inline)]

Automake provides a useful 'installcheck' implementation for
packages that install programs: It runs each program twice,
once with option '--help' and once with option '--version'.

Here is a sample output:

make[2]: Entering directory '/work/gettext-2025-06-19/build/gettext-tools/src'
bad=0; pid=$$; list="msgcmp msgfmt msgmerge msgunfmt xgettext msgattrib msgcat msgcomm msgconv msgen msgexec msgfilter msggrep msginit msguniq recode-sr-latin"; for p in $list; do \
  case '  ' in \
   *" $p "* | *" ../../../gettext-tools/src/$p "*) continue;; \
  esac; \
  f=`echo "$p" | \
     sed 's,^.*/,,;s/$//;s,x,x,;s/$//'`; \
  for opt in --help --version; do \
    if "/usr/local/bin/$f" $opt >c${pid}_.out \
         2>c${pid}_.err </dev/null \
	 && test -n "`cat c${pid}_.out`" \
	 && test -z "`cat c${pid}_.err`"; then :; \
    else echo "$f does not support $opt" 1>&2; bad=1; fi; \
  done; \
done; rm -f c${pid}_.???; exit $bad
msgcmp does not support --help
msgcmp does not support --version
msgfmt does not support --help
msgfmt does not support --version
...
make[2]: *** [Makefile:5076: installcheck-binPROGRAMS] Error 1

The problem with this output is that it gives no clue regarding the reason
of the failures. Each program invocation can fail due to
  - an exit status != 0,
  - an stdout output that is empty (not expected for --help or --version), or
  - some stderr output,
but it does not tell which of these actually caused the failure. And it
erases the log files just before the end of the rule execution ('exit $bad').

Nowadays it is quite frequent to do builds on machines in the "cloud", that
is, on machines that
  - are allocated to the developer only for the time of the build,
  - don't offer the possibility for interactive login and investigation
    to the developer.
In other words, the Automake rule above was written with the assumption
"if the program invocation fails, the developer can investigate it locally".
But this assumption is not valid any more. In my case, not only I can't
login to the cloud machine. I also cannot reproduce the issue locally,
because the environment is a complex setup with docker.

So, here's a proposed patch to fix this. It adds more output (on stderr)
in the case of failure. It does not change the output in the case of success.

In the case above, the output was changed to:

make[2]: Entering directory '/work/gettext-2025-06-20/build/gettext-tools/src'
bad=0; pid=$$; list="msgcmp msgfmt msgmerge msgunfmt xgettext msgattrib msgcat msgcomm msgconv msgen msgexec msgfilter msggrep msginit msguniq recode-sr-latin"; for p in $list; do \
  case '  ' in \
   *" $p "* | *" ../../../gettext-tools/src/$p "*) continue;; \
  esac; \
  f=`echo "$p" | \
     sed 's,^.*/,,;s/$//;s,x,x,;s/$//'`; \
  for opt in --help --version; do \
    "/usr/local/bin/$f" $opt \
      >c${pid}_.out 2>c${pid}_.err </dev/null; \
    xc=$?; \
    if test -n "`cat c${pid}_.err`"; then \
      echo "$f does not support $opt: error output" 1>&2; \
      cat c${pid}_.err 1>&2; \
      bad=1; \
    else \
      if test -z "`cat c${pid}_.out`"; then \
        echo "$f does not support $opt: no output" 1>&2; \
        bad=1; \
      else \
        if test $xc != 0; then \
          echo "$f does not support $opt: exit code $xc" 1>&2; \
          bad=1; \
        else \
          :; \
        fi; \
      fi; \
    fi; \
  done; \
done; rm -f c${pid}_.???; exit $bad
msgcmp does not support --help: error output
/usr/local/bin/msgcmp: error while loading shared libraries: libgettextsrc-2025-06-20.so: cannot open shared object file: No such file or directory
msgcmp does not support --version: error output
/usr/local/bin/msgcmp: error while loading shared libraries: libgettextsrc-2025-06-20.so: cannot open shared object file: No such file or directory
msgfmt does not support --help: error output
/usr/local/bin/msgfmt: error while loading shared libraries: libgettextsrc-2025-06-20.so: cannot open shared object file: No such file or directory
msgfmt does not support --version: error output
/usr/local/bin/msgfmt: error while loading shared libraries: libgettextsrc-2025-06-20.so: cannot open shared object file: No such file or directory
...
make[2]: *** [Makefile:5080: installcheck-binPROGRAMS] Error 1

This more detailed output allowed me to understand the problem
(a missing /etc/ld.so.conf entry or a missing 'ldconfig' invocation)
and add a fix/workaround.

Bruno

[0001-Improve-debuggability-of-installcheck-failures.patch (text/x-patch, attachment)]

Message #8 received at 78850 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: bruno <at> clisp.org
Cc: 78850 <at> debbugs.gnu.org
Subject: Re: [bug#78850] improve debuggability of installcheck failures
Date: Fri, 20 Jun 2025 19:07:24 -0600

    So, here's a proposed patch to fix this. It adds more output (on
    stderr) in the case of failure. It does not change the output in the
    case of success.

I installed it. Saw nothing to change, and make distcheck still
succeeded. Thanks as always, Bruno. -k

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.