GNU bug report logs - #53313
[ERT] EMACS_TEST_VERBOSE can not be explicitly disabled

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 53313 in the body.
You can then email your comments to 53313 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Max Nikulin <manikulin <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled
Date: Mon, 17 Jan 2022 17:59:33 +0700

At least empty value of EMACS_TEST_VERBOSE environment variable
should be considered as false value that disables verbose ERT summary.

Currently ERT implementation does not distinguish non-empty and empty
values of the variable and generates verbose summary when such
variable is just present in the process environment.

To improve user experience some strings widely used in configuration 
files as false values should be recognized by ERT as well. Examples 
(case insensitive): "0", "false", "N", "no", "off", "none", "nil".

Recently Ihor Radchenko proposed to enable verbose summary mode by 
default for Org Mode tests when it is built as a standalone project 
outside of Emacs source tree:
https://lists.gnu.org/archive/html/emacs-orgmode/2022-01/msg00022.html
I like the idea, but I can not say the same concerning implementation. 
Ihor introduced new make variable while I believe that it is not 
necessary and Org makefiles should be transparent in relation to 
EMACS_TEST_VERBOSE.

I expect that it should be enough to add
    EMACS_TEST_VERBOSE ?= yes
    export EMACS_TEST_VERBOSE
to the default.mk file
https://git.savannah.gnu.org/cgit/emacs/org-mode.git/tree/mk/default.mk

Actually with current implementation developer has no chance to disable
verbose summary mode from command line since empty string is considered
as true value by ERT implementation. The following does not work:

    EMACS_TEST_VERBOSE= make test
    make test EMACS_TEST_VERBOSE=

A workaround is "unexport EMACS_TEST_VERBOSE" either in the local.mk
file or wrapped with ifeq-else-endif in default.mk.

Another case is that a developer may have
"export EMACS_TEST_VERBOSE=yes" in their shell init file
but sometimes wants to disable verbose summary for a couple of commands
without changing of environment by "unset EMACS_TEST_VERBOSE"
that will affect following commands. The following commands
should cause different states of verbose summary setting
independently of current environment:

    make test EMACS_TEST_VERBOSE=
    make test EMACS_TEST_VERBOSE=yes

Standalone example: ert-sample.el
---- >8 ----
(require 'ert)
(ert-deftest ert-sample ()
  (should (equal 5 (* 2 2))))
(ert-run-tests-batch-and-exit)
---- 8< ----

    export EMACS_TEST_VERBOSE=yes  # in ~/.profile, ~/.bashrc, etc.
    EMACS_TEST_VERBOSE= emacs --batch -Q ert-sample.el

I have not found any discussion of design choices related to this variable.

To wrap up, I suppose that at least empty value of EMACS_TEST_VERBOSE
should be considered as a request to generate brief summary.
It would be great to have opportunity to explicitly say "no":
EMACS_TEST_VERBOSE=off (or "false", "0", etc.).
It will make it easier for various build tools to respect
developer's choice.

Message #8 received at 53313 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Max Nikulin <manikulin <at> gmail.com>
Cc: 53313 <at> debbugs.gnu.org
Subject: Re: bug#53313: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled
Date: Fri, 21 Jan 2022 12:32:20 +0100

Max Nikulin <manikulin <at> gmail.com> writes:

> At least empty value of EMACS_TEST_VERBOSE environment variable
> should be considered as false value that disables verbose ERT summary.

I've now fixed this in Emacs 29.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #13 received at 53313 <at> debbugs.gnu.org (full text, mbox):

From: Max Nikulin <manikulin <at> gmail.com>
To: 53313 <at> debbugs.gnu.org
Subject: Re: bug#53313: [ERT] EMACS_TEST_VERBOSE can not be explicitly disabled
Date: Fri, 21 Jan 2022 21:53:45 +0700

On 21/01/2022 18:32, Lars Ingebrigtsen wrote:
> Max Nikulin writes:
> 
>> At least empty value of EMACS_TEST_VERBOSE environment variable
>> should be considered as false value that disables verbose ERT summary.
> 
> I've now fixed this in Emacs 29.

Thank you, Lars. It works accordingly may expectation in its minimal 
variant.

Just to have it clearly stated in the bug tracker: strings like "0", 
"no", etc. are considered as true value as any other non-empty strings. 
Only unset variable or empty value disables verbose summary.

Commit 4d866fc0f5

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.