GNU logs - #59317, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Wed, 16 Nov 2022 18:53:01 +0000
Resent-Message-ID: <handler.59317.B.166862476418173 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 59317 <at> debbugs.gnu.org
From: Matt Armstrong <matt@HIDDEN>
Date: Wed, 16 Nov 2022 10:52:11 -0800
Message-ID: <87r0y268bo.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=2001:4b98:dc4:8::230; envelope-from=matt@HIDDEN;
 helo=relay10.mail.gandi.net
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

This is a follow up to bug#59028.

A good number of Emacs tests exercise parts of Emacs that call
`message'.

When ERT runs tests interactively the `message' output is not normally
visible in the "*ert*" buffer.  In fact ERT arranges for the messages to
not even appear in the minibuffer as they happen.  They are instead
available on demand with the
`ert-results-pop-to-messages-for-test-at-point' command, bound to 'm' in
"*ert*".

When ERT runs tests in batch mode, messages are printed to the console
intermixed with ERT's progress messages.

Idea: hide `message' output when running in batch mode, printing them
only for failed tests.

Rationale: for passing tests the output is not useful.  For failing
tests it can be useful as a kind of trace, so the writing tests that
avoid all calls to `message' is not necessarily the best option.

This would also remove the need for a long game of whack-a-mole to
suppress such output, which leads to specific fixes like Eli's commit:

0a26b26217 (Reduce buffer-tests noisiness even more, 2022-11-16)

In GNU Emacs 29.0.50 (build 9, x86_64-pc-linux-gnu, GTK+ Version
 3.24.34, cairo version 1.16.0) of 2022-11-07 built on naz
Repository revision: d04433b96215d7d3387573f19cc315de86f2341a
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12201003
System Description: Debian GNU/Linux bookworm/sid

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Matt Armstrong <matt@HIDDEN>
Subject: bug#59317: Acknowledgement (29.0.50; Feature idea: suppress
 `message' output in ert batch test output)
Message-ID: <handler.59317.B.166862476418173.ack <at> debbugs.gnu.org>
References: <87r0y268bo.fsf@HIDDEN>
X-Gnu-PR-Message: ack 59317
X-Gnu-PR-Package: emacs
Reply-To: 59317 <at> debbugs.gnu.org
Date: Wed, 16 Nov 2022 18:53:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 59317 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
59317: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D59317
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Wed, 16 Nov 2022 19:55:02 +0000
Resent-Message-ID: <handler.59317.B59317.1668628458753 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Matt Armstrong <matt@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Date: Wed, 16 Nov 2022 21:54:09 +0200
Message-Id: <834juyfzfi.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87r0y268bo.fsf@HIDDEN> (message from Matt Armstrong on Wed,
 16 Nov 2022 10:52:11 -0800)
References: <87r0y268bo.fsf@HIDDEN>
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

> From: Matt Armstrong <matt@HIDDEN>
> Date: Wed, 16 Nov 2022 10:52:11 -0800
> 
> A good number of Emacs tests exercise parts of Emacs that call
> `message'.

Are you sure?  Many Emacs features display text that doesn't go
through 'message'.

> Idea: hide `message' output when running in batch mode, printing them
> only for failed tests.

We have set-message-function that could be used for this purpose, I
think?  If indeed 'message' is the culprit.

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Stefan Kangas <stefankangas@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 00:13:02 +0000
Resent-Message-ID: <handler.59317.B59317.16686439651381 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Eli Zaretskii <eliz@HIDDEN>, Matt Armstrong <matt@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
From: Stefan Kangas <stefankangas@HIDDEN>
In-Reply-To: <834juyfzfi.fsf@HIDDEN>
References: <87r0y268bo.fsf@HIDDEN> <834juyfzfi.fsf@HIDDEN>
MIME-Version: 1.0
Date: Wed, 16 Nov 2022 16:12:36 -0800
Message-ID: <CADwFkmmUOLNvTqxv8=qBodqxZe77qTDkR5VCtMYgz=fdbttp4g@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Eli Zaretskii <eliz@HIDDEN> writes:

>> A good number of Emacs tests exercise parts of Emacs that call
>> `message'.
>
> Are you sure?  Many Emacs features display text that doesn't go
> through 'message'.
>
>> Idea: hide `message' output when running in batch mode, printing them
>> only for failed tests.
>
> We have set-message-function that could be used for this purpose, I
> think?  If indeed 'message' is the culprit.

Is there any way to capture and stop *all* output in the terminal for
passing tests?  Like `set-batch-output-function', or something to that
effect.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 05:53:02 +0000
Resent-Message-ID: <handler.59317.B59317.166866432110218 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166866432110218
          (code B ref 59317); Thu, 17 Nov 2022 05:53:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 05:52:01 +0000
Received: from localhost ([127.0.0.1]:58815 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovXof-0002ef-9B
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 00:52:01 -0500
Received: from relay5-d.mail.gandi.net ([217.70.183.197]:46909)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <matt@HIDDEN>) id 1ovXob-0002eN-PC
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 00:52:00 -0500
Received: (Authenticated sender: matt@HIDDEN)
 by mail.gandi.net (Postfix) with ESMTPSA id 37BB01C0002;
 Thu, 17 Nov 2022 05:51:48 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
 t=1668664311;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=/rNDqS65D8h+GbzCn/1MOKiEa78bLzC/ZKCwLC0vAxc=;
 b=GbFFOWUQTQPBBwaozu64V3NqAZf1bExRtUeQbVu6zd3I/3g7JG1fJliEXzcUJuCc21Le6b
 F7JkMK/EYBfJGyro4f65srBZU6yRmPCRDjo/zRdRKWnuag6n+kSoP9X/xGJXb7In+UGjBX
 zQFn0IMSLBfUvc0RSvsm4u6SYF2+W35hyvcKKxGQeEr10bjwe0Cj+pS9KtQLUqitflJj40
 zAf5tl+i2DF/QF0086ytnULviZQ38CgvGwWh3WnOe8iOeKfjTWkmZcrUHoRW+TNXfiK8l3
 1aN8tag2YrFJtrnKPjK+jUJpo53FFvatHQ7kgVyiFpWNW39XxJKhmTKiuzu7FA==
Received: by mac-mini.lan (Postfix) with ESMTPS id 3881139080;
 Wed, 16 Nov 2022 21:51:46 -0800 (PST)
Received: by naz.lan (Postfix, from userid 1000)
 id 12B434143152; Wed, 16 Nov 2022 21:51:46 -0800 (PST)
From: Matt Armstrong <matt@HIDDEN>
In-Reply-To: <834juyfzfi.fsf@HIDDEN>
References: <87r0y268bo.fsf@HIDDEN> <834juyfzfi.fsf@HIDDEN>
Date: Wed, 16 Nov 2022 21:51:46 -0800
Message-ID: <87mt8qruvh.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Eli Zaretskii <eliz@HIDDEN> writes:

>> From: Matt Armstrong <matt@HIDDEN>
>> Date: Wed, 16 Nov 2022 10:52:11 -0800
>>
>> A good number of Emacs tests exercise parts of Emacs that call
>> `message'.
>
> Are you sure?  Many Emacs features display text that doesn't go
> through 'message'.
>
>> Idea: hide `message' output when running in batch mode, printing them
>> only for failed tests.
>
> We have set-message-function that could be used for this purpose, I
> think?  If indeed 'message' is the culprit.

It looks like the set-message-function(s) hooks are bypassed in
non-interactive mode.  In that case message3_nolog in xdisp.c calls
message_to_stderr, instead of calling set_message.  It is set_message
that ultimately uses the set-message-function machinery.

After looking at this more I agree with your suspicions.  It looks like
good amount of C level code calls various message functions without
going through Fmessage.  Also, I spot no easy way to modify ert.el to
temporarily redirect stderr elsewhere.  In noninteractive mode Emacs
seems hard coded to print to stderr.

I proposed this thinking it would be an easy and simple change.  At this
point I'm thinking the effort and complexity here isn't worth it.  If
nobody else wants to champion this idea I'll go ahead and close it.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 06:07:02 +0000
Resent-Message-ID: <handler.59317.B59317.166866517911630 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Stefan Kangas <stefankangas@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166866517911630
          (code B ref 59317); Thu, 17 Nov 2022 06:07:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 06:06:19 +0000
Received: from localhost ([127.0.0.1]:58839 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovY2V-00031W-AH
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 01:06:19 -0500
Received: from relay2-d.mail.gandi.net ([217.70.183.194]:47985)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <matt@HIDDEN>) id 1ovY2T-00031H-Bg
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 01:06:17 -0500
Received: (Authenticated sender: matt@HIDDEN)
 by mail.gandi.net (Postfix) with ESMTPSA id 47BB040002;
 Thu, 17 Nov 2022 06:06:08 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
 t=1668665170;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=4CYZ1nPTi2qigdL8CfFnF1J9qETrhx8GQ6vGvcw1+mE=;
 b=lL/i5Cw6Th9a2Nsa+EK+xc85NOMtv1r/zyp0UXnGm8wQxTPJ09Oaup9SUqTL1C8ottLofJ
 VrYS1m6/FoFrZW6fTY1wwvYDjmjkd60lra9+gftTgVPHs7aRvy6IPhuZaR/237+a0twhI3
 +cFyttiN2xPUU7ln8a1KkcGpjL83oW/yJhcwrBlpgB4B9Vl/ay0Rboo/LmqffuGaazeJvG
 nmE3kD4qzGaZmL06JFzv5U99SYPPvVl0JSie1Fxfx+O7YHAfRciT7PGnitPXw44txZWTdo
 N0UYIHmqTD+lFwnLRionUtBJ53EhrZ2dxGYdBx0Ketp0AFiqBkTdDRk9p+ndfw==
Received: by mac-mini.lan (Postfix) with ESMTPS id 754D439081;
 Wed, 16 Nov 2022 22:06:05 -0800 (PST)
Received: by naz.lan (Postfix, from userid 1000)
 id 505E34143282; Wed, 16 Nov 2022 22:06:05 -0800 (PST)
From: Matt Armstrong <matt@HIDDEN>
In-Reply-To: <CADwFkmmUOLNvTqxv8=qBodqxZe77qTDkR5VCtMYgz=fdbttp4g@HIDDEN>
References: <87r0y268bo.fsf@HIDDEN> <834juyfzfi.fsf@HIDDEN>
 <CADwFkmmUOLNvTqxv8=qBodqxZe77qTDkR5VCtMYgz=fdbttp4g@HIDDEN>
Date: Wed, 16 Nov 2022 22:06:05 -0800
Message-ID: <87iljeru7m.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Stefan Kangas <stefankangas@HIDDEN> writes:

> Eli Zaretskii <eliz@HIDDEN> writes:
>
>>> A good number of Emacs tests exercise parts of Emacs that call
>>> `message'.
>>
>> Are you sure?  Many Emacs features display text that doesn't go
>> through 'message'.
>>
>>> Idea: hide `message' output when running in batch mode, printing them
>>> only for failed tests.
>>
>> We have set-message-function that could be used for this purpose, I
>> think?  If indeed 'message' is the culprit.
>
> Is there any way to capture and stop *all* output in the terminal for
> passing tests?  Like `set-batch-output-function', or something to that
> effect.

The call sequence to look at is:

message3_nolog:xdisp.c ->
   message_to_stderr:xdisp.c ->
   errwrite:sysdep.c

There is errstream:sysdep.c, which is where any temporary redirection of
error output could go, but there is no existing way to change it
temporarily from lisp or even C.  Further, we'd probably want to save
the output somewhere in case the test did fail (or we trust the
*Message* buffer...not sure if that is there in batch mode?).

A concern I have is suppressing useful diagnostic errors.  E.g. at any
point emacs_abort() can run kill-emacs hooks which can call `message'.
Is it worth the complexity of adding temporary output redirections, to
clean up test output, at the risk of making this kind of thing harder to
debug?

A simple approach is to run each test in an isolated subprocess, but
that would be much slower.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 06:40:02 +0000
Resent-Message-ID: <handler.59317.B59317.166866715324209 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Stefan Kangas <stefankangas@HIDDEN>
Cc: matt@HIDDEN, 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166866715324209
          (code B ref 59317); Thu, 17 Nov 2022 06:40:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 06:39:13 +0000
Received: from localhost ([127.0.0.1]:58893 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovYYK-0006IP-U8
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 01:39:13 -0500
Received: from eggs.gnu.org ([209.51.188.92]:43970)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1ovYYI-0006IB-Ih
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 01:39:11 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1ovYYB-0003RZ-GY; Thu, 17 Nov 2022 01:39:03 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=jo4CUavfSlYYpXDY6CkaMAI9KostsdtkAN+AfF7+rZQ=; b=ddNNZogWkCik
 09p2Tzj9lV8g2Z8qbdx7wYn3zD5/koA5ZaLTN8x2W2uI/tYEBUHgQpmWwmCvbhDMBrrIh/kBLYAu8
 zdq9CvZc/3ynLilq72FaSAcSJ/UVJ4G8Vz8iHhHpUjDzqkUvkZ8uNiHXaRStiXy4pDn8LnbdHh7aS
 lmbj6Bi8LE6e+BwPIDk6WyHLQkR5g8eWEzsrf8jNIAPA20T1WZ1Vp29F+R89AUr7bA3OeV8uIi99z
 5j8XTuU+aPI3WT8IgEmG9vMk9luF5k9sIzDU87zh4El6lIi9WSlEcOZCR6p+Klz5LaojLBHl/yWTz
 mrQebI9QFlzoz+WPrdRj8w==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1ovYYA-0006a7-Lm; Thu, 17 Nov 2022 01:39:03 -0500
Date: Thu, 17 Nov 2022 08:39:02 +0200
Message-Id: <83wn7udr09.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <CADwFkmmUOLNvTqxv8=qBodqxZe77qTDkR5VCtMYgz=fdbttp4g@HIDDEN>
 (message from Stefan Kangas on Wed, 16 Nov 2022 16:12:36 -0800)
References: <87r0y268bo.fsf@HIDDEN> <834juyfzfi.fsf@HIDDEN>
 <CADwFkmmUOLNvTqxv8=qBodqxZe77qTDkR5VCtMYgz=fdbttp4g@HIDDEN>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Stefan Kangas <stefankangas@HIDDEN>
> Date: Wed, 16 Nov 2022 16:12:36 -0800
> Cc: 59317 <at> debbugs.gnu.org
> 
> Is there any way to capture and stop *all* output in the terminal for
> passing tests?  Like `set-batch-output-function', or something to that
> effect.

No, not that I know of.  We have several functions that will write to
the terminal in batch mode, and they use different low-level
interfaces to do that.  Also, some of the output goes to stdout and
some to stderr.  You can always redirect these two streams to files,
of course.

Besides, we don't want to stop _everything_: the messages that
announce the tests that passed and the time it took to run each test
are useful and should not be shut up.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 07:14:02 +0000
Resent-Message-ID: <handler.59317.B59317.166866920227644 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Matt Armstrong <matt@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166866920227644
          (code B ref 59317); Thu, 17 Nov 2022 07:14:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 07:13:22 +0000
Received: from localhost ([127.0.0.1]:58946 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovZ5O-0007Bo-Fo
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 02:13:22 -0500
Received: from eggs.gnu.org ([209.51.188.92]:46592)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1ovZ5L-0007BY-Ey
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 02:13:20 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1ovZ5E-0002HH-KE; Thu, 17 Nov 2022 02:13:13 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=References:Subject:In-Reply-To:To:From:Date:
 mime-version; bh=zdfrGSAUoROMrKunLqny1QwkSlKQ9yo8REVuvoTfMAY=; b=B5Q9YiPjlK7Y
 OphY9Dmv/ATeucFm818vccU3phxi7F+Z4OAj2JO7IMxdpTEOdHiVo413sQSbBF1BRY8Dp4WkOdabb
 0OR+B/oUYwkRa20VlJzN71FwGti+N8jcKT8UTCKJTYeFWADFcijoA4N+mOUSLscwr0k2Gn/XiEOKK
 Y8X+ksgiOTLXFyU2BPui3QNPAvHTH6W9M7yLkWMTKS+z3dl5u8XKqIppK/u/vH0fT7KNE1HheTWCW
 Qrg0pHHwSW5rG4Q64d/fk9HKm4KWiuG2btbf9UVa1aCMNoaZFyaCG8Ydy8cvR1QS1Tc+kfefmPOeB
 cth3TDRFyueQDEhiqf3XbA==;
Received: from [87.69.77.57] (helo=home-c4e4a596f7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1ovZ5E-0001FA-2B; Thu, 17 Nov 2022 02:13:12 -0500
Date: Thu, 17 Nov 2022 09:13:12 +0200
Message-Id: <83mt8qdpfb.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <87mt8qruvh.fsf@HIDDEN> (message from Matt Armstrong on Wed,
 16 Nov 2022 21:51:46 -0800)
References: <87r0y268bo.fsf@HIDDEN> <834juyfzfi.fsf@HIDDEN>
 <87mt8qruvh.fsf@HIDDEN>
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: Matt Armstrong <matt@HIDDEN>
> Cc: 59317 <at> debbugs.gnu.org
> Date: Wed, 16 Nov 2022 21:51:46 -0800
> 
> It looks like the set-message-function(s) hooks are bypassed in
> non-interactive mode.  In that case message3_nolog in xdisp.c calls
> message_to_stderr, instead of calling set_message.  It is set_message
> that ultimately uses the set-message-function machinery.
> 
> After looking at this more I agree with your suspicions.  It looks like
> good amount of C level code calls various message functions without
> going through Fmessage.  Also, I spot no easy way to modify ert.el to
> temporarily redirect stderr elsewhere.  In noninteractive mode Emacs
> seems hard coded to print to stderr.
> 
> I proposed this thinking it would be an easy and simple change.  At this
> point I'm thinking the effort and complexity here isn't worth it.  If
> nobody else wants to champion this idea I'll go ahead and close it.

The only idea I have is to mock-out the relevant functions in the
test.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Stefan Kangas <stefankangas@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 07:29:02 +0000
Resent-Message-ID: <handler.59317.B59317.166867009029041 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Eli Zaretskii <eliz@HIDDEN>
Cc: matt@HIDDEN, 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166867009029041
          (code B ref 59317); Thu, 17 Nov 2022 07:29:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 07:28:10 +0000
Received: from localhost ([127.0.0.1]:58961 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovZJh-0007YL-QP
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 02:28:10 -0500
Received: from mail-oa1-f50.google.com ([209.85.160.50]:37750)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stefankangas@HIDDEN>) id 1ovZJf-0007Xw-EJ
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 02:28:07 -0500
Received: by mail-oa1-f50.google.com with SMTP id
 586e51a60fabf-12c8312131fso1258214fac.4
 for <59317 <at> debbugs.gnu.org>; Wed, 16 Nov 2022 23:28:07 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=cc:to:subject:message-id:date:mime-version:references:in-reply-to
 :from:from:to:cc:subject:date:message-id:reply-to;
 bh=tKo2N4UdVb7tELXWjU2kzNW3mrtdPe025zVMGAY6Egs=;
 b=P9eiW8JLbYbESTtlZkRx2AUEfA7NZL8pc7IOENlyVVYqIOLyZ971Vvm9tAw1FsJ1Ww
 K6cGRGN7oHM9XqMBCIiMdK6zAlDITuNj3J270QyOwj0vYUFXdEls4Y8wmztS2wiuyT9/
 CwFNzxBkn1h3n2SD0vmiDA4aTxwaml5GMgvtcxKOb+uiw6tPzzHX0R+UeOAAhtg3YrI0
 bryKgNXLUgQuVzNV/Zjm5ISt0+YCXTb/4577S6oaAMkmNC3JOWF8xI4BK9SvOKy7KnMI
 fFUczBlM7M+jM+hYJGlJUz0GNB21ZwpjpZRyz1rxFEdZtokcObTv4BTrxWqqKHNuCBg3
 FjGg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=cc:to:subject:message-id:date:mime-version:references:in-reply-to
 :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=tKo2N4UdVb7tELXWjU2kzNW3mrtdPe025zVMGAY6Egs=;
 b=7EKpcxEAvkYYXjuaAB1FTukiRGFOJb9o7zA4xLSVqJ5rYC4OLMISyxOwkfcGhVytI1
 YPTZnN/pUwLScUR/fGjMTTswwr1ezF16/xC8IbRJf4OcYTO4UL1aNBuDnJ2XcSMlKYgE
 CiS0DcGDxu/5ij6tRId8VJCksNqA9YQew2YMY+1cFvwTsf7jz6vm8Gq4dO2Vt1W+qKco
 Col5/oadP46+E9rplthZuVrKAaqKebbwS5ZExipWkzyR72NCHNcYStsGxqlfbXxCe64u
 wcNDtFqXGLt1KOWaX7Anpek6oSVfFVp2OtYmcUleYQDOizx94RD1iXOjMcZkpFo9Z07d
 XurA==
X-Gm-Message-State: ANoB5pmVKzVciQCimbKdzHysbyuecJ+in/5MrWMUCe7m8haHTLkOgH/4
 heTbxxKlAoNN9fsjCVK6jsskTFxAgHGTKorB4dY=
X-Google-Smtp-Source: AA0mqf6WHphiI/E/sy5mTzq1DH+FhmQWf+om8K11CiuCf4TTRACLqlepqOZHAH93BvOMQbsz9fw+dEeU6p8a1rXl/+Q=
X-Received: by 2002:a05:6870:c34a:b0:13b:8a07:2a1f with SMTP id
 e10-20020a056870c34a00b0013b8a072a1fmr3725252oak.199.1668670081703; Wed, 16
 Nov 2022 23:28:01 -0800 (PST)
Received: from 753933720722 named unknown by gmailapi.google.com with
 HTTPREST; Wed, 16 Nov 2022 23:28:01 -0800
From: Stefan Kangas <stefankangas@HIDDEN>
In-Reply-To: <83wn7udr09.fsf@HIDDEN>
References: <87r0y268bo.fsf@HIDDEN> <834juyfzfi.fsf@HIDDEN>
 <CADwFkmmUOLNvTqxv8=qBodqxZe77qTDkR5VCtMYgz=fdbttp4g@HIDDEN>
 <83wn7udr09.fsf@HIDDEN>
X-Hashcash: 1:20:221117:eliz@HIDDEN::aB3Pe8Agdi6fjCBy:4hTL
MIME-Version: 1.0
Date: Wed, 16 Nov 2022 23:28:01 -0800
Message-ID: <CADwFkmnvpYX6a4b_kKq4mCb8K5spZKGkckWffZwBe=ErterBiw@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Eli Zaretskii <eliz@HIDDEN> writes:

> No, not that I know of.  We have several functions that will write to
> the terminal in batch mode, and they use different low-level
> interfaces to do that.  Also, some of the output goes to stdout and
> some to stderr.  You can always redirect these two streams to files,
> of course.
>
> Besides, we don't want to stop _everything_: the messages that
> announce the tests that passed and the time it took to run each test
> are useful and should not be shut up.

I was thinking of temporarily setting stdout and stderr to a buffer
(maybe using something like open_memstream) for the duration of each
individual test, and then print the content of that buffer only if the
test failed.

Maybe it's not worth the effort, though.  I'm also not sure how to do
the above portably.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Michael Albinus <michael.albinus@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 10:37:01 +0000
Resent-Message-ID: <handler.59317.B59317.166868137215382 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Matt Armstrong <matt@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166868137215382
          (code B ref 59317); Thu, 17 Nov 2022 10:37:01 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 10:36:12 +0000
Received: from localhost ([127.0.0.1]:59254 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovcFg-000402-7u
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 05:36:12 -0500
Received: from mout.gmx.net ([212.227.15.18]:37709)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael.albinus@HIDDEN>) id 1ovcFb-0003zm-NJ
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 05:36:11 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417;
 t=1668681361; bh=XB+nTcVmU4L+erq6wQXLxPL2Tga7X6W+p/1ogh5cPYk=;
 h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date;
 b=C3Nv45iWYsGS7tSKbAkWhPIvfc0mhW4EaXAHl+bwlJZ65vTSimyubTCS/QGc1o4gi
 UK48z9fcsHQudM7YWi4oVh8DRWFTZIMsuluB21BmteU7jFzThrox1lJjpaL1gerqKl
 iOEwT252WVvQxd0Xy49Oeg38AAHaxvaobQV4xCrlNMFfFPEXocZx+wOUCyFTP7SLBq
 VjIKsDdfsiaYLjb70vEdc9ygdjP6h6Gdg6Rc4O+rD2hhIC1WOAh1RqL1LNnPxDdwx1
 fhvR9/5hIsYcwUsKi+DDDUMZnqZnWawRGcW+y4Vy8QnRnyLRLQRH8uvFU0FJklsumC
 6FSHpPWANtpvg==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from gandalf.gmx.de ([185.89.39.6]) by mail.gmx.net (mrgmx004
 [212.227.17.190]) with ESMTPSA (Nemesis) id 1MXp5a-1oS1eP1pe8-00Y7bH; Thu, 17
 Nov 2022 11:36:01 +0100
From: Michael Albinus <michael.albinus@HIDDEN>
In-Reply-To: <87r0y268bo.fsf@HIDDEN> (Matt Armstrong's message of "Wed, 16
 Nov 2022 10:52:11 -0800")
References: <87r0y268bo.fsf@HIDDEN>
Date: Thu, 17 Nov 2022 11:36:00 +0100
Message-ID: <871qq1yijz.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:z1kc1VN82IX+hjMeho6ntGI2pkOa52nniCOPwXunew+T9Hv03+H
 TLATrVIL/00PNknS/Q4BZUHiQNONjuT8fJN2SpUkJRNLHM0WhybEllZquxCxx1VodH2IlQ2
 ouIl/qixMO43QiyCx+GVW69kWLVCJgY5qsQFz2W8wDPMPSCNLz5OwfFe173tDBVm86EVZP4
 c1Qwta/fKkFL86Vhc+DvQ==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:78UCsXZS9A8=;PMEKbx6kp6YZQB7t6TFffEzBjTr
 pboUoiEmFBRODdYQKdKIzdMCp8Yq2u2FpRez6yMPYKac1BdxcTOwzrOs8AAcZh0Xbm1rGv/Lj
 YpnSRgi5U6busGPa1LlTyf77WTXmkpUoznDK6UM1hRAbwMsucMTSPi1DECfDDNJ1eCekNCVKj
 YKIgHBoJb7ST2sDfaUXtwbP0o9CxstGQ30BsMCNGaRL9fxDBBUq/eJCIV8f1RPQreX6uJmAir
 CSgBUnQO+JplCCmUZ8sFzmMPLJlH2gsCkzFws6ARF6W/QXpqqOg/izPQ77jyqIDjql3DQYUQA
 UzHkfYaL2N6MlOvk9hQj70mXZupxxhFGBlbzush960aameHOLXdpY/c+fWGDODMkpycv71itj
 Yn+SGdH/tWM6PpaIRfxtbbfRvTu5EfZJ1iSbsAGYU/UCpYf//Mfzt8XwrGOQ63XtOQfUZToNy
 kZ8VTxHbmQ8P1qX8twQ0kUp9X3oZHVGgcyOmV/hnIF2LioxdggqkwQIJReW3WNJ7wmH1gL9jC
 7q//KpT+wbskegdLiw8ACViyc4LuqnC99kenK+DE0q8rZSRGOotlIKnlewHZzeW65Gs5W371e
 B0BEUY7B0m0Blh8UPAzL2fD7NQrH4TsU6650wAElDyLRaK7Enbw9UCK/PkwNa5cNdYvak5obt
 qIDqbdpp2omsun/9RFKeyrEHb3Ma2uXO2kDgdKP8ddPQ7h/GYI4MRXvXZcW02vYytntBoe2SG
 Cq2Hmukj+xO6nLBa68Vmk+fPJ7n3JDdlJUJf1Nv/8S64CeqpySgjtN8aaHMvDgeamoeRN0mG1
 XfPfbSLK+dfEtchdq05ZIudyGcuEi7+yi7Fm4kINjqQfZFmbmdVt4mvofOsMTunR0GXoM/JBP
 eWUlmHjjyWwh9CejazN8ypP/IbzMwLE9oS6RfajPz33jREAzs2tOBPUCmoNoSBdlBsqyL9Xrh
 NMbEPw==
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Matt Armstrong <matt@HIDDEN> writes:

Hi Matt,

> When ERT runs tests interactively the `message' output is not normally
> visible in the "*ert*" buffer.  In fact ERT arranges for the messages to
> not even appear in the minibuffer as they happen.  They are instead
> available on demand with the
> `ert-results-pop-to-messages-for-test-at-point' command, bound to 'm' in
> "*ert*".
>
> When ERT runs tests in batch mode, messages are printed to the console
> intermixed with ERT's progress messages.
>
> Idea: hide `message' output when running in batch mode, printing them
> only for failed tests.
>
> Rationale: for passing tests the output is not useful.  For failing
> tests it can be useful as a kind of trace, so the writing tests that
> avoid all calls to `message' is not necessarily the best option.

Besides the other arguments already given in this thread, I'd like to
emphasize that messages are useful even in batch mode, for successful
tests. This is the only way to study ert tests in batch mode, for
example on our CI/CD machines hydra and emba.

If messages shall be suppressed, this must be controlled by a user
option. However, I'm not in favor to do it at all.

Best regards, Michael.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Stefan Kangas <stefankangas@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 10:50:02 +0000
Resent-Message-ID: <handler.59317.B59317.166868215716780 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Michael Albinus <michael.albinus@HIDDEN>, Matt Armstrong <matt@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166868215716780
          (code B ref 59317); Thu, 17 Nov 2022 10:50:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 10:49:17 +0000
Received: from localhost ([127.0.0.1]:59275 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovcSL-0004MZ-9f
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 05:49:17 -0500
Received: from mail-oa1-f44.google.com ([209.85.160.44]:42743)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stefankangas@HIDDEN>) id 1ovcSK-0004MM-Bh
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 05:49:16 -0500
Received: by mail-oa1-f44.google.com with SMTP id
 586e51a60fabf-13ae8117023so1680873fac.9
 for <59317 <at> debbugs.gnu.org>; Thu, 17 Nov 2022 02:49:16 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=cc:to:subject:message-id:date:mime-version:references:in-reply-to
 :from:from:to:cc:subject:date:message-id:reply-to;
 bh=zbWSzEGWCtxymzn7rqoy2OXJDkhaaz8KqVxSSolairo=;
 b=hFfPamA1w7THYpTYMrao+zG1cEbQUUnZdK5J/UTsbT8rFCtg+PL46UoJvhOfGTXVDC
 oQiRS33oCpljuVYRN7+vX008wK1Z3eK/7+SqtrnwQvPfBuPPIUXNuSGvbseV32lPw9jH
 Cmo6u8+2tbxDdUdNuzMDzX2WkQsUeAvKA8kwdG5AjRw9JByZYG2ZqIw/xLk2QZIiRgCR
 xVXnZeqoMG95KbcBTslQu3ksdt9ZJ829UeESrMjGwdcM6hPXk1J4uGKxAetVpvCoSyJ3
 cFBrnk/w2ZvVIPGjryl++jzIIP4qKeKRCngyQ2GU2FtI97kJPMZzQRWKdGGNAbEILlJH
 1rew==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=cc:to:subject:message-id:date:mime-version:references:in-reply-to
 :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=zbWSzEGWCtxymzn7rqoy2OXJDkhaaz8KqVxSSolairo=;
 b=b7I8ft/uLi054umkuo/7ZADzH17EhetkQJLCGxNN6fwwVC1yEcyymXjBCbgMJoEvtL
 HE3KkprwhNG7E3VKcYJJ13jHjcwQfo+ngJ9WNZvT3s4+GlY2w3uvwvMcR3bc3jK6EBFl
 9R5WtEcHqwvseUR0bYL0aMUKRUVko90x8Y9CooPlIaE6vyFi75crmqR6E7fMKIPOkJdB
 681wTv2jYv6FB8yjVb2zv3HaJecc5sF2LseWYabNhI9libq0SlWLpm9AmmGwmX7zdlmr
 kOA1C+AuJJ0VJgl/YpyeKzEdXrGmICEExoUTwb/UkU5Mx/Yd/XwM/thEhNHNJ+2yPOzb
 pvHw==
X-Gm-Message-State: ANoB5plOzZ6DiIpZOFFiVNFrna15+dR10yCOGvkZMgGRCfTb1YYvdQZT
 vMLAv5g6Z8595JxnhQMmWhC1d9S67TJaVokeiyM=
X-Google-Smtp-Source: AA0mqf64xWloleet3jEofAhpJxVPWCmbXa4cTtnzB1UTMwrP4RKy/q8/vmr8blpn9H723AIT8A2YCn+CnYvCmuMTAAw=
X-Received: by 2002:a05:6870:c34a:b0:13b:8a07:2a1f with SMTP id
 e10-20020a056870c34a00b0013b8a072a1fmr4003613oak.199.1668682150503; Thu, 17
 Nov 2022 02:49:10 -0800 (PST)
Received: from 753933720722 named unknown by gmailapi.google.com with
 HTTPREST; Thu, 17 Nov 2022 02:49:10 -0800
From: Stefan Kangas <stefankangas@HIDDEN>
In-Reply-To: <871qq1yijz.fsf@HIDDEN>
References: <87r0y268bo.fsf@HIDDEN> <871qq1yijz.fsf@HIDDEN>
X-Hashcash: 1:20:221117:michael.albinus@HIDDEN::Vx5/gnsGrk6Ulzxx:6oZY
MIME-Version: 1.0
Date: Thu, 17 Nov 2022 02:49:10 -0800
Message-ID: <CADwFkmk+FEJ_k_UBQ6RLztyEAMsdMd5cNamrEOYK-g4DLjos1A@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Michael Albinus <michael.albinus@HIDDEN> writes:

> Besides the other arguments already given in this thread, I'd like to
> emphasize that messages are useful even in batch mode, for successful
> tests. This is the only way to study ert tests in batch mode, for
> example on our CI/CD machines hydra and emba.

It would be less than useful in CI, indeed.

But in interactive sessions, and when running tests manually with
e.g. M-x compile, it helps to have less noise, to find the relevant
failures faster.

> If messages shall be suppressed, this must be controlled by a user
> option. However, I'm not in favor to do it at all.

Agreed.  Perhaps an environment variable could also be provided, so that
it is easier to control from the command line?

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Michael Albinus <michael.albinus@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 17 Nov 2022 12:37:02 +0000
Resent-Message-ID: <handler.59317.B59317.166868856521528 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Stefan Kangas <stefankangas@HIDDEN>
Cc: Matt Armstrong <matt@HIDDEN>, 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.166868856521528
          (code B ref 59317); Thu, 17 Nov 2022 12:37:02 +0000
Received: (at 59317) by debbugs.gnu.org; 17 Nov 2022 12:36:05 +0000
Received: from localhost ([127.0.0.1]:59503 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ove7g-0005bA-Jy
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 07:36:04 -0500
Received: from mout.gmx.net ([212.227.17.21]:58667)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <michael.albinus@HIDDEN>) id 1ove7d-0005af-QF
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 07:36:03 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=gmx.de; s=s31663417;
 t=1668688554; bh=RoJCCxXvNUDl3b0yufARBjwLXY0QHiMeo91mK03cGWs=;
 h=X-UI-Sender-Class:From:To:Cc:Subject:In-Reply-To:References:Date;
 b=B1tfmEnbTecTcHUuYh77sC/dP1hfi0EcLltMLS8zSmC0t7cdoDzPSTS5rHkfe4csS
 fm24e5nqLMeU2UmO671b2qnZcyqR+7M82OzJ6uxrFjag//GBXwOItrGlDNFrFrU/Xu
 wKz2CCKTBbIiMACOb6P3HGfyHAfRkoMAmSrOyUpT0cjANbV4s4/oPoKH1EaAO0vaOa
 GZ3Q4N47UAjeTXX0IHxq7JFkvQT6Ma6Ig3AjUnWnrMsLrMJ9hlBmzfKiXA2SEqx/ym
 9YoXB7izokzxBCWuQzG3LOQ0hRMniCmYe4i06ZVZzd+x8EDhCLH16/TIISwRPxR87T
 yIrQUmiBkmA2Q==
X-UI-Sender-Class: 724b4f7f-cbec-4199-ad4e-598c01a50d3a
Received: from gandalf.gmx.de ([185.89.39.6]) by mail.gmx.net (mrgmx105
 [212.227.17.168]) with ESMTPSA (Nemesis) id 1MhlKy-1pQb9u24u0-00dqot; Thu, 17
 Nov 2022 13:35:54 +0100
From: Michael Albinus <michael.albinus@HIDDEN>
In-Reply-To: <CADwFkmk+FEJ_k_UBQ6RLztyEAMsdMd5cNamrEOYK-g4DLjos1A@HIDDEN>
 (Stefan Kangas's message of "Thu, 17 Nov 2022 02:49:10 -0800")
References: <87r0y268bo.fsf@HIDDEN> <871qq1yijz.fsf@HIDDEN>
 <CADwFkmk+FEJ_k_UBQ6RLztyEAMsdMd5cNamrEOYK-g4DLjos1A@HIDDEN>
Date: Thu, 17 Nov 2022 13:35:53 +0100
Message-ID: <87wn7twyfq.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain
X-Provags-ID: V03:K1:YMfwEiZizgRym1zYafnLlcC4vLC2ZjF+ddH7Hy/GwbEDOpYg9b1
 ZlTvdX+i0w10nmuHPwuzR5IpSc+442SDiKspzWhDXyv0EB5hqFoqFvnsjDOWCtnHutkWaN+
 Uw2cpTBSCTZlpp3ZhbLdcanEUFBocMC495o3gSFQ7i3iP9Lgb9nCsVkAq3nqPdjGdlMt7BO
 F3f5/k5Xa11pfZ/XsagtA==
X-Spam-Flag: NO
UI-OutboundReport: notjunk:1;M01:P0:RmZgeyLhhN8=;TDNW9S4yel4sJowqcxxrAbRC2x9
 /CRaMeviuS7VLG5Pd4Tek4Ku+4E6EZ2xe4DctMY9+iZeuRk8/Tk1BIcgdiSCGobOuL6SD5r9P
 5HAWnekdOqVKB3SIPqMqoblsqum8s+bBMOsmTTXoRGdKWBCESOrdVTYQcaiHQALxmAAHwKEja
 CUKYTPN46UueUamuLh8me+jSgvTByt0NZtcMYLI9e4x9x9eczf0+lS1E7KnOvwExrVpp+i52d
 qLerpwtyl7+Oea3BVDuJcul5PLFdKwCoLP20xI67jTxDT/cgVbsaynEJaJ/0UxFmrdpNExjzu
 b71zhHP7Pyaw7HPsZWqk9MHY45Z+/+QIQfECXb4NYfLoTc0gqLh1tgAj6p095/w+m3cNxwODi
 AGUVlqU0jm+Bk2smCOegj6OA83qPoL+jeGTi0sHSplPC4sqckjPeiOh2lF4ropX6d+PZmkWIo
 jt+nmIhzRjauJZye9Krz3AGuZSznalW/64YcAy9zPfBjVuF8DRTYzffAOERZ6csP3AzWOr7tA
 GyRmtrtQgzcK1tS01Sgu7Sr7ZFW4t8HFrZ1eD2NxJQty1hZOFf1ZOV5jqViVIs5w0zrA26g0v
 571bldsym0FvMT/590NepArou8rH4fzANrYMgufHB0WgnYa4G5xh7/DtAjsvLKKYA2zM7iXYG
 AiZ3J+lCqrzMpbzodkBrL4dJVe6n2A3H1KZxL85E0xvpgwBDqWdRH4pX8vGb3dA+jCddoPMgQ
 +9fwN0RJ5vP6NTU/aMoBTy60CZG7jkxYx7nHl3QLkG+9PUgvAe37alEQfVD1XJyGCjacJ2YUg
 t6Ib22eMBYGBI0gPZu5GMD3GppqkXWRl3BDIhA9jlAYvFdACxtqPjArhhneZ+DxxvApHIVVI9
 g9SdJjbnEZJvtFqzE9g4xEbl+NH7/WBab1lRV23RxTKSGpfoV+5gu5aS4hfLi/ZzyW5eVD/+t
 sfHeAw==
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Stefan Kangas <stefankangas@HIDDEN> writes:

Hi Stefan,
>> Besides the other arguments already given in this thread, I'd like to
>> emphasize that messages are useful even in batch mode, for successful
>> tests. This is the only way to study ert tests in batch mode, for
>> example on our CI/CD machines hydra and emba.
>
> It would be less than useful in CI, indeed.
>
> But in interactive sessions, and when running tests manually with
> e.g. M-x compile, it helps to have less noise, to find the relevant
> failures faster.

Maybe, although the messages in interactive mode go to a special *ERT
Messages* buffer.

Btw, there are test cases which depend on proper messages. See for
example autorevert-tests.el. For those tests, messages must not be
suppressed at all.

>> If messages shall be suppressed, this must be controlled by a user
>> option. However, I'm not in favor to do it at all.
>
> Agreed.  Perhaps an environment variable could also be provided, so that
> it is easier to control from the command line?

Sure. There are already several $EMACS_TEST_* environment variables, see
test/README.

Best regards, Michael.

Message sent to bug-gnu-emacs@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#59317: 29.0.50; Feature idea: suppress `message' output in ert batch test output
Resent-From: Matt Armstrong <matt@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 18 Nov 2022 00:13:01 +0000
Resent-Message-ID: <handler.59317.B59317.16687303673144 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 59317
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Michael Albinus <michael.albinus@HIDDEN>, Stefan Kangas <stefankangas@HIDDEN>
Cc: 59317 <at> debbugs.gnu.org
Received: via spool by 59317-submit <at> debbugs.gnu.org id=B59317.16687303673144
          (code B ref 59317); Fri, 18 Nov 2022 00:13:01 +0000
Received: (at 59317) by debbugs.gnu.org; 18 Nov 2022 00:12:47 +0000
Received: from localhost ([127.0.0.1]:34661 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ovozu-0000od-QT
	for submit <at> debbugs.gnu.org; Thu, 17 Nov 2022 19:12:47 -0500
Received: from relay8-d.mail.gandi.net ([217.70.183.201]:33595)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <matt@HIDDEN>) id 1ovozs-0000oK-Mr
 for 59317 <at> debbugs.gnu.org; Thu, 17 Nov 2022 19:12:46 -0500
Received: (Authenticated sender: matt@HIDDEN)
 by mail.gandi.net (Postfix) with ESMTPSA id 93C521BF203;
 Fri, 18 Nov 2022 00:12:35 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
 t=1668730358;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=slTxIXKi8MIMg3tuGgg9rAtq6M3bessylVEjvfmfHCI=;
 b=Sm8m79qYIrFrU05b1YLGi6dh2D/eT7Ko8c29Fh5UwDESDousbhYrAwQ12nEFhCVqCNiK4R
 dcgnCaNLIX1EwkQuxKVIYDF72ZStiIEVUsl0nFG7Tejfw08U4cn5FjozADTZW68Tk8HqkO
 68qsi2NGAvm9Htma4/Gfz+f/Pf038wftwi+qPoDpWaRrrZbBJJ5GEdYLjLxT+tE2dZJQxa
 vlM7+27b8qBxnr6LreyIM9Tau6PCCazOCL3kX2XqGy/rTWE1d3eYj/8MxSEkWPT1P98U9u
 pdOklKiwsXqWYwxcz8RxG3UbDyPmmrChnb4y/0nuwTHwqun52+EcyqkVGzBwvQ==
Received: by mac-mini.lan (Postfix) with ESMTPS id 3F3C6393EB;
 Thu, 17 Nov 2022 16:12:32 -0800 (PST)
Received: by naz.lan (Postfix, from userid 1000)
 id 149C14143DEB; Thu, 17 Nov 2022 16:12:32 -0800 (PST)
From: Matt Armstrong <matt@HIDDEN>
In-Reply-To: <87wn7twyfq.fsf@HIDDEN>
References: <87r0y268bo.fsf@HIDDEN> <871qq1yijz.fsf@HIDDEN>
 <CADwFkmk+FEJ_k_UBQ6RLztyEAMsdMd5cNamrEOYK-g4DLjos1A@HIDDEN>
 <87wn7twyfq.fsf@HIDDEN>
Date: Thu, 17 Nov 2022 16:12:32 -0800
Message-ID: <87fsehruhb.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Michael Albinus <michael.albinus@HIDDEN> writes:

> Stefan Kangas <stefankangas@HIDDEN> writes:
>
> Hi Stefan,
>>> Besides the other arguments already given in this thread, I'd like to
>>> emphasize that messages are useful even in batch mode, for successful
>>> tests. This is the only way to study ert tests in batch mode, for
>>> example on our CI/CD machines hydra and emba.
>>
>> It would be less than useful in CI, indeed.
>>
>> But in interactive sessions, and when running tests manually with
>> e.g. M-x compile, it helps to have less noise, to find the relevant
>> failures faster.
>
> Maybe, although the messages in interactive mode go to a special *ERT
> Messages* buffer.

I believe Stefan's idea as running something like "make check" through
M-x compile, so the tests run in batch mode with output in *Compilation*
buffers.

As an implementation note, the *ERT Messages* buffer is not where
`message' output goes during a test.  Instead, the `ert-test' function
notes the `point-max-marker' of the *Messages* buffer, then runs the
test, and then extracts with `buffer-substring' the region of *Messages*
between that marker and its (point-max).  It then squirrels that string
away in a test result data structure.  Notably, ert doesn't suppress
message output in any way during a test.  They still appear in the
minibuffer, go to *Messages*, etc.


> Btw, there are test cases which depend on proper messages. See for
> example autorevert-tests.el. For those tests, messages must not be
> suppressed at all.

The way I imagine it, any lisp level introspection, mocking, etc., would
still work.


>>> If messages shall be suppressed, this must be controlled by a user
>>> option. However, I'm not in favor to do it at all.
>>
>> Agreed.  Perhaps an environment variable could also be provided, so
>> that it is easier to control from the command line?
>
> Sure. There are already several $EMACS_TEST_* environment variables,
> see test/README.

Good to know.

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 24 Nov 2022 18:27:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Nov 24 13:27:14 2022
Received: from localhost ([127.0.0.1]:60040 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1oyGwM-0006qo-JK
	for submit <at> debbugs.gnu.org; Thu, 24 Nov 2022 13:27:14 -0500
Received: from mail-ot1-f44.google.com ([209.85.210.44]:39870)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stefankangas@HIDDEN>) id 1oyGwH-0006po-Ab
 for control <at> debbugs.gnu.org; Thu, 24 Nov 2022 13:27:09 -0500
Received: by mail-ot1-f44.google.com with SMTP id
 g51-20020a9d12b6000000b0066dbea0d203so1364765otg.6
 for <control <at> debbugs.gnu.org>; Thu, 24 Nov 2022 10:27:09 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20210112;
 h=to:subject:message-id:date:mime-version:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=OcCbJ2BWq2sOqDNUvlJClbpVMWeEw8/uMa7rxa50NjA=;
 b=Xh0SfddOGBkm3VwHbDqKoYrzfO/RJH4QlzhqrlnjaNQBwRRN+LYlXHIR7KRM1Suwdn
 KBLe406yZd9QZUt+oECp/AkUZKTaqA3A4EH6ykWns10zPbtkeIjGMQoAVNtg9jP0r6pd
 yfEaB/ZBRHSR8b1xGezufBZ5UeuMkMrh7yhlWwc9to5Y37EubTECoEiBFrOB58W2IIxx
 OZ38h0/7ZE1FAOGG3HvSxnMORteDJB9eonNvwIFeaJCORMjWl3tYQKN769ZDbrb16m08
 X2+9be4LkFUWHgXc3p3g5cWAT4b9zobIS3xeaeAcEDUi3kYZCmabmQ0UknTmeCxI0kjb
 1yYw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=to:subject:message-id:date:mime-version:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=OcCbJ2BWq2sOqDNUvlJClbpVMWeEw8/uMa7rxa50NjA=;
 b=Y3t1wN904puSz6cYWGlJgd8YWP5FY+GG2xu9EO1YNHHi+8H6lnGbPDoMXxexdIRvRo
 uaP6gz/6k7ln/+JrR3rRijRu8lbvfpq9Pi3UGEorhKvbXh9wKBwr6XY5MSn17d8O8ky/
 Rd//SJlI6t9Saztym30VS4rh3Hsphj8KfgBwgsKxH9R4sAWV+78sa939xXFEEgwBkCWh
 7Ka5xFqclw8jDO1Q5uvqS8WGFlJdvNFGPVptMDKvqhBCk446awYTxHzahbAnMybsw/Mv
 j8WnvxEANyQfE/0xXPKlIOu1ao31DBnRS732VQKWIA2dArmcXVWeR2Qd/FQuRPWtykdc
 NpHA==
X-Gm-Message-State: ANoB5plyXm7ZoOPvz6gVXF3WFul+vxCAzAUVOT3CyicA9PkL96x+GH6O
 bqhQXWPHwUwsA57KWb0s+Yls4Q21227B/1y9oDRzvX7h
X-Google-Smtp-Source: AA0mqf6fBMGlBniEz23ZFUXQcu7hVS6QHjdKA6sf0KdhJF1CaoxEL8TNZHY3lPJh0gt3x1DiurS+d2XsPVRFI/fbGo8=
X-Received: by 2002:a9d:5c81:0:b0:661:c48b:12db with SMTP id
 a1-20020a9d5c81000000b00661c48b12dbmr8561249oti.105.1669314422373; Thu, 24
 Nov 2022 10:27:02 -0800 (PST)
Received: from 753933720722 named unknown by gmailapi.google.com with
 HTTPREST; Thu, 24 Nov 2022 10:27:02 -0800
From: Stefan Kangas <stefankangas@HIDDEN>
X-Hashcash: 1:20:221124:control <at> debbugs.gnu.org::6I+nbGl2AGtNGvIN:55Ak
MIME-Version: 1.0
Date: Thu, 24 Nov 2022 10:27:02 -0800
Message-ID: <CADwFkm=02GXJes71=0yN_o-kTu2GWCya5SjpYEYq0ivzCUm7yQ@HIDDEN>
Subject: control message for bug #59317
To: control <at> debbugs.gnu.org
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

severity 59317 wishlist
quit

Last modified: Thu, 24 Nov 2022 18:30:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.