GNU logs - #76447, boring messages

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#76447: 29.4; ert-run-tests-batch now aborts if a test signals an undefined error
Resent-From: Philipp <p.stephani2@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Thu, 20 Feb 2025 20:28:01 +0000
Resent-Message-ID: <handler.76447.B.174008322323887 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 76447 <at> debbugs.gnu.org
From: Philipp <p.stephani2@HIDDEN>
Date: Thu, 20 Feb 2025 21:26:41 +0100
Message-ID: <m2o6ywqs5a.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=2a00:1450:4864:20::32b;
 envelope-from=p.stephani2@HIDDEN; helo=mail-wm1-x32b.google.com
X-Spam_score_int: -17
X-Spam_score: -1.8
X-Spam_bar: -
X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.2 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  With Emacs 29.4: $ emacs -Q -batch -eval '(ert-deftest test-1
    () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
    Running 2 tests (2025-02-20 21:13:09+0100, selector =?UTF-8?Q?=E2=80=98t=E2=80=99?=) [...] 
 
 Content analysis details:   (1.2 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
  1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
  0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends
                             in digit (p.stephani2[at]gmail.com)
 -0.0 RCVD_IN_DNSWL_NONE     RBL: Sender listed at https://www.dnswl.org/,
                              no trust
                             [2001:470:142:0:0:0:0:17 listed in]
                             [list.dnswl.org]
 -0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
  0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail
                             provider (p.stephani2[at]gmail.com)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.2 (/)


With Emacs 29.4:

$ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' =
-eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
Running 2 tests (2025-02-20 21:13:09+0100, selector =E2=80=98t=E2=80=99)
Test test-1 backtrace:
  signal(foo nil)
  (closure (t) nil (signal 'foo nil))()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name test-1 :documentation nil :body (clos
  ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test=20
  ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
  ert-run-tests-batch(nil)
  ert-run-tests-batch-and-exit()
  command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo)=20
  command-line()
  normal-top-level()
Test test-1 condition:
    (foo)
   FAILED  1/2  test-1 (0.000060 sec)=20
   passed  2/2  test-2 (0.000030 sec)

Ran 2 tests, 1 results as expected, 1 unexpected (2025-02-20 21:13:09+0100,=
 0.019570 sec)

1 unexpected results:
   FAILED  test-1


With Emacs 30.1 release candidate 1:

$ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' =
-eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
Running 2 tests (2025-02-20 21:13:03+0100, selector =E2=80=98t=E2=80=99)
Test test-1 aborted with non-local exit
  ABORTED  1/2  test-1 (0.000073 sec)=20

Aborted: Ran 2 tests, 0 results as expected, 0 unexpected (2025-02-20 21:13=
:03+0100, 0.000140 sec)

Error running tests
  backtrace()
  #f(compiled-function () #<bytecode -0x174c28bd26a373a>)()
  ert-run-tests-batch-and-exit()
  command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) nil))=
" "-eval" "(ert-deftest test-2 ())" "-f" "ert-run-tests-batch-and-exit"))
  command-line()
  normal-top-level()


test-2 isn't executed.  I think the Emacs 29.4 behavior is more
"user-friendly", though this (signaling an error symbol that doesn't
derive from 'error') is arguably a corner case.

Probably the behavior change is fallout from switching to handler-bind.
To restore the Emacs 29.4 behavior, probably a construct such as

(condition-case err
    (handler-bind ((t ...)) ...)
  (t ...))

could be used.  The pseudo-condition t should be able to catch all
signals.


In GNU Emacs 29.4 (build 1, aarch64-apple-darwin24.3.0, NS
 appkit-2575.40 Version 15.3.1 (Build 24D70)) of 2025-02-20
Repository revision: 6a299b3caceb2c73b932ba73849738faa8c5d975
Repository branch: HEAD
Windowing system distributor 'Apple', version 10.3.2575
System Description:  macOS 15.3.1

Configured using:
 'configure --with-modules --with-xml2 --without-pop --with-mailutils
 'CFLAGS=3D-g -O3' LDFLAGS=3D-O3'

Configured features:
ACL GNUTLS JSON LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS PDUMPER PNG
SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER WEBP ZLIB

Important settings:
  value of $LANG: de_DE.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug phst skeleton pcase ffap thingatpt url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq
eieio eieio-core cl-macs json map byte-opt gv bytecomp byte-compile
url-vars rx message sendmail mailcap yank-media dired dired-loaddefs
rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config
gnus-util time-date mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils
gmm-utils mailheader gnutls puny elp dbus xml subr-x compile
text-property-search comint ansi-osc ansi-color ring cl-loaddefs cl-lib
rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue cocoa ns lcms2
multi-tty make-network-process emacs)

Memory information:
((conses 16 69214 10345)
 (symbols 48 8174 0)
 (strings 32 23097 2385)
 (string-bytes 1 704838)
 (vectors 16 16338)
 (vector-slots 8 223298 13523)
 (floats 8 29 54)
 (intervals 56 229 0)
 (buffers 984 10))

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Philipp <p.stephani2@HIDDEN>
Subject: bug#76447: Acknowledgement (29.4; ert-run-tests-batch now aborts
 if a test signals an undefined error)
Message-ID: <handler.76447.B.174008322323887.ack <at> debbugs.gnu.org>
References: <m2o6ywqs5a.fsf@HIDDEN>
X-Gnu-PR-Message: ack 76447
X-Gnu-PR-Package: emacs
Reply-To: 76447 <at> debbugs.gnu.org
Date: Thu, 20 Feb 2025 20:28:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 76447 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
76447: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D76447
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-gnu-emacs@HIDDEN:

Subject: bug#76447: 29.4; ert-run-tests-batch now aborts if a test signals an undefined error
Resent-From: Eli Zaretskii <eliz@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 01 Mar 2025 12:25:02 +0000
Resent-Message-ID: <handler.76447.B76447.174083189530232 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Philipp <p.stephani2@HIDDEN>, Stefan Monnier <monnier@HIDDEN>
Cc: 76447 <at> debbugs.gnu.org
Date: Sat, 01 Mar 2025 14:24:42 +0200
Message-Id: <86frjxos51.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
In-Reply-To: <m2o6ywqs5a.fsf@HIDDEN> (message from
 Philipp on Thu, 20 Feb 2025 21:26:41 +0100)
References: <m2o6ywqs5a.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

> From: Philipp <p.stephani2@HIDDEN>
> Date: Thu, 20 Feb 2025 21:26:41 +0100
> 
> 
> With Emacs 29.4:
> 
> $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> Running 2 tests (2025-02-20 21:13:09+0100, selector ‘t’)
> Test test-1 backtrace:
>   signal(foo nil)
>   (closure (t) nil (signal 'foo nil))()
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name test-1 :documentation nil :body (clos
>   ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test 
>   ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
>   ert-run-tests-batch(nil)
>   ert-run-tests-batch-and-exit()
>   command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) 
>   command-line()
>   normal-top-level()
> Test test-1 condition:
>     (foo)
>    FAILED  1/2  test-1 (0.000060 sec) 
>    passed  2/2  test-2 (0.000030 sec)
> 
> Ran 2 tests, 1 results as expected, 1 unexpected (2025-02-20 21:13:09+0100, 0.019570 sec)
> 
> 1 unexpected results:
>    FAILED  test-1
> 
> 
> With Emacs 30.1 release candidate 1:
> 
> $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> Running 2 tests (2025-02-20 21:13:03+0100, selector ‘t’)
> Test test-1 aborted with non-local exit
>   ABORTED  1/2  test-1 (0.000073 sec) 
> 
> Aborted: Ran 2 tests, 0 results as expected, 0 unexpected (2025-02-20 21:13:03+0100, 0.000140 sec)
> 
> Error running tests
>   backtrace()
>   #f(compiled-function () #<bytecode -0x174c28bd26a373a>)()
>   ert-run-tests-batch-and-exit()
>   command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) nil))" "-eval" "(ert-deftest test-2 ())" "-f" "ert-run-tests-batch-and-exit"))
>   command-line()
>   normal-top-level()
> 
> 
> test-2 isn't executed.  I think the Emacs 29.4 behavior is more
> "user-friendly", though this (signaling an error symbol that doesn't
> derive from 'error') is arguably a corner case.
> 
> Probably the behavior change is fallout from switching to handler-bind.
> To restore the Emacs 29.4 behavior, probably a construct such as
> 
> (condition-case err
>     (handler-bind ((t ...)) ...)
>   (t ...))
> 
> could be used.  The pseudo-condition t should be able to catch all
> signals.

Stefan, any comments?

Last modified: Sat, 1 Mar 2025 12:30:03 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.