Received: (at 76447) by debbugs.gnu.org; 1 Mar 2025 12:24:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Mar 01 07:24:55 2025
Received: from localhost ([127.0.0.1]:34892 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1toLtm-0007rW-Pn
for submit <at> debbugs.gnu.org; Sat, 01 Mar 2025 07:24:55 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:40322)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <eliz@HIDDEN>) id 1toLtj-0007qk-7n
for 76447 <at> debbugs.gnu.org; Sat, 01 Mar 2025 07:24:53 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
id 1toLtd-0001KH-O0; Sat, 01 Mar 2025 07:24:45 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
Date; bh=WAG2v2uK5vB3Rik0EMEhUQ/CGridqO+7u48Qwa9zNmA=; b=a4U2l48n239dSH9W97Xh
G/vsG/edrmrQfkhC9iMOslnQ1Xu9CZoobVo8ZN9R+9FaPwOpTytPXRBmfl0sHqVjKu9+rwtVwow11
3laSnveRCjep8EdiSSykjB2WJT8z/yP6oS/p6wR+fQgwu9jZaPPXpsR/eWySuqg1hqx/mikuAM/KR
neThsni8eelY/AmsNXFNB6U0CucE+oaxqkZ1RPBNfO5MBZh9xFCwi2X/3/b/P+BYZO5F3DHE+JQjb
q3UMA8GCCT+o2g+URb5wplZ/L2kSM9qVY7bcIpSuq4eGaLXGJeURF4G7hsxG3NB648JWOhobJTPhi
i6mrOM9XwRUrjQ==;
Date: Sat, 01 Mar 2025 14:24:42 +0200
Message-Id: <86frjxos51.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Philipp <p.stephani2@HIDDEN>, Stefan Monnier <monnier@HIDDEN>
In-Reply-To: <m2o6ywqs5a.fsf@HIDDEN> (message from
Philipp on Thu, 20 Feb 2025 21:26:41 +0100)
Subject: Re: bug#76447: 29.4;
ert-run-tests-batch now aborts if a test signals an undefined error
References: <m2o6ywqs5a.fsf@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 76447
Cc: 76447 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
> From: Philipp <p.stephani2@HIDDEN>
> Date: Thu, 20 Feb 2025 21:26:41 +0100
>
>
> With Emacs 29.4:
>
> $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> Running 2 tests (2025-02-20 21:13:09+0100, selector ‘t’)
> Test test-1 backtrace:
> signal(foo nil)
> (closure (t) nil (signal 'foo nil))()
> ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
> ert-run-test(#s(ert-test :name test-1 :documentation nil :body (clos
> ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test
> ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
> ert-run-tests-batch(nil)
> ert-run-tests-batch-and-exit()
> command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo)
> command-line()
> normal-top-level()
> Test test-1 condition:
> (foo)
> FAILED 1/2 test-1 (0.000060 sec)
> passed 2/2 test-2 (0.000030 sec)
>
> Ran 2 tests, 1 results as expected, 1 unexpected (2025-02-20 21:13:09+0100, 0.019570 sec)
>
> 1 unexpected results:
> FAILED test-1
>
>
> With Emacs 30.1 release candidate 1:
>
> $ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
> Running 2 tests (2025-02-20 21:13:03+0100, selector ‘t’)
> Test test-1 aborted with non-local exit
> ABORTED 1/2 test-1 (0.000073 sec)
>
> Aborted: Ran 2 tests, 0 results as expected, 0 unexpected (2025-02-20 21:13:03+0100, 0.000140 sec)
>
> Error running tests
> backtrace()
> #f(compiled-function () #<bytecode -0x174c28bd26a373a>)()
> ert-run-tests-batch-and-exit()
> command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) nil))" "-eval" "(ert-deftest test-2 ())" "-f" "ert-run-tests-batch-and-exit"))
> command-line()
> normal-top-level()
>
>
> test-2 isn't executed. I think the Emacs 29.4 behavior is more
> "user-friendly", though this (signaling an error symbol that doesn't
> derive from 'error') is arguably a corner case.
>
> Probably the behavior change is fallout from switching to handler-bind.
> To restore the Emacs 29.4 behavior, probably a construct such as
>
> (condition-case err
> (handler-bind ((t ...)) ...)
> (t ...))
>
> could be used. The pseudo-condition t should be able to catch all
> signals.
Stefan, any comments?
bug-gnu-emacs@HIDDEN:bug#76447; Package emacs.
Full text available.
Received: (at submit) by debbugs.gnu.org; 20 Feb 2025 20:27:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Feb 20 15:27:03 2025
Received: from localhost ([127.0.0.1]:42206 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1tlD8O-0006Ci-Gg
for submit <at> debbugs.gnu.org; Thu, 20 Feb 2025 15:27:03 -0500
Received: from lists.gnu.org ([2001:470:142::17]:40824)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <p.stephani2@HIDDEN>)
id 1tlD8L-0006Aw-9M
for submit <at> debbugs.gnu.org; Thu, 20 Feb 2025 15:26:58 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <p.stephani2@HIDDEN>)
id 1tlD8C-0004WO-QT
for bug-gnu-emacs@HIDDEN; Thu, 20 Feb 2025 15:26:49 -0500
Received: from mail-wm1-x32b.google.com ([2a00:1450:4864:20::32b])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.90_1) (envelope-from <p.stephani2@HIDDEN>)
id 1tlD8A-0003R0-Sb
for bug-gnu-emacs@HIDDEN; Thu, 20 Feb 2025 15:26:48 -0500
Received: by mail-wm1-x32b.google.com with SMTP id
5b1f17b1804b1-439999d2bbfso1944145e9.2
for <bug-gnu-emacs@HIDDEN>; Thu, 20 Feb 2025 12:26:43 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1740083203; x=1740688003; darn=gnu.org;
h=content-transfer-encoding:mime-version:message-id:date:subject:to
:from:from:to:cc:subject:date:message-id:reply-to;
bh=1+AuT8jIyXAhsyBvOELSidXX0ayAJgL3bYmaEhLEvrA=;
b=aBt11/6j1iAORi9ZhbsHB6IrnbN431EWvSqlqW2LvwhhMsqQS+YqAxT22qAxiQwL2l
nZgmETbr9PRxRMEqEVQ49m2EAvSmPxm6PZ1dqon56S/QI4Omps8WqcD+AJ2wNywPStdl
0oXwjMQhrnPXwLVmfpz0z9F4pvN2HeiHrHJr9bBW10prDWTKcsSZMNWNY5BcAzeXax4/
KONKbY6BkOlWt6tP93G637+0oaWpXSSnD4whhPGGS1ZDZR6TuQZU0mxrnrcSK4XoChnu
/JBhpoV+l2s2mYvNTvgpNDwe7zLc53N4VrUP+PWABIE6+4GkfAJ+88FLEexTHRlxa2s3
J1Kw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1740083203; x=1740688003;
h=content-transfer-encoding:mime-version:message-id:date:subject:to
:from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=1+AuT8jIyXAhsyBvOELSidXX0ayAJgL3bYmaEhLEvrA=;
b=NQo+k/XrK4wVI/aFQIRGEafTPWkWYHmhIgWbNVE6s9sKoOaC7Q74ktua2uGJ27ypM2
4JjBNMX3tvhH3cOAkbPV49EzymatxqmrEFcRFV1af97zICQblJ9AldhMVYMAZycc+wLg
ZSk0T9XlgUPGAaWA1/id83QcQQtOpHPNOpEi/N9GJOVvPyzyNfNVr5Qeb8JwzJFfVYRx
W06LjfB0LKPjctHZ1UkONYoTtoIJlkxPBW4aPdtJoJad7h0heNOxHBtCNzK6ovw4QfYB
6hkQSQlFBm2LkwOCPAFyxbZ3OT6Be3wLJBh5lZDuLYlG+LAdO6M1+l+jm3MEe0HYgP7n
2xXw==
X-Gm-Message-State: AOJu0YzlfhDZ2KXN2WJSOLFQw/XBvIIdGghw2FmluMKFrXyae47zpD1b
zc6NoUghcWTYGWTm8cEYcqE07AsX9uw1XYw6B9DzNV2FbLXEEdpsLcfM7Q==
X-Gm-Gg: ASbGncvCzeemOvTYWckF+LH7utKnuJDhW1DvCsqh6d/oJJL4kODTYsSgv/VVYxblW2B
TJetxDTO7cT77s1IrkhgfuDU1FMOTWqGo6soqOaKE3V8UCdMoSJlfrt4Usz64KhhLuhL0THNcRe
/s4mEBjm7915NTFb9ACTn8vvEAwIAYDdENFj9VW7umnBB28saGPYwJjOzn+U6XmrKS60E8BM8lM
EZoRRgCmM42Be1b+6/wxnCdYd9KqkpJHxYJIDY6PPw6pGA1YxazX67XzsAE/dosq5O3aLzhYqwB
7jegRaaqWoPnf+ZOApJuvYJCr9SjbgS61lTZP8T76Lw1ekR1h2I1PSLpICHL3WpN
X-Google-Smtp-Source: AGHT+IEm1jd3VIFpyXzd780m+INN1P8Xz08E+4oFtOjeTOKzOzXJEmlilm/SX3I0wQx0d0r9sK3CAg==
X-Received: by 2002:a05:6000:1889:b0:385:de67:229e with SMTP id
ffacd0b85a97d-38f6f0be804mr206331f8f.11.1740083202585;
Thu, 20 Feb 2025 12:26:42 -0800 (PST)
Received: from Philipps-MacBook-Pro.fritz.box
([2001:a61:3a5d:f801:d126:75a0:efc6:a006])
by smtp.gmail.com with ESMTPSA id
ffacd0b85a97d-38f258cccd3sm21451760f8f.23.2025.02.20.12.26.41
for <bug-gnu-emacs@HIDDEN>
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Thu, 20 Feb 2025 12:26:41 -0800 (PST)
From: Philipp <p.stephani2@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 29.4; ert-run-tests-batch now aborts if a test signals an undefined
error
Date: Thu, 20 Feb 2025 21:26:41 +0100
Message-ID: <m2o6ywqs5a.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=2a00:1450:4864:20::32b;
envelope-from=p.stephani2@HIDDEN; helo=mail-wm1-x32b.google.com
X-Spam_score_int: -17
X-Spam_score: -1.8
X-Spam_bar: -
X-Spam_report: (-1.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
FREEMAIL_ENVFROM_END_DIGIT=0.25, FREEMAIL_FROM=0.001,
RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.2 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
has NOT identified this incoming email as spam. The original
message has been attached to this so you can view it or label
similar future email. If you have any questions, see
the administrator of that system for details.
Content preview: With Emacs 29.4: $ emacs -Q -batch -eval '(ert-deftest test-1
() (signal (quote foo) nil))' -eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
Running 2 tests (2025-02-20 21:13:09+0100, selector ‘t’) [...]
Content analysis details: (1.2 points, 10.0 required)
pts rule name description
---- ---------------------- --------------------------------------------------
1.0 SPF_SOFTFAIL SPF: sender does not match SPF record (softfail)
0.2 FREEMAIL_ENVFROM_END_DIGIT Envelope-from freemail username ends
in digit (p.stephani2[at]gmail.com)
-0.0 RCVD_IN_DNSWL_NONE RBL: Sender listed at https://www.dnswl.org/,
no trust
[2001:470:142:0:0:0:0:17 listed in]
[list.dnswl.org]
-0.0 SPF_HELO_PASS SPF: HELO matches SPF record
0.0 FREEMAIL_FROM Sender email is commonly abused enduser mail
provider (p.stephani2[at]gmail.com)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.2 (/)
With Emacs 29.4:
$ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' =
-eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
Running 2 tests (2025-02-20 21:13:09+0100, selector =E2=80=98t=E2=80=99)
Test test-1 backtrace:
signal(foo nil)
(closure (t) nil (signal 'foo nil))()
ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
ert-run-test(#s(ert-test :name test-1 :documentation nil :body (clos
ert-run-or-rerun-test(#s(ert--stats :selector t :tests [#s(ert-test=20
ert-run-tests(t #f(compiled-function (event-type &rest event-args) #
ert-run-tests-batch(nil)
ert-run-tests-batch-and-exit()
command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo)=20
command-line()
normal-top-level()
Test test-1 condition:
(foo)
FAILED 1/2 test-1 (0.000060 sec)=20
passed 2/2 test-2 (0.000030 sec)
Ran 2 tests, 1 results as expected, 1 unexpected (2025-02-20 21:13:09+0100,=
0.019570 sec)
1 unexpected results:
FAILED test-1
With Emacs 30.1 release candidate 1:
$ emacs -Q -batch -eval '(ert-deftest test-1 () (signal (quote foo) nil))' =
-eval '(ert-deftest test-2 ())' -f ert-run-tests-batch-and-exit
Running 2 tests (2025-02-20 21:13:03+0100, selector =E2=80=98t=E2=80=99)
Test test-1 aborted with non-local exit
ABORTED 1/2 test-1 (0.000073 sec)=20
Aborted: Ran 2 tests, 0 results as expected, 0 unexpected (2025-02-20 21:13=
:03+0100, 0.000140 sec)
Error running tests
backtrace()
#f(compiled-function () #<bytecode -0x174c28bd26a373a>)()
ert-run-tests-batch-and-exit()
command-line-1(("-eval" "(ert-deftest test-1 () (signal (quote foo) nil))=
" "-eval" "(ert-deftest test-2 ())" "-f" "ert-run-tests-batch-and-exit"))
command-line()
normal-top-level()
test-2 isn't executed. I think the Emacs 29.4 behavior is more
"user-friendly", though this (signaling an error symbol that doesn't
derive from 'error') is arguably a corner case.
Probably the behavior change is fallout from switching to handler-bind.
To restore the Emacs 29.4 behavior, probably a construct such as
(condition-case err
(handler-bind ((t ...)) ...)
(t ...))
could be used. The pseudo-condition t should be able to catch all
signals.
In GNU Emacs 29.4 (build 1, aarch64-apple-darwin24.3.0, NS
appkit-2575.40 Version 15.3.1 (Build 24D70)) of 2025-02-20
Repository revision: 6a299b3caceb2c73b932ba73849738faa8c5d975
Repository branch: HEAD
Windowing system distributor 'Apple', version 10.3.2575
System Description: macOS 15.3.1
Configured using:
'configure --with-modules --with-xml2 --without-pop --with-mailutils
'CFLAGS=3D-g -O3' LDFLAGS=3D-O3'
Configured features:
ACL GNUTLS JSON LCMS2 LIBXML2 MODULES NOTIFY KQUEUE NS PDUMPER PNG
SQLITE3 THREADS TOOLKIT_SCROLL_BARS TREE_SITTER WEBP ZLIB
Important settings:
value of $LANG: de_DE.UTF-8
locale-coding-system: utf-8-unix
Major mode: Lisp Interaction
Minor modes in effect:
tooltip-mode: t
global-eldoc-mode: t
eldoc-mode: t
show-paren-mode: t
electric-indent-mode: t
mouse-wheel-mode: t
tool-bar-mode: t
menu-bar-mode: t
file-name-shadow-mode: t
global-font-lock-mode: t
font-lock-mode: t
blink-cursor-mode: t
line-number-mode: t
indent-tabs-mode: t
transient-mark-mode: t
auto-composition-mode: t
auto-encryption-mode: t
auto-compression-mode: t
Load-path shadows:
None found.
Features:
(shadow sort mail-extr emacsbug phst skeleton pcase ffap thingatpt url
url-proxy url-privacy url-expand url-methods url-history url-cookie
generate-lisp-file url-domsuf url-util url-parse auth-source cl-seq
eieio eieio-core cl-macs json map byte-opt gv bytecomp byte-compile
url-vars rx message sendmail mailcap yank-media dired dired-loaddefs
rfc822 mml mml-sec password-cache epa derived epg rfc6068 epg-config
gnus-util time-date mm-decode mm-bodies mm-encode mail-parse rfc2231
rfc2047 rfc2045 mm-util ietf-drums mail-prsvr mailabbrev mail-utils
gmm-utils mailheader gnutls puny elp dbus xml subr-x compile
text-property-search comint ansi-osc ansi-color ring cl-loaddefs cl-lib
rmc iso-transl tooltip cconv eldoc paren electric uniquify ediff-hook
vc-hooks lisp-float-type elisp-mode mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer nadvice seq simple cl-generic
indonesian philippine cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese eucjp-ms cp51932 hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
composite emoji-zwj charscript charprop case-table epa-hook
jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button loaddefs
theme-loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads kqueue cocoa ns lcms2
multi-tty make-network-process emacs)
Memory information:
((conses 16 69214 10345)
(symbols 48 8174 0)
(strings 32 23097 2385)
(string-bytes 1 704838)
(vectors 16 16338)
(vector-slots 8 223298 13523)
(floats 8 29 54)
(intervals 56 229 0)
(buffers 984 10))
Philipp <p.stephani2@HIDDEN>:bug-gnu-emacs@HIDDEN.
Full text available.bug-gnu-emacs@HIDDEN:bug#76447; Package emacs.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.