GNU bug report logs - #22471
ert batch should print compile parsable error messages

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Severity: wishlist; Reported by: phillip.lord@HIDDEN (Phillip Lord); dated Wed, 27 Jan 2016 09:17:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 24 Oct 2021 14:27:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 24 10:27:40 2021
Received: from localhost ([127.0.0.1]:40490 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1meeTM-0000Pf-4r
	for submit <at> debbugs.gnu.org; Sun, 24 Oct 2021 10:27:40 -0400
Received: from mail-pf1-f170.google.com ([209.85.210.170]:38904)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stefankangas@HIDDEN>) id 1meeTJ-0000PS-VI
 for 22471 <at> debbugs.gnu.org; Sun, 24 Oct 2021 10:27:38 -0400
Received: by mail-pf1-f170.google.com with SMTP id k26so8244594pfi.5
 for <22471 <at> debbugs.gnu.org>; Sun, 24 Oct 2021 07:27:37 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:in-reply-to:references:mime-version:date
 :message-id:subject:to:cc;
 bh=DjxpqJfCIbDBQl7u19zNla2aysldIvhOvJOOgHURaoc=;
 b=aCIfvAwXa8m3FqK9aVZF219oD73mlOvNsbzI4ELck4QN0QJO/3gfGvLzhrr53O5i4h
 9zNlQtMzGAuyZ0AKnGJidYf3p+cgvbhGCQb329iw5K0p91p/i4nUAMYcaDnNQNpArlvE
 +eR6XQvhi8teTYPow7gz5xjqecfSqrFK52YJ7YZMpL5aeEXWNRjfSKMnASy1PAiPYBUK
 BEVAQ8NdWeJUW7T7uRVIlLPL1F63X1lrauBL4pTnDuHdmGsUZuf5+iadVacm6dYjPGdN
 Ivk8j2KCdqMUvqkNs+erGJL0E6HnMlZfKNPXYEtUe8RYBlaN8ZBsbmRsbRz1WPZdXKVo
 OoPw==
X-Gm-Message-State: AOAM532NoXB5ndWrc2S/2T4QDkJ85XS3pF4j5heNwZ5hMqGNhDufInKi
 rzL+PyOOOz/li36YotwkKtgK4BSPBAyGNSqmTWYzeU+MdbY=
X-Google-Smtp-Source: ABdhPJytA77MTRi5AB5+vmfE7oP/XaAuHdq8prSrcIR7G5+c0ptIExH/sXZ/XOiEFDL2EKWECz4ujHUh6fLNKhZBxnk=
X-Received: by 2002:a05:6a00:1950:b0:44d:9402:3396 with SMTP id
 s16-20020a056a00195000b0044d94023396mr12613196pfk.70.1635085652297; Sun, 24
 Oct 2021 07:27:32 -0700 (PDT)
Received: from 753933720722 named unknown by gmailapi.google.com with
 HTTPREST; Sun, 24 Oct 2021 07:27:31 -0700
From: Stefan Kangas <stefan@HIDDEN>
In-Reply-To: <871r4abmwr.fsf@HIDDEN>
References: <87bn878jsv.fsf@HIDDEN> <87wo1tp8z2.fsf@HIDDEN>
 <CADwFkmnChMB0b3S3i2Oqvr8K+vZpnYxnJE2CFUv9cWFYN+yOEg@HIDDEN>
 <871r4abmwr.fsf@HIDDEN>
MIME-Version: 1.0
Date: Sun, 24 Oct 2021 07:27:31 -0700
Message-ID: <CADwFkmnDM-8161nYvW6FSCJyEnSZSLqPFpu37ap57Q_q5E5Ezg@HIDDEN>
Subject: Re: bug#22471: ert batch should print compile parsable error messages
To: Lars Ingebrigtsen <larsi@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org, Phillip Lord <phillip.lord@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)

Lars Ingebrigtsen <larsi@HIDDEN> writes:

> Well -- ert-deftest doesn't record what file it's from, I think?
> Looking at the definition, it just adds the test to the symbol-plist.
> So that macro would also have to stash the file name somewhere.
> load-history would be possible (but sounds unclean), so stashing it in
> the symbol-plist would be better.
>
> Then find-definition-noselect could be extended to use that for this
> type.

Right, I'm just wondering what has changed as presumably the patch
worked when Phillip Lord first wrote it?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 24 Oct 2021 12:57:54 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 24 08:57:54 2021
Received: from localhost ([127.0.0.1]:38143 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1med4T-0001jn-QW
	for submit <at> debbugs.gnu.org; Sun, 24 Oct 2021 08:57:53 -0400
Received: from quimby.gnus.org ([95.216.78.240]:43762)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1med4R-0001ja-Ic
 for 22471 <at> debbugs.gnu.org; Sun, 24 Oct 2021 08:57:52 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org;
 s=20200322; h=Content-Transfer-Encoding:Content-Type:MIME-Version:Message-ID
 :In-Reply-To:Date:References:Subject:Cc:To:From:Sender:Reply-To:Content-ID:
 Content-Description:Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc
 :Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:List-Subscribe:
 List-Post:List-Owner:List-Archive;
 bh=eiXFjMMjimA60j5NqATNMUPDqohTQfyhQEFfuh5GWnw=; b=ZQI9Lc4ow5eQF1Rl+YCZbosIqF
 HVzFucktWq9bNBXVlXZfcF6y8uXqHllU12sqH6dodDo5qzQZX0Nnjz63jpbpA4+CNHZzIVLuQwpFv
 cisRmmteuTGS2BDYEut03mrk8TYmfnZgKbCgDwZTompim6WTLlCNIFozilKpKv7JitBk=;
Received: from [84.212.220.105] (helo=elva)
 by quimby.gnus.org with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1med4H-00077j-Ak; Sun, 24 Oct 2021 14:57:44 +0200
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: Stefan Kangas <stefan@HIDDEN>
Subject: Re: bug#22471: ert batch should print compile parsable error messages
References: <87bn878jsv.fsf@HIDDEN> <87wo1tp8z2.fsf@HIDDEN>
 <CADwFkmnChMB0b3S3i2Oqvr8K+vZpnYxnJE2CFUv9cWFYN+yOEg@HIDDEN>
Face: iVBORw0KGgoAAAANSUhEUgAAADAAAAAwAgMAAAAqbBEUAAAABGdBTUEAALGPC/xhBQAAACBj
 SFJNAAB6JgAAgIQAAPoAAACA6AAAdTAAAOpgAAA6mAAAF3CculE8AAAADFBMVEUwGA+kTy7olFT/
 //+IyZgpAAAAAWJLR0QDEQxM8gAAAAd0SU1FB+UKGAw3DMXrUjsAAADuSURBVCjPjc8xbsMwDABA
 ETB/EP6HAerdQyjAmtMhekWf0KFTpw42EOuVJSlZdjrVg8QzRZEKYf9IpMeXM3DUGHaB3HpGIf9B
 mOkEeoEfQ+EzwrGRdxto9l8OpIet6CXgB4i4Zu5eUxG/bGvT2tR9JqvRF16nBk0IOea79pXJqzCt
 DqjYDOygVKyeR4PkzSBivfEhFUu4eD9DXIJo/G4XSFrAIA0otvsE+Yk3R9ClPHGihmiQivituDbk
 ohgbSilLzyh+GsDwiXzgAxg7GJh6DcNEaY0H8tbwZljzDrvgD+AVKZcBGBwp520Ae2T5BQcodEfp
 GrRSAAAAOGVYSWZNTQAqAAAACAABh2kABAAAAAEAAAAaAAAAAAACoAIABAAAAAEAAAGQoAMABAAA
 AAEAAAGQAAAAADj5TfYAAAAldEVYdGRhdGU6Y3JlYXRlADIwMjEtMTAtMjRUMTI6NTU6MTErMDA6
 MDB+Lt0lAAAAJXRFWHRkYXRlOm1vZGlmeQAyMDIxLTEwLTI0VDEyOjU1OjExKzAwOjAwD3NlmQAA
 ABJ0RVh0ZXhpZjpFeGlmT2Zmc2V0ADI2UxuiZQAAABh0RVh0ZXhpZjpQaXhlbFhEaW1lbnNpb24A
 NDAwQPmChgAAABh0RVh0ZXhpZjpQaXhlbFlEaW1lbnNpb24ANDAw3fZj8AAAAABJRU5ErkJggg==
Date: Sun, 24 Oct 2021 14:57:40 +0200
In-Reply-To: <CADwFkmnChMB0b3S3i2Oqvr8K+vZpnYxnJE2CFUv9cWFYN+yOEg@HIDDEN>
 (Stefan Kangas's message of "Fri, 22 Oct 2021 17:15:35 -0700")
Message-ID: <871r4abmwr.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 
 Content preview:  Stefan Kangas <stefan@HIDDEN> writes: > I got the following
    backtrace, but gave up after looking at it for a > while: > > Debugger entered--Lisp
    error: (error "Don’t know where > ‘fns-tests-string-bytes’ is defin...")
    > signal(error [...] 
 
 Content analysis details:   (-2.9 points, 5.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
                             [score: 0.0000]
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org, Phillip Lord <phillip.lord@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Stefan Kangas <stefan@HIDDEN> writes:

> I got the following backtrace, but gave up after looking at it for a
> while:
>
> Debugger entered--Lisp error: (error "Don=E2=80=99t know where
> =E2=80=98fns-tests-string-bytes=E2=80=99 is defin...")
>   signal(error ("Don=E2=80=99t know where =E2=80=98fns-tests-string-bytes=
=E2=80=99 is defin..."))

Well -- ert-deftest doesn't record what file it's from, I think?
Looking at the definition, it just adds the test to the symbol-plist.
So that macro would also have to stash the file name somewhere.
load-history would be possible (but sounds unclean), so stashing it in
the symbol-plist would be better.

Then find-definition-noselect could be extended to use that for this
type.

--=20
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 23 Oct 2021 00:15:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 22 20:15:45 2021
Received: from localhost ([127.0.0.1]:34003 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1me4hM-0001iK-Tj
	for submit <at> debbugs.gnu.org; Fri, 22 Oct 2021 20:15:45 -0400
Received: from mail-pj1-f52.google.com ([209.85.216.52]:54799)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <stefankangas@HIDDEN>) id 1me4hK-0001i5-1o
 for 22471 <at> debbugs.gnu.org; Fri, 22 Oct 2021 20:15:43 -0400
Received: by mail-pj1-f52.google.com with SMTP id np13so3999672pjb.4
 for <22471 <at> debbugs.gnu.org>; Fri, 22 Oct 2021 17:15:42 -0700 (PDT)
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20210112;
 h=x-gm-message-state:from:in-reply-to:references:user-agent
 :mime-version:date:message-id:subject:to:cc;
 bh=2s9rFJ87CC81sat2Jur2wtwLhzx9cQtGIKm4i8kjBoo=;
 b=nP9WrMBI3OntRzzIuuhaiqvBv+Yh2A6pqbEUYNzEFxgDOIG5kBJC7rQpidf97VtHEr
 eBpLWOkq3dPbR45/i60qpuuN+XGQm3QbQV1crudl9atPeNmr/Yr2+0ALMt2CYeK2jqI3
 DV3gH/13kHqjMqgqwhfsFTTuCO5xetjk6cwyB+eBgsz8PJvGGTm5KimILetqQWCGCB1g
 KUiYRkDgV2/FE5ZZXTSMrUBeXr0X8i6li2yVIYMQsvmqDBYc9qh5SuXob0kCS2ZnwQf6
 yAqkKNHaO+7qY6pdX4hg8jPzoCcLWwJyzMbnxFY2R4/M1V1N6HGUQ6ErHwoxwy9Htb2k
 Dryw==
X-Gm-Message-State: AOAM532bzVkZj7KLj4b+meavAtthvl6JfCg3rx1KfMyBR3lg+RTCfV5V
 z3w5gFH+Yog6hsfXi5cRHxanIP7yaSjXQVxp0ew=
X-Google-Smtp-Source: ABdhPJxTWd49sHX4BY2m0TRQPylq3TiYStKNMjxB/mtXUK7Aweuu04SbpIz5TlQCX8o/Dgsy6ygd3fIJdfxJsGTUoQs=
X-Received: by 2002:a17:90a:245:: with SMTP id
 t5mr3290512pje.133.1634948136222; 
 Fri, 22 Oct 2021 17:15:36 -0700 (PDT)
Received: from 753933720722 named unknown by gmailapi.google.com with
 HTTPREST; Fri, 22 Oct 2021 17:15:35 -0700
From: Stefan Kangas <stefan@HIDDEN>
In-Reply-To: <87wo1tp8z2.fsf@HIDDEN> (Lars Ingebrigtsen's message of "Thu,
 20 Aug 2020 18:39:45 +0200")
References: <87bn878jsv.fsf@HIDDEN> <87wo1tp8z2.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/29.0.50 (gnu/linux)
MIME-Version: 1.0
Date: Fri, 22 Oct 2021 17:15:35 -0700
Message-ID: <CADwFkmnChMB0b3S3i2Oqvr8K+vZpnYxnJE2CFUv9cWFYN+yOEg@HIDDEN>
Subject: Re: bug#22471: ert batch should print compile parsable error messages
To: Lars Ingebrigtsen <larsi@HIDDEN>
Content-Type: multipart/mixed; boundary="000000000000e2ddb405cefa06ff"
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org, Phillip Lord <phillip.lord@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)

--000000000000e2ddb405cefa06ff
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Lars Ingebrigtsen <larsi@HIDDEN> writes:

> phillip.lord@HIDDEN (Phillip Lord) writes:
>
>> Currently, when running in batch, ert prints out messages like so:
>>
>> Running 24 tests (2016-01-27 09:05:17+0000)
>>    passed   1/24  buffer-string=3D
>>    passed   2/24  buffer=3D
>> ...
>>    passed  22/24  sisyphus-test-with-find-file
>>    passed  23/24  to-string
>>    passed  24/24  with-temp-buffers
>>
>> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+=
0000)
>>
>> 1 unexpected results:
>>    FAILED  crash-out
>>
>> It would be nice to add some compilation-mode parsable data to this. So,
>> something like
>>
>>    passed   1/24  buffer-string=3D  (in ./test.sisyphus-test.el:22:)
>
> I think that would look rather cluttered for the "passed" lines, but it
> would indeed be helpful on the FAILED lines.

This would in my opinion be very useful and a great timesaver.  But yes,
for FAILED results only, much like how the compiler only gives us line
numbers if there is a problem.  (AFAIR, that's how the test frameworks
I've used do it.)

So I updated the patch from the feature/parsable-ert-output against
current master (patch attached), made sure that one of our tests failed,
and then tested it with "make check".

I got the following backtrace, but gave up after looking at it for a
while:

Debugger entered--Lisp error: (error "Don=E2=80=99t know where
=E2=80=98fns-tests-string-bytes=E2=80=99 is defin...")
  signal(error ("Don=E2=80=99t know where =E2=80=98fns-tests-string-bytes=
=E2=80=99 is defin..."))
  error("Don't know where `%s' is defined" fns-tests-string-bytes)
  find-function-search-for-symbol(fns-tests-string-bytes ert-deftest nil)
  find-definition-noselect(fns-tests-string-bytes ert-deftest)
  ert-test-location([...])
[...]

--000000000000e2ddb405cefa06ff
Content-Type: text/x-diff; charset="US-ASCII"; 
	name="0001-Add-source-information-to-ERT-batch-mode.patch"
Content-Disposition: attachment; 
	filename="0001-Add-source-information-to-ERT-batch-mode.patch"
Content-Transfer-Encoding: base64
X-Attachment-Id: 78b2d87a32e4a3fc_0.1

RnJvbSBlNmMxODVmMjgwZTJjMzMwODI1YmVlYmFkN2QyMzkzYzIxMTk5NTY0IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBQaGlsbGlwIExvcmQgPHBoaWxsaXAubG9yZEBydXNzZXQub3Jn
LnVrPgpEYXRlOiBXZWQsIDI3IEphbiAyMDE2IDIwOjU5OjUwICswMDAwClN1YmplY3Q6IFtQQVRD
SF0gQWRkIHNvdXJjZSBpbmZvcm1hdGlvbiB0byBFUlQgYmF0Y2ggbW9kZQoKKiBsaXNwL2VtYWNz
LWxpc3AvZXJ0LmVsIChlcnQtdGVzdC1sb2NhdGlvbik6IE5ldyBmdW5jdGlvbi4KKGVydC1ydW4t
dGVzdHMtYmF0Y2gpOiBBZGQgdGVzdCBsb2NhdGlvbiB0byBmYWlsZWQgdGVzdHMuICAoQnVnIzIy
NDcxKQotLS0KIGxpc3AvZW1hY3MtbGlzcC9lcnQuZWwgfCAyNCArKysrKysrKysrKysrKysrKysr
KysrLS0KIDEgZmlsZSBjaGFuZ2VkLCAyMiBpbnNlcnRpb25zKCspLCAyIGRlbGV0aW9ucygtKQoK
ZGlmZiAtLWdpdCBhL2xpc3AvZW1hY3MtbGlzcC9lcnQuZWwgYi9saXNwL2VtYWNzLWxpc3AvZXJ0
LmVsCmluZGV4IDU3NjU1NDAzYzIuLjM2ZjUwNjJlODMgMTAwNjQ0Ci0tLSBhL2xpc3AvZW1hY3Mt
bGlzcC9lcnQuZWwKKysrIGIvbGlzcC9lbWFjcy1saXNwL2VydC5lbApAQCAtMTM1Myw2ICsxMzUz
LDIzIEBAIGVydC0taW5zZXJ0LWluZm9zCiAoZGVmdmFyIGVydC1xdWlldCBuaWwKICAgIk5vbi1u
aWwgbWFrZXMgRVJUIG9ubHkgcHJpbnQgaW1wb3J0YW50IGluZm9ybWF0aW9uIGluIGJhdGNoIG1v
ZGUuIikKIAorKGRlZnVuIGVydC10ZXN0LWxvY2F0aW9uICh0ZXN0KQorICAiUmV0dXJuIGEgc3Ry
aW5nIGRlc2NyaXB0aW9uIHRoZSBzb3VyY2UgbG9jYXRpb24gb2YgVEVTVC4iCisgIChsZXQqICgo
bG9jCisgICAgICAgICAgKGZpbmQtZGVmaW5pdGlvbi1ub3NlbGVjdCAoZXJ0LXRlc3QtbmFtZSB0
ZXN0KSAnZXJ0LWRlZnRlc3QpKQorICAgICAgICAgKGJ1ZmZlcgorICAgICAgICAgIChjYXIgbG9j
KSkKKyAgICAgICAgIChwb2ludCAoY2RyIGxvYykpCisgICAgICAgICAoZmlsZQorICAgICAgICAg
IChmaWxlLXJlbGF0aXZlLW5hbWUKKyAgICAgICAgICAgKGJ1ZmZlci1maWxlLW5hbWUgYnVmZmVy
KSkpCisgICAgICAgICAobGluZSAod2l0aC1jdXJyZW50LWJ1ZmZlciBidWZmZXIKKyAgICAgICAg
ICAgICAgICAgKGxpbmUtbnVtYmVyLWF0LXBvcyBwb2ludCkpKSkKKyAgICAoZm9ybWF0ICJhdCAl
cyBsaW5lICVzLiIgZmlsZSBsaW5lKSkpCisKKyhkZWZ2YXIgZXJ0LWJhdGNoLWJhY2t0cmFjZS1y
aWdodC1tYXJnaW4gNzAKKyAgIlRoZSBtYXhpbXVtIGxpbmUgbGVuZ3RoIGZvciBwcmludGluZyBi
YWNrdHJhY2VzIGluIGBlcnQtcnVuLXRlc3RzLWJhdGNoJy4iKQorCiA7OzsjIyNhdXRvbG9hZAog
KGRlZnVuIGVydC1ydW4tdGVzdHMtYmF0Y2ggKCZvcHRpb25hbCBzZWxlY3RvcikKICAgIlJ1biB0
aGUgdGVzdHMgc3BlY2lmaWVkIGJ5IFNFTEVDVE9SLCBwcmludGluZyByZXN1bHRzIHRvIHRoZSB0
ZXJtaW5hbC4KQEAgLTE0NzEsMTQgKzE0ODgsMTcgQEAgZXJ0LXJ1bi10ZXN0cy1iYXRjaAogICAg
ICAgICAgICAgKGxldCogKChtYXggKHByaW4xLXRvLXN0cmluZyAobGVuZ3RoIChlcnQtLXN0YXRz
LXRlc3RzIHN0YXRzKSkpKQogICAgICAgICAgICAgICAgICAgIChmb3JtYXQtc3RyaW5nIChjb25j
YXQgIiU5cyAgJSIKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIChw
cmluMS10by1zdHJpbmcgKGxlbmd0aCBtYXgpKQotICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgInMvIiBtYXggIiAgJVMgKCVmIHNlYykiKSkpCisgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAicy8iIG1heCAiICAlUyAoJWYgc2VjKSVzIikp
KQogICAgICAgICAgICAgICAobWVzc2FnZSBmb3JtYXQtc3RyaW5nCiAgICAgICAgICAgICAgICAg
ICAgICAgIChlcnQtc3RyaW5nLWZvci10ZXN0LXJlc3VsdCByZXN1bHQKICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIChlcnQtdGVzdC1yZXN1bHQtZXhw
ZWN0ZWQtcAogICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIHRlc3QgcmVzdWx0KSkKICAgICAgICAgICAgICAgICAgICAgICAgKDErIChlcnQtLXN0YXRz
LXRlc3QtcG9zIHN0YXRzIHRlc3QpKQogICAgICAgICAgICAgICAgICAgICAgICAoZXJ0LXRlc3Qt
bmFtZSB0ZXN0KQotICAgICAgICAgICAgICAgICAgICAgICAoZXJ0LXRlc3QtcmVzdWx0LWR1cmF0
aW9uIHJlc3VsdCkpKSkpKSkpCisgICAgICAgICAgICAgICAgICAgICAgIChlcnQtdGVzdC1yZXN1
bHQtZHVyYXRpb24gcmVzdWx0KQorICAgICAgICAgICAgICAgICAgICAgICAoaWYgKGVydC10ZXN0
LXJlc3VsdC1leHBlY3RlZC1wIHRlc3QgcmVzdWx0KQorICAgICAgICAgICAgICAgICAgICAgICAg
ICAgIiIKKyAgICAgICAgICAgICAgICAgICAgICAgICAoY29uY2F0ICIgIiAoZXJ0LXRlc3QtbG9j
YXRpb24gdGVzdCkpKSkpKSkpKSkKICAgIG5pbCkpCiAKIDs7OyMjI2F1dG9sb2FkCi0tIAoyLjMw
LjIKCg==
--000000000000e2ddb405cefa06ff--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 20 Aug 2020 16:51:51 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Aug 20 12:51:51 2020
Received: from localhost ([127.0.0.1]:43825 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1k8nn4-0000qj-WE
	for submit <at> debbugs.gnu.org; Thu, 20 Aug 2020 12:51:51 -0400
Received: from mail-ot1-f53.google.com ([209.85.210.53]:42877)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <p.stephani2@HIDDEN>) id 1k8nn3-0000qW-2w
 for 22471 <at> debbugs.gnu.org; Thu, 20 Aug 2020 12:51:50 -0400
Received: by mail-ot1-f53.google.com with SMTP id v21so1977374otj.9
 for <22471 <at> debbugs.gnu.org>; Thu, 20 Aug 2020 09:51:48 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=yJlU8QuWYQiZqyGc4Dw9F8yql2pHBDtkHE+dPMAi9Nk=;
 b=szTdYNq7eY9+uZwAcvR6TVvyayP9dapv//uLRcOGv6DTYd8Ft1Wc9v0e0hFziMckC2
 6cX5eVjMcPjwzKplPDhb15DdWb4nAksGPkss4/1grMqgQwKNnllzfHg94H3mINP9Vml8
 03ykkdi2SkZ1ak4BHhnYST6yflJEjomQZ1N0Untfp0wXi869tyLhmo/S2M7eP2UFL7we
 kcetSI0wqxdTwSJWtasTq81N8IEOf/Yk6lExOwGUFh03Bbl3nf4nmKm+ii95QeSEl8jm
 xTS2LV3SNlpuUbHYqDchuuoPgijKTd9ku3c98lyOfst2GqlJDog33v+fcSYvRZ+Dp+0B
 bYdg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=yJlU8QuWYQiZqyGc4Dw9F8yql2pHBDtkHE+dPMAi9Nk=;
 b=QA0tfnPPcN0qawejXZAMuXIT9KP3YmSerTPk6v3E256oHrRNPUhaIw5Qr/So+8FP+E
 fGjyqUfvUqfPGTUIHIQctSTC6+cvYnM3UStZH7HiuL+68rrpmQJgcAvGF+VMXQXoFGFx
 yZlndEstiqu8u7GN4a8dNcEqPkRqaa4h/EaW+/wQ0nxKAo1xgqQsc+aThlcQMGAm6i80
 Ry/hXC3J/x2qnzugBJJHxrj5R4XxNw1Y+X/aHbHhbLRqsmgXAg5XHGiiqHT6b8Opr6M1
 pD/u0QD1KDRys/I9XXlaiilYpo4MZ+Jtlqrwl56JBJLKNkutBmbbj/t/Kx1Uv++F5F7r
 dX5w==
X-Gm-Message-State: AOAM530oPR34hwdp2jwrk4xXTpI1r+VSpsL1xbDQ01gCC+kRrTnOQ2zg
 xySAY2YY+i74ogD6+o6aEoNF6IkdwKuYo/w576A=
X-Google-Smtp-Source: ABdhPJwDTaQcAXHT1jA4Uv72YazY9mcuDQqd9wFHGMivOuzuhcBkUG30cvWBTUgInyQyKu47KU8sE2v0Ze7Brp93Vv0=
X-Received: by 2002:a05:6830:4032:: with SMTP id
 i18mr2619404ots.174.1597942303159; 
 Thu, 20 Aug 2020 09:51:43 -0700 (PDT)
MIME-Version: 1.0
References: <87bn878jsv.fsf@HIDDEN> <87wo1tp8z2.fsf@HIDDEN>
In-Reply-To: <87wo1tp8z2.fsf@HIDDEN>
From: Philipp Stephani <p.stephani2@HIDDEN>
Date: Thu, 20 Aug 2020 18:51:32 +0200
Message-ID: <CAArVCkTzqS_RmHua2zpQFQwTG_0tKT9p8c7vwjcP8RNZ08ip_Q@HIDDEN>
Subject: Re: bug#22471: ert batch should print compile parsable error messages
To: Lars Ingebrigtsen <larsi@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
X-Spam-Score: 0.3 (/)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org, Phillip Lord <phillip.lord@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)

Am Do., 20. Aug. 2020 um 18:41 Uhr schrieb Lars Ingebrigtsen <larsi@HIDDEN>:
>
> phillip.lord@HIDDEN (Phillip Lord) writes:
>
> > Currently, when running in batch, ert prints out messages like so:
> >
> > Running 24 tests (2016-01-27 09:05:17+0000)
> >    passed   1/24  buffer-string=
> >    passed   2/24  buffer=
> > ...
> >    passed  22/24  sisyphus-test-with-find-file
> >    passed  23/24  to-string
> >    passed  24/24  with-temp-buffers
> >
> > Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
> >
> > 1 unexpected results:
> >    FAILED  crash-out
> >
> > It would be nice to add some compilation-mode parsable data to this. So,
> > something like
> >
> >    passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)
>
> I think that would look rather cluttered for the "passed" lines, but it
> would indeed be helpful on the FAILED lines.
>

I'm doing something similar in my Bazel ruleset for Elisp
(https://github.com/phst/rules_elisp/blob/0b24aa1660af2f6c668899bdd78aaba383d7ac18/elisp/ert/runner.el#L446).
It would be nice to have an API for this as my code needs a few hacks
and subtleties.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 20 Aug 2020 16:40:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Aug 20 12:40:04 2020
Received: from localhost ([127.0.0.1]:43773 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1k8nbg-0006lA-Cc
	for submit <at> debbugs.gnu.org; Thu, 20 Aug 2020 12:40:04 -0400
Received: from quimby.gnus.org ([95.216.78.240]:59802)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <larsi@HIDDEN>) id 1k8nbf-0006kc-NE
 for 22471 <at> debbugs.gnu.org; Thu, 20 Aug 2020 12:40:04 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnus.org;
 s=20200322; h=Content-Type:MIME-Version:Message-ID:In-Reply-To:Date:
 References:Subject:Cc:To:From:Sender:Reply-To:Content-Transfer-Encoding:
 Content-ID:Content-Description:Resent-Date:Resent-From:Resent-Sender:
 Resent-To:Resent-Cc:Resent-Message-ID:List-Id:List-Help:List-Unsubscribe:
 List-Subscribe:List-Post:List-Owner:List-Archive;
 bh=KhuXMs5WJlmJ2DWesrrLviJeI4ut/V09Ul9HUZsekug=; b=OgMgjC5ywfsaDakaQ2cyblAqeV
 c983JCuChJ3gYg5kbzCzim9F+k8f8Yg4h/Tcf1CLRCeelMUV1U/zl/chJ+GJ+3jaWqqo3TAzCERdV
 6laoOAKKxoivBjkw8KHD3plQHjZ9UUv/CIcxpOLR7k9On055xoyZ5uXhdSWiIZTJjeiE=;
Received: from cm-84.212.202.86.getinternet.no ([84.212.202.86] helo=xo)
 by quimby with esmtpsa (TLS1.3:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.92) (envelope-from <larsi@HIDDEN>)
 id 1k8nbO-0006v8-CV; Thu, 20 Aug 2020 18:39:57 +0200
From: Lars Ingebrigtsen <larsi@HIDDEN>
To: phillip.lord@HIDDEN (Phillip Lord)
Subject: Re: bug#22471: ert batch should print compile parsable error messages
References: <87bn878jsv.fsf@HIDDEN>
X-Now-Playing: The Declining Winter's _Endless Scenery_: ".....On the Mantle"
Date: Thu, 20 Aug 2020 18:39:45 +0200
In-Reply-To: <87bn878jsv.fsf@HIDDEN> (Phillip Lord's message of "Wed,
 27 Jan 2016 09:15:44 +0000")
Message-ID: <87wo1tp8z2.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/28.0.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Report: Spam detection software, running on the system "quimby.gnus.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 @@CONTACT_ADDRESS@@ for details.
 Content preview: phillip.lord@HIDDEN (Phillip Lord) writes: > Currently,
 when running in batch, ert prints out messages like so: > > Running 24 tests
 (2016-01-27 09:05:17+0000) > passed 1/24 buffer-string= > passed 2/24 buffer=
 > ... > passed 22/24 sisyphus-te [...] 
 Content analysis details:   (-2.9 points, 5.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -1.0 ALL_TRUSTED            Passed through trusted hosts only via SMTP
 -1.9 BAYES_00               BODY: Bayes spam probability is 0 to 1%
 [score: 0.0000]
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

phillip.lord@HIDDEN (Phillip Lord) writes:

> Currently, when running in batch, ert prints out messages like so:
>
> Running 24 tests (2016-01-27 09:05:17+0000)
>    passed   1/24  buffer-string=
>    passed   2/24  buffer=
> ...
>    passed  22/24  sisyphus-test-with-find-file
>    passed  23/24  to-string
>    passed  24/24  with-temp-buffers
>
> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
>
> 1 unexpected results:
>    FAILED  crash-out
>
> It would be nice to add some compilation-mode parsable data to this. So,
> something like
>
>    passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)

I think that would look rather cluttered for the "passed" lines, but it
would indeed be helpful on the FAILED lines.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 28 Jan 2016 17:15:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jan 28 12:15:02 2016
Received: from localhost ([127.0.0.1]:39995 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aOq9u-0003ai-HK
	for submit <at> debbugs.gnu.org; Thu, 28 Jan 2016 12:15:02 -0500
Received: from cheviot12.ncl.ac.uk ([128.240.234.12]:37704)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOq9r-0003aA-FI
 for 22471 <at> debbugs.gnu.org; Thu, 28 Jan 2016 12:15:00 -0500
Received: from smtpauth-vm.ncl.ac.uk ([10.8.233.129] helo=smtpauth.ncl.ac.uk)
 by cheviot12.ncl.ac.uk with esmtp (Exim 4.63)
 (envelope-from <phillip.lord@HIDDEN>)
 id 1aOq9q-0006M3-9x; Thu, 28 Jan 2016 17:14:58 +0000
Received: from jangai.ncl.ac.uk ([10.66.67.223] helo=localhost)
 by smtpauth.ncl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63)
 (envelope-from <phillip.lord@HIDDEN>)
 id 1aOq9p-0002Kk-PI; Thu, 28 Jan 2016 17:14:57 +0000
From: phillip.lord@HIDDEN (Phillip Lord)
To: Glenn Morris <rgm@HIDDEN>
Subject: Re: bug#22471: ert batch should print compile parsable error messages
References: <87bn878jsv.fsf@HIDDEN> <83pownyplf.fsf@HIDDEN>
 <87twlyd6yx.fsf@HIDDEN> <egmvrqa3wp.fsf@HIDDEN>
Date: Thu, 28 Jan 2016 17:14:57 +0000
In-Reply-To: <egmvrqa3wp.fsf@HIDDEN> (Glenn Morris's message of
 "Wed, 27 Jan 2016 20:28:22 -0500")
Message-ID: <87lh79iq26.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 22471
Cc: Eli Zaretskii <eliz@HIDDEN>, 22471 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Glenn Morris <rgm@HIDDEN> writes:

> Why would you call the batch version of ert from within Emacs,
> when M-x ert exists and does colour, hyperlinking to source, etc?

M-x ert is nice, but has one main failure. It's not running in a clean,
controllable Emacs; it's running in my Emacs. I use M-x ert only when I
am debugging failing tests.

The rest of the time, I run tests in a clean, newly launched Emacs with
a sandboxed .emacs.d/elpa. As a secondary advantage, it can be easier to
test two or three Emacs versions at the same time.

I've been writing a library to help get around this problem, but
fundamentally, the only clean way to test code is not to run it in the
same Emacs as is launching it. At some point, I'd like to add the
ability to launch an emacs for every test.

Phil





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 28 Jan 2016 01:29:05 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 27 20:29:05 2016
Received: from localhost ([127.0.0.1]:39044 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aObOT-0002x8-LK
	for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 20:29:05 -0500
Received: from eggs.gnu.org ([208.118.235.92]:42414)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <rgm@HIDDEN>) id 1aObOR-0002wd-QN
 for 22471 <at> debbugs.gnu.org; Wed, 27 Jan 2016 20:29:04 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <rgm@HIDDEN>) id 1aObOL-0008TY-Cl
 for 22471 <at> debbugs.gnu.org; Wed, 27 Jan 2016 20:28:58 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:41038)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <rgm@HIDDEN>)
 id 1aObNq-0008Nb-6k; Wed, 27 Jan 2016 20:28:26 -0500
Received: from rgm by fencepost.gnu.org with local (Exim 4.82)
 (envelope-from <rgm@HIDDEN>)
 id 1aObNn-0005sw-FR; Wed, 27 Jan 2016 20:28:24 -0500
From: Glenn Morris <rgm@HIDDEN>
To: phillip.lord@HIDDEN (Phillip Lord)
Subject: Re: bug#22471: ert batch should print compile parsable error messages
References: <87bn878jsv.fsf@HIDDEN> <83pownyplf.fsf@HIDDEN>
 <87twlyd6yx.fsf@HIDDEN>
X-Spook: Methamphetamine Domestic nuclear detection Standoff
X-Ran: &\2"$6G,2UGlcnh2\B^CgoRoKa$0bKMB"\0fC1Uj9-Vg/PmtmnvQA^kyw{*#f-j'5qHgH@
X-Hue: white
X-Debbugs-No-Ack: yes
X-Attribution: GM
Date: Wed, 27 Jan 2016 20:28:22 -0500
In-Reply-To: <87twlyd6yx.fsf@HIDDEN> (Phillip Lord's message of "Wed,
 27 Jan 2016 21:53:58 +0000")
Message-ID: <egmvrqa3wp.fsf@HIDDEN>
User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/)
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 22471
Cc: Eli Zaretskii <eliz@HIDDEN>, 22471 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)


Why would you call the batch version of ert from within Emacs,
when M-x ert exists and does colour, hyperlinking to source, etc?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 27 Jan 2016 21:54:03 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 27 16:54:03 2016
Received: from localhost ([127.0.0.1]:38987 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aOY2N-0006JI-0Z
	for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 16:54:03 -0500
Received: from cheviot12.ncl.ac.uk ([128.240.234.12]:35342)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOY2K-0006It-KT
 for 22471 <at> debbugs.gnu.org; Wed, 27 Jan 2016 16:54:01 -0500
Received: from smtpauth-vm.ncl.ac.uk ([10.8.233.129] helo=smtpauth.ncl.ac.uk)
 by cheviot12.ncl.ac.uk with esmtp (Exim 4.63)
 (envelope-from <phillip.lord@HIDDEN>)
 id 1aOY2J-0006bp-BV; Wed, 27 Jan 2016 21:53:59 +0000
Received: from cpc1-benw10-2-0-cust373.gate.cable.virginm.net ([77.98.219.118]
 helo=localhost)
 by smtpauth.ncl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63)
 (envelope-from <phillip.lord@HIDDEN>)
 id 1aOY2J-0000GH-DV; Wed, 27 Jan 2016 21:53:59 +0000
From: phillip.lord@HIDDEN (Phillip Lord)
To: Eli Zaretskii <eliz@HIDDEN>
Subject: Re: bug#22471: ert batch should print compile parsable error messages
References: <87bn878jsv.fsf@HIDDEN> <83pownyplf.fsf@HIDDEN>
Date: Wed, 27 Jan 2016 21:53:58 +0000
In-Reply-To: <83pownyplf.fsf@HIDDEN> (Eli Zaretskii's message of "Wed, 27 Jan
 2016 18:06:04 +0200")
Message-ID: <87twlyd6yx.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Eli Zaretskii <eliz@HIDDEN> writes:

>> From: phillip.lord@HIDDEN (Phillip Lord)
>> Date: Wed, 27 Jan 2016 09:15:44 +0000
>> 
>> 
>> Currently, when running in batch, ert prints out messages like so:
>> 
>> 
>> Running 24 tests (2016-01-27 09:05:17+0000)
>>    passed   1/24  buffer-string=
>>    passed   2/24  buffer=
>> ...
>>    passed  22/24  sisyphus-test-with-find-file
>>    passed  23/24  to-string
>>    passed  24/24  with-temp-buffers
>> 
>> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
>> 
>> 1 unexpected results:
>>    FAILED  crash-out
>> 
>> It would be nice to add some compilation-mode parsable data to this. So,
>> something like
>> 
>> 
>>    passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)
>
> FWIW, I have never seen a test suite that displayed in
> compilation-mode compatible format.  If we do that (and I'm
> necessarily opposed to that), we'd be the odd one out, I think.

There's a couple in the compilation mode regexp -- cucumber, ruby and
perl test harnesses and so forth.

I put a quick attempt at this up on feature/parsable-ert-output.

   passed  22/24  sisyphus-test-with-find-file at test/sisyphus-test.el line 277.
   passed  23/24  to-string at test/sisyphus-test.el line 96.
   passed  24/24  with-temp-buffers at test/sisyphus-test.el line 232.

Works, but makes for rather a cluttered output.

Thoughts welcome.

Phil




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at 22471 <at> debbugs.gnu.org:


Received: (at 22471) by debbugs.gnu.org; 27 Jan 2016 16:05:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 27 11:05:46 2016
Received: from localhost ([127.0.0.1]:38870 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aOSbJ-0004Rc-TH
	for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 11:05:46 -0500
Received: from eggs.gnu.org ([208.118.235.92]:55491)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <eliz@HIDDEN>) id 1aOSbH-0004RO-DP
 for 22471 <at> debbugs.gnu.org; Wed, 27 Jan 2016 11:05:43 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <eliz@HIDDEN>) id 1aOSb8-0000cE-BN
 for 22471 <at> debbugs.gnu.org; Wed, 27 Jan 2016 11:05:38 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:60243)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <eliz@HIDDEN>)
 id 1aOSb8-0000c1-7u; Wed, 27 Jan 2016 11:05:34 -0500
Received: from 84.94.185.246.cable.012.net.il ([84.94.185.246]:2008
 helo=HOME-C4E4A596F7)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128)
 (Exim 4.82) (envelope-from <eliz@HIDDEN>)
 id 1aOSb7-0002t4-4f; Wed, 27 Jan 2016 11:05:33 -0500
Date: Wed, 27 Jan 2016 18:06:04 +0200
Message-Id: <83pownyplf.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: phillip.lord@HIDDEN (Phillip Lord)
In-reply-to: <87bn878jsv.fsf@HIDDEN> (phillip.lord@HIDDEN)
Subject: Re: bug#22471: ert batch should print compile parsable error messages
References: <87bn878jsv.fsf@HIDDEN>
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: 22471
Cc: 22471 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: Eli Zaretskii <eliz@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

> From: phillip.lord@HIDDEN (Phillip Lord)
> Date: Wed, 27 Jan 2016 09:15:44 +0000
> 
> 
> Currently, when running in batch, ert prints out messages like so:
> 
> 
> Running 24 tests (2016-01-27 09:05:17+0000)
>    passed   1/24  buffer-string=
>    passed   2/24  buffer=
> ...
>    passed  22/24  sisyphus-test-with-find-file
>    passed  23/24  to-string
>    passed  24/24  with-temp-buffers
> 
> Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)
> 
> 1 unexpected results:
>    FAILED  crash-out
> 
> It would be nice to add some compilation-mode parsable data to this. So,
> something like
> 
> 
>    passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)

FWIW, I have never seen a test suite that displayed in
compilation-mode compatible format.  If we do that (and I'm
necessarily opposed to that), we'd be the odd one out, I think.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 27 Jan 2016 09:16:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Jan 27 04:16:01 2016
Received: from localhost ([127.0.0.1]:38066 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1aOMCn-0001bR-A1
	for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 04:16:01 -0500
Received: from eggs.gnu.org ([208.118.235.92]:54897)
 by debbugs.gnu.org with esmtp (Exim 4.84)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCl-0001b8-Kb
 for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 04:16:00 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCf-0000zd-Rd
 for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 04:15:54 -0500
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:36293)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCf-0000zZ-P1
 for submit <at> debbugs.gnu.org; Wed, 27 Jan 2016 04:15:53 -0500
Received: from eggs.gnu.org ([2001:4830:134:3::10]:45867)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCe-0001M2-UC
 for bug-gnu-emacs@HIDDEN; Wed, 27 Jan 2016 04:15:53 -0500
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCZ-0000yr-G6
 for bug-gnu-emacs@HIDDEN; Wed, 27 Jan 2016 04:15:52 -0500
Received: from cheviot12.ncl.ac.uk ([128.240.234.12]:38517)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCZ-0000yl-AL
 for bug-gnu-emacs@HIDDEN; Wed, 27 Jan 2016 04:15:47 -0500
Received: from smtpauth-vm.ncl.ac.uk ([10.8.233.129] helo=smtpauth.ncl.ac.uk)
 by cheviot12.ncl.ac.uk with esmtp (Exim 4.63)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCX-0005S5-BN
 for bug-gnu-emacs@HIDDEN; Wed, 27 Jan 2016 09:15:45 +0000
Received: from cpc1-benw10-2-0-cust373.gate.cable.virginm.net ([77.98.219.118]
 helo=localhost)
 by smtpauth.ncl.ac.uk with esmtpsa (TLSv1:AES128-SHA:128) (Exim 4.63)
 (envelope-from <phillip.lord@HIDDEN>) id 1aOMCX-00036x-CK
 for bug-gnu-emacs@HIDDEN; Wed, 27 Jan 2016 09:15:45 +0000
From: phillip.lord@HIDDEN (Phillip Lord)
To: bug-gnu-emacs@HIDDEN
Subject: ert batch should print compile parsable error messages
Date: Wed, 27 Jan 2016 09:15:44 +0000
Message-ID: <87bn878jsv.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.0 (----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.0 (----)


Currently, when running in batch, ert prints out messages like so:


Running 24 tests (2016-01-27 09:05:17+0000)
   passed   1/24  buffer-string=
   passed   2/24  buffer=
...
   passed  22/24  sisyphus-test-with-find-file
   passed  23/24  to-string
   passed  24/24  with-temp-buffers

Ran 24 tests, 23 results as expected, 1 unexpected (2016-01-27 09:05:18+0000)

1 unexpected results:
   FAILED  crash-out

It would be nice to add some compilation-mode parsable data to this. So,
something like


   passed   1/24  buffer-string=  (in ./test.sisyphus-test.el:22:)

It would also be nice to add colourisation: red and green for passed and
failed. In the idea world, these could be added as ansi colour codes to
the batch output of ERT. They should work on the command line then also.

More of an RFE than a bug, but it is odd that compilation mode works
with lots and lots of test frameworks, but not Emacs'

Phil




Acknowledgement sent to phillip.lord@HIDDEN (Phillip Lord):
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#22471; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 24 Oct 2021 14:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.