GNU bug report logs - #43190
feature/native-comp; failure to compile json-mode leaves Emacs broken

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Tom Gillespie <tgbugs@HIDDEN>; dated Fri, 4 Sep 2020 01:07:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 17 Sep 2020 17:49:45 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 17 13:49:45 2020
Received: from localhost ([127.0.0.1]:38945 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kIy2S-0001g6-Pu
	for submit <at> debbugs.gnu.org; Thu, 17 Sep 2020 13:49:45 -0400
Received: from lists.gnu.org ([209.51.188.17]:39990)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <tgbugs@HIDDEN>) id 1kIy2R-0001fw-2Y
 for submit <at> debbugs.gnu.org; Thu, 17 Sep 2020 13:49:43 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:45964)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <tgbugs@HIDDEN>) id 1kIy2Q-0004sX-Pt
 for bug-gnu-emacs@HIDDEN; Thu, 17 Sep 2020 13:49:42 -0400
Received: from mail-wr1-x443.google.com ([2a00:1450:4864:20::443]:36208)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <tgbugs@HIDDEN>) id 1kIy2O-0003ue-Qj
 for bug-gnu-emacs@HIDDEN; Thu, 17 Sep 2020 13:49:42 -0400
Received: by mail-wr1-x443.google.com with SMTP id z1so3005298wrt.3
 for <bug-gnu-emacs@HIDDEN>; Thu, 17 Sep 2020 10:49:40 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:references:in-reply-to:from:date:message-id:subject:to
 :cc; bh=19Yz+trWa8pKSE5GDGoQht8tnxtPMuQ2BGt75NxW6eA=;
 b=sJI9uxgpfEz+kty010fQasutK34tEux3MMaIU2J8ftV10d7JpVqH9OTD1CM3/K+BKy
 gx3zE9ZM2uU/3bETv81aDQGCF5p/avLlU4rawfz3MlTAQk4qHAFh7ZPcKRE6rwf7X/wI
 bJ77P9sDNpxorYSLWBBhmDXLS2/7Abs/UgCjcqpUKqX95alLF5clt8AE7Uv3ge0FfWLg
 IAr7m7ugGIuyahUFHEXmng2NjH1Iep8Yrvj1Bk4OSGjtKdr1POvxWUZycEqlRhpc/zDj
 pOxnArSflQRdxJFGzvy3Nvi0JXm9Wg3v332ekN+usvlDHRCi4KDygzyW82cSbbpxK8zy
 0kMQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:references:in-reply-to:from:date
 :message-id:subject:to:cc;
 bh=19Yz+trWa8pKSE5GDGoQht8tnxtPMuQ2BGt75NxW6eA=;
 b=CArLZ02+JkIpbNM+wv+pbAwH0smgDwmc7MBssu2oRZtY1lF12bqIEaJmSnhR5rcqSu
 La9Z3/evE94RzO0MDAQqLdMoZT3P7E8lag2cL5Qo4KuARwgN9TE+LGwATJ/+uOFp/Ixt
 v91BPkrl7uzf7wkZc+TEZLusfyr2XTMRGP0c/npVmh/tgwnsX14vopYgnbxtMujHBLgZ
 QJ8mT2oxk/rN0LrlTuQ76mwFsq2HVt9XoBONpoYYmnsiigMPnupEjc3c+XC//q8fVpHv
 dF3Fj944nC8/ujSGlchBjqewFBfqqbqJwKG1kvcxmOfSeVxYDGKEyx+aGzEQvnZ1UWac
 jrOg==
X-Gm-Message-State: AOAM533UKs4DCfxEg6Acuco4FLvz+8TEbFp5n+hPhn1te7QNFHMq9NTA
 1mF8hYzR6f1FjMxIZcmM/w/9lXBCyo1z01rUVsk=
X-Google-Smtp-Source: ABdhPJyFCGmQFNApvFqjjB4r2ASMw94mOKFGjBevTn8rqB8+rSiQ7aQS2rNvp5aWyOslQF3vmBwS45R8QbbI7AA8k24=
X-Received: by 2002:adf:e44b:: with SMTP id t11mr10925240wrm.101.1600364979173; 
 Thu, 17 Sep 2020 10:49:39 -0700 (PDT)
MIME-Version: 1.0
References: <CA+G3_PMYen1zbSXk0UySyxRadveFATVr-O-AQXSjgQ0SP5z0Ag@HIDDEN>
 <xjfblimt1p6.fsf@HIDDEN>
In-Reply-To: <xjfblimt1p6.fsf@HIDDEN>
From: Tom Gillespie <tgbugs@HIDDEN>
Date: Thu, 17 Sep 2020 13:49:28 -0400
Message-ID: <CA+G3_PP_bgNd+GnSAxtG4AfN5yfknJZ2J9O-yhuMdQT_P-Dmag@HIDDEN>
Subject: Re: feature/native-comp;
 failure to compile json-mode leaves Emacs broken
To: Andrea Corallo <akrl@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Received-SPF: pass client-ip=2a00:1450:4864:20::443;
 envelope-from=tgbugs@HIDDEN; helo=mail-wr1-x443.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
 That's all we know.
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
Cc: Emacs Bug Report <bug-gnu-emacs@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Hi Andrea,
    Having poked about a bit more I agree. I don't think it has
anything to do with the native compilation directly. I suspect that it
has to do with how or exactly when the native compiled code is swapped
in. When byte compiling on the command line I get an error at the same
place that I get one when native compiling stating that json-snatcher
could not be required. However when byte compiling the file, the code
to add to kill-buffer-hook is not run. I'm wondering whether the load
starts, native comp kicks in, succeeds on the first couple of forms,
then fails for the same reason as bytecomp, except that the first
forms were successfully loaded? This assumes that native-comp compiles
and loads single expressions at a time. The other possibility that
comes to mind is that maybe the regular el file forms are loaded, and
then there is a step where the forms that would be replaced are
unloaded? My ignorance of how you do the swap doesn't help here. Note
that if you try to repro this now you will have to install an older
version of json-snacher since the patch to fix the issue has been
merged. Best,
Tom




On Fri, Sep 4, 2020 at 12:56 AM Andrea Corallo <akrl@HIDDEN> wrote:
>
> Hi Tom,
>
> thanks for the detailed report.
>
> Tom Gillespie <tgbugs@HIDDEN> writes:
>
> > While this is clearly a bug in the way that json-snatcher is written
> > (the symbol should never have been added to the hook before the
> > function was defined), it is only causing an issue with the new native
> > comp code.
>
> had no time to reproduce but to a very first look I'm quite unconvinced
> this is a native-comp bug.  I think you'd get the same exact error
> byte-compiling the file from shell using a vanilla Emacs.
>
> The fact is that native async compilations are child processes so they
> start with a clean environment.  This makes the native compiler a little
> more picky on bugs when the compilation is relaying silently on some
> definition sneaked in from the environment.
>
> This lead already to a number of packages to fix their requires.
>
> The proof is typically trying to byte-compile from shell, this give
> systematically the same error, this because the front-end of the native
> compiler *is* the byte-compiler :)
>
> Each compilation unit should be consistent and compilable autonomously
> without relaying on previous modifications of the environment.
>
> Thanks
>
>   Andrea




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#43190; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Sep 2020 07:56:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Sep 04 03:56:32 2020
Received: from localhost ([127.0.0.1]:37163 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kE6aG-0006yI-Ju
	for submit <at> debbugs.gnu.org; Fri, 04 Sep 2020 03:56:32 -0400
Received: from lists.gnu.org ([209.51.188.17]:52490)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <akrl@HIDDEN>) id 1kE6aE-0006yB-Py
 for submit <at> debbugs.gnu.org; Fri, 04 Sep 2020 03:56:31 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:40726)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <akrl@HIDDEN>) id 1kE6aE-0007AQ-Hl
 for bug-gnu-emacs@HIDDEN; Fri, 04 Sep 2020 03:56:30 -0400
Received: from mx.sdf.org ([205.166.94.24]:64062)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <akrl@HIDDEN>) id 1kE6aC-0000SF-GX
 for bug-gnu-emacs@HIDDEN; Fri, 04 Sep 2020 03:56:30 -0400
Received: from mab (ma.sdf.org [205.166.94.33])
 by mx.sdf.org (8.15.2/8.14.5) with ESMTP id 0847uLFc009204;
 Fri, 4 Sep 2020 07:56:21 GMT
From: Andrea Corallo <akrl@HIDDEN>
To: Tom Gillespie <tgbugs@HIDDEN>
Subject: Re: feature/native-comp;
 failure to compile json-mode leaves Emacs broken
References: <CA+G3_PMYen1zbSXk0UySyxRadveFATVr-O-AQXSjgQ0SP5z0Ag@HIDDEN>
Date: Fri, 04 Sep 2020 07:56:21 +0000
In-Reply-To: <CA+G3_PMYen1zbSXk0UySyxRadveFATVr-O-AQXSjgQ0SP5z0Ag@HIDDEN>
 (Tom Gillespie's message of "Thu, 3 Sep 2020 18:06:00 -0700")
Message-ID: <xjfblimt1p6.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=205.166.94.24; envelope-from=akrl@HIDDEN;
 helo=mx.sdf.org
X-detected-operating-system: by eggs.gnu.org: First seen = 2020/09/04 03:56:21
X-ACL-Warn: Detected OS   = ???
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
Cc: Emacs Bug Report <bug-gnu-emacs@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

Hi Tom,

thanks for the detailed report.

Tom Gillespie <tgbugs@HIDDEN> writes:

> While this is clearly a bug in the way that json-snatcher is written
> (the symbol should never have been added to the hook before the
> function was defined), it is only causing an issue with the new native
> comp code.

had no time to reproduce but to a very first look I'm quite unconvinced
this is a native-comp bug.  I think you'd get the same exact error
byte-compiling the file from shell using a vanilla Emacs.

The fact is that native async compilations are child processes so they
start with a clean environment.  This makes the native compiler a little
more picky on bugs when the compilation is relaying silently on some
definition sneaked in from the environment.

This lead already to a number of packages to fix their requires.

The proof is typically trying to byte-compile from shell, this give
systematically the same error, this because the front-end of the native
compiler *is* the byte-compiler :)

Each compilation unit should be consistent and compilable autonomously
without relaying on previous modifications of the environment.

Thanks

  Andrea




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#43190; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 4 Sep 2020 01:06:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 03 21:06:22 2020
Received: from localhost ([127.0.0.1]:36575 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kE0BI-0008PU-9Y
	for submit <at> debbugs.gnu.org; Thu, 03 Sep 2020 21:06:22 -0400
Received: from lists.gnu.org ([209.51.188.17]:36450)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <tgbugs@HIDDEN>) id 1kE0BE-0008PH-Cy
 for submit <at> debbugs.gnu.org; Thu, 03 Sep 2020 21:06:18 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54184)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <tgbugs@HIDDEN>) id 1kE0BE-00055U-7g
 for bug-gnu-emacs@HIDDEN; Thu, 03 Sep 2020 21:06:16 -0400
Received: from mail-wm1-x32f.google.com ([2a00:1450:4864:20::32f]:50379)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <tgbugs@HIDDEN>) id 1kE0BB-0002yP-Be
 for bug-gnu-emacs@HIDDEN; Thu, 03 Sep 2020 21:06:15 -0400
Received: by mail-wm1-x32f.google.com with SMTP id e17so4567205wme.0
 for <bug-gnu-emacs@HIDDEN>; Thu, 03 Sep 2020 18:06:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=mime-version:from:date:message-id:subject:to:cc
 :content-transfer-encoding;
 bh=ECll1aaavKUUl50un30zgtLlTssRolPHyxY8UOjZkZc=;
 b=RqjPTeXxZ7czDC12WrZ8naZcOP0yoD4hZU48XwmShpSpq+uF5BKZ+rCVD+p3QmeB8q
 ic81vIDvB395cgqZ3InepPtdJH2IkOUEwd17HMPUHVc0bqhUAnCZUojf0FVSS7RKzHJ4
 Y67R5DUB/tuNKq/Wl/eVa/I6frQsXqVr0LV/LxsLS9R645H98uo8zOzLuIbqLjszWF+G
 6ddLdVES/Di2lJUDNzTv5qOG98IIot95aOEBlW3GuDwT/cBfaJQhJNjsrCojKCdsm29u
 gHuoySeiiABaZzRJ0U+r8H5SjKZi3DVyxeq/7KmWde4ucHlkJ2BYVKWJtcnkad7tanDX
 Yubw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:from:date:message-id:subject:to:cc
 :content-transfer-encoding;
 bh=ECll1aaavKUUl50un30zgtLlTssRolPHyxY8UOjZkZc=;
 b=NFUfcv7oPPAxGRqx+6YiMBIOXpMCnBtfH+hWi7qCxhL5HHq8abD8vhL+xbA96C1YHd
 KHuIXUTygfuqr70nhu2aQQYFpHuBYYGCC5XNLUMLqj5Hn0bkFp3m30uuUB/zVfhFdn7H
 KIxxvNJEGR0gJeSH3C7eczoi5mn8WyyMAquwkIDWAewWphmgAzh8GoH0R9qrgkrVv7RI
 QBbR8sJglN26NqpnTtNvcpXU1iPTEWCvnq/MEMhXjNivtKZqGtjhFZWUEEzRT5+2oVf/
 vO112nut0xbzsFfgGf3vxMVIcb/Hspn4U9lmORLj+PAGEThEbAp0h7ORfmvLXzIZzFZR
 R4Lg==
X-Gm-Message-State: AOAM530EPHqAdzWuqKvyhht5cvBn21wZ63sXG3xEkUo1tM7m5UNw9qAO
 56p4nn+k1LIoScfBXOa40em/XIOItUHQ83veNh9Pxc+Hiet1iw==
X-Google-Smtp-Source: ABdhPJyQiZbycWDfLpmkGBiT4vE9tspHdg9vA9f/t/qrTYfbX8ZCNRn+R4ekKdnPInhlRVMxSY5askKAyafSm8xCJn0=
X-Received: by 2002:a1c:3d44:: with SMTP id k65mr4767279wma.132.1599181571023; 
 Thu, 03 Sep 2020 18:06:11 -0700 (PDT)
MIME-Version: 1.0
From: Tom Gillespie <tgbugs@HIDDEN>
Date: Thu, 3 Sep 2020 18:06:00 -0700
Message-ID: <CA+G3_PMYen1zbSXk0UySyxRadveFATVr-O-AQXSjgQ0SP5z0Ag@HIDDEN>
Subject: feature/native-comp; failure to compile json-mode leaves Emacs broken
To: Emacs Bug Report <bug-gnu-emacs@HIDDEN>
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=2a00:1450:4864:20::32f;
 envelope-from=tgbugs@HIDDEN; helo=mail-wm1-x32f.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
 That's all we know.
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: submit
Cc: Andrea Corallo <akrl@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Hi Andrea,
   Found another one. I don't know what is going wrong on
the compiler side that induces it, but hopefully the repro
can get you pointed in the right direction. Also sent to the
bug tracker so it should show up there shortly. I have left
the section on issues with kill-buffer in the report so that
anyone encountering the effect of the bug won't have to spend
the same amount of time I did figuring out what is happening.
Best!
Tom


This issue occurs at 3023eb569213a3dd5183640f6e322acd00ea536a.
It does not happen in emacs-27 nor on master at
ae6daa680a5f5f5fb9c6a15296e5e88c97cd770a.

When trying to load json-mode for the first time it fails to compile
due to some issue in json-snatcher, the error that is displayed is
json-mode.el:33:1: Error: Symbol=E2=80=99s function definition is void:
jsons-remove-buffer

I suspect that something goes wrong inside json-snatcher.el and the
end result is that =3D(add-hook 'kill-buffer-hook 'jsons-remove-buffer)=3D
on line json-snatcher.el:85 manages to run, but then the defun for
that function is never called. The fact that that function cannot be
found means that anything that calls kill-buffer will be broken, which
includes a terrifyingly large portion of Emacs' debugging
functionality (see footnote 1).  On a bad day this also breaks things
like the ability to exit Emacs via =3DC-x C-c=3D.

Fortunately in the repro below this state is not triggered so you can
at least exit.  Just in case I included the escape hatches that can be
evaluated using =3DC-x C-e=3D in the scratch buffer most relevantly =3D(pop
kill-buffer-hook)=3D.

Here is a simple reproduction that is hopefully enough
to track down what is going wrong with native compile.
I have not been able to produce a simple visible repro
using --batch, but I think the problem is clear enough.
#+begin_src bash
[[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \
emacs -q \
--eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \
--eval "(require 'package)" \
--eval "(add-to-list 'package-archives '(\"melpa\" .
\"https://melpa.org/packages/\") t)" \
--eval "(package-refresh-contents)" \
--eval "(package-install 'json-mode)"
#+end_src

The repro for emacs-27 without the --eval options.
#+begin_src bash
[[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \
emacs-27 -q \
#+end_src

The repro for the master branch without the --eval options.
#+begin_src bash
./autogen.sh
./configure
make -j 8
[[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \
src/emacs -q \
#+end_src

A longer reproduction that controls for a number of variables and
shows the larger issues with kill-buffer is below.

Run this and hit enter in the ielm buffer and you will experience
and odd hang. Hit enter again and then switch to the scratch buffer
if you want to play around with the additional issues.
#+begin_src bash
[[ -d "/tmp/test/.emacs.d/" ]] && rm -r /tmp/test/.emacs.d/; \
emacs -q \
--eval "(setq user-emacs-directory \"/tmp/test/.emacs.d/\")" \
--eval "(when (boundp 'comp-eln-load-path) (pop comp-eln-load-path)
(add-to-list 'comp-eln-load-path (concat user-emacs-directory
\"eln-cache\")))" \
--eval "(require 'package)" \
--eval "(add-to-list 'package-archives '(\"melpa\" .
\"https://melpa.org/packages/\") t)" \
--eval "(package-refresh-contents)" \
--eval "(ignore-errors (package-install 'json-mode))" \
--eval "(ielm)" \
--eval "(with-current-buffer \"*ielm*\" (insert \"(setq asdf (+ 1 2))\"))" =
\
--eval "(with-current-buffer \"*scratch*\"
(insert \"(pop kill-buffer-hook) ; calling this will revert the massive bre=
akage
asdf  ; C-x C-e this to see more brokeness with native comp or working with=
 27
(fmakunbound 'jsons-remove-buffer)  ; use to repro the effects without
native comp
C-x-C-e-me-i-am-not-bound  ; asdf will be bound if testing with 27
;; run these blocks to be able to safely enter the debugger
(setq debug-on-message \\\"break\\\")
(defvar break-message \\\"break\\\")
(defun jsons-remove-buffer ()
  (let ((bm break-message))
    (setq break-message \\\"\\\")
    (message bm)))\"))"
#+end_src

Backtraces.

#+begin_example elisp
Debugger entered--Lisp error: "break"
  message("break")
  (let ((bm break-message)) (setq break-message "") (message bm))
  jsons-remove-buffer()
  kill-buffer(#<buffer  *temp*-833840>)
  ielm-eval-input(#("(+ 1 2)" 0 7 (fontified t)) nil)
  ielm-send-input(nil)
  ielm-return()
  funcall-interactively(ielm-return)
  command-execute(ielm-return)
#+end_example

#+begin_example elisp
Debugger entered--Lisp error: "break"
  message("break")
  (let ((bm break-message)) (setq break-message "") (message bm))
  jsons-remove-buffer()
  kill-buffer(#<buffer  *temp*-61838>)
  #f(compiled-function () #<bytecode 0x11ac346e22dcf22>)()
  cl-print-to-string-with-limit(backtrace--print (void-variable asdf) 5000)
  backtrace--print-to-string((void-variable asdf) nil)
  backtrace-print-to-string((void-variable asdf))
  debugger--insert-header((error (void-variable asdf)))
  #f(compiled-function () #<bytecode 0x1f401d22beba>)()
  backtrace-print()
  debugger-setup-buffer((error (void-variable asdf)))
  debug(error (void-variable asdf))
  (progn asdf)
  elisp--eval-last-sexp(nil)
  eval-last-sexp(nil)
  funcall-interactively(eval-last-sexp nil)
  command-execute(eval-last-sexp)
#+end_example

Thoughts on the presence of a larger issue.

While this is clearly a bug in the way that json-snatcher is written
(the symbol should never have been added to the hook before the
function was defined), it is only causing an issue with the new native
comp code.  Furthermore this example reveals a number of systematic
issues with when and where kill-buffer is being called inside core
Emacs code, and with how hook failures on kill-buffer are being
handled. This was an absolute nightmare to debug and I think I will
submit a separate issue about the utterly undebuggable nature of
issues with kill-buffer-hook.

Thoughts on potential solutions while they are fresh in my mind. It
might be possible to add a check in add-hook that could be invoked for
hooks that want to require any symbol added to the hook list to
already be bound at the time that it is added to the hook list. This
would prevent the issue that we see with json-snatcher in the future.
However this does not prevent issues if someone was doing something
evil like using push to add to kill-buffer-hook. The error message
that is currently emitted for these kinds of failures is almost
good enough, I think that it just needs a note that the symbol was
originally found the kill-buffer-hook list and maybe provide
instructions on how to remove the symbol from the hook. For example
"kill-buffer: Symbol=E2=80=99s function definition is void: jsons-remove-bu=
ffer
Symbol jsons-remove-buffer in kill-buffer-hook list is void
you can run (remove-hook 'kill-buffer-hook 'jsons-remove-buffer)
to restore kill-buffer functionality"

Footnotes.

1. For example toggle-debug-on-error completely fails to function
   correctly in this case, as do things like ielm-return. Anything
   which uses with-temp-buffer and tries to print, such as
   cl-print-to-string-with-limit will have its output silenced or
   ignored due to the failure of kill-buffer in the unwind forms.
   Normal approaches to debugging nearly all wind up stuck in infinite
   loops since most of them make a call to kill-buffer somewhere.




Acknowledgement sent to Tom Gillespie <tgbugs@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#43190; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 17 Sep 2020 18:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.