X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Alan Mackenzie <acm@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 27 Apr 2018 21:15:02 +0000
Resent-Message-ID: <handler.31290.B.152486364720568 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: 31290 <at> debbugs.gnu.org
X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.152486364720568
(code B ref -1); Fri, 27 Apr 2018 21:15:02 +0000
Received: (at submit) by debbugs.gnu.org; 27 Apr 2018 21:14:07 +0000
Received: from localhost ([127.0.0.1]:41390 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1fCAgx-0005Lg-FH
for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:14:07 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55182)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1fCAgv-0005L3-6Q
for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:14:06 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <acm@HIDDEN>) id 1fCAgp-0003ka-2K
for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level:
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:48842)
by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
(Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgo-0003kQ-VJ
for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:46147)
by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <acm@HIDDEN>)
id 1fCAgn-0002Jr-Ub
for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <acm@HIDDEN>) id 1fCAgi-0003iQ-Vj
for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:57 -0400
Received: from colin.muc.de ([193.149.48.1]:31573 helo=mail.muc.de)
by eggs.gnu.org with smtp (Exim 4.71) (envelope-from <acm@HIDDEN>)
id 1fCAgi-0003gc-KK
for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:52 -0400
Received: (qmail 77206 invoked by uid 3782); 27 Apr 2018 21:13:47 -0000
Received: from acm.muc.de (p5B14720D.dip0.t-ipconnect.de [91.20.114.13]) by
colin.muc.de (tmda-ofmipd) with ESMTP;
Fri, 27 Apr 2018 23:13:46 +0200
Received: (qmail 6051 invoked by uid 1000); 27 Apr 2018 21:08:59 -0000
Date: Fri, 27 Apr 2018 21:08:59 +0000
Message-ID: <20180427210859.GA6023@ACM>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.9.4 (2018-02-28)
X-Delivery-Agent: TMDA/1.1.12 (Macallan)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.3 (----)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.3 (-----)
Hello, Emacs.
There are fundamental bugs in syntax-propertize and
syntax-propertize-function. The doc string of the latter states:
The specified function may call `syntax-ppss' on any position before
END, ....
This is untrue. True is that syntax-ppss can be called on a position
only up to syntax-propertize--done. After this point, the syntax-table
properties haven't been applied, so calling syntax-ppss is, in general,
going to give a false result.
At least that would be true if syntax-propertize--done hadn't been
prematurely and spuriously increased, crudely to prevent an infinite
recursion, falsely indicating to the syntax-ppss infrastructure that the
syntax-table properties have already been applied to the region (BEGIN
END).
.... but it should not call `syntax-ppss-flush-cache', ....
Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done
back to its true value, allowing the wrongly allowed syntax-ppss calls at
a later position to cause a recursive loop.
.... which means that it should not call `syntax-ppss' on some
position and later modify the buffer on some earlier position.
This is a bad restriction, because sometimes syntax-table properties can
only be correctly determined by examining the syntax of later buffer
positions. An example of this is giving the string-fence syntax-table
text property to an unbalanced opening string quote, but not to correctly
matched quotes.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
The plain fact is that (syntax-ppss pos) calls (syntax-propertize pos),
so syntax-propertize cannot itself use syntax-ppss because of the
recursive loop thus created.
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
Proposed solutions:
1. Major modes' syntax-propertize-function's are somehow given read
access to syntax-propertize--done, and may call syntax-ppss up to that
point only. syntax-propertize--done is updated only after the
syntax-table properties have been applied. Or....
2. syntax-propertize-function's are banned from using syntax-ppss, the
documentation instead directing them to use parse-partial-sexp directly.
In either solution, the restriction on using syntax-ppss-flush-cache
would no longer be necessary, and there would be no restriction on
setting syntax-table text properties at an earlier position than the one
currently being analysed.
I think solution 2 is the better one.
--
Alan Mackenzie (Nuremberg, Germany).
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Alan Mackenzie <acm@HIDDEN> Subject: bug#31290: Acknowledgement (Fundamental bugs in syntax-propertize) Message-ID: <handler.31290.B.152486364720568.ack <at> debbugs.gnu.org> References: <20180427210859.GA6023@ACM> X-Gnu-PR-Message: ack 31290 X-Gnu-PR-Package: emacs Reply-To: 31290 <at> debbugs.gnu.org Date: Fri, 27 Apr 2018 21:15:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): bug-gnu-emacs@HIDDEN If you wish to submit further information on this problem, please send it to 31290 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 31290: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D31290 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Dmitry Gutov <dgutov@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 08 May 2018 12:36:02 +0000
Resent-Message-ID: <handler.31290.B31290.152578292814648 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: Alan Mackenzie <acm@HIDDEN>, 31290 <at> debbugs.gnu.org
Received: via spool by 31290-submit <at> debbugs.gnu.org id=B31290.152578292814648
(code B ref 31290); Tue, 08 May 2018 12:36:02 +0000
Received: (at 31290) by debbugs.gnu.org; 8 May 2018 12:35:28 +0000
Received: from localhost ([127.0.0.1]:53540 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1fG1q3-0003oC-Sb
for submit <at> debbugs.gnu.org; Tue, 08 May 2018 08:35:28 -0400
Received: from mail-wr0-f173.google.com ([209.85.128.173]:46771)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <raaahh@HIDDEN>) id 1fG1pz-0003nw-F7
for 31290 <at> debbugs.gnu.org; Tue, 08 May 2018 08:35:24 -0400
Received: by mail-wr0-f173.google.com with SMTP id a12-v6so1246492wrn.13
for <31290 <at> debbugs.gnu.org>; Tue, 08 May 2018 05:35:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
h=sender:subject:to:references:from:message-id:date:user-agent
:mime-version:in-reply-to:content-language:content-transfer-encoding;
bh=VYOIVEaIRrc4TFp82f2wDek7DlL8StKt4BkR4d74PTQ=;
b=mBuczawT4iEmzk3SgRZNvOGXdAukcQaHOUxrT24lNbUlt96YjdhPsy4t+BrbrskPuN
kF9nzgM8Kaz4yYP4CUy+KxcSVe3D9yQJu1mAFH4j0ruUsdMZdM/us/jYG1iLMQVUn4So
/FBgaUuYwblAP8X+ATTnXJcFDbk+ihJQGfPVkJ+E+MNF5mDaxJu1t4HpMHaLKjsRpvgn
mEN9gVjBKS8Qe2WfhRbcPnbdNLBpTR01xA/2g4Te8xwkzdrG8M8Hiigo5E3iSn7y39g/
wuh46xbIUrLCQ0+WQ9fzHFZmSiAb9a6URJLR6M353/E8lWjbvffv9blE9FMc/VpxPrFY
ZacQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:sender:subject:to:references:from:message-id
:date:user-agent:mime-version:in-reply-to:content-language
:content-transfer-encoding;
bh=VYOIVEaIRrc4TFp82f2wDek7DlL8StKt4BkR4d74PTQ=;
b=gJ8TsstQ9bA+oNJfNyIzn5rgvoyMIFqPyQt22x3NqfaGYd4ZrcWki6L6xUaVe4V556
6w3o+yYSidudHgSiz5HBHYj56HYJaf6+5dMWsuep3tL2JmJ5pkFF3FJDfECfV434OAGM
7TLDAI/wwkVnCQPXINIAN6YM4gIBwq5f8zy6BF57zcQG0pc8VkBD6nxfoknl1T0Ccz0F
mIkb0t6CpD5xlG4oEcTw8LIPD3X7WimbAUteNZWNqpugeovX7uhPtRxxgVT6inKbpi1R
eqqfuqBJwFW9IX8pyoXxc2piJyZThNDlGkNVkrTXEmJ+RlN80S93FvVyvQtuEx/YbKBz
bFMA==
X-Gm-Message-State: ALQs6tDxS8uqYvu8Zf69wHQUVyTu3Q9jLIHJUNb+ADf7ERHoPVW2nYQD
hmXAIF8xwWudh/0gLIpfXhNR7RDz
X-Google-Smtp-Source: AB8JxZoHbA1vlFJmZXO2KGHVnMZM2QfPcNZW5FPh2e/EflgVZ3VLKZE1Bf3mKTRf2Uo+y9fPzW247w==
X-Received: by 2002:adf:b78b:: with SMTP id
s11-v6mr26222145wre.247.1525782917366;
Tue, 08 May 2018 05:35:17 -0700 (PDT)
Received: from [192.168.0.200] ([212.50.99.193])
by smtp.googlemail.com with ESMTPSA id
h133sm11716321wmf.47.2018.05.08.05.35.15
(version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
Tue, 08 May 2018 05:35:16 -0700 (PDT)
References: <20180427210859.GA6023@ACM>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN>
Date: Tue, 8 May 2018 15:35:14 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
Thunderbird/60.0
MIME-Version: 1.0
In-Reply-To: <20180427210859.GA6023@ACM>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.5 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)
On 4/28/18 12:08 AM, Alan Mackenzie wrote:
> At least that would be true if syntax-propertize--done hadn't been
> prematurely and spuriously increased, crudely to prevent an infinite
> recursion, falsely indicating to the syntax-ppss infrastructure that the
> syntax-table properties have already been applied to the region (BEGIN
> END).
>
> .... but it should not call `syntax-ppss-flush-cache', ....
>
> Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done
> back to its true value, allowing the wrongly allowed syntax-ppss calls at
> a later position to cause a recursive loop.
Maybe we should "allow" it to loop, in certain cases? Leaving it to be
the responsibility of the programmer, to make sure the result doesn't
infloop, even if these rules are violated.
> .... which means that it should not call `syntax-ppss' on some
> position and later modify the buffer on some earlier position.
>
> This is a bad restriction, because sometimes syntax-table properties can
> only be correctly determined by examining the syntax of later buffer
> positions. An example of this is giving the string-fence syntax-table
> text property to an unbalanced opening string quote, but not to correctly
> matched quotes.
I'm not exactly convinced by the given example (why would we use the
string-fence in that case?), but it might be better if something like
this was possible, indeed.
> 2. syntax-propertize-function's are banned from using syntax-ppss, the
> documentation instead directing them to use parse-partial-sexp directly.
The ones that currently call syntax-ppss, can't simply switch over to
parse-partial-sexp without becoming slower due to the lack of cache.
Before tackling this bug, I'd rather we see a real-world problem that it
caused, and pick a particular approach based on it.
But off the top of my head, we could introduce a "stricter but somewhat
slower" variation of syntax-ppss to be called inside
syntax-propertize-function's, which would treat the values in question
more carefully, somehow.
X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Alan Mackenzie <acm@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 12 May 2018 11:34:01 +0000
Resent-Message-ID: <handler.31290.B31290.152612483114047 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: Dmitry Gutov <dgutov@HIDDEN>
Cc: 31290 <at> debbugs.gnu.org
Received: via spool by 31290-submit <at> debbugs.gnu.org id=B31290.152612483114047
(code B ref 31290); Sat, 12 May 2018 11:34:01 +0000
Received: (at 31290) by debbugs.gnu.org; 12 May 2018 11:33:51 +0000
Received: from localhost ([127.0.0.1]:58989 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1fHSmc-0003eV-UA
for submit <at> debbugs.gnu.org; Sat, 12 May 2018 07:33:51 -0400
Received: from colin.muc.de ([193.149.48.1]:38221 helo=mail.muc.de)
by debbugs.gnu.org with smtp (Exim 4.84_2)
(envelope-from <acm@HIDDEN>) id 1fHSma-0003eL-Fr
for 31290 <at> debbugs.gnu.org; Sat, 12 May 2018 07:33:49 -0400
Received: (qmail 81713 invoked by uid 3782); 12 May 2018 11:33:47 -0000
Received: from acm.muc.de (p5B146231.dip0.t-ipconnect.de [91.20.98.49]) by
colin.muc.de (tmda-ofmipd) with ESMTP;
Sat, 12 May 2018 13:33:45 +0200
Received: (qmail 6270 invoked by uid 1000); 12 May 2018 11:26:12 -0000
Date: Sat, 12 May 2018 11:26:12 +0000
Message-ID: <20180512112612.GA4766@ACM>
References: <20180427210859.GA6023@ACM>
<e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN>
User-Agent: Mutt/1.9.4 (2018-02-28)
X-Delivery-Agent: TMDA/1.1.12 (Macallan)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello, Dmitry.
On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote:
> On 4/28/18 12:08 AM, Alan Mackenzie wrote:
> > At least that would be true if syntax-propertize--done hadn't been
> > prematurely and spuriously increased, crudely to prevent an infinite
> > recursion, falsely indicating to the syntax-ppss infrastructure that the
> > syntax-table properties have already been applied to the region (BEGIN
> > END).
> > .... but it should not call `syntax-ppss-flush-cache', ....
> > Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done
> > back to its true value, allowing the wrongly allowed syntax-ppss calls at
> > a later position to cause a recursive loop.
> Maybe we should "allow" it to loop, in certain cases? Leaving it to be
> the responsibility of the programmer, to make sure the result doesn't
> infloop, even if these rules are violated.
I'm not sure how this could work. We would need to formalise the rules
very carefully, to avoid the need to read syntax.{c,el}'s source code.
> > .... which means that it should not call `syntax-ppss' on some
> > position and later modify the buffer on some earlier position.
> > This is a bad restriction, because sometimes syntax-table properties can
> > only be correctly determined by examining the syntax of later buffer
> > positions. An example of this is giving the string-fence syntax-table
> > text property to an unbalanced opening string quote, but not to correctly
> > matched quotes.
> I'm not exactly convinced by the given example (why would we use the
> string-fence in that case?), but it might be better if something like
> this was possible, indeed.
String fence can be used to signal to font lock that the delimiter
(together with the "mismatching" unescaped EOL) should be fontified in
warning face.
A better example might be C++ Mode's marking of a "< ... >" pair with
paren syntax. This isn't done with syntax-propertize-function (as you
know), but it would be nice if this were possible.
> > 2. syntax-propertize-function's are banned from using syntax-ppss, the
> > documentation instead directing them to use parse-partial-sexp directly.
> The ones that currently call syntax-ppss, can't simply switch over to
> parse-partial-sexp without becoming slower due to the lack of cache.
The cache at the pertinent buffer position doesn't exist at the time:
consistent syntax-table properties aren't on the preceding buffer
positions.
> Before tackling this bug, I'd rather we see a real-world problem that it
> caused, and pick a particular approach based on it.
My enhancements for bug#30393: "24.4; cperl-mode: indentation failure -
Documentation enhancements", where (almost) any change which affects the
syntactic state is programmed to call syntax-ppss-flush-cache from the C
level, clashes with the mechanism in this bug report. Most of the time
it's fine, but when a change affecting the syntactic state is made from
inside a synax-propertize-function, Emacs goes into an infinite recursive
loop.
This isn't good.
> But off the top of my head, we could introduce a "stricter but somewhat
> slower" variation of syntax-ppss to be called inside
> syntax-propertize-function's, which would treat the values in question
> more carefully, somehow.
That's an idea worth exploring.
--
Alan Mackenzie (Nuremberg, Germany).
X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Andreas =?UTF-8?Q?R=C3=B6hler?= <andreas.roehler@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sun, 13 May 2018 07:33:01 +0000
Resent-Message-ID: <handler.31290.B.15261967324782 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords:
To: 31290 <at> debbugs.gnu.org
X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.15261967324782
(code B ref -1); Sun, 13 May 2018 07:33:01 +0000
Received: (at submit) by debbugs.gnu.org; 13 May 2018 07:32:12 +0000
Received: from localhost ([127.0.0.1]:60116 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1fHlUK-0001F4-8N
for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:12 -0400
Received: from eggs.gnu.org ([208.118.235.92]:58068)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <andreas.roehler@HIDDEN>) id 1fHlUJ-0001Er-0r
for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:11 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <andreas.roehler@HIDDEN>) id 1fHlUC-0004lJ-J7
for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:05 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level:
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:58346)
by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
(Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>)
id 1fHlUC-0004lC-Fl
for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:04 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:49027)
by lists.gnu.org with esmtp (Exim 4.71)
(envelope-from <andreas.roehler@HIDDEN>) id 1fHlUB-0002Fc-1I
for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:32:04 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
(envelope-from <andreas.roehler@HIDDEN>) id 1fHlU7-0004jq-SG
for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:32:03 -0400
Received: from mout.kundenserver.de ([212.227.126.130]:38883)
by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
(Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>)
id 1fHlU7-0004jL-I8
for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:31:59 -0400
Received: from [192.168.178.35] ([188.103.30.197]) by mrelayeu.kundenserver.de
(mreue006 [212.227.15.167]) with ESMTPSA (Nemesis) id
0MPeS5-1fD6ej3sUD-004kdl for <bug-gnu-emacs@HIDDEN>; Sun, 13 May 2018
09:31:57 +0200
References: <20180427210859.GA6023@ACM>
<e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN> <20180512112612.GA4766@ACM>
From: Andreas =?UTF-8?Q?R=C3=B6hler?= <andreas.roehler@HIDDEN>
Message-ID: <79bd61c9-1d9c-d2f4-9206-3623a0553ff1@HIDDEN>
Date: Sun, 13 May 2018 09:33:20 +0200
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:52.0) Gecko/20100101
Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <20180512112612.GA4766@ACM>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K1:4xXONJZrL2d6kiMkNZaUsOONQr6go7o5lUXxB99/1i4sr/elsDP
EjXZ3/4Wi3N3/KZeg4fKcCcGoTP5MO0alvUZuIUwEDLSWrgQMad0cHk9iROW2+J+sq3/5n/
AHcYxPnfVqAi72F8E9++IUcX8webunvTT1zaMU7jPMks6ePqfnYJHsXtbTDrwLXyEkre3oJ
jrPcwTaRu9egEXyhJtmUA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:vCBxJ2fpsPY=:HTM9dqgxmqPcnCQwbXQWif
+5TxWJGJ2dUjXGrE2Yx1BewEZIBSie67aIoK4F4IBqhmyKeoLeXbei7RbpMiHJVHVxRh4WG3N
LOuoiVHBhgab6khCAubVuvLBnd3+kZ4kh6/YkID0NWqf1uUB686EydgR4nZKYXrSDrjH3w7Tq
XW0/YoGi3kYCZdRx+63nqalSHjeoUdy8L68tTMeLMlmA9iH1bTzgwoA38RuqAcL/XlHdwJiaA
KQcWt1hkphNr7JfOI8IGkNJr5XzIb8UzSV0f7vPtZkaWRnCCT/t6geg4UceL+8Qe4smd3yJ//
pQQE3JsIwcw79qbHF3m9T6q2TGC3QWtMzwYnTstLQMBO55nBxOcrlu6Z6hkGtV7gzJwFlJmTr
WBQ2oxTSn4K3MPwyNuX52GkAFlalntq3dPPB1Qnn3QbKJRSD4JXEL3rqZZJgvYJBL34TPd0ki
xdN4g0xFIMpe2HZYsqWwnkDFlhPEsq+wYaTReZIRgQEDXjQZL/vBLJRlP7e6VYCn8hV5JJh5t
jGzUtJE894vyn/g67qKwVOaYmnIzROC0JwHtVLp4NEONnY6LRff87Wxv77Dj1/bX/9YSyooIa
SYZnGKgiRaFtmbGdXTQAxAWCoECIs2A2ZvJOJVpbKs/RUb01XJfqozaSv42sPwv+vhRE55ITh
pRCaDmxloiRiQmbuSb1JpF9as54uZeMnK2zWyxrm1Ch40wO3pm/gaiTqlURFIHxv0ScqwqpgM
lUuv38zXk8FRTNhL
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
[fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)
On 12.05.2018 13:26, Alan Mackenzie wrote:
> Hello, Dmitry.
>
> On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote:
>> On 4/28/18 12:08 AM, Alan Mackenzie wrote:
>
>>> At least that would be true if syntax-propertize--done hadn't been
>>> prematurely and spuriously increased, crudely to prevent an infinite
>>> recursion, falsely indicating to the syntax-ppss infrastructure that the
>>> syntax-table properties have already been applied to the region (BEGIN
>>> END).
>
>>> .... but it should not call `syntax-ppss-flush-cache', ....
>
>>> Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done
>>> back to its true value, allowing the wrongly allowed syntax-ppss calls at
>>> a later position to cause a recursive loop.
>
>> Maybe we should "allow" it to loop, in certain cases? Leaving it to be
>> the responsibility of the programmer, to make sure the result doesn't
>> infloop, even if these rules are violated.
>
> I'm not sure how this could work. We would need to formalise the rules
> very carefully, to avoid the need to read syntax.{c,el}'s source code.
>
>>> .... which means that it should not call `syntax-ppss' on some
>>> position and later modify the buffer on some earlier position.
>
>>> This is a bad restriction, because sometimes syntax-table properties can
>>> only be correctly determined by examining the syntax of later buffer
>>> positions. An example of this is giving the string-fence syntax-table
>>> text property to an unbalanced opening string quote, but not to correctly
>>> matched quotes.
>
>> I'm not exactly convinced by the given example (why would we use the
>> string-fence in that case?), but it might be better if something like
>> this was possible, indeed.
>
> String fence can be used to signal to font lock that the delimiter
> (together with the "mismatching" unescaped EOL) should be fontified in
> warning face.
>
> A better example might be C++ Mode's marking of a "< ... >" pair with
> paren syntax. This isn't done with syntax-propertize-function (as you
> know), but it would be nice if this were possible.
>
>>> 2. syntax-propertize-function's are banned from using syntax-ppss, the
>>> documentation instead directing them to use parse-partial-sexp directly.
>
>> The ones that currently call syntax-ppss, can't simply switch over to
>> parse-partial-sexp without becoming slower due to the lack of cache.
>
> The cache at the pertinent buffer position doesn't exist at the time:
> consistent syntax-table properties aren't on the preceding buffer
> positions.
>
>> Before tackling this bug, I'd rather we see a real-world problem that it
>> caused, and pick a particular approach based on it.
>
> My enhancements for bug#30393: "24.4; cperl-mode: indentation failure -
> Documentation enhancements", where (almost) any change which affects the
> syntactic state is programmed to call syntax-ppss-flush-cache from the C
> level, clashes with the mechanism in this bug report. Most of the time
> it's fine, but when a change affecting the syntactic state is made from
> inside a synax-propertize-function, Emacs goes into an infinite recursive
> loop.
>
> This isn't good.
>
>> But off the top of my head, we could introduce a "stricter but somewhat
>> slower" variation of syntax-ppss to be called inside
>> syntax-propertize-function's, which would treat the values in question
>> more carefully, somehow.
>
> That's an idea worth exploring.
>
Hi folks,
from what I've seen month ago just may stress the term fundamental.
Gave up to follow details WRT to check-ins made.
That part needs some person treating the gordic knot according to its
quality...
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.