GNU logs - #31290, boring messages


Message sent to bug-gnu-emacs@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Alan Mackenzie <acm@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Fri, 27 Apr 2018 21:15:02 +0000
Resent-Message-ID: <handler.31290.B.152486364720568 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: 31290 <at> debbugs.gnu.org
X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.152486364720568
          (code B ref -1); Fri, 27 Apr 2018 21:15:02 +0000
Received: (at submit) by debbugs.gnu.org; 27 Apr 2018 21:14:07 +0000
Received: from localhost ([127.0.0.1]:41390 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1fCAgx-0005Lg-FH
	for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:14:07 -0400
Received: from eggs.gnu.org ([208.118.235.92]:55182)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <acm@HIDDEN>) id 1fCAgv-0005L3-6Q
 for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:14:06 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <acm@HIDDEN>) id 1fCAgp-0003ka-2K
 for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:48842)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgo-0003kQ-VJ
 for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:46147)
 by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <acm@HIDDEN>)
 id 1fCAgn-0002Jr-Ub
 for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:58 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <acm@HIDDEN>) id 1fCAgi-0003iQ-Vj
 for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:57 -0400
Received: from colin.muc.de ([193.149.48.1]:31573 helo=mail.muc.de)
 by eggs.gnu.org with smtp (Exim 4.71) (envelope-from <acm@HIDDEN>)
 id 1fCAgi-0003gc-KK
 for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:52 -0400
Received: (qmail 77206 invoked by uid 3782); 27 Apr 2018 21:13:47 -0000
Received: from acm.muc.de (p5B14720D.dip0.t-ipconnect.de [91.20.114.13]) by
 colin.muc.de (tmda-ofmipd) with ESMTP;
 Fri, 27 Apr 2018 23:13:46 +0200
Received: (qmail 6051 invoked by uid 1000); 27 Apr 2018 21:08:59 -0000
Date: Fri, 27 Apr 2018 21:08:59 +0000
Message-ID: <20180427210859.GA6023@ACM>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
User-Agent: Mutt/1.9.4 (2018-02-28)
X-Delivery-Agent: TMDA/1.1.12 (Macallan)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.3 (----)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.3 (-----)

Hello, Emacs.

There are fundamental bugs in syntax-propertize and
syntax-propertize-function.  The doc string of the latter states:

    The specified function may call `syntax-ppss' on any position before
    END, ....

This is untrue.  True is that syntax-ppss can be called on a position
only up to syntax-propertize--done.  After this point, the syntax-table
properties haven't been applied, so calling syntax-ppss is, in general,
going to give a false result.

At least that would be true if syntax-propertize--done hadn't been
prematurely and spuriously increased, crudely to prevent an infinite
recursion, falsely indicating to the syntax-ppss infrastructure that the
syntax-table properties have already been applied to the region (BEGIN
END).

    .... but it should not call `syntax-ppss-flush-cache', ....

Why not?  Because syntax-ppss-flush-cache sets syntax-propertize--done
back to its true value, allowing the wrongly allowed syntax-ppss calls at
a later position to cause a recursive loop.

    .... which means that it should not call `syntax-ppss' on some
    position and later modify the buffer on some earlier position.

This is a bad restriction, because sometimes syntax-table properties can
only be correctly determined by examining the syntax of later buffer
positions.  An example of this is giving the string-fence syntax-table
text property to an unbalanced opening string quote, but not to correctly
matched quotes.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

The plain fact is that (syntax-ppss pos) calls (syntax-propertize pos),
so syntax-propertize cannot itself use syntax-ppss because of the
recursive loop thus created.

;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;

Proposed solutions:

1. Major modes' syntax-propertize-function's are somehow given read
access to syntax-propertize--done, and may call syntax-ppss up to that
point only.  syntax-propertize--done is updated only after the
syntax-table properties have been applied.  Or....

2. syntax-propertize-function's are banned from using syntax-ppss, the
documentation instead directing them to use parse-partial-sexp directly.

In either solution, the restriction on using syntax-ppss-flush-cache
would no longer be necessary, and there would be no restriction on
setting syntax-table text properties at an earlier position than the one
currently being analysed.

I think solution 2 is the better one.

--
Alan Mackenzie (Nuremberg, Germany).




Message sent:


Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Alan Mackenzie <acm@HIDDEN>
Subject: bug#31290: Acknowledgement (Fundamental bugs in syntax-propertize)
Message-ID: <handler.31290.B.152486364720568.ack <at> debbugs.gnu.org>
References: <20180427210859.GA6023@ACM>
X-Gnu-PR-Message: ack 31290
X-Gnu-PR-Package: emacs
Reply-To: 31290 <at> debbugs.gnu.org
Date: Fri, 27 Apr 2018 21:15:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-gnu-emacs@HIDDEN

If you wish to submit further information on this problem, please
send it to 31290 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
31290: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D31290
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems


Message sent to bug-gnu-emacs@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Dmitry Gutov <dgutov@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Tue, 08 May 2018 12:36:02 +0000
Resent-Message-ID: <handler.31290.B31290.152578292814648 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Alan Mackenzie <acm@HIDDEN>, 31290 <at> debbugs.gnu.org
Received: via spool by 31290-submit <at> debbugs.gnu.org id=B31290.152578292814648
          (code B ref 31290); Tue, 08 May 2018 12:36:02 +0000
Received: (at 31290) by debbugs.gnu.org; 8 May 2018 12:35:28 +0000
Received: from localhost ([127.0.0.1]:53540 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1fG1q3-0003oC-Sb
	for submit <at> debbugs.gnu.org; Tue, 08 May 2018 08:35:28 -0400
Received: from mail-wr0-f173.google.com ([209.85.128.173]:46771)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raaahh@HIDDEN>) id 1fG1pz-0003nw-F7
 for 31290 <at> debbugs.gnu.org; Tue, 08 May 2018 08:35:24 -0400
Received: by mail-wr0-f173.google.com with SMTP id a12-v6so1246492wrn.13
 for <31290 <at> debbugs.gnu.org>; Tue, 08 May 2018 05:35:23 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:subject:to:references:from:message-id:date:user-agent
 :mime-version:in-reply-to:content-language:content-transfer-encoding;
 bh=VYOIVEaIRrc4TFp82f2wDek7DlL8StKt4BkR4d74PTQ=;
 b=mBuczawT4iEmzk3SgRZNvOGXdAukcQaHOUxrT24lNbUlt96YjdhPsy4t+BrbrskPuN
 kF9nzgM8Kaz4yYP4CUy+KxcSVe3D9yQJu1mAFH4j0ruUsdMZdM/us/jYG1iLMQVUn4So
 /FBgaUuYwblAP8X+ATTnXJcFDbk+ihJQGfPVkJ+E+MNF5mDaxJu1t4HpMHaLKjsRpvgn
 mEN9gVjBKS8Qe2WfhRbcPnbdNLBpTR01xA/2g4Te8xwkzdrG8M8Hiigo5E3iSn7y39g/
 wuh46xbIUrLCQ0+WQ9fzHFZmSiAb9a6URJLR6M353/E8lWjbvffv9blE9FMc/VpxPrFY
 ZacQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:subject:to:references:from:message-id
 :date:user-agent:mime-version:in-reply-to:content-language
 :content-transfer-encoding;
 bh=VYOIVEaIRrc4TFp82f2wDek7DlL8StKt4BkR4d74PTQ=;
 b=gJ8TsstQ9bA+oNJfNyIzn5rgvoyMIFqPyQt22x3NqfaGYd4ZrcWki6L6xUaVe4V556
 6w3o+yYSidudHgSiz5HBHYj56HYJaf6+5dMWsuep3tL2JmJ5pkFF3FJDfECfV434OAGM
 7TLDAI/wwkVnCQPXINIAN6YM4gIBwq5f8zy6BF57zcQG0pc8VkBD6nxfoknl1T0Ccz0F
 mIkb0t6CpD5xlG4oEcTw8LIPD3X7WimbAUteNZWNqpugeovX7uhPtRxxgVT6inKbpi1R
 eqqfuqBJwFW9IX8pyoXxc2piJyZThNDlGkNVkrTXEmJ+RlN80S93FvVyvQtuEx/YbKBz
 bFMA==
X-Gm-Message-State: ALQs6tDxS8uqYvu8Zf69wHQUVyTu3Q9jLIHJUNb+ADf7ERHoPVW2nYQD
 hmXAIF8xwWudh/0gLIpfXhNR7RDz
X-Google-Smtp-Source: AB8JxZoHbA1vlFJmZXO2KGHVnMZM2QfPcNZW5FPh2e/EflgVZ3VLKZE1Bf3mKTRf2Uo+y9fPzW247w==
X-Received: by 2002:adf:b78b:: with SMTP id
 s11-v6mr26222145wre.247.1525782917366; 
 Tue, 08 May 2018 05:35:17 -0700 (PDT)
Received: from [192.168.0.200] ([212.50.99.193])
 by smtp.googlemail.com with ESMTPSA id
 h133sm11716321wmf.47.2018.05.08.05.35.15
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 08 May 2018 05:35:16 -0700 (PDT)
References: <20180427210859.GA6023@ACM>
From: Dmitry Gutov <dgutov@HIDDEN>
Message-ID: <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN>
Date: Tue, 8 May 2018 15:35:14 +0300
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101
 Thunderbird/60.0
MIME-Version: 1.0
In-Reply-To: <20180427210859.GA6023@ACM>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Spam-Score: 0.5 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)

On 4/28/18 12:08 AM, Alan Mackenzie wrote:

> At least that would be true if syntax-propertize--done hadn't been
> prematurely and spuriously increased, crudely to prevent an infinite
> recursion, falsely indicating to the syntax-ppss infrastructure that the
> syntax-table properties have already been applied to the region (BEGIN
> END).
> 
>      .... but it should not call `syntax-ppss-flush-cache', ....
> 
> Why not?  Because syntax-ppss-flush-cache sets syntax-propertize--done
> back to its true value, allowing the wrongly allowed syntax-ppss calls at
> a later position to cause a recursive loop.

Maybe we should "allow" it to loop, in certain cases? Leaving it to be 
the responsibility of the programmer, to make sure the result doesn't 
infloop, even if these rules are violated.

>      .... which means that it should not call `syntax-ppss' on some
>      position and later modify the buffer on some earlier position.
> 
> This is a bad restriction, because sometimes syntax-table properties can
> only be correctly determined by examining the syntax of later buffer
> positions.  An example of this is giving the string-fence syntax-table
> text property to an unbalanced opening string quote, but not to correctly
> matched quotes.

I'm not exactly convinced by the given example (why would we use the 
string-fence in that case?), but it might be better if something like 
this was possible, indeed.

> 2. syntax-propertize-function's are banned from using syntax-ppss, the
> documentation instead directing them to use parse-partial-sexp directly.

The ones that currently call syntax-ppss, can't simply switch over to 
parse-partial-sexp without becoming slower due to the lack of cache.

Before tackling this bug, I'd rather we see a real-world problem that it 
caused, and pick a particular approach based on it.

But off the top of my head, we could introduce a "stricter but somewhat 
slower" variation of syntax-ppss to be called inside 
syntax-propertize-function's, which would treat the values in question 
more carefully, somehow.




Message sent to bug-gnu-emacs@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Alan Mackenzie <acm@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sat, 12 May 2018 11:34:01 +0000
Resent-Message-ID: <handler.31290.B31290.152612483114047 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: Dmitry Gutov <dgutov@HIDDEN>
Cc: 31290 <at> debbugs.gnu.org
Received: via spool by 31290-submit <at> debbugs.gnu.org id=B31290.152612483114047
          (code B ref 31290); Sat, 12 May 2018 11:34:01 +0000
Received: (at 31290) by debbugs.gnu.org; 12 May 2018 11:33:51 +0000
Received: from localhost ([127.0.0.1]:58989 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1fHSmc-0003eV-UA
	for submit <at> debbugs.gnu.org; Sat, 12 May 2018 07:33:51 -0400
Received: from colin.muc.de ([193.149.48.1]:38221 helo=mail.muc.de)
 by debbugs.gnu.org with smtp (Exim 4.84_2)
 (envelope-from <acm@HIDDEN>) id 1fHSma-0003eL-Fr
 for 31290 <at> debbugs.gnu.org; Sat, 12 May 2018 07:33:49 -0400
Received: (qmail 81713 invoked by uid 3782); 12 May 2018 11:33:47 -0000
Received: from acm.muc.de (p5B146231.dip0.t-ipconnect.de [91.20.98.49]) by
 colin.muc.de (tmda-ofmipd) with ESMTP;
 Sat, 12 May 2018 13:33:45 +0200
Received: (qmail 6270 invoked by uid 1000); 12 May 2018 11:26:12 -0000
Date: Sat, 12 May 2018 11:26:12 +0000
Message-ID: <20180512112612.GA4766@ACM>
References: <20180427210859.GA6023@ACM>
 <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN>
User-Agent: Mutt/1.9.4 (2018-02-28)
X-Delivery-Agent: TMDA/1.1.12 (Macallan)
From: Alan Mackenzie <acm@HIDDEN>
X-Primary-Address: acm@HIDDEN
X-Spam-Score: -0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello, Dmitry.

On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote:
> On 4/28/18 12:08 AM, Alan Mackenzie wrote:

> > At least that would be true if syntax-propertize--done hadn't been
> > prematurely and spuriously increased, crudely to prevent an infinite
> > recursion, falsely indicating to the syntax-ppss infrastructure that the
> > syntax-table properties have already been applied to the region (BEGIN
> > END).

> >      .... but it should not call `syntax-ppss-flush-cache', ....

> > Why not?  Because syntax-ppss-flush-cache sets syntax-propertize--done
> > back to its true value, allowing the wrongly allowed syntax-ppss calls at
> > a later position to cause a recursive loop.

> Maybe we should "allow" it to loop, in certain cases? Leaving it to be 
> the responsibility of the programmer, to make sure the result doesn't 
> infloop, even if these rules are violated.

I'm not sure how this could work.  We would need to formalise the rules
very carefully, to avoid the need to read syntax.{c,el}'s source code.

> >      .... which means that it should not call `syntax-ppss' on some
> >      position and later modify the buffer on some earlier position.

> > This is a bad restriction, because sometimes syntax-table properties can
> > only be correctly determined by examining the syntax of later buffer
> > positions.  An example of this is giving the string-fence syntax-table
> > text property to an unbalanced opening string quote, but not to correctly
> > matched quotes.

> I'm not exactly convinced by the given example (why would we use the 
> string-fence in that case?), but it might be better if something like 
> this was possible, indeed.

String fence can be used to signal to font lock that the delimiter
(together with the "mismatching" unescaped EOL) should be fontified in
warning face.

A better example might be C++ Mode's marking of a "< ... >" pair with
paren syntax.  This isn't done with syntax-propertize-function (as you
know), but it would be nice if this were possible.

> > 2. syntax-propertize-function's are banned from using syntax-ppss, the
> > documentation instead directing them to use parse-partial-sexp directly.

> The ones that currently call syntax-ppss, can't simply switch over to 
> parse-partial-sexp without becoming slower due to the lack of cache.

The cache at the pertinent buffer position doesn't exist at the time:
consistent syntax-table properties aren't on the preceding buffer
positions.

> Before tackling this bug, I'd rather we see a real-world problem that it 
> caused, and pick a particular approach based on it.

My enhancements for bug#30393: "24.4; cperl-mode: indentation failure -
Documentation enhancements", where (almost) any change which affects the
syntactic state is programmed to call syntax-ppss-flush-cache from the C
level, clashes with the mechanism in this bug report.  Most of the time
it's fine, but when a change affecting the syntactic state is made from
inside a synax-propertize-function, Emacs goes into an infinite recursive
loop.

This isn't good.

> But off the top of my head, we could introduce a "stricter but somewhat 
> slower" variation of syntax-ppss to be called inside 
> syntax-propertize-function's, which would treat the values in question 
> more carefully, somehow.

That's an idea worth exploring.

-- 
Alan Mackenzie (Nuremberg, Germany).




Message sent to bug-gnu-emacs@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: bug#31290: Fundamental bugs in syntax-propertize
Resent-From: Andreas =?UTF-8?Q?R=C3=B6hler?= <andreas.roehler@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-gnu-emacs@HIDDEN
Resent-Date: Sun, 13 May 2018 07:33:01 +0000
Resent-Message-ID: <handler.31290.B.15261967324782 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 31290
X-GNU-PR-Package: emacs
X-GNU-PR-Keywords: 
To: 31290 <at> debbugs.gnu.org
X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.15261967324782
          (code B ref -1); Sun, 13 May 2018 07:33:01 +0000
Received: (at submit) by debbugs.gnu.org; 13 May 2018 07:32:12 +0000
Received: from localhost ([127.0.0.1]:60116 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1fHlUK-0001F4-8N
	for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:12 -0400
Received: from eggs.gnu.org ([208.118.235.92]:58068)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUJ-0001Er-0r
 for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:11 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUC-0004lJ-J7
 for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:05 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled
 version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:58346)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>)
 id 1fHlUC-0004lC-Fl
 for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:04 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:49027)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUB-0002Fc-1I
 for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:32:04 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <andreas.roehler@HIDDEN>) id 1fHlU7-0004jq-SG
 for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:32:03 -0400
Received: from mout.kundenserver.de ([212.227.126.130]:38883)
 by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>)
 id 1fHlU7-0004jL-I8
 for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:31:59 -0400
Received: from [192.168.178.35] ([188.103.30.197]) by mrelayeu.kundenserver.de
 (mreue006 [212.227.15.167]) with ESMTPSA (Nemesis) id
 0MPeS5-1fD6ej3sUD-004kdl for <bug-gnu-emacs@HIDDEN>; Sun, 13 May 2018
 09:31:57 +0200
References: <20180427210859.GA6023@ACM>
 <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN> <20180512112612.GA4766@ACM>
From: Andreas =?UTF-8?Q?R=C3=B6hler?= <andreas.roehler@HIDDEN>
Message-ID: <79bd61c9-1d9c-d2f4-9206-3623a0553ff1@HIDDEN>
Date: Sun, 13 May 2018 09:33:20 +0200
User-Agent: Mozilla/5.0 (X11; Linux i686; rv:52.0) Gecko/20100101
 Thunderbird/52.7.0
MIME-Version: 1.0
In-Reply-To: <20180512112612.GA4766@ACM>
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Language: en-US
Content-Transfer-Encoding: 7bit
X-Provags-ID: V03:K1:4xXONJZrL2d6kiMkNZaUsOONQr6go7o5lUXxB99/1i4sr/elsDP
 EjXZ3/4Wi3N3/KZeg4fKcCcGoTP5MO0alvUZuIUwEDLSWrgQMad0cHk9iROW2+J+sq3/5n/
 AHcYxPnfVqAi72F8E9++IUcX8webunvTT1zaMU7jPMks6ePqfnYJHsXtbTDrwLXyEkre3oJ
 jrPcwTaRu9egEXyhJtmUA==
X-UI-Out-Filterresults: notjunk:1;V01:K0:vCBxJ2fpsPY=:HTM9dqgxmqPcnCQwbXQWif
 +5TxWJGJ2dUjXGrE2Yx1BewEZIBSie67aIoK4F4IBqhmyKeoLeXbei7RbpMiHJVHVxRh4WG3N
 LOuoiVHBhgab6khCAubVuvLBnd3+kZ4kh6/YkID0NWqf1uUB686EydgR4nZKYXrSDrjH3w7Tq
 XW0/YoGi3kYCZdRx+63nqalSHjeoUdy8L68tTMeLMlmA9iH1bTzgwoA38RuqAcL/XlHdwJiaA
 KQcWt1hkphNr7JfOI8IGkNJr5XzIb8UzSV0f7vPtZkaWRnCCT/t6geg4UceL+8Qe4smd3yJ//
 pQQE3JsIwcw79qbHF3m9T6q2TGC3QWtMzwYnTstLQMBO55nBxOcrlu6Z6hkGtV7gzJwFlJmTr
 WBQ2oxTSn4K3MPwyNuX52GkAFlalntq3dPPB1Qnn3QbKJRSD4JXEL3rqZZJgvYJBL34TPd0ki
 xdN4g0xFIMpe2HZYsqWwnkDFlhPEsq+wYaTReZIRgQEDXjQZL/vBLJRlP7e6VYCn8hV5JJh5t
 jGzUtJE894vyn/g67qKwVOaYmnIzROC0JwHtVLp4NEONnY6LRff87Wxv77Dj1/bX/9YSyooIa
 SYZnGKgiRaFtmbGdXTQAxAWCoECIs2A2ZvJOJVpbKs/RUb01XJfqozaSv42sPwv+vhRE55ITh
 pRCaDmxloiRiQmbuSb1JpF9as54uZeMnK2zWyxrm1Ch40wO3pm/gaiTqlURFIHxv0ScqwqpgM
 lUuv38zXk8FRTNhL
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
 [fuzzy]
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -6.0 (------)

On 12.05.2018 13:26, Alan Mackenzie wrote:
> Hello, Dmitry.
> 
> On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote:
>> On 4/28/18 12:08 AM, Alan Mackenzie wrote:
> 
>>> At least that would be true if syntax-propertize--done hadn't been
>>> prematurely and spuriously increased, crudely to prevent an infinite
>>> recursion, falsely indicating to the syntax-ppss infrastructure that the
>>> syntax-table properties have already been applied to the region (BEGIN
>>> END).
> 
>>>       .... but it should not call `syntax-ppss-flush-cache', ....
> 
>>> Why not?  Because syntax-ppss-flush-cache sets syntax-propertize--done
>>> back to its true value, allowing the wrongly allowed syntax-ppss calls at
>>> a later position to cause a recursive loop.
> 
>> Maybe we should "allow" it to loop, in certain cases? Leaving it to be
>> the responsibility of the programmer, to make sure the result doesn't
>> infloop, even if these rules are violated.
> 
> I'm not sure how this could work.  We would need to formalise the rules
> very carefully, to avoid the need to read syntax.{c,el}'s source code.
> 
>>>       .... which means that it should not call `syntax-ppss' on some
>>>       position and later modify the buffer on some earlier position.
> 
>>> This is a bad restriction, because sometimes syntax-table properties can
>>> only be correctly determined by examining the syntax of later buffer
>>> positions.  An example of this is giving the string-fence syntax-table
>>> text property to an unbalanced opening string quote, but not to correctly
>>> matched quotes.
> 
>> I'm not exactly convinced by the given example (why would we use the
>> string-fence in that case?), but it might be better if something like
>> this was possible, indeed.
> 
> String fence can be used to signal to font lock that the delimiter
> (together with the "mismatching" unescaped EOL) should be fontified in
> warning face.
> 
> A better example might be C++ Mode's marking of a "< ... >" pair with
> paren syntax.  This isn't done with syntax-propertize-function (as you
> know), but it would be nice if this were possible.
> 
>>> 2. syntax-propertize-function's are banned from using syntax-ppss, the
>>> documentation instead directing them to use parse-partial-sexp directly.
> 
>> The ones that currently call syntax-ppss, can't simply switch over to
>> parse-partial-sexp without becoming slower due to the lack of cache.
> 
> The cache at the pertinent buffer position doesn't exist at the time:
> consistent syntax-table properties aren't on the preceding buffer
> positions.
> 
>> Before tackling this bug, I'd rather we see a real-world problem that it
>> caused, and pick a particular approach based on it.
> 
> My enhancements for bug#30393: "24.4; cperl-mode: indentation failure -
> Documentation enhancements", where (almost) any change which affects the
> syntactic state is programmed to call syntax-ppss-flush-cache from the C
> level, clashes with the mechanism in this bug report.  Most of the time
> it's fine, but when a change affecting the syntactic state is made from
> inside a synax-propertize-function, Emacs goes into an infinite recursive
> loop.
> 
> This isn't good.
> 
>> But off the top of my head, we could introduce a "stricter but somewhat
>> slower" variation of syntax-ppss to be called inside
>> syntax-propertize-function's, which would treat the values in question
>> more carefully, somehow.
> 
> That's an idea worth exploring.
> 

Hi folks,

from what I've seen month ago just may stress the term fundamental.
Gave up to follow details WRT to check-ins made.

That part needs some person treating the gordic knot according to its 
quality...








Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.