X-Loop: help-debbugs@HIDDEN Subject: bug#31290: Fundamental bugs in syntax-propertize Resent-From: Alan Mackenzie <acm@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-gnu-emacs@HIDDEN Resent-Date: Fri, 27 Apr 2018 21:15:02 +0000 Resent-Message-ID: <handler.31290.B.152486364720568 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: report 31290 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 31290 <at> debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.152486364720568 (code B ref -1); Fri, 27 Apr 2018 21:15:02 +0000 Received: (at submit) by debbugs.gnu.org; 27 Apr 2018 21:14:07 +0000 Received: from localhost ([127.0.0.1]:41390 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1fCAgx-0005Lg-FH for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:14:07 -0400 Received: from eggs.gnu.org ([208.118.235.92]:55182) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1fCAgv-0005L3-6Q for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:14:06 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgp-0003ka-2K for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:48842) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgo-0003kQ-VJ for submit <at> debbugs.gnu.org; Fri, 27 Apr 2018 17:13:59 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:46147) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgn-0002Jr-Ub for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:58 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgi-0003iQ-Vj for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:57 -0400 Received: from colin.muc.de ([193.149.48.1]:31573 helo=mail.muc.de) by eggs.gnu.org with smtp (Exim 4.71) (envelope-from <acm@HIDDEN>) id 1fCAgi-0003gc-KK for bug-gnu-emacs@HIDDEN; Fri, 27 Apr 2018 17:13:52 -0400 Received: (qmail 77206 invoked by uid 3782); 27 Apr 2018 21:13:47 -0000 Received: from acm.muc.de (p5B14720D.dip0.t-ipconnect.de [91.20.114.13]) by colin.muc.de (tmda-ofmipd) with ESMTP; Fri, 27 Apr 2018 23:13:46 +0200 Received: (qmail 6051 invoked by uid 1000); 27 Apr 2018 21:08:59 -0000 Date: Fri, 27 Apr 2018 21:08:59 +0000 Message-ID: <20180427210859.GA6023@ACM> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline User-Agent: Mutt/1.9.4 (2018-02-28) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-detected-operating-system: by eggs.gnu.org: FreeBSD 9.x [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -4.3 (----) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -5.3 (-----) Hello, Emacs. There are fundamental bugs in syntax-propertize and syntax-propertize-function. The doc string of the latter states: The specified function may call `syntax-ppss' on any position before END, .... This is untrue. True is that syntax-ppss can be called on a position only up to syntax-propertize--done. After this point, the syntax-table properties haven't been applied, so calling syntax-ppss is, in general, going to give a false result. At least that would be true if syntax-propertize--done hadn't been prematurely and spuriously increased, crudely to prevent an infinite recursion, falsely indicating to the syntax-ppss infrastructure that the syntax-table properties have already been applied to the region (BEGIN END). .... but it should not call `syntax-ppss-flush-cache', .... Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done back to its true value, allowing the wrongly allowed syntax-ppss calls at a later position to cause a recursive loop. .... which means that it should not call `syntax-ppss' on some position and later modify the buffer on some earlier position. This is a bad restriction, because sometimes syntax-table properties can only be correctly determined by examining the syntax of later buffer positions. An example of this is giving the string-fence syntax-table text property to an unbalanced opening string quote, but not to correctly matched quotes. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; The plain fact is that (syntax-ppss pos) calls (syntax-propertize pos), so syntax-propertize cannot itself use syntax-ppss because of the recursive loop thus created. ;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;; Proposed solutions: 1. Major modes' syntax-propertize-function's are somehow given read access to syntax-propertize--done, and may call syntax-ppss up to that point only. syntax-propertize--done is updated only after the syntax-table properties have been applied. Or.... 2. syntax-propertize-function's are banned from using syntax-ppss, the documentation instead directing them to use parse-partial-sexp directly. In either solution, the restriction on using syntax-ppss-flush-cache would no longer be necessary, and there would be no restriction on setting syntax-table text properties at an earlier position than the one currently being analysed. I think solution 2 is the better one. -- Alan Mackenzie (Nuremberg, Germany).
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Alan Mackenzie <acm@HIDDEN> Subject: bug#31290: Acknowledgement (Fundamental bugs in syntax-propertize) Message-ID: <handler.31290.B.152486364720568.ack <at> debbugs.gnu.org> References: <20180427210859.GA6023@ACM> X-Gnu-PR-Message: ack 31290 X-Gnu-PR-Package: emacs Reply-To: 31290 <at> debbugs.gnu.org Date: Fri, 27 Apr 2018 21:15:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): bug-gnu-emacs@HIDDEN If you wish to submit further information on this problem, please send it to 31290 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 31290: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D31290 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN Subject: bug#31290: Fundamental bugs in syntax-propertize Resent-From: Dmitry Gutov <dgutov@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-gnu-emacs@HIDDEN Resent-Date: Tue, 08 May 2018 12:36:02 +0000 Resent-Message-ID: <handler.31290.B31290.152578292814648 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 31290 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Alan Mackenzie <acm@HIDDEN>, 31290 <at> debbugs.gnu.org Received: via spool by 31290-submit <at> debbugs.gnu.org id=B31290.152578292814648 (code B ref 31290); Tue, 08 May 2018 12:36:02 +0000 Received: (at 31290) by debbugs.gnu.org; 8 May 2018 12:35:28 +0000 Received: from localhost ([127.0.0.1]:53540 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1fG1q3-0003oC-Sb for submit <at> debbugs.gnu.org; Tue, 08 May 2018 08:35:28 -0400 Received: from mail-wr0-f173.google.com ([209.85.128.173]:46771) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <raaahh@HIDDEN>) id 1fG1pz-0003nw-F7 for 31290 <at> debbugs.gnu.org; Tue, 08 May 2018 08:35:24 -0400 Received: by mail-wr0-f173.google.com with SMTP id a12-v6so1246492wrn.13 for <31290 <at> debbugs.gnu.org>; Tue, 08 May 2018 05:35:23 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=sender:subject:to:references:from:message-id:date:user-agent :mime-version:in-reply-to:content-language:content-transfer-encoding; bh=VYOIVEaIRrc4TFp82f2wDek7DlL8StKt4BkR4d74PTQ=; b=mBuczawT4iEmzk3SgRZNvOGXdAukcQaHOUxrT24lNbUlt96YjdhPsy4t+BrbrskPuN kF9nzgM8Kaz4yYP4CUy+KxcSVe3D9yQJu1mAFH4j0ruUsdMZdM/us/jYG1iLMQVUn4So /FBgaUuYwblAP8X+ATTnXJcFDbk+ihJQGfPVkJ+E+MNF5mDaxJu1t4HpMHaLKjsRpvgn mEN9gVjBKS8Qe2WfhRbcPnbdNLBpTR01xA/2g4Te8xwkzdrG8M8Hiigo5E3iSn7y39g/ wuh46xbIUrLCQ0+WQ9fzHFZmSiAb9a6URJLR6M353/E8lWjbvffv9blE9FMc/VpxPrFY ZacQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:sender:subject:to:references:from:message-id :date:user-agent:mime-version:in-reply-to:content-language :content-transfer-encoding; bh=VYOIVEaIRrc4TFp82f2wDek7DlL8StKt4BkR4d74PTQ=; b=gJ8TsstQ9bA+oNJfNyIzn5rgvoyMIFqPyQt22x3NqfaGYd4ZrcWki6L6xUaVe4V556 6w3o+yYSidudHgSiz5HBHYj56HYJaf6+5dMWsuep3tL2JmJ5pkFF3FJDfECfV434OAGM 7TLDAI/wwkVnCQPXINIAN6YM4gIBwq5f8zy6BF57zcQG0pc8VkBD6nxfoknl1T0Ccz0F mIkb0t6CpD5xlG4oEcTw8LIPD3X7WimbAUteNZWNqpugeovX7uhPtRxxgVT6inKbpi1R eqqfuqBJwFW9IX8pyoXxc2piJyZThNDlGkNVkrTXEmJ+RlN80S93FvVyvQtuEx/YbKBz bFMA== X-Gm-Message-State: ALQs6tDxS8uqYvu8Zf69wHQUVyTu3Q9jLIHJUNb+ADf7ERHoPVW2nYQD hmXAIF8xwWudh/0gLIpfXhNR7RDz X-Google-Smtp-Source: AB8JxZoHbA1vlFJmZXO2KGHVnMZM2QfPcNZW5FPh2e/EflgVZ3VLKZE1Bf3mKTRf2Uo+y9fPzW247w== X-Received: by 2002:adf:b78b:: with SMTP id s11-v6mr26222145wre.247.1525782917366; Tue, 08 May 2018 05:35:17 -0700 (PDT) Received: from [192.168.0.200] ([212.50.99.193]) by smtp.googlemail.com with ESMTPSA id h133sm11716321wmf.47.2018.05.08.05.35.15 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128); Tue, 08 May 2018 05:35:16 -0700 (PDT) References: <20180427210859.GA6023@ACM> From: Dmitry Gutov <dgutov@HIDDEN> Message-ID: <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN> Date: Tue, 8 May 2018 15:35:14 +0300 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:60.0) Gecko/20100101 Thunderbird/60.0 MIME-Version: 1.0 In-Reply-To: <20180427210859.GA6023@ACM> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Spam-Score: 0.5 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.5 (/) On 4/28/18 12:08 AM, Alan Mackenzie wrote: > At least that would be true if syntax-propertize--done hadn't been > prematurely and spuriously increased, crudely to prevent an infinite > recursion, falsely indicating to the syntax-ppss infrastructure that the > syntax-table properties have already been applied to the region (BEGIN > END). > > .... but it should not call `syntax-ppss-flush-cache', .... > > Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done > back to its true value, allowing the wrongly allowed syntax-ppss calls at > a later position to cause a recursive loop. Maybe we should "allow" it to loop, in certain cases? Leaving it to be the responsibility of the programmer, to make sure the result doesn't infloop, even if these rules are violated. > .... which means that it should not call `syntax-ppss' on some > position and later modify the buffer on some earlier position. > > This is a bad restriction, because sometimes syntax-table properties can > only be correctly determined by examining the syntax of later buffer > positions. An example of this is giving the string-fence syntax-table > text property to an unbalanced opening string quote, but not to correctly > matched quotes. I'm not exactly convinced by the given example (why would we use the string-fence in that case?), but it might be better if something like this was possible, indeed. > 2. syntax-propertize-function's are banned from using syntax-ppss, the > documentation instead directing them to use parse-partial-sexp directly. The ones that currently call syntax-ppss, can't simply switch over to parse-partial-sexp without becoming slower due to the lack of cache. Before tackling this bug, I'd rather we see a real-world problem that it caused, and pick a particular approach based on it. But off the top of my head, we could introduce a "stricter but somewhat slower" variation of syntax-ppss to be called inside syntax-propertize-function's, which would treat the values in question more carefully, somehow.
X-Loop: help-debbugs@HIDDEN Subject: bug#31290: Fundamental bugs in syntax-propertize Resent-From: Alan Mackenzie <acm@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-gnu-emacs@HIDDEN Resent-Date: Sat, 12 May 2018 11:34:01 +0000 Resent-Message-ID: <handler.31290.B31290.152612483114047 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 31290 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: Dmitry Gutov <dgutov@HIDDEN> Cc: 31290 <at> debbugs.gnu.org Received: via spool by 31290-submit <at> debbugs.gnu.org id=B31290.152612483114047 (code B ref 31290); Sat, 12 May 2018 11:34:01 +0000 Received: (at 31290) by debbugs.gnu.org; 12 May 2018 11:33:51 +0000 Received: from localhost ([127.0.0.1]:58989 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1fHSmc-0003eV-UA for submit <at> debbugs.gnu.org; Sat, 12 May 2018 07:33:51 -0400 Received: from colin.muc.de ([193.149.48.1]:38221 helo=mail.muc.de) by debbugs.gnu.org with smtp (Exim 4.84_2) (envelope-from <acm@HIDDEN>) id 1fHSma-0003eL-Fr for 31290 <at> debbugs.gnu.org; Sat, 12 May 2018 07:33:49 -0400 Received: (qmail 81713 invoked by uid 3782); 12 May 2018 11:33:47 -0000 Received: from acm.muc.de (p5B146231.dip0.t-ipconnect.de [91.20.98.49]) by colin.muc.de (tmda-ofmipd) with ESMTP; Sat, 12 May 2018 13:33:45 +0200 Received: (qmail 6270 invoked by uid 1000); 12 May 2018 11:26:12 -0000 Date: Sat, 12 May 2018 11:26:12 +0000 Message-ID: <20180512112612.GA4766@ACM> References: <20180427210859.GA6023@ACM> <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Disposition: inline In-Reply-To: <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN> User-Agent: Mutt/1.9.4 (2018-02-28) X-Delivery-Agent: TMDA/1.1.12 (Macallan) From: Alan Mackenzie <acm@HIDDEN> X-Primary-Address: acm@HIDDEN X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, Dmitry. On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote: > On 4/28/18 12:08 AM, Alan Mackenzie wrote: > > At least that would be true if syntax-propertize--done hadn't been > > prematurely and spuriously increased, crudely to prevent an infinite > > recursion, falsely indicating to the syntax-ppss infrastructure that the > > syntax-table properties have already been applied to the region (BEGIN > > END). > > .... but it should not call `syntax-ppss-flush-cache', .... > > Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done > > back to its true value, allowing the wrongly allowed syntax-ppss calls at > > a later position to cause a recursive loop. > Maybe we should "allow" it to loop, in certain cases? Leaving it to be > the responsibility of the programmer, to make sure the result doesn't > infloop, even if these rules are violated. I'm not sure how this could work. We would need to formalise the rules very carefully, to avoid the need to read syntax.{c,el}'s source code. > > .... which means that it should not call `syntax-ppss' on some > > position and later modify the buffer on some earlier position. > > This is a bad restriction, because sometimes syntax-table properties can > > only be correctly determined by examining the syntax of later buffer > > positions. An example of this is giving the string-fence syntax-table > > text property to an unbalanced opening string quote, but not to correctly > > matched quotes. > I'm not exactly convinced by the given example (why would we use the > string-fence in that case?), but it might be better if something like > this was possible, indeed. String fence can be used to signal to font lock that the delimiter (together with the "mismatching" unescaped EOL) should be fontified in warning face. A better example might be C++ Mode's marking of a "< ... >" pair with paren syntax. This isn't done with syntax-propertize-function (as you know), but it would be nice if this were possible. > > 2. syntax-propertize-function's are banned from using syntax-ppss, the > > documentation instead directing them to use parse-partial-sexp directly. > The ones that currently call syntax-ppss, can't simply switch over to > parse-partial-sexp without becoming slower due to the lack of cache. The cache at the pertinent buffer position doesn't exist at the time: consistent syntax-table properties aren't on the preceding buffer positions. > Before tackling this bug, I'd rather we see a real-world problem that it > caused, and pick a particular approach based on it. My enhancements for bug#30393: "24.4; cperl-mode: indentation failure - Documentation enhancements", where (almost) any change which affects the syntactic state is programmed to call syntax-ppss-flush-cache from the C level, clashes with the mechanism in this bug report. Most of the time it's fine, but when a change affecting the syntactic state is made from inside a synax-propertize-function, Emacs goes into an infinite recursive loop. This isn't good. > But off the top of my head, we could introduce a "stricter but somewhat > slower" variation of syntax-ppss to be called inside > syntax-propertize-function's, which would treat the values in question > more carefully, somehow. That's an idea worth exploring. -- Alan Mackenzie (Nuremberg, Germany).
X-Loop: help-debbugs@HIDDEN Subject: bug#31290: Fundamental bugs in syntax-propertize Resent-From: Andreas =?UTF-8?Q?R=C3=B6hler?= <andreas.roehler@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-gnu-emacs@HIDDEN Resent-Date: Sun, 13 May 2018 07:33:01 +0000 Resent-Message-ID: <handler.31290.B.15261967324782 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 31290 X-GNU-PR-Package: emacs X-GNU-PR-Keywords: To: 31290 <at> debbugs.gnu.org X-Debbugs-Original-To: bug-gnu-emacs@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.15261967324782 (code B ref -1); Sun, 13 May 2018 07:33:01 +0000 Received: (at submit) by debbugs.gnu.org; 13 May 2018 07:32:12 +0000 Received: from localhost ([127.0.0.1]:60116 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1fHlUK-0001F4-8N for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:12 -0400 Received: from eggs.gnu.org ([208.118.235.92]:58068) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUJ-0001Er-0r for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:11 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUC-0004lJ-J7 for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:05 -0400 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50 autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:58346) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUC-0004lC-Fl for submit <at> debbugs.gnu.org; Sun, 13 May 2018 03:32:04 -0400 Received: from eggs.gnu.org ([2001:4830:134:3::10]:49027) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>) id 1fHlUB-0002Fc-1I for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:32:04 -0400 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>) id 1fHlU7-0004jq-SG for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:32:03 -0400 Received: from mout.kundenserver.de ([212.227.126.130]:38883) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_128_CBC_SHA1:16) (Exim 4.71) (envelope-from <andreas.roehler@HIDDEN>) id 1fHlU7-0004jL-I8 for bug-gnu-emacs@HIDDEN; Sun, 13 May 2018 03:31:59 -0400 Received: from [192.168.178.35] ([188.103.30.197]) by mrelayeu.kundenserver.de (mreue006 [212.227.15.167]) with ESMTPSA (Nemesis) id 0MPeS5-1fD6ej3sUD-004kdl for <bug-gnu-emacs@HIDDEN>; Sun, 13 May 2018 09:31:57 +0200 References: <20180427210859.GA6023@ACM> <e8658e46-da92-6310-7b1e-33c55f6255d2@HIDDEN> <20180512112612.GA4766@ACM> From: Andreas =?UTF-8?Q?R=C3=B6hler?= <andreas.roehler@HIDDEN> Message-ID: <79bd61c9-1d9c-d2f4-9206-3623a0553ff1@HIDDEN> Date: Sun, 13 May 2018 09:33:20 +0200 User-Agent: Mozilla/5.0 (X11; Linux i686; rv:52.0) Gecko/20100101 Thunderbird/52.7.0 MIME-Version: 1.0 In-Reply-To: <20180512112612.GA4766@ACM> Content-Type: text/plain; charset=utf-8; format=flowed Content-Language: en-US Content-Transfer-Encoding: 7bit X-Provags-ID: V03:K1:4xXONJZrL2d6kiMkNZaUsOONQr6go7o5lUXxB99/1i4sr/elsDP EjXZ3/4Wi3N3/KZeg4fKcCcGoTP5MO0alvUZuIUwEDLSWrgQMad0cHk9iROW2+J+sq3/5n/ AHcYxPnfVqAi72F8E9++IUcX8webunvTT1zaMU7jPMks6ePqfnYJHsXtbTDrwLXyEkre3oJ jrPcwTaRu9egEXyhJtmUA== X-UI-Out-Filterresults: notjunk:1;V01:K0:vCBxJ2fpsPY=:HTM9dqgxmqPcnCQwbXQWif +5TxWJGJ2dUjXGrE2Yx1BewEZIBSie67aIoK4F4IBqhmyKeoLeXbei7RbpMiHJVHVxRh4WG3N LOuoiVHBhgab6khCAubVuvLBnd3+kZ4kh6/YkID0NWqf1uUB686EydgR4nZKYXrSDrjH3w7Tq XW0/YoGi3kYCZdRx+63nqalSHjeoUdy8L68tTMeLMlmA9iH1bTzgwoA38RuqAcL/XlHdwJiaA KQcWt1hkphNr7JfOI8IGkNJr5XzIb8UzSV0f7vPtZkaWRnCCT/t6geg4UceL+8Qe4smd3yJ// pQQE3JsIwcw79qbHF3m9T6q2TGC3QWtMzwYnTstLQMBO55nBxOcrlu6Z6hkGtV7gzJwFlJmTr WBQ2oxTSn4K3MPwyNuX52GkAFlalntq3dPPB1Qnn3QbKJRSD4JXEL3rqZZJgvYJBL34TPd0ki xdN4g0xFIMpe2HZYsqWwnkDFlhPEsq+wYaTReZIRgQEDXjQZL/vBLJRlP7e6VYCn8hV5JJh5t jGzUtJE894vyn/g67qKwVOaYmnIzROC0JwHtVLp4NEONnY6LRff87Wxv77Dj1/bX/9YSyooIa SYZnGKgiRaFtmbGdXTQAxAWCoECIs2A2ZvJOJVpbKs/RUb01XJfqozaSv42sPwv+vhRE55ITh pRCaDmxloiRiQmbuSb1JpF9as54uZeMnK2zWyxrm1Ch40wO3pm/gaiTqlURFIHxv0ScqwqpgM lUuv38zXk8FRTNhL X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] [fuzzy] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.0 (-----) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -6.0 (------) On 12.05.2018 13:26, Alan Mackenzie wrote: > Hello, Dmitry. > > On Tue, May 08, 2018 at 15:35:14 +0300, Dmitry Gutov wrote: >> On 4/28/18 12:08 AM, Alan Mackenzie wrote: > >>> At least that would be true if syntax-propertize--done hadn't been >>> prematurely and spuriously increased, crudely to prevent an infinite >>> recursion, falsely indicating to the syntax-ppss infrastructure that the >>> syntax-table properties have already been applied to the region (BEGIN >>> END). > >>> .... but it should not call `syntax-ppss-flush-cache', .... > >>> Why not? Because syntax-ppss-flush-cache sets syntax-propertize--done >>> back to its true value, allowing the wrongly allowed syntax-ppss calls at >>> a later position to cause a recursive loop. > >> Maybe we should "allow" it to loop, in certain cases? Leaving it to be >> the responsibility of the programmer, to make sure the result doesn't >> infloop, even if these rules are violated. > > I'm not sure how this could work. We would need to formalise the rules > very carefully, to avoid the need to read syntax.{c,el}'s source code. > >>> .... which means that it should not call `syntax-ppss' on some >>> position and later modify the buffer on some earlier position. > >>> This is a bad restriction, because sometimes syntax-table properties can >>> only be correctly determined by examining the syntax of later buffer >>> positions. An example of this is giving the string-fence syntax-table >>> text property to an unbalanced opening string quote, but not to correctly >>> matched quotes. > >> I'm not exactly convinced by the given example (why would we use the >> string-fence in that case?), but it might be better if something like >> this was possible, indeed. > > String fence can be used to signal to font lock that the delimiter > (together with the "mismatching" unescaped EOL) should be fontified in > warning face. > > A better example might be C++ Mode's marking of a "< ... >" pair with > paren syntax. This isn't done with syntax-propertize-function (as you > know), but it would be nice if this were possible. > >>> 2. syntax-propertize-function's are banned from using syntax-ppss, the >>> documentation instead directing them to use parse-partial-sexp directly. > >> The ones that currently call syntax-ppss, can't simply switch over to >> parse-partial-sexp without becoming slower due to the lack of cache. > > The cache at the pertinent buffer position doesn't exist at the time: > consistent syntax-table properties aren't on the preceding buffer > positions. > >> Before tackling this bug, I'd rather we see a real-world problem that it >> caused, and pick a particular approach based on it. > > My enhancements for bug#30393: "24.4; cperl-mode: indentation failure - > Documentation enhancements", where (almost) any change which affects the > syntactic state is programmed to call syntax-ppss-flush-cache from the C > level, clashes with the mechanism in this bug report. Most of the time > it's fine, but when a change affecting the syntactic state is made from > inside a synax-propertize-function, Emacs goes into an infinite recursive > loop. > > This isn't good. > >> But off the top of my head, we could introduce a "stricter but somewhat >> slower" variation of syntax-ppss to be called inside >> syntax-propertize-function's, which would treat the values in question >> more carefully, somehow. > > That's an idea worth exploring. > Hi folks, from what I've seen month ago just may stress the term fundamental. Gave up to follow details WRT to check-ins made. That part needs some person treating the gordic knot according to its quality...
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.