GNU bug report logs - #67262
python-ts-mode cannot identify triple-quoted-strings

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: JD Smith <jdtsmith@HIDDEN>; merged with #67394; dated Sat, 18 Nov 2023 15:53:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.
Merged 67262 67394. Request was from Eli Zaretskii <eliz@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 27 Nov 2023 00:05:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Nov 26 19:05:30 2023
Received: from localhost ([127.0.0.1]:42911 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r7P7y-000051-GY
	for submit <at> debbugs.gnu.org; Sun, 26 Nov 2023 19:05:30 -0500
Received: from wout2-smtp.messagingengine.com ([64.147.123.25]:43983)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dmitry@HIDDEN>) id 1r7P7v-0008WJ-PH
 for 67262 <at> debbugs.gnu.org; Sun, 26 Nov 2023 19:05:28 -0500
Received: from compute3.internal (compute3.nyi.internal [10.202.2.43])
 by mailout.west.internal (Postfix) with ESMTP id A43513200A5B;
 Sun, 26 Nov 2023 19:05:15 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute3.internal (MEProxy); Sun, 26 Nov 2023 19:05:16 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
 :cc:content-transfer-encoding:content-type:content-type:date
 :date:from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:sender:subject:subject:to:to; s=fm3; t=
 1701043515; x=1701129915; bh=SaA2idlh9VCrhYUQOTPkECkSMWtpLJG7suJ
 z65+TwJU=; b=FZQ4VsOJES7lwWGWIOj3HPLkDotfKhFHTnXAQrU76+e9ufLjdmR
 jK9WrPzRZjQnu5cSt5IMoTT5a+/1zC5+XuVq9P0B6iKoqlQLhdYZuT1yzAWqk1NM
 xEFQMwl/ZQ0nOv+n1Y6qiPcO6gSIajBiul7FgMJAu+KOUQXRKr2PTle4f24+II81
 agfrTe4gEjTK4DDkVOJ95W9f3CxChQK0Mls5we2jt6OjFHjcDC8mIFJ4CtPlAccK
 K0V3T6g8fXEi2XhktawNEOOuR/XG5HWzKhVBKNy1FdT7D9zAuczX5Wgt4FdxcC3E
 Im+uSCEnXC2KrsVuQbzhECbLKVLlWpzqVnQ==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:cc:content-transfer-encoding
 :content-type:content-type:date:date:feedback-id:feedback-id
 :from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:sender:subject:subject:to:to:x-me-proxy
 :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
 1701043515; x=1701129915; bh=SaA2idlh9VCrhYUQOTPkECkSMWtpLJG7suJ
 z65+TwJU=; b=gkuNUbrA9QKxU538/DlLG98nnWUnAb9PZ/fHNM2vGH0qWh+2mTG
 xI7ZauDhz6xrzQ05hDb6L3u0A/OmysCHOGdYUBUWzQvJlY6FWVQ4934pLgmFDOyd
 b4jrAd79GVyOCZwJ7o4B1BqgazZMVblR99OkE++CpER4gsMW+waKHJrRMPbJebvN
 kyI7q6SThMaRXBUtFCI8Yhf443UDSXfqDLJ/PtAmHtNsQ5rTmCAf8ZDdir51BcW5
 VE0ZkY2+Mp2JKfVTn6lF5y/BxPo+Ab3acfcLWInnUlX9EZlQoMtEMgzeYSoZmBoC
 qejXDDauaLiCRsVKEyUtHpQ6fIPS1Xvm3QQ==
X-ME-Sender: <xms:Ot1jZR3kBxSu_nKD41L7s-7FX4zZUapJbmbWcsOyhHAb7OsB9UvyUA>
 <xme:Ot1jZYFKAFd1lz0BM0ykK_HyFIeQir_V7msOYBgaIr1WUkFC1xpAOkvyi3-qBOquH
 T30n50SaWsmARq2tyQ>
X-ME-Received: <xmr:Ot1jZR73W9K330DAbGj71Hqr9bUKofG4s9HowfWdPahg3R5UrJduzveD8FX_LgBnFDxw1Q>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudeitddgudelucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
 cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi
 thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth
 htvghrnhepiefgteevheevveffheeltdeukeeiieekueefgedugfefgefhudelgfefveel
 vdevnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug
 hmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:Ot1jZe2XuEnbeitCkGZ0ApGg6oc6FvVVxho_1_mKVQuOgQljV80ftg>
 <xmx:Ot1jZUFi1wztEInpHCl4-ojRgije9fnIzxhEpblXtjY6l97bJWZ0BA>
 <xmx:Ot1jZf_XzWyYc6nWk9G5LRoV0iJaWnSpix6dsK7WhbtIfkxHyFkE8A>
 <xmx:O91jZZQW-rx3bEOzV3Qo-JHN6h1Rd8Z11sAT3QlnyHjGTIZxv_gZig>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun,
 26 Nov 2023 19:05:13 -0500 (EST)
Message-ID: <a6119652-a44c-f7d4-2e2b-cd56a53d0b51@HIDDEN>
Date: Mon, 27 Nov 2023 02:05:10 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Content-Language: en-US
To: Yuan Fu <casouri@HIDDEN>, JD Smith <jdtsmith@HIDDEN>,
 Eli Zaretskii <eliz@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
 <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN> <83v89qcfsb.fsf@HIDDEN>
 <9B8C904A-3729-44AF-82F7-3BEA849F46D0@HIDDEN>
 <b7ba3a23-6b8b-efba-9b76-0c49abd0fd03@HIDDEN>
 <ec6230a6-38dc-02cf-3a8f-9dafda4a8ab7@HIDDEN>
 <e4fd3420-8715-4bbc-ab33-7f3235fea564@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <e4fd3420-8715-4bbc-ab33-7f3235fea564@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.9 (--)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.9 (---)

On 27/11/2023 01:43, Yuan Fu wrote:
> 
> On 11/26/23 6:58 AM, Dmitry Gutov wrote:
>> On 26/11/2023 04:04, Dmitry Gutov wrote:
>>> As for what to do about this one -- probably something involving 
>>> syntax-propertize-extend-region-functions, adding an entry which 
>>> would initialize the parser, but not call syntax-ppss-flush-cache 
>>> directly (or at least not just that). It would signal the earlier 
>>> position to extend to through some dynamic variable. This is getting 
>>> tricky enough to move from the individual major modes into treesit.el 
>>> proper, I think.
>>
>> Alternatively, we'd trigger updates eagerly from within 
>> treesit_record_change -- that would make it slower, invalidating the 
>> comment above it. Not sure by how much, though.
> 
> It seems to me that what we need is to force a re-parse at the beginning 
> of syntax-propertize or in syntax-ppss-flush-cache; the re-parse would 
> cause the notifier to run, which runs python--treesit-parser-after-change.

syntax-ppss-flush-cache is called by edits (and by the re-parse). It 
seems like it will be odd to have execution the other way around and/or 
add some hook into it which would call the re-parse and extend the 
region to be invalidated.

syntax-propertize could have another hook added, yes. Or an advice.

But it seems better to reuse some of the existing hooks, such as 
syntax-propertize-extend-region-functions. It treesit.c provided a way 
to fetch the newly-invalidated region, the treesit-major-mode-setup 
could add a new function to syntax-propertize-extend-region-functions 
which would invoke that feature. But even now it can instantiate the 
parse, which would call treesit-force-reparse internally, and then 
collect the info from the callbacks.

And yet another way - is to extend the region to be propertized from 
inside the major mode's syntax-propertize-function, invalidating some 
earlier entries too. The main problem with that, I think, is that every 
ts mode will have to repeat that trick. And that authors would have to 
know to do that. How to make that easier and more obvious, is a question.

Finally, if I'm right that bug#66732 has a similar cause, then a shared 
solution that can be reused by syntax and font-lock (or preferably just 
fix both in the same place) would be ideal.

> I'm not quite sure about how do we cause this re-parse. The 
> straightforward approach would be calling treesit-force-reparse[1] in 
> syntax-propertize/syntax-ppss-flush-cache. But ideally I'd like to keep 
> tree-sitter transparent for syntax.el. Maybe we can add a hook in 
> syntax-propertize/syntax-ppss-flush-cache.
> 
> [1] This function doesn't exist yet, but it's easy to define in lisp.

treesit-parser-root-node calls it anyway and does little else, so we 
could get by with just using it.





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 26 Nov 2023 23:43:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Nov 26 18:43:40 2023
Received: from localhost ([127.0.0.1]:42899 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r7Omq-0007db-89
	for submit <at> debbugs.gnu.org; Sun, 26 Nov 2023 18:43:40 -0500
Received: from mail-pj1-x1036.google.com ([2607:f8b0:4864:20::1036]:45473)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <casouri@HIDDEN>) id 1r7Omm-0007dK-Js
 for 67262 <at> debbugs.gnu.org; Sun, 26 Nov 2023 18:43:39 -0500
Received: by mail-pj1-x1036.google.com with SMTP id
 98e67ed59e1d1-2851a2b30a2so2350948a91.3
 for <67262 <at> debbugs.gnu.org>; Sun, 26 Nov 2023 15:43:30 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1701042204; x=1701647004; darn=debbugs.gnu.org;
 h=content-transfer-encoding:in-reply-to:from:references:cc:to
 :content-language:subject:user-agent:mime-version:date:message-id
 :from:to:cc:subject:date:message-id:reply-to;
 bh=d0jwQ5H2+gY9BA7UuH5TACFS1IuRdZ1WlFiGQ1wDnjw=;
 b=MGz2lfnWgDSpnClrmBlRVvlanFVCHaZlNMDy5jkpwiqfW/U1ceTK69c/6DRroEMZ4t
 R+DE/S2u4dJGWRTFoUp5MkXz6BQhLx+5c43dfjWE/SKwc3jb9D3unfQxYtVy1ZqIO2Ev
 FcWDYAzcMtjOqiogDpNBXzOlgORj857QvHmVnJnMgDjJ9uAlojIFrAcnE6wgpw9adoFw
 +m12vU7WNyzLcEfUmmZi1oHtEsp7OT38FISO6vR3c94HpWnY/02XES2phRWJr+bR11Ce
 so62TeZiStbWBZvazFbL9warueDya/XZKFuE8K16S9CXKEzgVIKkGtFMp/s3S1xBHuC5
 vPPw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1701042204; x=1701647004;
 h=content-transfer-encoding:in-reply-to:from:references:cc:to
 :content-language:subject:user-agent:mime-version:date:message-id
 :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=d0jwQ5H2+gY9BA7UuH5TACFS1IuRdZ1WlFiGQ1wDnjw=;
 b=ejJ66ua90CCtXlS6xtUnRtXfWqmPzx0PZgTtFafccA03NO872tjc2mbYZNLdvkXy7r
 lbktTu9YQy1iTy0RB3biZJn1TsYDdC6JcHsyLu+uqPL+pqxxQu2OdxRqzAjvmXK6VwoT
 GM6jl2CkDxZRDOU1hCcbM8CMVy5x6HovlJpqido6bVZAPFwghxJahqRqRmd5aRaZM0TU
 Swwwj8bGuQfTNt/H2V23WvCIHkHFSrs4jnZu7dQ/sz/mvtV/1ZfVjDOVkws8lJYnFbUc
 brwchOUGyQonHQJZCbTjvWkLkM21XuqXfkvehM8fOdYhS/ZeNNaOiYJK/H9vpON9Q+tW
 5NZQ==
X-Gm-Message-State: AOJu0Yw925VyMvoCGFZwu9Ffxzvfc2bnvkl+YjKNuHL6djyS73/KisLK
 imm1Rof89Zyw6rkhOG1fASI=
X-Google-Smtp-Source: AGHT+IH+e2nDJHfe0nxF2MH3oLr5vgSAnEK8gC6g2kO8C2Qu3qjdjLXz/NApUpLSDJUeGTXfxXvYTA==
X-Received: by 2002:a17:90b:1d0f:b0:285:a2af:31aa with SMTP id
 on15-20020a17090b1d0f00b00285a2af31aamr3930702pjb.37.1701042204514; 
 Sun, 26 Nov 2023 15:43:24 -0800 (PST)
Received: from [192.168.1.7] (cpe-172-117-161-177.socal.res.rr.com.
 [172.117.161.177]) by smtp.gmail.com with ESMTPSA id
 gb20-20020a17090b061400b002808c9e3095sm6094106pjb.26.2023.11.26.15.43.23
 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128);
 Sun, 26 Nov 2023 15:43:24 -0800 (PST)
Message-ID: <e4fd3420-8715-4bbc-ab33-7f3235fea564@HIDDEN>
Date: Sun, 26 Nov 2023 15:43:22 -0800
MIME-Version: 1.0
User-Agent: Mozilla Thunderbird
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Content-Language: en-US
To: Dmitry Gutov <dmitry@HIDDEN>, JD Smith <jdtsmith@HIDDEN>,
 Eli Zaretskii <eliz@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
 <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN> <83v89qcfsb.fsf@HIDDEN>
 <9B8C904A-3729-44AF-82F7-3BEA849F46D0@HIDDEN>
 <b7ba3a23-6b8b-efba-9b76-0c49abd0fd03@HIDDEN>
 <ec6230a6-38dc-02cf-3a8f-9dafda4a8ab7@HIDDEN>
From: Yuan Fu <casouri@HIDDEN>
In-Reply-To: <ec6230a6-38dc-02cf-3a8f-9dafda4a8ab7@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


On 11/26/23 6:58 AM, Dmitry Gutov wrote:
> On 26/11/2023 04:04, Dmitry Gutov wrote:
>> As for what to do about this one -- probably something involving 
>> syntax-propertize-extend-region-functions, adding an entry which 
>> would initialize the parser, but not call syntax-ppss-flush-cache 
>> directly (or at least not just that). It would signal the earlier 
>> position to extend to through some dynamic variable. This is getting 
>> tricky enough to move from the individual major modes into treesit.el 
>> proper, I think.
>
> Alternatively, we'd trigger updates eagerly from within 
> treesit_record_change -- that would make it slower, invalidating the 
> comment above it. Not sure by how much, though.

It seems to me that what we need is to force a re-parse at the beginning 
of syntax-propertize or in syntax-ppss-flush-cache; the re-parse would 
cause the notifier to run, which runs python--treesit-parser-after-change.

I'm not quite sure about how do we cause this re-parse. The 
straightforward approach would be calling treesit-force-reparse[1] in 
syntax-propertize/syntax-ppss-flush-cache. But ideally I'd like to keep 
tree-sitter transparent for syntax.el. Maybe we can add a hook in 
syntax-propertize/syntax-ppss-flush-cache.

[1] This function doesn't exist yet, but it's easy to define in lisp.

Yuan






Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 26 Nov 2023 14:59:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Nov 26 09:59:00 2023
Received: from localhost ([127.0.0.1]:42558 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r7Gb6-0001Eu-KH
	for submit <at> debbugs.gnu.org; Sun, 26 Nov 2023 09:59:00 -0500
Received: from out3-smtp.messagingengine.com ([66.111.4.27]:43757)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dmitry@HIDDEN>) id 1r7Gb3-0001Ee-AH
 for 67262 <at> debbugs.gnu.org; Sun, 26 Nov 2023 09:58:59 -0500
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailout.nyi.internal (Postfix) with ESMTP id 57C315C011D;
 Sun, 26 Nov 2023 09:58:46 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Sun, 26 Nov 2023 09:58:46 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
 :cc:content-transfer-encoding:content-type:content-type:date
 :date:from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:sender:subject:subject:to:to; s=fm3; t=
 1701010726; x=1701097126; bh=e2xAuD7gOAUM8OfGtdae+BtjLwHNiZQBir5
 buE9N3tI=; b=wvkD6ren/7P5o5OpDNrDYJ9pty8HiSjkkA3tPVK/MPpCxIFip5/
 TuL4bx32Z/7KUcytaT0eFndeZO9cT+s7FNz6bdmHfVdl2GKst2O9qvLSs4fAp3b/
 QuC46/GJMuQBZ1gzeLb/droXnPTXCnxGQLVOyZHAZDStNFsGlc5qbj7wcv5lAJ7O
 84o8OPgcKFcjlfZcEQbKfset6LmAUkmDlSQxMJZ17z/G/DMZoqn6jCtHeRCH+Hmr
 uFryKivWml82phixUcM6kQf0KqrX2ggDLzSWH/IoMr35oB9Nm1D/TW6xrSOfhEjP
 SNNXW4ba7it2XpHkIkNrgfUVkCPF0YI/s4w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:cc:content-transfer-encoding
 :content-type:content-type:date:date:feedback-id:feedback-id
 :from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:sender:subject:subject:to:to:x-me-proxy
 :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
 1701010726; x=1701097126; bh=e2xAuD7gOAUM8OfGtdae+BtjLwHNiZQBir5
 buE9N3tI=; b=PiFz7EYjcJMZ/sR7ee/r7xvNru6rSRn52bKCk9UgCoLQqJp5KQA
 L4hYS2rl88hdMvPVirOXOVFHxK3MuJdPvR6Wj0Y7/8g2f1iRLJx3R6OepZRf4h8s
 4vZhFYJJqITgExqlOMnzKRe6w3ImT2uM/hxVqHEZmC9Xyha+nh+FlJWITBh6odKt
 dRSruzrRi6V9xHZM5e6+um2aVlmhv2rYjV18TF1s/de+o7o3Q1DUgnLf3r0Y/mMh
 oIL4f/MzKRkGb/PaK4GbRTRSXHZ7cpdeS8l2SG+90a6h/av6kpJVV7+x1+TmUwCu
 ENHt+BZXIt1dVMAImXp4kXTXwh5g8eU5fgQ==
X-ME-Sender: <xms:Jl1jZdLzuIqAu3TQwOxa4QFe4-oXsVuA3bAoh2AwIhTzFxb5jX4-8Q>
 <xme:Jl1jZZKYXGWZ7A1S9n6OeuCLBimyCW_GnSZ1-Upq4CtGf_vPABz5q19V2ETLdjtE3
 3GXyMPyxFWNuMwKNM0>
X-ME-Received: <xmr:Jl1jZVt6eJOHqPjATvPlVclgHOfRADR0U5g0YWGJ-WlrpfPlOTNwXiZ-5MdwT7CYEGX4hQ>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehledgjeegucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
 cujfgurhepkfffgggfuffhvfevfhgjtgfgsehtjeertddtfeejnecuhfhrohhmpeffmhhi
 thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth
 htvghrnhepjeegiedvfefghfduteeuveejgfefleeltedvtdefveekheevffeuuefhheej
 fffhnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug
 hmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:Jl1jZeZGAQMep_8w0hNVMuoQOOvGU61jOTyAJPZq6tUY-Uv_1gIEDw>
 <xmx:Jl1jZUaYiXkbl13Kuc8bNhDco0SRzPQKNvF_XTsXCNTCovQWFuemcg>
 <xmx:Jl1jZSBW0_PxybY4xPKthDWWbdk7lh-1ICORgJyYbIVC5lIs20pHug>
 <xmx:Jl1jZTkoOVl5eBDywWHh7-pVA_bG8PiZCXJchwoxmoai6HbgA5jj6Q>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sun,
 26 Nov 2023 09:58:44 -0500 (EST)
Message-ID: <ec6230a6-38dc-02cf-3a8f-9dafda4a8ab7@HIDDEN>
Date: Sun, 26 Nov 2023 16:58:41 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Content-Language: en-US
From: Dmitry Gutov <dmitry@HIDDEN>
To: JD Smith <jdtsmith@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>,
 Yuan Fu <casouri@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
 <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN> <83v89qcfsb.fsf@HIDDEN>
 <9B8C904A-3729-44AF-82F7-3BEA849F46D0@HIDDEN>
 <b7ba3a23-6b8b-efba-9b76-0c49abd0fd03@HIDDEN>
In-Reply-To: <b7ba3a23-6b8b-efba-9b76-0c49abd0fd03@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 7bit
X-Spam-Score: -2.9 (--)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.9 (---)

On 26/11/2023 04:04, Dmitry Gutov wrote:
> As for what to do about this one -- probably something involving 
> syntax-propertize-extend-region-functions, adding an entry which would 
> initialize the parser, but not call syntax-ppss-flush-cache directly (or 
> at least not just that). It would signal the earlier position to extend 
> to through some dynamic variable. This is getting tricky enough to move 
> from the individual major modes into treesit.el proper, I think.

Alternatively, we'd trigger updates eagerly from within 
treesit_record_change -- that would make it slower, invalidating the 
comment above it. Not sure by how much, though.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 26 Nov 2023 02:04:27 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 25 21:04:27 2023
Received: from localhost ([127.0.0.1]:40514 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r74VX-00067f-27
	for submit <at> debbugs.gnu.org; Sat, 25 Nov 2023 21:04:27 -0500
Received: from wout1-smtp.messagingengine.com ([64.147.123.24]:38751)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dmitry@HIDDEN>) id 1r74VU-00067O-UI
 for 67262 <at> debbugs.gnu.org; Sat, 25 Nov 2023 21:04:25 -0500
Received: from compute1.internal (compute1.nyi.internal [10.202.2.41])
 by mailout.west.internal (Postfix) with ESMTP id 4F9573200A29;
 Sat, 25 Nov 2023 21:04:13 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute1.internal (MEProxy); Sat, 25 Nov 2023 21:04:13 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
 :cc:content-type:content-type:date:date:from:from:in-reply-to
 :in-reply-to:message-id:mime-version:references:reply-to:sender
 :subject:subject:to:to; s=fm3; t=1700964252; x=1701050652; bh=a9
 Lyz/C1Q7pCc+s0vAFxWUT3th/1UmmH+obTnOAqC7c=; b=NXUJCAE5E7psMVGs4e
 R7ckz+cpGO5GV9XPl4ME+opls6HQB1mJLVbUisBYibHliUq3+xCnu8cQojqiEdmx
 +ceKFfe133hq3LS1XrnSCp+vCY4lfbiOutG0330xRbW/Tkn9DOY5PpnWRkx/vgdV
 hr8FZrWIT3ua/uW254lJbYFg9inuoQKvZ2K8kCaaIIxwMPDjwcaWc/tsveUP42zm
 mj6limlnk3kgJRuNLxs+oj/kaZnY4JfHW0prDdB4TH52F06WQAFrS9UHNMX11Gsk
 R3YfxYEC3VpYGUEc92/LgPtaGjMZRT+Fpm71jhoDIylV8FGDmJ3wt6jxP6MJ8iS6
 5b+w==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:cc:content-type:content-type:date:date
 :feedback-id:feedback-id:from:from:in-reply-to:in-reply-to
 :message-id:mime-version:references:reply-to:sender:subject
 :subject:to:to:x-me-proxy:x-me-proxy:x-me-sender:x-me-sender
 :x-sasl-enc; s=fm1; t=1700964252; x=1701050652; bh=a9Lyz/C1Q7pCc
 +s0vAFxWUT3th/1UmmH+obTnOAqC7c=; b=E2drv21aSFQbZiyIKR1EQqw90x5um
 PlvAhFNO0vKVAGj2hI175YHfEUPGQCCGRaAqDMubGKpX2XjJ+IduViPb/Y94P/Z/
 ey/VHzv0E23KuK4s1S49mTZJ3gYGVUjhQBIXSp+xbhNTRmPBg5g2gIPXLASMnqbI
 LN7dC4yoZS/hxilit0SWFRkMafrI9f3StpSB6dwwchVLKwFRNxoDgpQsxQ6hNNWG
 MJtvE95DL5IaPU0aZARDa0qU6e7xpf+O5dpKVo9dOrl34Xic6uy35NRannWoqYRA
 ZmMmxT6TqFXGMChgELL7SoU4CHaMzOcXHvTE/AZQQXoAePYrNiHq70c+A==
X-ME-Sender: <xms:nKdiZVfFMCazIlZvP92h5k_Pxk6SnKzJkcyUDDZbELw7igYXbav8bg>
 <xme:nKdiZTNMOrruFCGGK20QhWCMqMBFhlKs2bkWblq-FvY8Bjn7MMmqknO6b2faK2kjv
 IT3Nqw04XT13CZoOuc>
X-ME-Received: <xmr:nKdiZeibrBuJy07HbPHNO6UVbYESLxjiz0sWBYx_pvB8rGa9ivCjkvmH1Kwf15S6fFEx1g>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudehkedggeefucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
 cujfgurheptgfkffggfgfuvfevfhfhjgesmhdtreertdefjeenucfhrhhomhepffhmihht
 rhihucfiuhhtohhvuceoughmihhtrhihsehguhhtohhvrdguvghvqeenucggtffrrghtth
 gvrhhnpeehteekgfetieeujeeuvddtvdelteffleejteduvdefffejieehheeuteffveei
 jeenucevlhhushhtvghrufhiiigvpedtnecurfgrrhgrmhepmhgrihhlfhhrohhmpegumh
 hithhrhiesghhuthhovhdruggvvh
X-ME-Proxy: <xmx:nKdiZe89VAtO7M1HpW5WgSRmoGBXHxt6-T9QfwnORHRMWgC9ZCi2PA>
 <xmx:nKdiZRvdGhlzBh6tkiCtHUjlTY20Sb9i_dkRHDalGu_I8tQ9DvXsWg>
 <xmx:nKdiZdEC4ZIqdUfQGHX4HIRjVtpiwdz8K6SmhVDXtoTBnR0j5BdfXA>
 <xmx:nKdiZa5SfGKKIaTaaq225OSuID_zTxp9A4U6jNxwT5MRYlU2bynjmQ>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
 25 Nov 2023 21:04:11 -0500 (EST)
Content-Type: multipart/mixed; boundary="------------Q0aIrxjQwHlh0z0jWLCDsSeL"
Message-ID: <b7ba3a23-6b8b-efba-9b76-0c49abd0fd03@HIDDEN>
Date: Sun, 26 Nov 2023 04:04:07 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Content-Language: en-US
To: JD Smith <jdtsmith@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>,
 Yuan Fu <casouri@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
 <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN> <83v89qcfsb.fsf@HIDDEN>
 <9B8C904A-3729-44AF-82F7-3BEA849F46D0@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <9B8C904A-3729-44AF-82F7-3BEA849F46D0@HIDDEN>
X-Spam-Score: -2.9 (--)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.9 (---)

This is a multi-part message in MIME format.
--------------Q0aIrxjQwHlh0z0jWLCDsSeL
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit

On 25/11/2023 16:42, JD Smith wrote:
> Bridging emacs syntax to treesitter in a robust way seems like it could be a subtle enterprise, so I’d prefer to leave that to one of the experts.  Right now the syntax-propertize-function in python-mode does one simple thing: ensure triple quotes are properly marked as strings.  Since the treesitter grammar doesn’t distinguish between different flavors of strings, something similar would still be needed, if we want to continue to treat various string flavors distinctly using syntax.
> 
> Is moving all syntactification (beyond just font-lock) over to TS an explicit goal for all the *-ts-mode’s?

It would make sense - since this way we would only have one source of 
syntax-recognition bugs, rather than two (both the grammar and the 
definition in Elisp).

Attached is a patch you can try (that uses treesit for s-p-f).

Unfortunately, it's not quite perfect (nor is python-syntax-stringify, 
according to its FIXME inside): after certain modifications, the 
syntax-table property is not applied.

I've done some print-debugging in python--treesit-parser-after-change, 
and it looks like the problem is this: in certain cases (e.g. when 
electric-pair-post-self-insert-function fires) the parser notifier fires 
only after syntax-propertize has been called -- and it fires inside of 
it. Meaning it's too late to flush the syntax-propertize cache at that 
point.

The reason for it is, overall, the fast that we're trigger parser's 
after-change notifiers lazily: only after some other feature has to 
initialize the parser, calling treesit_ensure_parsed from 
treesit-parser-root-node.

I think bug#66732 might also be a variation of this problem.

As for what to do about this one -- probably something involving 
syntax-propertize-extend-region-functions, adding an entry which would 
initialize the parser, but not call syntax-ppss-flush-cache directly (or 
at least not just that). It would signal the earlier position to extend 
to through some dynamic variable. This is getting tricky enough to move 
from the individual major modes into treesit.el proper, I think.

Yuan and others, thoughts welcome.

JD, I do believe the attached patch is TRT (or close to it), but 
depending on how it works for you, and how quickly we deal with the 
above problem, it might make sense to enact your original suggestion first.

And finally, here's the backtrace that led me to the above conclusions:

   backtrace()
   (message "in progress, backtrace %s" (backtrace))
   (progn (message "in progress, backtrace %s" (backtrace)))
   (if (syntax-propertize--in-process-p) (progn (message "in progress, 
backtrace %s" (backtrace))))
   (save-current-buffer (set-buffer (treesit-parser-buffer parser)) 
(message "flushing %s up to %s" ranges (let* ((--cl-var-- ranges) (r 
nil) (--cl-var-- nil)) (while (consp --cl-var--) (setq r (car 
--cl-var--)) (let* ((temp (car r))) (setq --cl-var-- (if --cl-var-- (min 
--cl-var-- temp) temp))) (setq --cl-var-- (cdr --cl-var--))) 
--cl-var--)) (syntax-ppss-flush-cache (let* ((--cl-var-- ranges) (r nil) 
(--cl-var-- nil)) (while (consp --cl-var--) (setq r (car --cl-var--)) 
(let* ((temp (car r))) (setq --cl-var-- (if --cl-var-- (min --cl-var-- 
temp) temp))) (setq --cl-var-- (cdr --cl-var--))) --cl-var--)) (if 
(syntax-propertize--in-process-p) (progn (message "in progress, 
backtrace %s" (backtrace)))) (message "flushed up to %d, %s" 
syntax-propertize--done syntax-ppss-wide))
   (progn (save-current-buffer (set-buffer (treesit-parser-buffer 
parser)) (message "flushing %s up to %s" ranges (let* ((--cl-var-- 
ranges) (r nil) (--cl-var-- nil)) (while (consp --cl-var--) (setq r (car 
--cl-var--)) (let* ((temp ...)) (setq --cl-var-- (if --cl-var-- ... 
temp))) (setq --cl-var-- (cdr --cl-var--))) --cl-var--)) 
(syntax-ppss-flush-cache (let* ((--cl-var-- ranges) (r nil) (--cl-var-- 
nil)) (while (consp --cl-var--) (setq r (car --cl-var--)) (let* ((temp 
...)) (setq --cl-var-- (if --cl-var-- ... temp))) (setq --cl-var-- (cdr 
--cl-var--))) --cl-var--)) (if (syntax-propertize--in-process-p) (progn 
(message "in progress, backtrace %s" (backtrace)))) (message "flushed up 
to %d, %s" syntax-propertize--done syntax-ppss-wide)))
   (if ranges (progn (save-current-buffer (set-buffer 
(treesit-parser-buffer parser)) (message "flushing %s up to %s" ranges 
(let* ((--cl-var-- ranges) (r nil) (--cl-var-- nil)) (while (consp 
--cl-var--) (setq r (car --cl-var--)) (let* (...) (setq --cl-var-- ...)) 
(setq --cl-var-- (cdr --cl-var--))) --cl-var--)) 
(syntax-ppss-flush-cache (let* ((--cl-var-- ranges) (r nil) (--cl-var-- 
nil)) (while (consp --cl-var--) (setq r (car --cl-var--)) (let* (...) 
(setq --cl-var-- ...)) (setq --cl-var-- (cdr --cl-var--))) --cl-var--)) 
(if (syntax-propertize--in-process-p) (progn (message "in progress, 
backtrace %s" (backtrace)))) (message "flushed up to %d, %s" 
syntax-propertize--done syntax-ppss-wide))))
   python--treesit-parser-after-change(((27 . 50)) #<treesit-parser for 
python>)
   treesit-buffer-root-node(python)
   treesit-node-at(42)
   (let ((node (treesit-node-at (point)))) (cond ((equal 
(treesit-node-type node) "string_content") (put-text-property (- (point) 
3) (- (point) 2) 'syntax-table (string-to-syntax "|"))) ((and (equal 
(treesit-node-type node) "\"") (= (treesit-node-start node) (- (point) 
3))) (put-text-property (1- (point)) (point) 'syntax-table 
(string-to-syntax "|")))))
   (cond (t (message "pt %s" (point)) (let ((node (treesit-node-at 
(point)))) (cond ((equal (treesit-node-type node) "string_content") 
(put-text-property (- (point) 3) (- (point) 2) 'syntax-table 
(string-to-syntax "|"))) ((and (equal (treesit-node-type node) "\"") (= 
(treesit-node-start node) (- ... 3))) (put-text-property (1- (point)) 
(point) 'syntax-table (string-to-syntax "|")))))))
   (while (and (< (point) end) (re-search-forward "\\(?:\"\"\"\\|'''\\)" 
end t)) (cond (t (message "pt %s" (point)) (let ((node (treesit-node-at 
(point)))) (cond ((equal (treesit-node-type node) "string_content") 
(put-text-property (- ... 3) (- ... 2) 'syntax-table (string-to-syntax 
"|"))) ((and (equal ... "\"") (= ... ...)) (put-text-property (1- ...) 
(point) 'syntax-table (string-to-syntax "|"))))))))
   (closure (t) (start end) (goto-char start) (while (and (< (point) 
end) (re-search-forward "\\(?:\"\"\"\\|'''\\)" end t)) (cond (t (message 
"pt %s" (point)) (let ((node ...)) (cond (... ...) (... ...)))))))(39 50)
   funcall((closure (t) (start end) (goto-char start) (while (and (< 
(point) end) (re-search-forward "\\(?:\"\"\"\\|'''\\)" end t)) (cond (t 
(message "pt %s" (point)) (let ((node ...)) (cond (... ...) (... 
...))))))) 39 50)
   python--treesit-syntax-propertize-function-1(39 50)
   syntax-propertize(42)
   syntax-ppss(42)
   electric-pair-syntax-info(39)
   electric-pair-post-self-insert-function()
   self-insert-command(1 39)
   funcall-interactively(self-insert-command 1 39)
   #<subr call-interactively>(self-insert-command nil nil)
   call-interactively@ido-cr+-record-current-command(#<subr 
call-interactively> self-insert-command nil nil)
   apply(call-interactively@ido-cr+-record-current-command #<subr 
call-interactively> (self-insert-command nil nil))
   call-interactively(self-insert-command nil nil)
   command-execute(self-insert-command)
--------------Q0aIrxjQwHlh0z0jWLCDsSeL
Content-Type: text/x-patch; charset=UTF-8;
 name="python--treesit-syntax-propertize-function.diff"
Content-Disposition: attachment;
 filename="python--treesit-syntax-propertize-function.diff"
Content-Transfer-Encoding: base64

ZGlmZiAtLWdpdCBhL2xpc3AvcHJvZ21vZGVzL3B5dGhvbi5lbCBiL2xpc3AvcHJvZ21vZGVz
L3B5dGhvbi5lbAppbmRleCBhYjNiZjFiNGVjMC4uNjU5ZGVmMTg5OTkgMTAwNjQ0Ci0tLSBh
L2xpc3AvcHJvZ21vZGVzL3B5dGhvbi5lbAorKysgYi9saXNwL3Byb2dtb2Rlcy9weXRob24u
ZWwKQEAgLTEyMzcsNiArMTIzNywyOSBAQCBweXRob24tLXRyZWVzaXQtZm9udGlmeS12YXJp
YWJsZQogICAgICAodHJlZXNpdC1ub2RlLXN0YXJ0IG5vZGUpICh0cmVlc2l0LW5vZGUtZW5k
IG5vZGUpCiAgICAgICdmb250LWxvY2stdmFyaWFibGUtdXNlLWZhY2Ugb3ZlcnJpZGUgc3Rh
cnQgZW5kKSkpCiAKKyhkZWZjb25zdCBweXRob24tLXRyZWVzaXQtc3ludGF4LXByb3BlcnRp
emUtZnVuY3Rpb24KKyAgKHN5bnRheC1wcm9wZXJ0aXplLXJ1bGVzCisgICAoKHJ4IChvciAi
XCJcIlwiIiAiJycnIikpCisgICAgKDAgKGlnbm9yZQorICAgICAgICAobGV0ICgobm9kZSAo
dHJlZXNpdC1ub2RlLWF0IChwb2ludCkpKSkKKyAgICAgICAgICAoY29uZAorICAgICAgICAg
ICAoKGVxdWFsICh0cmVlc2l0LW5vZGUtdHlwZSBub2RlKSAic3RyaW5nX2NvbnRlbnQiKQor
ICAgICAgICAgICAgKHB1dC10ZXh0LXByb3BlcnR5ICgtIChwb2ludCkgMykgKC0gKHBvaW50
KSAyKQorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICdzeW50YXgtdGFibGUgKHN0
cmluZy10by1zeW50YXggInwiKSkpCisgICAgICAgICAgICgoYW5kIChlcXVhbCAodHJlZXNp
dC1ub2RlLXR5cGUgbm9kZSkgIlwiIikKKyAgICAgICAgICAgICAgICAgKD0gKHRyZWVzaXQt
bm9kZS1zdGFydCBub2RlKSAoLSAocG9pbnQpIDMpKSkKKyAgICAgICAgICAgIChwdXQtdGV4
dC1wcm9wZXJ0eSAoMS0gKHBvaW50KSkgKHBvaW50KQorICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICdzeW50YXgtdGFibGUgKHN0cmluZy10by1zeW50YXggInwiKSkpKSkpKSkp
KQorCisoZGVmdW4gcHl0aG9uLS10cmVlc2l0LXBhcnNlci1hZnRlci1jaGFuZ2UgKHJhbmdl
cyBwYXJzZXIpCisgIDs7IE1ha2Ugc3VyZSB3ZSByZS1zeW50YXgtcHJvcGVydGl6ZSB0aGUg
ZnVsbCBub2RlIHRoYXQgaXMgYmVpbmcKKyAgOzsgZWRpdGVkLiAgRm9yIHRyaXBsZS1xdW90
ZWQgc3RyaW5ncy4KKyAgKHdoZW4gcmFuZ2VzCisgICAgKHdpdGgtY3VycmVudC1idWZmZXIg
KHRyZWVzaXQtcGFyc2VyLWJ1ZmZlciBwYXJzZXIpCisgICAgICAoc3ludGF4LXBwc3MtZmx1
c2gtY2FjaGUgKGNsLWxvb3AgZm9yIHIgaW4gcmFuZ2VzCisgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICAgICAgICAgbWluaW1pemUgKGNhciByKSkpKSkpCisKIAwKIDs7OyBJ
bmRlbnRhdGlvbgogCkBAIC02ODUxLDYgKzY4NzQsMTQgQEAgcHl0aG9uLXRzLW1vZGUKICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiX2RlZmluaXRp
b24iKSkKICAgICAoc2V0cS1sb2NhbCB0cmVlc2l0LWRlZnVuLW5hbWUtZnVuY3Rpb24KICAg
ICAgICAgICAgICAgICAjJ3B5dGhvbi0tdHJlZXNpdC1kZWZ1bi1uYW1lKQorCisgICAgKHNl
dHEtbG9jYWwgc3ludGF4LXByb3BlcnRpemUtZnVuY3Rpb24KKyAgICAgICAgICAgICAgICBw
eXRob24tLXRyZWVzaXQtc3ludGF4LXByb3BlcnRpemUtZnVuY3Rpb24pCisKKyAgICA7OyBN
YWtlIHN1cmUgaXQncyBwbGFjZWQgYmVmb3JlIGZvbnQtbG9jaydzIG5vdGlmaWVyLgorICAg
ICh0cmVlc2l0LXBhcnNlci1hZGQtbm90aWZpZXIgKGNhciAodHJlZXNpdC1wYXJzZXItbGlz
dCkpCisgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAjJ3B5dGhvbi0tdHJlZXNp
dC1wYXJzZXItYWZ0ZXItY2hhbmdlKQorCiAgICAgKHRyZWVzaXQtbWFqb3ItbW9kZS1zZXR1
cCkKIAogICAgIChweXRob24tc2tlbGV0b24tYWRkLW1lbnUtaXRlbXMpCg==

--------------Q0aIrxjQwHlh0z0jWLCDsSeL--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 25 Nov 2023 14:42:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 25 09:42:48 2023
Received: from localhost ([127.0.0.1]:38256 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r6trs-0001Rt-15
	for submit <at> debbugs.gnu.org; Sat, 25 Nov 2023 09:42:48 -0500
Received: from mail-qt1-x835.google.com ([2607:f8b0:4864:20::835]:44431)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jdtsmith@HIDDEN>) id 1r6trq-0001Rg-Bg
 for 67262 <at> debbugs.gnu.org; Sat, 25 Nov 2023 09:42:46 -0500
Received: by mail-qt1-x835.google.com with SMTP id
 d75a77b69052e-41cd4cc515fso15538461cf.1
 for <67262 <at> debbugs.gnu.org>; Sat, 25 Nov 2023 06:42:41 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1700923355; x=1701528155; darn=debbugs.gnu.org;
 h=to:references:message-id:content-transfer-encoding:cc:date
 :in-reply-to:from:subject:mime-version:from:to:cc:subject:date
 :message-id:reply-to;
 bh=dIuoqavLS6ieFo+4s9nlUxcmHM+w60AtzPdwEBfo9QU=;
 b=jQu+KeR4CyLNcz/N2grL8GU46+5uZzbex32VL7DwGlaZN8dEDy5uS13nC10cAvBnL8
 S1IVEV5YEcW9NTa7tC7ElFfEDJmzNYGJzAtayo43ahL6fW2B3THGku+XF3nE1PmPTQw+
 UnsIngLhMv3x7jfwex5n7h1pxabVuSjQoYcu/lSDZNYgeyYQSQLlOyXaujFhCUO7SJVe
 M41g+MiB1jeCsfDdTQcFVtpTIMe+pLmDKnAgtDIwQacfEg/2pVv+nx8xJCw/zm9C9D7X
 DxtsTbnXNgIQa3BM+36CsDMXhht6lkd1bmOmdDe2eZ9bSrEebPDRPVYtJY0q4oan6Lzi
 xbtA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1700923355; x=1701528155;
 h=to:references:message-id:content-transfer-encoding:cc:date
 :in-reply-to:from:subject:mime-version:x-gm-message-state:from:to:cc
 :subject:date:message-id:reply-to;
 bh=dIuoqavLS6ieFo+4s9nlUxcmHM+w60AtzPdwEBfo9QU=;
 b=h07eFs2V8arOEPUE2/Fz+yyvYl5CL4o2QWcjAmGQ5sg8E4JePzi168wcet4+elDXWN
 yLy8swXtGcU3U168ShTCjGXd0dypr9k0G3R8x5/RPfC5Je4Sgv8KXH+ipe5uWQzLkxos
 LgNJbo0jaAfds5tWowtr+qdi73/pIUhD3+jMajZkG/qwJerXP/tMxgl8mOqkMjz03alz
 VpW7crKq6FtT/C6vzSqDXi+lXlyHuQhJ7usJ0jAuDTM4FMNot6eMYBxI7YcJNnAlykR8
 AewlJTsDWJmdp8VGnqIeBPwxjnLhx9XhFO9FsmE9+Hsvy9C2ViPXIeo3Exyyc7HbTmii
 qAPQ==
X-Gm-Message-State: AOJu0Yx7iTvhUY4Z+X7ez/e10CLMd2YmGI32nLfxQDJ+p8H5t9pwVz7Z
 MyqVxC5TrTtkb3qV2DhxTHU=
X-Google-Smtp-Source: AGHT+IGUZjEzf+PHT6wGiZZ5rUE3O9A/9z4GDHK83VzOnyF7p+cIfF4lfHbaO19ZrBK8R5iDav1uIQ==
X-Received: by 2002:a05:622a:514:b0:421:f47f:d758 with SMTP id
 l20-20020a05622a051400b00421f47fd758mr8362303qtx.8.1700923355608; 
 Sat, 25 Nov 2023 06:42:35 -0800 (PST)
Received: from smtpclient.apple (cm-24-53-187-34.buckeyecom.net.
 [24.53.187.34]) by smtp.gmail.com with ESMTPSA id
 z25-20020ac86b99000000b003f6ac526568sm2216670qts.39.2023.11.25.06.42.34
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 25 Nov 2023 06:42:34 -0800 (PST)
Content-Type: text/plain;
	charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3774.200.91.1.1\))
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
From: JD Smith <jdtsmith@HIDDEN>
In-Reply-To: <83v89qcfsb.fsf@HIDDEN>
Date: Sat, 25 Nov 2023 09:42:23 -0500
Content-Transfer-Encoding: quoted-printable
Message-Id: <9B8C904A-3729-44AF-82F7-3BEA849F46D0@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
 <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN> <83v89qcfsb.fsf@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
X-Mailer: Apple Mail (2.3774.200.91.1.1)
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67262
Cc: Dmitry Gutov <dmitry@HIDDEN>, 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Bridging emacs syntax to treesitter in a robust way seems like it could =
be a subtle enterprise, so I=E2=80=99d prefer to leave that to one of =
the experts.  Right now the syntax-propertize-function in python-mode =
does one simple thing: ensure triple quotes are properly marked as =
strings.  Since the treesitter grammar doesn=E2=80=99t distinguish =
between different flavors of strings, something similar would still be =
needed, if we want to continue to treat various string flavors =
distinctly using syntax. =20

Is moving all syntactification (beyond just font-lock) over to TS an =
explicit goal for all the *-ts-mode=E2=80=99s?

> On Nov 25, 2023, at 5:01=E2=80=AFAM, Eli Zaretskii <eliz@HIDDEN> =
wrote:
>=20
>> Date: Sun, 19 Nov 2023 00:52:06 +0200
>> Cc: 67262 <at> debbugs.gnu.org
>> From: Dmitry Gutov <dmitry@HIDDEN>
>>=20
>> On 18/11/2023 19:18, JD Smith wrote:
>>>   (setq-local syntax-propertize-function=20
>>> python-syntax-propertize-function)
>>=20
>> This is not a bad choice, but even better would be to write a s-p-f=20=

>> based on the tree-sitter parse tree.
>>=20
>> There are examples in ruby-ts-mode and js-ts-mode.
>=20
> JD, would you like to try writing such a syntax-propertize-function?





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 25 Nov 2023 10:02:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 25 05:02:11 2023
Received: from localhost ([127.0.0.1]:37878 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r6pUI-0004jC-UE
	for submit <at> debbugs.gnu.org; Sat, 25 Nov 2023 05:02:11 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:34346)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1r6pUG-0004iz-Ql
 for 67262 <at> debbugs.gnu.org; Sat, 25 Nov 2023 05:02:09 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1r6pU6-0005Oh-4Y; Sat, 25 Nov 2023 05:01:58 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
 Date; bh=QoKCSirePNGI/IVJC9I637IlbxEq6G8hl9Y5TeG6Mco=; b=VeH3+wVnlFVSYelJQiFd
 1BTyd24a67JM2L3tmQFdl1/OHwUrvlTAu3EdYNfq6elMeGEex605p2sZYKCvj1fg3lvMl937XevRj
 gjKblYAgrqCDbU671pPvJlO8ke03ueWKux6yCNqj8okhLd3ZoM+o0L+RrHZYcSv2uqBdnz72d28eE
 8Pi50e6DrcXn2ndbjg91N7jvlMu9V+BqTAA8Dwm0eKC7l/E1pesKedYZLIrw/w9c8VrVX1GqjhkHG
 zhkN/i1/l21dwY5BstA2G18w8ElJE1DCTAmu4hy+8x55R6kp0oFVO3ph8j3xR7gLohohR8kCwUpJZ
 +2qNch+3k6QJrg==;
Date: Sat, 25 Nov 2023 12:01:56 +0200
Message-Id: <83v89qcfsb.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN> (message from
 Dmitry Gutov on Sun, 19 Nov 2023 00:52:06 +0200)
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
 <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org, jdtsmith@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> Date: Sun, 19 Nov 2023 00:52:06 +0200
> Cc: 67262 <at> debbugs.gnu.org
> From: Dmitry Gutov <dmitry@HIDDEN>
> 
> On 18/11/2023 19:18, JD Smith wrote:
> >    (setq-local syntax-propertize-function 
> > python-syntax-propertize-function)
> 
> This is not a bad choice, but even better would be to write a s-p-f 
> based on the tree-sitter parse tree.
> 
> There are examples in ruby-ts-mode and js-ts-mode.

JD, would you like to try writing such a syntax-propertize-function?




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 18 Nov 2023 22:52:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 18 17:52:25 2023
Received: from localhost ([127.0.0.1]:49631 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r4UAq-0007GS-NM
	for submit <at> debbugs.gnu.org; Sat, 18 Nov 2023 17:52:24 -0500
Received: from wout4-smtp.messagingengine.com ([64.147.123.20]:60401)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dmitry@HIDDEN>) id 1r4UAl-0007GA-DN
 for 67262 <at> debbugs.gnu.org; Sat, 18 Nov 2023 17:52:23 -0500
Received: from compute2.internal (compute2.nyi.internal [10.202.2.46])
 by mailout.west.internal (Postfix) with ESMTP id 382AB32007F1;
 Sat, 18 Nov 2023 17:52:11 -0500 (EST)
Received: from mailfrontend1 ([10.202.2.162])
 by compute2.internal (MEProxy); Sat, 18 Nov 2023 17:52:11 -0500
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gutov.dev; h=cc
 :cc:content-transfer-encoding:content-type:content-type:date
 :date:from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:sender:subject:subject:to:to; s=fm2; t=
 1700347930; x=1700434330; bh=lBDF5ma+0dk1QKWQvy4XVo0nAptBy9Gifp+
 AKpgP0XA=; b=TSngVlDQYmx7V8x57oQE4VP9RAlEfSHOVR9nY1XzA+Tw7IvwH8F
 5AQmdAUR6xBch4cR8tEQgvNsFU0KjS+vwsA1jdxrdnapWyD6Od2/h80eH+60phZn
 gHWqibvhmVZ1vDaqNAyZz6I/+4I3hgdW06yYJb/w4jekFOQ8svbhLRn/0xX4Ql6p
 Yr8Wa4gi2ZSfK5or06+M2RuIp3gy9YWwwZ0oKmt7UWNiRpnoGu96Fj4afuP6lxWN
 IIXOOj07lKeww4sHbPdh4rbT/N94c3iw1D58+9uShHFh/Ws9HAPFrqi5KwFDMUoj
 VmlvDuNzDjddiFCC+vSxD292IxvOPdVUUYg==
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=
 messagingengine.com; h=cc:cc:content-transfer-encoding
 :content-type:content-type:date:date:feedback-id:feedback-id
 :from:from:in-reply-to:in-reply-to:message-id:mime-version
 :references:reply-to:sender:subject:subject:to:to:x-me-proxy
 :x-me-proxy:x-me-sender:x-me-sender:x-sasl-enc; s=fm1; t=
 1700347930; x=1700434330; bh=lBDF5ma+0dk1QKWQvy4XVo0nAptBy9Gifp+
 AKpgP0XA=; b=DqeBZLkngKnZSd9eSsByUdXJAzs6ePiEtWMyu2mS8zRMURqmp0c
 CNGF7JqCChIRKv/4ayz1zCMjxJ+6fPZn2mqAqi9mw/VwuJrHJnBtnLsQUPbBdHWZ
 HOg7xqSWXGUP3EHkA4BoweA8QBu6QXV5fz2r5jfmBL8RMBxEslL3zTaRSYurtaJd
 6Zqea/rhxT2wLUVpuXKHEniM3XhaaTMvCbqZl4tnK5LoLfiEE8ve10xyc5kCWSTB
 AMy56GvY8KGN6QBq2NAsqbTFHXBQMebyfpC5e8kgivySpDdpYI1ycbRaAB2+pUJp
 fGV2IhYvK1VP304eMNlzmY3Pa26kE5hm6BA==
X-ME-Sender: <xms:GkBZZb_GMQq2J1pMA5vk6eNR0XT0pBQdYzcyM8uCymxVM-3QsV8_2Q>
 <xme:GkBZZXtVyzEfUMqT-BEQNldvH8h8ummheU17YEinhjtIkrVvW03lqe1fNSJikbzZ3
 fWhg3ka4_oCJ3hYbGU>
X-ME-Received: <xmr:GkBZZZA_JFFMwzS9Wl9LTzeEtWHvlqGmners1-nwqVmEo9qxjP_a7QOf78kzpcXUXD3a0w>
X-ME-Proxy-Cause: gggruggvucftvghtrhhoucdtuddrgedvkedrudegfedgtddvucetufdoteggodetrfdotf
 fvucfrrhhofhhilhgvmecuhfgrshhtofgrihhlpdfqfgfvpdfurfetoffkrfgpnffqhgen
 uceurghilhhouhhtmecufedttdenucesvcftvggtihhpihgvnhhtshculddquddttddmne
 cujfgurhepkfffgggfuffvvehfhfgjtgfgsehtkeertddtfeejnecuhfhrohhmpeffmhhi
 thhrhicuifhuthhovhcuoegumhhithhrhiesghhuthhovhdruggvvheqnecuggftrfgrth
 htvghrnhephfffheeljeffgeffueeghfekkedtfffgheejvdegjeettdduheeufffggfef
 jeehnecuvehluhhsthgvrhfuihiivgeptdenucfrrghrrghmpehmrghilhhfrhhomhepug
 hmihhtrhihsehguhhtohhvrdguvghv
X-ME-Proxy: <xmx:GkBZZXfcLds2Lhxnh9lyZAyEL4nj2K092IXt23vVdRf4i_GaT5gpFA>
 <xmx:GkBZZQONkTIczEZte14FVkVLeNov9M1K-n-5T44-1Q691bwLvVRcrw>
 <xmx:GkBZZZlYVGY1_OKRdEoI98YEhD7ldgsEyYcjvzSiOtyB91mBfjO60g>
 <xmx:GkBZZa1fYZ40nkHZ6Q6HocLh4bAJmqRKc1vV6ymBsErvY71a_jf5Vw>
Feedback-ID: i0e71465a:Fastmail
Received: by mail.messagingengine.com (Postfix) with ESMTPA; Sat,
 18 Nov 2023 17:52:09 -0500 (EST)
Message-ID: <9bfc5e6f-3612-115f-a59d-35ad629bdf9e@HIDDEN>
Date: Sun, 19 Nov 2023 00:52:06 +0200
MIME-Version: 1.0
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:102.0) Gecko/20100101
 Thunderbird/102.13.0
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Content-Language: en-US
To: JD Smith <jdtsmith@HIDDEN>, Eli Zaretskii <eliz@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN> <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
From: Dmitry Gutov <dmitry@HIDDEN>
In-Reply-To: <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
Content-Type: text/plain; charset=UTF-8; format=flowed
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.9 (--)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.9 (---)

On 18/11/2023 19:18, JD Smith wrote:
>    (setq-local syntax-propertize-function 
> python-syntax-propertize-function)

This is not a bad choice, but even better would be to write a s-p-f 
based on the tree-sitter parse tree.

There are examples in ruby-ts-mode and js-ts-mode.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 18 Nov 2023 17:18:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 18 12:18:24 2023
Received: from localhost ([127.0.0.1]:49447 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r4Oxb-0006S5-QH
	for submit <at> debbugs.gnu.org; Sat, 18 Nov 2023 12:18:24 -0500
Received: from mail-qt1-x82c.google.com ([2607:f8b0:4864:20::82c]:51636)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jdtsmith@HIDDEN>) id 1r4OxY-0006Rr-NL
 for 67262 <at> debbugs.gnu.org; Sat, 18 Nov 2023 12:18:22 -0500
Received: by mail-qt1-x82c.google.com with SMTP id
 d75a77b69052e-41cc56255e3so18514071cf.3
 for <67262 <at> debbugs.gnu.org>; Sat, 18 Nov 2023 09:18:19 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1700327894; x=1700932694; darn=debbugs.gnu.org;
 h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
 :from:from:to:cc:subject:date:message-id:reply-to;
 bh=mtuERcL1BAErYeklFADqdJ9vFPj+BsbUkr1Zx3qZZqc=;
 b=aetLCvz1qiYYEuUVxdC75yKreKEzQxGLwajowRLXfc3BVUfoSu7kC89MLLb2KGrMA0
 zMlZqUy//t2a8if7JH8sa6u2L0ZqWiC1fKlZFIFYYyPeyA8+NSInXxgFsX8RSCISo5i9
 S8ZdWzGkeITDlkXrTX/GkIk3fawvfnjD9VRY/C+On1VqSmdQfaF+mYMgZgJXtm3sMZic
 jAYOo22FGwd26/+Av6kOlyTzn+nisTkDPsMBCFXZHACBMeRWFuuUbPZBHcRwwD9MID2f
 0O0glodXJcU/do6s49F/7z/fNUnVoOCuZh4dmRyvLUbazGlGgzq217/Bz1knYaWoEmOz
 4XTw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1700327894; x=1700932694;
 h=references:to:cc:in-reply-to:date:subject:mime-version:message-id
 :from:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
 bh=mtuERcL1BAErYeklFADqdJ9vFPj+BsbUkr1Zx3qZZqc=;
 b=dqC32K0+YQquf4GKh0358gdLeKm3Ct7u4uBZEiHMZfiga32voR9ENOmvoJJpJMY6Fn
 t7E7dRFHaINVMBtYvXXPM3FT6MOpp6GSX/nhIGmpNE9y8mL9qiWngWzBYTbyfVafTLsT
 S1MF7Up0Rukd4ewWPXCj7g45QJGTmOzywf8rzzXGmQfVBqU3LhomEGhWsgjcxvXBTu8W
 1rYdYpuKuPRbjuim2Vb2lxP4awcIfY24k7fjigcGgRBVcZPu4noHDJkrY4Y7ED7LyE2u
 X2y7/qRml643WZCKRAyqPKUitr/CnNm3SlsK8S+Qzp5ApXKbXACpp48AWIg9oA3Vx/f/
 fLQw==
X-Gm-Message-State: AOJu0Yxqb0W2apUZe0Y1MzwsB8rhLUHXvYrjz+lDQgGEq+KWUmb7aiNT
 LcZVPeYfV+fV8Oc1L1kRiYo=
X-Google-Smtp-Source: AGHT+IGHUcO2aFV/Cy2Vg2OfJ+2XYS3kDwK5TdWnTi44NCRo+u0+KnVjEzuHoT5Y5D94Q3qAsu2drA==
X-Received: by 2002:ac8:7fcf:0:b0:41e:288a:d99e with SMTP id
 b15-20020ac87fcf000000b0041e288ad99emr4151380qtk.39.1700327894205; 
 Sat, 18 Nov 2023 09:18:14 -0800 (PST)
Received: from smtpclient.apple (cm-24-53-187-34.buckeyecom.net.
 [24.53.187.34]) by smtp.gmail.com with ESMTPSA id
 df25-20020a05622a0ed900b00419c39dd28fsm1400170qtb.20.2023.11.18.09.18.12
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 18 Nov 2023 09:18:13 -0800 (PST)
From: JD Smith <jdtsmith@HIDDEN>
Message-Id: <C4289F88-F4B0-4DB8-BAEF-191490D2717A@HIDDEN>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_46145E57-47AD-416E-AD39-436D1147BD68"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\))
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
Date: Sat, 18 Nov 2023 12:18:02 -0500
In-Reply-To: <838r6vm3dj.fsf@HIDDEN>
To: Eli Zaretskii <eliz@HIDDEN>
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
 <838r6vm3dj.fsf@HIDDEN>
X-Mailer: Apple Mail (2.3731.700.6)
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


--Apple-Mail=_46145E57-47AD-416E-AD39-436D1147BD68
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8



> On Nov 18, 2023, at 11:29 AM, Eli Zaretskii <eliz@HIDDEN> wrote:
>=20
>> From: JD Smith <jdtsmith@HIDDEN>
>> Date: Sat, 18 Nov 2023 10:52:05 -0500
>>=20
>> Inside this triple-quoted string, in a python buffer:
>>=20
>> a =3D """This is a test"""
>>=20
>> python-mode yields (python-info-triple-quoted-string-p)=3Dt, whereas =
python-ts-mode gives nil,
>=20
> Can you figure out which part of python-mode's initialization makes
> the above work correctly, and why?  Then we could discuss whether
> moving that part into python-base-mode is TRT.

I took a look. It=E2=80=99s `syntax-propertize-function' that is not =
being setup.  =46rom a buffer in `python-ts-mode', this reenables =
triple-quote recognition:

(progn=20
  (setq-local syntax-propertize-function =
python-syntax-propertize-function)=20
  (syntax-ppss-flush-cache (point-min)))

Note that `python-syntax-propertize-function' mentions =
`python-syntax-stringify', which scans the syntax for triple quotes and =
marks their 'syntax-table.  I am not sure whether this was an oversight, =
or was omitted purposefully from the body of python-ts-mode.=

--Apple-Mail=_46145E57-47AD-416E-AD39-436D1147BD68
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: =
after-white-space;"><br><div><br><blockquote type=3D"cite"><div>On Nov =
18, 2023, at 11:29 AM, Eli Zaretskii &lt;eliz@HIDDEN&gt; =
wrote:</div><br class=3D"Apple-interchange-newline"><div><meta =
charset=3D"UTF-8"><blockquote type=3D"cite" style=3D"font-family: =
Helvetica; font-size: 12px; font-style: normal; font-variant-caps: =
normal; font-weight: 400; letter-spacing: normal; orphans: auto; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; widows: auto; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;">From: JD Smith =
&lt;jdtsmith@HIDDEN&gt;<br>Date: Sat, 18 Nov 2023 10:52:05 =
-0500<br><br>Inside this triple-quoted string, in a python =
buffer:<br><br>a =3D """This is a test"""<br><br>python-mode yields =
(python-info-triple-quoted-string-p)=3Dt, whereas python-ts-mode gives =
nil,<br></blockquote><br style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;">Can you =
figure out which part of python-mode's initialization makes</span><br =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none;"><span =
style=3D"caret-color: rgb(0, 0, 0); font-family: Helvetica; font-size: =
12px; font-style: normal; font-variant-caps: normal; font-weight: 400; =
letter-spacing: normal; text-align: start; text-indent: 0px; =
text-transform: none; white-space: normal; word-spacing: 0px; =
-webkit-text-stroke-width: 0px; text-decoration: none; float: none; =
display: inline !important;">the above work correctly, and why? =
&nbsp;Then we could discuss whether</span><br style=3D"caret-color: =
rgb(0, 0, 0); font-family: Helvetica; font-size: 12px; font-style: =
normal; font-variant-caps: normal; font-weight: 400; letter-spacing: =
normal; text-align: start; text-indent: 0px; text-transform: none; =
white-space: normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none;"><span style=3D"caret-color: rgb(0, 0, 0); =
font-family: Helvetica; font-size: 12px; font-style: normal; =
font-variant-caps: normal; font-weight: 400; letter-spacing: normal; =
text-align: start; text-indent: 0px; text-transform: none; white-space: =
normal; word-spacing: 0px; -webkit-text-stroke-width: 0px; =
text-decoration: none; float: none; display: inline !important;">moving =
that part into python-base-mode is =
TRT.</span><br></div></blockquote><div><br></div>I took a look. It=E2=80=99=
s `syntax-propertize-function' that is not being setup. &nbsp;=46rom a =
buffer in `python-ts-mode', this reenables triple-quote =
recognition:</div><div><br></div><blockquote style=3D"margin: 0 0 0 =
40px; border: none; padding: 0px;"><div>(progn&nbsp;</div><div>&nbsp; =
(setq-local syntax-propertize-function =
python-syntax-propertize-function)&nbsp;</div><div>&nbsp; =
(syntax-ppss-flush-cache =
(point-min)))</div></blockquote><div><br></div>Note that `<span =
style=3D"caret-color: rgb(0, 0, 0); color: rgb(0, 0, =
0);">python-syntax-propertize-function' mentions =
`</span>python-syntax-stringify', which scans the syntax for triple =
quotes and marks their 'syntax-table. &nbsp;I am not sure whether this =
was an oversight, or was omitted purposefully from the body of =
python-ts-mode.</body></html>=

--Apple-Mail=_46145E57-47AD-416E-AD39-436D1147BD68--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at 67262 <at> debbugs.gnu.org:


Received: (at 67262) by debbugs.gnu.org; 18 Nov 2023 16:29:48 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 18 11:29:48 2023
Received: from localhost ([127.0.0.1]:49402 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r4OCZ-0004zR-MF
	for submit <at> debbugs.gnu.org; Sat, 18 Nov 2023 11:29:47 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10]:40088)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <eliz@HIDDEN>) id 1r4OCX-0004zD-38
 for 67262 <at> debbugs.gnu.org; Sat, 18 Nov 2023 11:29:45 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <eliz@HIDDEN>)
 id 1r4OCQ-0001Cu-OI; Sat, 18 Nov 2023 11:29:38 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:References:Subject:In-Reply-To:To:From:
 Date; bh=3N5kiEouUsOUU53pek3zu+FXLkVk2+PZKngie1QPaTw=; b=UuS358qnk7B9DR/Q4o8F
 jYWAPT5GQ4maPEC9gZfThXUNRb9tdPluEhuBX23PibMASIhHlXJ84dTBr5wR54t9rvivglc8iEKNt
 S+2aV7aFY5vwYhiljdrvhModAM+9HxQnV+UY7/x5IE+nIIy+ODJwfFha/cDqiduIijgK2YOER5jzi
 CqAbB1+lvUFw20dKm8gPuvnGL8qKLmkGplq9oXN5GKbAd21dGL0kZ5oGCGObGFXnwvlfTONN+AD+P
 7B4GYNlOdW+RdmF80Zkd85yRDy91LYg6NnwhM7RNJQoAWXTd0Ic/WuRFUcv4xqMgfLJ7zVANhcDkj
 /8o+fVB8l9T8Tw==;
Date: Sat, 18 Nov 2023 18:29:28 +0200
Message-Id: <838r6vm3dj.fsf@HIDDEN>
From: Eli Zaretskii <eliz@HIDDEN>
To: JD Smith <jdtsmith@HIDDEN>
In-Reply-To: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN> (message from JD
 Smith on Sat, 18 Nov 2023 10:52:05 -0500)
Subject: Re: bug#67262: python-ts-mode cannot identify triple-quoted-strings
References: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 67262
Cc: 67262 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

> From: JD Smith <jdtsmith@HIDDEN>
> Date: Sat, 18 Nov 2023 10:52:05 -0500
> 
> Inside this triple-quoted string, in a python buffer:
> 
> a = """This is a test"""
> 
> python-mode yields (python-info-triple-quoted-string-p)=t, whereas python-ts-mode gives nil,
> defeating the fancy doc string folding both modes implement.
> 
> The reason seems to be that (syntax-ppss) returns something different in position 3 (which is "non-nil
> if inside a string”) between these modes: 
> 
> * t for python-mode (which signals a triple quote)
> *  ?34=" in python-ts-mode 
> 
> If you first load python-mode, then load python-ts-mode, the syntax parse becomes equal between the
> modes, and this bug vanishes.  

Can you figure out which part of python-mode's initialization makes
the above work correctly, and why?  Then we could discuss whether
moving that part into python-base-mode is TRT.

Thanks.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 18 Nov 2023 15:52:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 18 10:52:34 2023
Received: from localhost ([127.0.0.1]:49331 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1r4NcX-00040I-M1
	for submit <at> debbugs.gnu.org; Sat, 18 Nov 2023 10:52:33 -0500
Received: from lists.gnu.org ([2001:470:142::17]:52300)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jdtsmith@HIDDEN>) id 1r4NcT-0003zs-6f
 for submit <at> debbugs.gnu.org; Sat, 18 Nov 2023 10:52:31 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <jdtsmith@HIDDEN>)
 id 1r4NcM-0003HX-IQ
 for bug-gnu-emacs@HIDDEN; Sat, 18 Nov 2023 10:52:22 -0500
Received: from mail-oi1-x229.google.com ([2607:f8b0:4864:20::229])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <jdtsmith@HIDDEN>)
 id 1r4NcK-0007xQ-UQ
 for bug-gnu-emacs@HIDDEN; Sat, 18 Nov 2023 10:52:22 -0500
Received: by mail-oi1-x229.google.com with SMTP id
 5614622812f47-3b2e72fe47fso2049381b6e.1
 for <bug-gnu-emacs@HIDDEN>; Sat, 18 Nov 2023 07:52:20 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=gmail.com; s=20230601; t=1700322738; x=1700927538; darn=gnu.org;
 h=to:date:message-id:subject:mime-version:from:from:to:cc:subject
 :date:message-id:reply-to;
 bh=4UbjX5EVUORu93hsIT4MXdL9at4+ArZnoNRFkCC7jSY=;
 b=mLOwcl5FKFTOA695G8eoNv+NQ56J28JPx3pyDA0F8/fuqDlACmdcxV3NE2CBHovgab
 wOfGkYcKsUsugj3mm/rU03LnIPJVFoUGDZIZco9j6RbTVafc/tsIXLuXWtCDxTDVUGhd
 RVlQvfxmzN7aNAKmzoOswNCS+ccmFOG2LsIRvEqCVBig5Dp17j7IasP/bX1imwliyPSR
 Lt0454RH3MIYmO+i9c9gqS4RSc9jxSWilMh/xkRzEErJYLMeDesFnXFdczYbIaeUckOQ
 0goJQaTrN1Gnk5N1rZjVR48ek2yHKGvL93RWCIZckCZqbuC7WxU6aq5AzB+g1LJBMpXK
 0cNw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20230601; t=1700322738; x=1700927538;
 h=to:date:message-id:subject:mime-version:from:x-gm-message-state
 :from:to:cc:subject:date:message-id:reply-to;
 bh=4UbjX5EVUORu93hsIT4MXdL9at4+ArZnoNRFkCC7jSY=;
 b=WMsPCMfXtRpbd0g9GG2+j9bHl9EbfQzfAXpChiRAzCRZOyIwEh4uiIxWviqiYiS8Nc
 qqtj58giyBbHUI9WnWo+aTwfg174oCbrYKHzUZSRpX5yqRrHndZRtWBW6lbYnm37He9J
 oU4uyAZ9FqKWUj6j5AcwAPMHtgHTUFOJrXNP/0sEleSz94iP56XK+quaMpiaK0QsJJFV
 kd248GSr6p0AKFlrFTLKHT4y+BIQjwO68QIWWoGwPF6xUXsmNjtCJJffoGBMMjlHFboN
 5uEApwv94fT9IxDfLnOolPehHN91cyD0YMvnbDt2SVq+DlWyAazKwTvdC2dZOoXgoYo7
 Docg==
X-Gm-Message-State: AOJu0YyshBQraK2r6WOZzkpTOlumwLF05Uic5umCgQ0uUo2WbhfKpGg5
 kskrGbf7taM9iE1uhVlOKOfKl/UQYSY=
X-Google-Smtp-Source: AGHT+IHzxGgNjHEsWfvUzKoGU0eRTqpr75MvBFUpbCwvZ5uuYzXNihR0vuk2RnxPnMScv01zF77Y2w==
X-Received: by 2002:a05:6808:17:b0:3a3:b39d:a8bf with SMTP id
 u23-20020a056808001700b003a3b39da8bfmr3001984oic.45.1700322738112; 
 Sat, 18 Nov 2023 07:52:18 -0800 (PST)
Received: from smtpclient.apple (cm-24-53-187-34.buckeyecom.net.
 [24.53.187.34]) by smtp.gmail.com with ESMTPSA id
 sn1-20020a05620a948100b00770f2a690a8sm1383453qkn.53.2023.11.18.07.52.16
 for <bug-gnu-emacs@HIDDEN>
 (version=TLS1_2 cipher=ECDHE-ECDSA-AES128-GCM-SHA256 bits=128/128);
 Sat, 18 Nov 2023 07:52:16 -0800 (PST)
From: JD Smith <jdtsmith@HIDDEN>
Content-Type: multipart/alternative;
 boundary="Apple-Mail=_90D71A41-AA9E-4CF7-95B2-67A477C0C181"
Mime-Version: 1.0 (Mac OS X Mail 16.0 \(3731.700.6\))
Subject: python-ts-mode cannot identify triple-quoted-strings
Message-Id: <66A741A1-38B8-40C9-BE84-AF99F74A079F@HIDDEN>
Date: Sat, 18 Nov 2023 10:52:05 -0500
To: bug-gnu-emacs@HIDDEN
X-Mailer: Apple Mail (2.3731.700.6)
Received-SPF: pass client-ip=2607:f8b0:4864:20::229;
 envelope-from=jdtsmith@HIDDEN; helo=mail-oi1-x229.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 HTML_MESSAGE=0.001, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001, T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.0 (/)


--Apple-Mail=_90D71A41-AA9E-4CF7-95B2-67A477C0C181
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain;
	charset=utf-8


Inside this triple-quoted string, in a python buffer:

a =3D """This is a test"""

python-mode yields (python-info-triple-quoted-string-p)=3Dt, whereas =
python-ts-mode gives nil, defeating the fancy doc string folding both =
modes implement.

The reason seems to be that (syntax-ppss) returns something different in =
position 3 (which is "non-nil if inside a string=E2=80=9D) between these =
modes:=20

t for python-mode (which signals a triple quote)
 ?34=3D" in python-ts-mode=20

If you first load python-mode, then load python-ts-mode, the syntax =
parse becomes equal between the modes, and this bug vanishes. =20

python.el v0.28, Emacs v29.1=

--Apple-Mail=_90D71A41-AA9E-4CF7-95B2-67A477C0C181
Content-Transfer-Encoding: quoted-printable
Content-Type: text/html;
	charset=utf-8

<html><head><meta http-equiv=3D"content-type" content=3D"text/html; =
charset=3Dutf-8"></head><body style=3D"overflow-wrap: break-word; =
-webkit-nbsp-mode: space; line-break: =
after-white-space;"><div><br></div><div>Inside this triple-quoted =
string, in a python buffer:</div><div><br></div><div><div>a =3D """This =
is a test"""</div></div><div><br></div><div>python-mode =
yields&nbsp;(python-info-triple-quoted-string-p)=3Dt, whereas =
python-ts-mode gives nil, defeating the fancy doc string folding both =
modes implement.</div><div><br></div><div>The reason seems to be that =
(syntax-ppss) returns something different in position 3 (which is =
"non-nil if inside a string=E2=80=9D) between these =
modes:&nbsp;</div><div><br></div><div><ul class=3D"MailOutline"><li>t =
for python-mode (which signals a triple quote)</li><li>&nbsp;?34=3D" in =
python-ts-mode&nbsp;</li></ul><div><br></div></div><div>If you first =
load python-mode, then load python-ts-mode, the syntax parse becomes =
equal between the modes, and this bug vanishes. =
&nbsp;</div><div><br></div><div>python.el v0.28, Emacs =
v29.1</div></body></html>=

--Apple-Mail=_90D71A41-AA9E-4CF7-95B2-67A477C0C181--




Acknowledgement sent to JD Smith <jdtsmith@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#67262; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 30 Nov 2023 06:30:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.