GNU bug report logs - #72478
29.2.50; down-list isn't customizable by modes like backward-up-list

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 5 Aug 2024 15:29:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 05 11:29:24 2024
Received: from localhost ([127.0.0.1]:59303 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1sazeG-0000uG-8S
	for submit <at> debbugs.gnu.org; Mon, 05 Aug 2024 11:29:24 -0400
Received: from lists.gnu.org ([209.51.188.17]:40040)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <sbaugh@HIDDEN>) id 1sazeE-0000u8-8Q
 for submit <at> debbugs.gnu.org; Mon, 05 Aug 2024 11:29:23 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <sbaugh@HIDDEN>)
 id 1sazdr-0004LW-En
 for bug-gnu-emacs@HIDDEN; Mon, 05 Aug 2024 11:28:59 -0400
Received: from mxout6.mail.janestreet.com ([64.215.233.21])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <sbaugh@HIDDEN>)
 id 1sazdn-0006eP-Vi
 for bug-gnu-emacs@HIDDEN; Mon, 05 Aug 2024 11:28:58 -0400
From: Spencer Baugh <sbaugh@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 29.2.50; down-list isn't customizable by modes like backward-up-list
Date: Mon, 05 Aug 2024 11:28:48 -0400
Message-ID: <iera5hrugy7.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=janestreet.com;
 s=waixah; t=1722871731;
 bh=8AOYnyNUTIQDpK4AotiY1lKgomvDoVoi8AZEczflkRs=;
 h=From:To:Cc:Subject:Date;
 b=J8CFfFjfdvdxo0DbAZpJEJ/9shTazCXW2Z33daUdWP5oiEU+h8/KJNx2GQeJnxeQs
 fa3KaAUHvNhZWLDGwYEJOiCcDb/fdcttdi2shkBP8eQ+5yLP9FjMdoSSbq+2/cvw5Q
 d4i6f2D5jCeIxMp+HhC1iT7hltU+uyV2qOPnUrf6WNWjEemXwRecVnEi4/znDmaBcx
 6rPLPQ9iJeIWpCok0D8l14eRNrPXK3FfFDfpTOsfgGF6Uw1Dp8GTTCyD2/VIuOE47V
 +LPBSqST36dJSwFs6VV9T18g+tDQRWcCNUvwlwmB6/ko0za+qr70yK/0v+ghlbIqd5
 l4O+6c/qg0EBA==
Received-SPF: pass client-ip=64.215.233.21; envelope-from=sbaugh@HIDDEN;
 helo=mxout6.mail.janestreet.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
Cc: Lars Ingebrigtsen <larsi@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)


The docstrings of both backward-up-list (C-M-u) and down-list (C-M-d)
say:

"This command will also work on other parentheses-like expressions
defined by the current language mode."

However, this is misleading:
- backward-up-list (and up-list) will use forward-sexp-function
  if that variable is non-nil, and otherwise use scan-lists
- down-list does not check forward-sexp-function and always uses
  scan-lists.

So, the two commands can have significantly different behavior in modes
that configure forward-sexp-function, in particular in modes which give
forward-sexp-function awareness of language-specific syntax which
scan-lists can't handle.

For example, python-mode sets forward-sexp-function to a function which
can operate on both lists and Python syntax expressions.  This results
in backward-up-list and down-list having completely different behavior
from each other.

I suggest we should take one of these options:

A. Update the docstring of up-list/down-list to indicate that while
   they're customizable by the mode, they aren't likely to work the same
   if they are customized by the mode.

B. Add a down-list-function buffer-local variable so that modes which
   are already configuring forward-sexp-function can also configure
   down-list.

C. Add a scan-expressions-function buffer-local variable, and have both
   backward-up-list and down-list check this variable.  Again, modes
   which are already configuring forward-sexp-function to work with
   language-specific syntax can configure this variable.

I personally suggests C.  My major-mode currently implements
forward-sexp-function, and it would be easy for it to also set
scan-expressions-function.  This would also fit well with
tree-sitter-based modes: a tree-sitter based mode could easily have a
tree-sitter-based scan-expressions-function.

I'm interested in implementing this scan-expressions-function variable,
if that sounds like a reasonable solution.


In GNU Emacs 29.2.50 (build 2, x86_64-pc-linux-gnu, X toolkit, cairo
 version 1.15.12, Xaw scroll bars) of 2024-08-04 built on
 igm-qws-hydra336
Windowing system distributor 'The X.Org Foundation', version 11.0.12011000
System Description: Rocky Linux 8.10 (Green Obsidian)

Configured using:
 'configure --config-cache --with-x-toolkit=lucid --without-gpm
 --without-gconf --without-selinux --without-imagemagick --with-modules
 --with-gif=no --with-cairo --with-rsvg --without-compress-install
 --with-tree-sitter --prefix=/j/office/app/emacs/store/20240804-224533
 PKG_CONFIG_PATH=/usr/local/home/garnish/libtree-sitter/0.22.6-1/lib/pkgconfig/'

Configured features:
CAIRO DBUS FREETYPE GLIB GMP GNUTLS GSETTINGS HARFBUZZ JPEG JSON
LIBSYSTEMD LIBXML2 MODULES NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND
SQLITE3 THREADS TIFF TOOLKIT_SCROLL_BARS TREE_SITTER X11 XDBE XIM
XINPUT2 XPM LUCID ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Fundamental