GNU bug report logs - #9148
Make `comment-normalize-vars' more syntax-aware

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9148 in the body.
You can then email your comments to 9148 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Štěpán Němec <stepnem <at> gmail.com>
To: bug-gnu-emacs <bug-gnu-emacs <at> gnu.org>
Subject: Make `comment-normalize-vars' more syntax-aware
Date: Fri, 22 Jul 2011 16:52:10 +0200

Problem:
--------

Emacs currently provides relatively decent support for major modes with
multiple syntaxes, mainly via the `syntax-table' text property.

Unfortunately, even though there is a `comment-use-syntax' variable
which is apparently supposed to indicate that the comment functions
should rely on syntax information instead of `comment-prefix' and
friends, one piece is missing -- the function `comment-normalize-vars',
called at invocation of functions like {comment,uncomment}-region, still
requires `comment-start' to be set explicitly even though the syntax
information is sufficient and `comment-use-syntax' set.

Suggested solution:
-------------------

Modify `comment-normalize-vars' so that it uses the syntax information
(current syntax table, `syntax-table' text property) _only_, whenever
`comment-use-syntax' is set and this is possible.

Use case:
---------

Consider a major mode with two or more clearly separate syntaxes (Org
mode source blocks could be one example), including comments.

Even though it is possible to provide the different text extents with
different values of the `syntax-table' text property, the only way to
work around `comment-normalize-vars' not relying on them currently seems
something like the following:

(defadvice comment-normalize-vars (around use-syntax-info activate)
  (set (make-local-variable 'comment-start)
       (determine-comment-start-based-on-syntax-info))
  ad-do-it)

...which is evil, ugly and doesn't scale.

-- 
Štěpán

Message #8 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Štěpán Němec <stepnem <at> gmail.com>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Tue, 02 Aug 2011 00:04:03 -0400

> Unfortunately, even though there is a `comment-use-syntax' variable
> which is apparently supposed to indicate that the comment functions
> should rely on syntax information instead of `comment-prefix' and
> friends, one piece is missing -- the function `comment-normalize-vars',
> called at invocation of functions like {comment,uncomment}-region, still
> requires `comment-start' to be set explicitly even though the syntax
> information is sufficient and `comment-use-syntax' set.

I'm not sure I understand the problem.  Can you give a more concrete
example where you get undesirable behavior?
Setting comment-start is usually needed to tell Emacs *which* of the
major mode's comment syntaxes to use (so it's only theoretically
unneeded if the syntax-table only include a single comment syntax).


        Stefan

Message #11 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Štěpán Němec <stepnem <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Tue, 02 Aug 2011 12:35:03 +0200

On Tue, 02 Aug 2011 06:04:03 +0200
Stefan Monnier wrote:

>> Unfortunately, even though there is a `comment-use-syntax' variable
>> which is apparently supposed to indicate that the comment functions
>> should rely on syntax information instead of `comment-prefix' and
>> friends, one piece is missing -- the function `comment-normalize-vars',
>> called at invocation of functions like {comment,uncomment}-region, still
>> requires `comment-start' to be set explicitly even though the syntax
>> information is sufficient and `comment-use-syntax' set.
>
> I'm not sure I understand the problem.  Can you give a more concrete
> example where you get undesirable behavior?
> Setting comment-start is usually needed to tell Emacs *which* of the
> major mode's comment syntaxes to use (so it's only theoretically
> unneeded if the syntax-table only include a single comment syntax).

The point is that you can define a major mode with different syntax
tables in different parts of the text (using the syntax-table text
property), but instead of using that information to determine
comment-start and the other variables, `comment-normalize-vars' still
relies on `comment-start' being hard-coded (which doesn't really make
sense in a buffer with multiple comment syntaxes), even when
`comment-use-syntax' is set and the syntax-table information is
sufficient.

-- 
Štěpán

Message #14 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Štěpán Němec <stepnem <at> gmail.com>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Tue, 02 Aug 2011 15:03:54 -0400

> The point is that you can define a major mode with different syntax
> tables in different parts of the text (using the syntax-table text
> property), but instead of using that information to determine
> comment-start and the other variables, `comment-normalize-vars' still
> relies on `comment-start' being hard-coded (which doesn't really make
> sense in a buffer with multiple comment syntaxes), even when
> `comment-use-syntax' is set and the syntax-table information is
> sufficient.

But this is part of a much larger problem (multiple major modes in the
same buffer).

Basically you're suggesting to use the syntax-table property as a way to
specify the major mode used in each part and then teach newcomment.el to
never use buffer-local variables but only the syntax-table around point.

It might be a good idea, but it's a non-trivial change (and still
leaves open the question of how to specify that I want //...\n for the
C chunks and (*...*) for the Pascal chunks).


        Stefan

Message #17 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Štěpán Němec <stepnem <at> gmail.com>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Tue, 02 Aug 2011 21:49:24 +0200

On Tue, 02 Aug 2011 21:03:54 +0200
Stefan Monnier wrote:

>> The point is that you can define a major mode with different syntax
>> tables in different parts of the text (using the syntax-table text
>> property), but instead of using that information to determine
>> comment-start and the other variables, `comment-normalize-vars' still
>> relies on `comment-start' being hard-coded (which doesn't really make
>> sense in a buffer with multiple comment syntaxes), even when
>> `comment-use-syntax' is set and the syntax-table information is
>> sufficient.
>
> But this is part of a much larger problem (multiple major modes in the
> same buffer).

Maybe, but the specific change to `comment-normalize-vars' (gather the
appropriate value for `comment-start' from the syntax-table at point
instead of prompting the user when `comment-use-syntax' is set) I
proposed would seem easy enough and fix this very problem.

> Basically you're suggesting to use the syntax-table property as a way to
> specify the major mode used in each part and then teach newcomment.el to
> never use buffer-local variables but only the syntax-table around
> point.

More precisely and less ambitiously, I'm suggesting to do what
`comment-use-syntax' seems to be claiming already (that's also why I
still consider this a bug, not an enhancement request) -- to rely on
syntax-table instead of the regexp variables when manipulating comments.

> It might be a good idea, but it's a non-trivial change (and still
> leaves open the question of how to specify that I want //...\n for the
> C chunks and (*...*) for the Pascal chunks).

I don't understand. Again: the case I've described, the comment syntax
is unambiguously specified for every part of the buffer using the
syntax-table (either the buffer-local value, or the text property
value). The C chunks have the C syntax table as its value, the Pascal
chunks have the Pascal syntax table as its value (if there is such a
thing -- I don't use C or Pascal; in my specific case it was JavaScript
on one hand and a VimL[1]-like custom syntax with double quote comment
start and end-of-line comment end on the other).

The only thing that's needed to fix this very use case is make
`comment-normalize-vars' set `comment-start' from the syntax-table when
`comment-use-syntax' is set, instead of forcing me to advise it and do
it myself. It'd still be ugly, but not more ugly than it already is now,
and certainly better than using advice.


[1] The "language" used for Vim scripting.

-- 
Štěpán

Message #20 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Štěpán Němec <stepnem <at> gmail.com>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Tue, 02 Aug 2011 16:45:48 -0400

>>> The point is that you can define a major mode with different syntax
>>> tables in different parts of the text (using the syntax-table text
>>> property), but instead of using that information to determine
>>> comment-start and the other variables, `comment-normalize-vars' still
>>> relies on `comment-start' being hard-coded (which doesn't really make
>>> sense in a buffer with multiple comment syntaxes), even when
>>> `comment-use-syntax' is set and the syntax-table information is
>>> sufficient.
>> But this is part of a much larger problem (multiple major modes in the
>> same buffer).
> Maybe, but the specific change to `comment-normalize-vars' (gather the
> appropriate value for `comment-start' from the syntax-table at point
> instead of prompting the user when `comment-use-syntax' is set) I
> proposed would seem easy enough and fix this very problem.

We could do that, yes, but unless we regularly flush the comment-start
value we got, it will then keep using the first value found even on
chunks using another language.

>> Basically you're suggesting to use the syntax-table property as a way to
>> specify the major mode used in each part and then teach newcomment.el to
>> never use buffer-local variables but only the syntax-table around
>> point.
> More precisely and less ambitiously, I'm suggesting to do what
> `comment-use-syntax' seems to be claiming already (that's also why I
> still consider this a bug, not an enhancement request) -- to rely on
> syntax-table instead of the regexp variables when manipulating comments.

comment-use-syntax was less ambitious: it was mostly about using
forward-comment.  It does try to build some of the vars from the
syntax-table, but mostly because comment-end-skip was new in
newcomment.el and I needed the code to work with old packages ;-)

But yes, comment-start could sometimes also be set from the
syntax-table.  I might accept a patch for it, tho I'm not completely
sure how useful that would be.

>> It might be a good idea, but it's a non-trivial change (and still
>> leaves open the question of how to specify that I want //...\n for the
>> C chunks and (*...*) for the Pascal chunks).
> I don't understand. Again: the case I've described, the comment syntax
> is unambiguously specified for every part of the buffer using the
> syntax-table (either the buffer-local value, or the text property
> value). The C chunks have the C syntax table as its value, the Pascal

That's not unambiguous: the C syntax-table specifies several comment
syntaxes (/*..*/ and //..\n) and so does the Pascal syntax table (even
more combinations, IIRC).  So comment-start-skip can be built from the
syntax-table since it needs to match "any comment starter", but
comment-start is more problematic since it would need to choose whose
comment syntax to favor.

> The only thing that's needed to fix this very use case is make
> `comment-normalize-vars' set `comment-start' from the syntax-table when
> `comment-use-syntax' is set, instead of forcing me to advise it and do
> it myself. It'd still be ugly, but not more ugly than it already is now,
> and certainly better than using advice.

But if it's not a major mode used over the whole file, when do you flush
the comment-start value you computed?


        Stefan

Message #25 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Štěpán Němec <stepnem <at> gmail.com>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Thu, 27 Jan 2022 18:57:09 +0100

Štěpán Němec <stepnem <at> gmail.com> writes:

> Suggested solution:
> -------------------
>
> Modify `comment-normalize-vars' so that it uses the syntax information
> (current syntax table, `syntax-table' text property) _only_, whenever
> `comment-use-syntax' is set and this is possible.

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

We now have some multi-mode modes in Emacs, so I'm not sure what's being
proposed here is necessary these days (or indeed it has been fixed in
the decade since this was reported).  Is this something that is still
relevant?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #30 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Štěpán Němec <stepnem <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Mon, 31 Jan 2022 22:39:50 +0100

On Thu, 27 Jan 2022 18:57:09 +0100
Lars Ingebrigtsen wrote:

> Štěpán Němec <stepnem <at> gmail.com> writes:
>
>> Suggested solution:
>> -------------------
>>
>> Modify `comment-normalize-vars' so that it uses the syntax information
>> (current syntax table, `syntax-table' text property) _only_, whenever
>> `comment-use-syntax' is set and this is possible.
>
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
>
> We now have some multi-mode modes in Emacs, so I'm not sure what's being
> proposed here is necessary these days (or indeed it has been fixed in
> the decade since this was reported).  Is this something that is still
> relevant?

Honestly, I don't know: I ended up using the defadvice hack those 10
years ago, but a few years after that the major mode that needed it
became obsolete along with the application whose config files it
supported, and I haven't needed anything similar since then.

Given that nobody other than Stefan, who expressed reservations to my
suggestions, has chimed in during those 10 years and, as you say, the
landscape of possible solutions has likely changed, in its current state
this bug report will probably not be less useful closed than it is
opened, should you so prefer.

  Thanks,

  Štěpán

Message #33 received at 9148 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Štěpán Němec <stepnem <at> gmail.com>
Cc: 9148 <at> debbugs.gnu.org
Subject: Re: bug#9148: Make `comment-normalize-vars' more syntax-aware
Date: Wed, 02 Feb 2022 18:21:27 +0100

Štěpán Němec <stepnem <at> gmail.com> writes:

> Given that nobody other than Stefan, who expressed reservations to my
> suggestions, has chimed in during those 10 years and, as you say, the
> landscape of possible solutions has likely changed, in its current state
> this bug report will probably not be less useful closed than it is
> opened, should you so prefer.

OK; closing this bug report, then.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.