GNU bug report logs - #63957
29.0.91; c-ts-mode: incorrect fontification in alloc.c

Previous Next

To reply to this bug, email your comments to 63957 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Thu, 08 Jun 2023 08:56:51 +0300

To reproduce:

  emacs -Q
  C-x C-f src/alloc.c RET
  M-x c-ts-mode RET
  C-u 3184 M-g g

Observe that several "else if" clauses in the following fragment are not
fontified correctly:

  #ifdef HAVE_TREE_SITTER
    else if (PSEUDOVECTOR_TYPEP (&vector->header, PVEC_TS_PARSER))
      treesit_delete_parser (PSEUDOVEC_STRUCT (vector, Lisp_TS_Parser));
    else if (PSEUDOVECTOR_TYPEP (&vector->header, PVEC_TS_COMPILED_QUERY))
      treesit_delete_query (PSEUDOVEC_STRUCT (vector, Lisp_TS_Query));
  #endif
  #ifdef HAVE_MODULES
    else if (PSEUDOVECTOR_TYPEP (&vector->header, PVEC_MODULE_FUNCTION))
      {
        ATTRIBUTE_MAY_ALIAS struct Lisp_Module_Function *function
          = (struct Lisp_Module_Function *) vector;
        module_finalize_function (function);
      }
  #endif
  #ifdef HAVE_NATIVE_COMP
    else if (PSEUDOVECTOR_TYPEP (&vector->header, PVEC_NATIVE_COMP_UNIT))
      {
        struct Lisp_Native_Comp_Unit *cu =
          PSEUDOVEC_STRUCT (vector, Lisp_Native_Comp_Unit);
        unload_comp_unit (cu);
      }
    else if (PSEUDOVECTOR_TYPEP (&vector->header, PVEC_SUBR))
      {
        struct Lisp_Subr *subr =
          PSEUDOVEC_STRUCT (vector, Lisp_Subr);
        if (!NILP (subr->native_comp_u))
          {
            /* FIXME Alternative and non invasive solution to this
               cast?  */
            xfree ((char *)subr->symbol_name);
            xfree (subr->native_c_name);
          }
      }
  #endif

In this fragment, "else" has the font-lock-type-face, and "if" has the
font-lock-function-name-face.

I'm guessing this is caused by the #ifdef's there, but can this be
fixed, please?

FTR, I'm using the latest tree-sitter-c grammar library, built from
the HEAD of their Git repository.

In GNU Emacs 29.0.91 (build 42, i686-pc-mingw32) of 2023-06-06 built on
 HOME-C4E4A596F7
Repository revision: bcc222251e1a750a11e365f2faa641cc56c1169d
Repository branch: emacs-29
Windowing system distributor 'Microsoft Corp.', version 5.1.2600
System Description: Microsoft Windows XP Service Pack 3 (v5.1.0.2600)

Configured using:
 'configure -C --prefix=/d/usr --with-wide-int
 --enable-checking=yes,glyphs 'CFLAGS=-O0 -gdwarf-4 -g3''

Configured features:
ACL GIF GMP GNUTLS HARFBUZZ JPEG JSON LCMS2 LIBXML2 MODULES NOTIFY
W32NOTIFY PDUMPER PNG RSVG SOUND SQLITE3 THREADS TIFF
TOOLKIT_SCROLL_BARS TREE_SITTER WEBP XPM ZLIB

Important settings:
  value of $LANG: ENU
  locale-coding-system: cp1255

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  line-number-mode: t
  indent-tabs-mode: t
  transient-mark-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message mailcap yank-media puny dired
dired-loaddefs rfc822 mml mml-sec password-cache epa derived epg rfc6068
epg-config gnus-util text-property-search time-date subr-x mm-decode
mm-bodies mm-encode mail-parse rfc2231 mailabbrev gmm-utils mailheader
cl-loaddefs cl-lib sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils rmc iso-transl tooltip cconv eldoc paren electric
uniquify ediff-hook vc-hooks lisp-float-type elisp-mode mwheel dos-w32
ls-lisp disp-table term/w32-win w32-win w32-vars term/common-win
tool-bar dnd fontset image regexp-opt fringe tabulated-list replace
newcomment text-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch easymenu timer select scroll-bar mouse jit-lock
font-lock syntax font-core term/tty-colors frame minibuffer nadvice seq
simple cl-generic indonesian philippine cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help abbrev obarray oclosure cl-preloaded button
loaddefs theme-loaddefs faces cus-face macroexp files window
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget keymap hashtable-print-readable backquote threads
w32notify w32 lcms2 multi-tty make-network-process emacs)

Memory information:
((conses 16 42750 9786)
 (symbols 48 6287 0)
 (strings 16 16570 3023)
 (string-bytes 1 399211)
 (vectors 16 9324)
 (vector-slots 8 147662 13516)
 (floats 8 23 27)
 (intervals 40 273 98)
 (buffers 888 10))

Message #8 received at 63957 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Theodor Thornhill <theo <at> thornhill.no>, Yuan Fu <casouri <at> gmail.com>
Cc: 63957 <at> debbugs.gnu.org
Subject: Re: bug#63957: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Thu, 08 Jun 2023 09:00:52 +0300

> Date: Thu, 08 Jun 2023 08:56:51 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
> 
> To reproduce:
> 
>   emacs -Q
>   C-x C-f src/alloc.c RET
>   M-x c-ts-mode RET
>   C-u 3184 M-g g
> 
> Observe that several "else if" clauses in the following fragment are not
> fontified correctly:

Adding the relevant folks.

Could you guys please look into this issue?

Message #11 received at 63957 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 63957 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#63957: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Thu, 8 Jun 2023 00:19:04 -0700

> On Jun 7, 2023, at 11:00 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
> 
>> Date: Thu, 08 Jun 2023 08:56:51 +0300
>> From: Eli Zaretskii <eliz <at> gnu.org>
>> 
>> 
>> To reproduce:
>> 
>>  emacs -Q
>>  C-x C-f src/alloc.c RET
>>  M-x c-ts-mode RET
>>  C-u 3184 M-g g
>> 
>> Observe that several "else if" clauses in the following fragment are not
>> fontified correctly:
> 
> Adding the relevant folks.
> 
> Could you guys please look into this issue?

Yep, I’ll look into it tomorrow.

Yuan

Message #14 received at 63957 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 63957 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#63957: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Fri, 9 Jun 2023 23:51:46 -0700

> On Jun 8, 2023, at 12:19 AM, Yuan Fu <casouri <at> gmail.com> wrote:
> 
> 
> 
>> On Jun 7, 2023, at 11:00 PM, Eli Zaretskii <eliz <at> gnu.org> wrote:
>> 
>>> Date: Thu, 08 Jun 2023 08:56:51 +0300
>>> From: Eli Zaretskii <eliz <at> gnu.org>
>>> 
>>> 
>>> To reproduce:
>>> 
>>> emacs -Q
>>> C-x C-f src/alloc.c RET
>>> M-x c-ts-mode RET
>>> C-u 3184 M-g g
>>> 
>>> Observe that several "else if" clauses in the following fragment are not
>>> fontified correctly:
>> 
>> Adding the relevant folks.
>> 
>> Could you guys please look into this issue?

Ok, so this is one of such cases where the preproc directives severs the code and the parser can’t recover very well. We can cover it over by just fontifying “else if” with keyword face, but there are a million ways for the preproc directive to mess up the parser, I don’t think we can cover every case.

Yuan

Message #17 received at 63957 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 63957 <at> debbugs.gnu.org, theo <at> thornhill.no
Subject: Re: bug#63957: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Sat, 10 Jun 2023 11:11:24 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Fri, 9 Jun 2023 23:51:46 -0700
> Cc: Theodor Thornhill <theo <at> thornhill.no>,
>  63957 <at> debbugs.gnu.org
> 
> >>> emacs -Q
> >>> C-x C-f src/alloc.c RET
> >>> M-x c-ts-mode RET
> >>> C-u 3184 M-g g
> >>> 
> >>> Observe that several "else if" clauses in the following fragment are not
> >>> fontified correctly:
> >> 
> >> Adding the relevant folks.
> >> 
> >> Could you guys please look into this issue?
> 
> Ok, so this is one of such cases where the preproc directives severs the code and the parser can’t recover very well. We can cover it over by just fontifying “else if” with keyword face, but there are a million ways for the preproc directive to mess up the parser, I don’t think we can cover every case.

Can you explain what is special in this particular case that is
different from other preprocessor directives?  I'd like to think if
this case is important enough to try harder.

Thanks.

Message #20 received at 63957 <at> debbugs.gnu.org (full text, mbox):

From: Yuan Fu <casouri <at> gmail.com>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 63957 <at> debbugs.gnu.org, Theodor Thornhill <theo <at> thornhill.no>
Subject: Re: bug#63957: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Mon, 12 Jun 2023 02:16:51 -0700

>> 
>>>>> emacs -Q
>>>>> C-x C-f src/alloc.c RET
>>>>> M-x c-ts-mode RET
>>>>> C-u 3184 M-g g
>>>>> 
>>>>> Observe that several "else if" clauses in the following fragment are not
>>>>> fontified correctly:
>>>> 
>>>> Adding the relevant folks.
>>>> 
>>>> Could you guys please look into this issue?
>> 
>> Ok, so this is one of such cases where the preproc directives severs the code and the parser can’t recover very well. We can cover it over by just fontifying “else if” with keyword face, but there are a million ways for the preproc directive to mess up the parser, I don’t think we can cover every case.
> 
> Can you explain what is special in this particular case that is
> different from other preprocessor directives?  I'd like to think if
> this case is important enough to try harder.

I wouldn’t say that this case is special, actually the cases we were able to more or less fix are special. The problem with preproc directives is that they can appear anywhere and break whatever construct they appear in. (Because tree-sitter-c parses preproc constructs as top-level constructs, higher than anything else.)

Say there’s a struct:

struct A
{
  int a;
  int b;
}

If we add preproc directives:

struct A
{
#if A
  int a;
#else  
  int b;
}
#endif

Now the parser will parse the "struct A {“ individually; parse “int a;” individually; and parse “int b; }” individually.

So in general, if a preproc directives butchers some construct, the first part is usually fine (eg, the “struct A {“ part), but the rest often have problems. Like a dangling “else if {}” in if-else-if, or a dangling “xxx }” in a function definition, or maybe a “default: xxx }” in a switch-case.

The Emacs-specific macros we were able to “fix” all have a specific pattern, so we can find them and expect them to have a certain shape, but the breakage caused by preproc directives don’t really have a pattern. I can’t think of a good way to handle them.

I’m not against fixing these case-by-case, if the cases becomes too many and not scalable, we can give up.

Yuan

Message #23 received at 63957 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Yuan Fu <casouri <at> gmail.com>
Cc: 63957 <at> debbugs.gnu.org, theo <at> thornhill.no
Subject: Re: bug#63957: 29.0.91; c-ts-mode: incorrect fontification in alloc.c
Date: Mon, 12 Jun 2023 15:38:22 +0300

> From: Yuan Fu <casouri <at> gmail.com>
> Date: Mon, 12 Jun 2023 02:16:51 -0700
> Cc: Theodor Thornhill <theo <at> thornhill.no>,
>  63957 <at> debbugs.gnu.org
> 
> If we add preproc directives:
> 
> struct A
> {
> #if A
>   int a;
> #else  
>   int b;
> }
> #endif
> 
> Now the parser will parse the "struct A {“ individually; parse “int a;” individually; and parse “int b; }” individually.
> 
> So in general, if a preproc directives butchers some construct, the first part is usually fine (eg, the “struct A {“ part), but the rest often have problems. Like a dangling “else if {}” in if-else-if, or a dangling “xxx }” in a function definition, or maybe a “default: xxx }” in a switch-case.

Thanks.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.