GNU bug report logs - #3824
23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Taylor R Campbell <campbell@HIDDEN>; dated Sat, 11 Jul 2009 16:40:06 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 3824 <at> debbugs.gnu.org:


Received: (at 3824) by debbugs.gnu.org; 5 Jun 2016 02:59:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 04 22:59:19 2016
Received: from localhost ([127.0.0.1]:54822 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1b9OHX-0008Dh-Lz
	for submit <at> debbugs.gnu.org; Sat, 04 Jun 2016 22:59:19 -0400
Received: from mail-it0-f50.google.com ([209.85.214.50]:38790)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <agrambot@HIDDEN>) id 1b9OHW-0008DT-C8
 for 3824 <at> debbugs.gnu.org; Sat, 04 Jun 2016 22:59:18 -0400
Received: by mail-it0-f50.google.com with SMTP id i65so12234205ith.1
 for <3824 <at> debbugs.gnu.org>; Sat, 04 Jun 2016 19:59:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=from:to:cc:subject:user-agent:date:message-id:mime-version;
 bh=rRN+uT9SjQi66mA384qT5xqFqfWr4/dIMU/3bxb8+9M=;
 b=ePXNIcORx81shzp+knez+vLDnqlKjuQgylHBkIbsRy5jp2nGVMBz/xuuJI5nlO4kNw
 uLPYHnFRx/F8sBF4EPStplEu+g3kvcWsH0VxKRpiZOPUApf8bbJbV1AyyjOEvSFTWVqx
 eNnYp4jATHaflWABETVcRqekjeu4hq11SKrFVPD9RzPisObclfmWfkXu7U0sNA2yOHYv
 cGoQYj37K+LnPEHueXZKHlznZew1O6Jhb5hegEVxnvpLc3+cJKB4R0lrc24H+YR6gtNf
 8/bNVk5Sg1GqYSouhNFeN1VAy07aZwW5OTj166Q29AVYfo/EPmXhoNLaZZRD0xrRiYQB
 NCHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20130820;
 h=x-gm-message-state:from:to:cc:subject:user-agent:date:message-id
 :mime-version;
 bh=rRN+uT9SjQi66mA384qT5xqFqfWr4/dIMU/3bxb8+9M=;
 b=W02dm/Ll4FNxe3DA8TIyI41mYpIFvfTc3bicb2eMoN0sTodwzjnLRt9x42oKKO6tYN
 TEHz3cqDjJjxrr2u/d3sxpeJUJo0PYnKF73n1NuIFHHoOvOuOCJrRn/NksyXosavdTb5
 ntDlt5dFaqSSDtvhlXOje6TcwWg9s4AMsaw+WkfX6bCs8Ll9pXsdzL5enyaLoWRVGsUq
 TLuj20d5W3548QPrZIjTOCsU2hPgpVNhpsx30SZGRk2HBCCAIfl7HoIQ/4C3h0xRaffd
 Im551RdHGV0OG6ng5OMEfWzGxzi+BwVo3izICfTWb0GgEVeIKdHvMyYVyppG1lAPE3MX
 F9zQ==
X-Gm-Message-State: ALyK8tIn5l03qyXXZkAuciwEgUMEmlpRgsAjFcDgnYqKFJp4Y3k2r3Isua8JLpBsk95Fcg==
X-Received: by 10.36.79.150 with SMTP id c144mr8731290itb.2.1465095552896;
 Sat, 04 Jun 2016 19:59:12 -0700 (PDT)
Received: from lylat (S01061859339e9903.ss.shawcable.net. [174.2.107.88])
 by smtp.gmail.com with ESMTPSA id 198sm3183528itk.5.2016.06.04.19.59.12
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sat, 04 Jun 2016 19:59:12 -0700 (PDT)
From: Alex <agrambot@HIDDEN>
To: monnier@HIDDEN
Subject: Re: bug#3824: 23.1.50;
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
Date: Sat, 04 Jun 2016 20:59:02 -0600
Message-ID: <87wpm4wdc9.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 3824
Cc: 3824 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.7 (/)


Is there any update on this? I don't mind so much the C-M-f behaviour
(I agree that ideally it should be fixed), but the nested #; not being
font-locked correctly is a bit sad.

I was recently wondering why Emacs Lisp didn't have sexp comments, but I
suppose I now get why that is.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#3824; Package emacs. Full text available.

Message received at 3824 <at> debbugs.gnu.org:


Received: (at 3824) by debbugs.gnu.org; 15 Apr 2014 20:57:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Apr 15 16:57:16 2014
Received: from localhost ([127.0.0.1]:49197 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1WaAPs-0005Kx-41
	for submit <at> debbugs.gnu.org; Tue, 15 Apr 2014 16:57:16 -0400
Received: from mercure.iro.umontreal.ca ([132.204.24.67]:38153)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <monnier@HIDDEN>) id 1WaAPp-0005Kn-FN
 for 3824 <at> debbugs.gnu.org; Tue, 15 Apr 2014 16:57:14 -0400
Received: from hidalgo.iro.umontreal.ca (hidalgo.iro.umontreal.ca
 [132.204.27.50])
 by mercure.iro.umontreal.ca (Postfix) with ESMTP id B6E5B84DB5;
 Tue, 15 Apr 2014 16:57:12 -0400 (EDT)
Received: from lechon.iro.umontreal.ca (lechon.iro.umontreal.ca
 [132.204.27.242])
 by hidalgo.iro.umontreal.ca (Postfix) with ESMTP id 246641A97E4;
 Tue, 15 Apr 2014 16:56:41 -0400 (EDT)
Received: by lechon.iro.umontreal.ca (Postfix, from userid 20848)
 id EFC1DB426B; Tue, 15 Apr 2014 16:56:40 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: "J. Ian Johnson" <ianj@HIDDEN>
Subject: Re: bug#3824: This problem persists
Message-ID: <jwvr44y71nh.fsf-monnier+emacsbugs@HIDDEN>
References: <20090711163545.1FF939828B@HIDDEN>
 <11168152.67691397576392365.JavaMail.root@zimbra>
Date: Tue, 15 Apr 2014 16:56:40 -0400
In-Reply-To: <11168152.67691397576392365.JavaMail.root@zimbra> (J. Ian
 Johnson's message of "Tue, 15 Apr 2014 11:39:52 -0400 (EDT)")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-DIRO-MailScanner-Information: Please contact the ISP for more information
X-DIRO-MailScanner: Found to be clean
X-DIRO-MailScanner-SpamCheck: n'est pas un polluriel,
 SpamAssassin (score=-2.82, requis 5, autolearn=not spam,
 ALL_TRUSTED -2.82, MC_TSTLAST 0.00)
X-DIRO-MailScanner-From: monnier@HIDDEN
X-Spam-Status: No
X-Spam-Score: -3.0 (---)
X-Debbugs-Envelope-To: 3824
Cc: 3824 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.0 (---)

> I use #; comments extensively in my Racket code, and have been bitten
> by emacs's weird handling of it. Taylor pointed me to this bug to
> follow up.

Does the patch below fix it for you?


        Stefan


=== modified file 'lisp/progmodes/scheme.el'
--- lisp/progmodes/scheme.el	2014-03-17 06:22:58 +0000
+++ lisp/progmodes/scheme.el	2014-04-15 20:53:34 +0000
@@ -99,7 +99,7 @@
     (modify-syntax-entry ?\( "()  " st)
     (modify-syntax-entry ?\) ")(  " st)
     ;; It's used for single-line comments as well as for #;(...) sexp-comments.
-    (modify-syntax-entry ?\; "< 2 " st)
+    (modify-syntax-entry ?\; "<"    st)
     (modify-syntax-entry ?\" "\"   " st)
     (modify-syntax-entry ?' "'   " st)
     (modify-syntax-entry ?` "'   " st)
@@ -147,19 +147,15 @@
   (setq-local lisp-indent-function 'scheme-indent-function)
   (setq mode-line-process '("" scheme-mode-line-process))
   (setq-local imenu-case-fold-search t)
-  (setq imenu-generic-expression scheme-imenu-generic-expression)
-  (setq-local imenu-syntax-alist
-	'(("+-*/.<>=?!$%_&~^:" . "w")))
+  (setq-local imenu-generic-expression scheme-imenu-generic-expression)
+  (setq-local imenu-syntax-alist '(("+-*/.<>=?!$%_&~^:" . "w")))
+  (setq-local syntax-propertize-function #'scheme-syntax-propertize)
   (setq font-lock-defaults
 	'((scheme-font-lock-keywords
 	   scheme-font-lock-keywords-1 scheme-font-lock-keywords-2)
 	  nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
 	  beginning-of-defun
-	  (font-lock-mark-block-function . mark-defun)
-	  (font-lock-syntactic-face-function
-	   . scheme-font-lock-syntactic-face-function)
-	  (parse-sexp-lookup-properties . t)
-	  (font-lock-extra-managed-props syntax-table)))
+	  (font-lock-mark-block-function . mark-defun)))
   (setq-local lisp-doc-string-elt-property 'scheme-doc-string-elt))
 
 (defvar scheme-mode-line-process "")
@@ -352,28 +348,28 @@
        (forward-comment (point-max))
        (if (eq (char-after) ?\() 2 0)))
 
-(defun scheme-font-lock-syntactic-face-function (state)
-  (when (and (null (nth 3 state))
-             (eq (char-after (nth 8 state)) ?#)
-             (eq (char-after (1+ (nth 8 state))) ?\;))
+(defun scheme-syntax-propertize (beg end)
+  (goto-char beg)
+  (scheme-syntax-propertize-sexp-comment (point) end)
+  (funcall
+   (syntax-propertize-rules
+    ("\\(#\\);" (1 (prog1 "< cn"
+                     (scheme-syntax-propertize-sexp-comment (point) end)))))
+   (point) end))
+
+(defun scheme-syntax-propertize-sexp-comment (_ end)
+  (let ((state (syntax-ppss)))
+    (when (eq 2 (nth 7 state))
     ;; It's a sexp-comment.  Tell parse-partial-sexp where it ends.
-    (save-excursion
-      (let ((pos (point))
-            (end
-             (condition-case err
-                 (let ((parse-sexp-lookup-properties nil))
+      (condition-case nil
+          (progn
                    (goto-char (+ 2 (nth 8 state)))
                    ;; FIXME: this doesn't handle the case where the sexp
                    ;; itself contains a #; comment.
                    (forward-sexp 1)
-                   (point))
-               (scan-error (nth 2 err)))))
-        (when (< pos (- end 2))
-          (put-text-property pos (- end 2)
-                             'syntax-table scheme-sexp-comment-syntax-table))
-        (put-text-property (- end 1) end 'syntax-table '(12)))))
-  ;; Choose the face to use.
-  (lisp-font-lock-syntactic-face-function state))
+            (put-text-property (1- (point)) (point)
+                               'syntax-table (string-to-syntax "> cn")))
+        (scan-error (goto-char end))))))
 
 ;;;###autoload
 (define-derived-mode dsssl-mode scheme-mode "DSSSL"





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#3824; Package emacs. Full text available.

Message received at 3824 <at> debbugs.gnu.org:


Received: (at 3824) by debbugs.gnu.org; 15 Apr 2014 18:35:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Apr 15 14:35:46 2014
Received: from localhost ([127.0.0.1]:49132 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1Wa8Cv-00089H-6W
	for submit <at> debbugs.gnu.org; Tue, 15 Apr 2014 14:35:46 -0400
Received: from amber.ccs.neu.edu ([129.10.116.51]:55327)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <ianj@HIDDEN>) id 1Wa5Sq-0002hS-IA
 for 3824 <at> debbugs.gnu.org; Tue, 15 Apr 2014 11:40:01 -0400
Received: from zimbra.ccs.neu.edu ([129.10.116.59])
 by amber.ccs.neu.edu with esmtp (Exim 4.69)
 (envelope-from <ianj@HIDDEN>) id 1Wa5Sq-0005Au-0A
 for 3824 <at> debbugs.gnu.org; Tue, 15 Apr 2014 11:40:00 -0400
Received: from localhost (localhost [127.0.0.1])
 by zimbra.ccs.neu.edu (Postfix) with ESMTP id EB77CF5C012
 for <3824 <at> debbugs.gnu.org>; Tue, 15 Apr 2014 11:39:59 -0400 (EDT)
X-Virus-Scanned: amavisd-new at zimbra.ccs.neu.edu
Received: from zimbra.ccs.neu.edu ([127.0.0.1])
 by localhost (zimbra.ccs.neu.edu [127.0.0.1]) (amavisd-new, port 10024)
 with ESMTP id qdRfPu2RnPUw for <3824 <at> debbugs.gnu.org>;
 Tue, 15 Apr 2014 11:39:52 -0400 (EDT)
Received: from zimbra.ccs.neu.edu (zimbra.ccs.neu.edu [129.10.116.59])
 by zimbra.ccs.neu.edu (Postfix) with ESMTP id 752C0F5C010
 for <3824 <at> debbugs.gnu.org>; Tue, 15 Apr 2014 11:39:52 -0400 (EDT)
Date: Tue, 15 Apr 2014 11:39:52 -0400 (EDT)
From: "J. Ian Johnson" <ianj@HIDDEN>
To: 3824 <at> debbugs.gnu.org
Message-ID: <11168152.67691397576392365.JavaMail.root@zimbra>
In-Reply-To: <14931252.67511397575947384.JavaMail.root@zimbra>
Subject: This problem persists
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Originating-IP: [129.10.110.48]
X-Mailer: Zimbra 5.0.22_GA_3210.UBUNTU6 (ZimbraWebClient - FF3.0
 (Linux)/5.0.22_GA_3210.UBUNTU6)
X-Spam-Score: -3.0 (---)
X-Debbugs-Envelope-To: 3824
X-Mailman-Approved-At: Tue, 15 Apr 2014 14:35:40 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.0 (---)

I use #; comments extensively in my Racket code, and have been bitten by em=
acs's weird handling of it. Taylor pointed me to this bug to follow up.
The following is a snippet from one of my projects, with a single #; commen=
t in it.
When you copy and paste it into emacs, it will likely match the parens corr=
ectly. If you save it into a file and reopen it, it will not. If you M-> C-=
M-b, then it will mark them matching.
My delay in reporting this is because the problem with #; really only manif=
ests in large (more than a screen) sexps. Once I navigate it /enough/, then=
 things match and I can keep working. I don't have a good qualification for=
 "enough," i.e., what navigation is necessary for the parens to be marked m=
atching; I only know that this should be seen as incorrect/buggy behavior.
I do hope that this can be fixed for later releases of emacs23/24.
Thanks,
-Ian

(define (a/equal? d=E2=82=80 d=E2=82=81 store-spaces =CE=BC)
  (define/match (egal-equal? a=E2=82=80 a=E2=82=81)
    [((Address-Egal space a) (Address-Egal space a))
     (match (hash-ref =CE=BC a=E2=82=80 '=CF=89)
       [1 #t]
       ['=CF=89 'b.=E2=8A=A4]
       [0 (error 'a/match "Live address with count 0: ~a (Counts ~a) (Store=
 ~a)" a=E2=82=80 =CE=BC store-spaces)])]
    [(_ _) #f])

  (define (ffun-equal? f=E2=82=80 f=E2=82=81)
    (if abs?
        (b=E2=88=A7 (ffun-=E2=8A=91? f=E2=82=80 f=E2=82=81)
            (ffun-=E2=8A=91? f=E2=82=81 f=E2=82=80))
        (concrete-ffun-equal? f=E2=82=80 f=E2=82=81)))

  ;; Slow path: linearly look for a key "equal" to k with "equal" values.
  (define (slow-equal k v f)
    (for/b=E2=88=A8 ([(k=E2=82=81 v=E2=82=81) (in-dict f)])
            (b=E2=88=A7 (a/equal? k k=E2=82=81)
                (a/equal? v v=E2=82=81))))

  (define (ffun-=E2=8A=91? dom f=E2=82=80 f=E2=82=81)
    (for/b=E2=88=A7 ([(k v) (in-dict f=E2=82=80)])
            (match (dict-ref f=E2=82=81 k -unmapped)
              [(=3D=3D -unmapped eq?) (slow-equal k v f=E2=82=81)]
              [v=E2=82=81 ;; fast path: check the structurally equal key
               (b=E2=88=A8 (a/equal? v=E2=82=80 v=E2=82=81)
                   (slow-equal k v f=E2=82=81))])))

  (define (concrete-ffun-equal? m=E2=82=80 m=E2=82=81)
    (and (=3D (dict-count m=E2=82=80) (dict-count m=E2=82=81))
         (for/b=E2=88=A7 ([(k=E2=82=80 v=E2=82=80) (in-dict m=E2=82=80)])
                 (match (dict-ref m=E2=82=81 k=E2=82=80 -unmapped)
                   ;; Concrete domains w/o structural equality are actually=
 abstract.
                   ;; Note this is different from the concrete semantics.
                   [(=3D=3D -unmapped eq?) #f]
                   ;; Note we don't use b=E2=88=A8 with the slow path
                   [v=E2=82=81 (a/equal? v=E2=82=80 v=E2=82=81)]))))

  (define (discrete-ffun-equal? m=E2=82=80 m=E2=82=81)
    (and (=3D (dict-count m=E2=82=80) (dict-count m=E2=82=81))
         (for/b=E2=88=A7 ([(k=E2=82=80 v=E2=82=80) (in-dict m=E2=82=80)])
                 (match (dict-ref m=E2=82=81 k=E2=82=80 -unmapped)
                   [(=3D=3D -unmapped eq?) #f]
                   [v=E2=82=81 (b=E2=88=A7
                        ;; Discrete maps get structural equality on keys, b=
ut can only be=20
                        ;; truly equal if the key has cardinality 1.
                        (if (=E2=88=A3=CE=B3=E2=88=A3>1 k=E2=82=80 =CE=BC) =
'b.=E2=8A=A4 #t)
                        (a/equal? v=E2=82=80 v=E2=82=81))]))))

  (define (equal-step d=E2=82=80 d=E2=82=81)
    (match* (d=E2=82=80 d=E2=82=81)
      [((variant v ds=E2=82=80) (variant v ds=E2=82=81))
       (for/b=E2=88=A7 ([d=E2=82=80 (in-vector ds=E2=82=80)]
                [d=E2=82=81 (in-vector ds=E2=82=81)])
               (a/equal? d=E2=82=80 d=E2=82=81))]

      ;; Addresses are the same if they have cardinality 1. Distinct addres=
ses don't overlap.
      [((? Address-Egal?) (? Address-Egal?))
       (egal-equal? d=E2=82=80 d=E2=82=81)]

      [((? Address-Structural? a=E2=82=80) (? Address-Structural? a=E2=82=
=81))
       (if (eq? (egal-equal? a=E2=82=80 a=E2=82=81) #t)
           #t
           ;; INVARIANT: not possible to be -unmapped since there must be
           ;; at least one value mapped in a store's address.
           (for*/b=CE=B4 ([d=E2=82=80 (in-set (store-ref store-spaces a=E2=
=82=80))]
                     [d=E2=82=81 (in-set (store-ref store-spaces a=E2=82=81=
))])
                    (a/equal? d=E2=82=80 d=E2=82=81)))]

      [((? dict? m=E2=82=80) (? dict? m=E2=82=81)) (concrete-ffun-equal? m=
=E2=82=80 m=E2=82=81)]

      ;; If at least one map has qualification, we can check the other with=
 the expectation of the same.
      ;; We log the incident for future debugging, since it seems like we s=
houldn't get this far.
      [((? dict? m=E2=82=80) (abstract-ffun m=E2=82=81))
       (log-info (format "Qualified/unqualified dictionary equality check ~=
a ~a" d=E2=82=80 d=E2=82=81))
       (ffun-equal? m=E2=82=80 m=E2=82=81)]
      [((abstract-ffun m=E2=82=80) (? dict? m=E2=82=81))
       (log-info (format "Qualified/unqualified dictionary equality check ~=
a ~a" d=E2=82=80 d=E2=82=81))
       (ffun-equal? m=E2=82=80 m=E2=82=81)]
      [((abstract-ffun m=E2=82=80) (abstract-ffun m=E2=82=81)) (ffun-equal?=
 m=E2=82=80 m=E2=82=81)]
      ;; Discrete cases
      [((discrete-ffun m=E2=82=80) (? dict? m=E2=82=81))
       (log-info (format "Qualified/unqualified (discrete) dictionary equal=
ity check ~a ~a" d=E2=82=80 d=E2=82=81))
       (discrete-ffun-equal? m=E2=82=80 m=E2=82=81)]
      [((? dict? m=E2=82=80) (discrete-ffun m=E2=82=81))
       (log-info (format "Qualified/unqualified (discrete) dictionary equal=
ity check ~a ~a" d=E2=82=80 d=E2=82=81))
       (discrete-ffun-equal? m=E2=82=80 m=E2=82=81)]
      [((discrete-ffun m=E2=82=80) (discrete-ffun m=E2=82=81))
       (discrete-ffun-equal? m=E2=82=80 m=E2=82=81)]

      ;; OPT-OP: This has no information on discrete abstractions, thus n=
=C2=B2logn instead of sometimes nlogn
      [((? set? s=E2=82=80) (? set? s=E2=82=81))
       (define (=E2=8A=86? s=E2=82=80 s=E2=82=81)
         (for/b=E2=88=A7 ([v (in-set s=E2=82=80)])
                 (for/b=E2=88=A8 ([v* (in-set s=E2=82=81)])
                         (a/equal? v v*))))
       (b=E2=88=A7 (=E2=8A=86? s=E2=82=80 s=E2=82=81) (=E2=8A=86? s=E2=82=
=81 s=E2=82=80))]

      [(atom atom) #t]

      [((external ex v=E2=82=80) (external ex v=E2=82=81))
       (match-define (External-Space _ card precision special-equality) ex)
       (if special-equality
           (special-equality v=E2=82=80 v=E2=82=81 =CE=BC #;a/equal?)=20
           (match precision
             ['concrete (equal? v=E2=82=80 v=E2=82=81)]
             ['discrete-abstraction (b=E2=88=A7 (equal? v=E2=82=80 v=E2=82=
=81) (implies (eq? (card v=E2=82=80 =CE=BC)) 'b.=E2=8A=A4))]
             ['abstract (error 'a/match "Cannot have non-discrete abstracti=
on of external values without a custom equality relation ~a" d=E2=82=80)]))=
]
      [(_ _) #f]))

  ;; Circular addresses are possible
  ;; OPT-OP?: Racket impl of equal? uses union-find instead of Map[_,Set[_]=
].
  ;;          Is that applicable here?
  (define seen (make-hasheq))
  (define (a/equal? d=E2=82=80 d=E2=82=81)
    (define checked-against (hash-ref! seen d=E2=82=80 mutable-seteq))
    ;; already checked =E2=87=92 assume equal
    ;; XXX: should this be #t or 'b.=E2=8A=A4?
    (or (set-member? checked-against d=E2=82=81)
        (begin (set-add! checked-against d=E2=82=81)
               (equal-step d=E2=82=80 d=E2=82=81))))

  (a/equal? d=E2=82=80 d=E2=82=81))




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#3824; Package emacs. Full text available.

Message received at 3824 <at> debbugs.gnu.org:


Received: (at 3824) by debbugs.gnu.org; 3 May 2011 21:00:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 03 17:00:46 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QHMi6-0005UD-4U
	for submit <at> debbugs.gnu.org; Tue, 03 May 2011 17:00:46 -0400
Received: from fencepost.gnu.org ([140.186.70.10])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <rgm@HIDDEN>) id 1QHMi5-0005U1-2A
	for 3824 <at> debbugs.gnu.org; Tue, 03 May 2011 17:00:45 -0400
Received: from localhost ([127.0.0.1]:34448)
	by fencepost.gnu.org with esmtp (Exim 4.71)
	(envelope-from <rgm@HIDDEN>)
	id 1QHMhy-0001VT-GS; Tue, 03 May 2011 17:00:38 -0400
From: Glenn Morris <rgm@HIDDEN>
To: Taylor R Campbell <campbell@HIDDEN>
Subject: Re: bug#3824: 23.1.50;
	too much effort is put into handling Scheme S-expression comments,
	causing problems
References: <20090711163545.1FF939828B@HIDDEN>
	<20110503172138.03F3C982B7@HIDDEN>
X-Spook: Bosnia Rubin crypto anarchy AIMSX [Hello to all my
X-Ran: +q3~z`2lzl-R<2CCe+nHk}66m>)Qg,DXd$m>TX){qo+?&|yS:3#amA*kVt}T_L(9WI|0Fa
X-Hue: yellow
X-Debbugs-No-Ack: yes
X-Attribution: GM
Date: Tue, 03 May 2011 17:00:37 -0400
Message-ID: <s5sjsvsdmy.fsf@HIDDEN>
User-Agent: Gnus (www.gnus.org), GNU Emacs (www.gnu.org/software/emacs/)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-Spam-Score: -6.3 (------)
X-Debbugs-Envelope-To: 3824
Cc: 3824 <at> debbugs.gnu.org, Stefan Monnier <monnier@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -6.3 (------)


For the record: please send any future correspondence on this to 3824 AT
debbugs.gnu.org. emacsbugs.donarmstrong.com has been retired (although
it still works as an alias, so no need to resend anything). Also, do not
include emacs-pretest-bug and bug-gnu-emacs as well, since all these
addresses are essentially the same, and it means we get 3 copies of
everything.




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#3824; Package emacs. Full text available.

Message received at 3824 <at> debbugs.gnu.org:


Received: (at 3824) by debbugs.gnu.org; 3 May 2011 17:25:10 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue May 03 13:25:10 2011
Received: from localhost ([127.0.0.1] helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1QHJLR-0000c7-QB
	for submit <at> debbugs.gnu.org; Tue, 03 May 2011 13:25:10 -0400
Received: from pluto.mumble.net ([206.123.75.20])
	by debbugs.gnu.org with esmtp (Exim 4.69)
	(envelope-from <campbell@HIDDEN>) id 1QHJHX-0000VO-0B
	for 3824@HIDDEN; Tue, 03 May 2011 13:21:11 -0400
Received: by pluto.mumble.net (Postfix, from userid 1014)
	id 03F3C982B7; Tue,  3 May 2011 17:21:38 +0000 (UTC)
To: Stefan Monnier <monnier@HIDDEN>
In-reply-to: <jwvljmr6q3e.fsf-monnier+emacsbugreports@HIDDEN>
Subject: Re: bug#3824: 23.1.50;
	too much effort is put into handling Scheme S-expression comments,
	causing problems
Date: Tue, 3 May 2011 17:20:52 +0000
From: Taylor R Campbell <campbell@HIDDEN>
User-Agent: IMAIL/1.21; Edwin/3.116; MIT-Scheme/9.1
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20110503172138.03F3C982B7@HIDDEN>
X-Spam-Score: 0.4 (/)
X-Debbugs-Envelope-To: 3824
X-Mailman-Approved-At: Tue, 03 May 2011 13:25:08 -0400
Cc: emacs-pretest-bug@HIDDEN, bug-gnu-emacs@HIDDEN,
	3824 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.11
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/pipermail/debbugs-submit>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
	<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Sender: debbugs-submit-bounces <at> debbugs.gnu.org
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
X-Spam-Score: -0.6 (/)

Gosh, this must have gotten buried in my inbox a long time ago, and I
completely forgot about it...  Sorry about that!

   Date: Tue, 14 Jul 2009 15:52:52 -0400
   From: Stefan Monnier <monnier@HIDDEN>

   > Emacs should treat `#;' as whitespace, nothing more.

   Of course, that's also incorrect with its own set of problems (mostly
   indentation), but admittedly, those problems are easier for the user to
   understand/predict.

What problems?  I think it would be perfectly reasonable for #; not to
affect indentation except inasmuch as it moves code following it on
the same line by two columns just like two spaces would.  For example,
I think the following is reasonable indentation:

(cond ((foo? bar) baz)
  #;
      ((quux? zot) mumble)  #;
      ((grog? swat) zoogle)
      #;((blort? gronk)
         (frobnozzle))
   #; (else (frotz)))

I wouldn't use the second two examples myself, because it takes zero
effort to make an editor treat #; as the start of a line comment
(e.g., Edwin does this) and that works perfectly well as long as you
don't put anything interesting on the line.

   Can you try the patch below?

I tried it.  Seems to work on a couple simple cases I threw at it, but
I haven't tried it in anger yet.




Information forwarded to owner <at> debbugs.gnu.org, bug-gnu-emacs@HIDDEN:
bug#3824; Package emacs. Full text available.
Disconnected #3825 from all other report(s). Request was from Glenn Morris <rgm@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 3824@HIDDEN:


Received: (at 3824) by emacsbugs.donarmstrong.com; 14 Jul 2009 19:53:19 +0000
From monnier@HIDDEN Tue Jul 14 12:53:19 2009
X-Spam-Checker-Version: SpamAssassin 3.2.5-bugs.debian.org_2005_01_02
	(2008-06-10) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available.
	hammytokens:Tokens not available.
X-Spam-Status: No, score=-3.8 required=4.0 tests=AWL,HAS_BUG_NUMBER,
	MURPHY_DRUGS_REL8 autolearn=ham version=3.2.5-bugs.debian.org_2005_01_02
Received: from ironport2-out.teksavvy.com (ironport2-out.teksavvy.com [206.248.154.182])
	by rzlab.ucr.edu (8.14.3/8.14.3/Debian-5) with ESMTP id n6EJrCZj031080
	for <3824@HIDDEN>; Tue, 14 Jul 2009 12:53:13 -0700
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Au4EAHN/XEpMCpbp/2dsb2JhbACBUdBRhAgFhm4
X-IronPort-AV: E=Sophos;i="4.42,399,1243828800"; 
   d="scan'208";a="41588200"
Received: from 76-10-150-233.dsl.teksavvy.com (HELO pastel.home) ([76.10.150.233])
  by ironport2-out.teksavvy.com with ESMTP; 14 Jul 2009 15:52:38 -0400
Received: by pastel.home (Postfix, from userid 20848)
	id 8C2FB7EF7; Tue, 14 Jul 2009 15:52:52 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Taylor R Campbell <campbell@HIDDEN>
Cc: 3824 <at> debbugs.gnu.org, emacs-pretest-bug@HIDDEN,
        bug-gnu-emacs@HIDDEN
Subject: Re: bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems
Message-ID: <jwvljmr6q3e.fsf-monnier+emacsbugreports@HIDDEN>
References: <20090711163545.1FF939828B@HIDDEN>
Date: Tue, 14 Jul 2009 15:52:52 -0400
In-Reply-To: <20090711163545.1FF939828B@HIDDEN> (Taylor
	R. Campbell's message of "Sat, 11 Jul 2009 12:35:45 -0400")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.94 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii

> on imbalanced parentheses.  This is because Emacs goes to excessive
> effort to handle S-expression comments in Scheme Mode, which causes
> more problems than it solves.

Thank you for appreciating my efforts.  The first part of the problem is
known (quoting from the code:)

                   ;; FIXME: this doesn't handle the case where the sexp
                   ;; itself contains a #; comment.

The second part is indeed new: by marking the closing-paren as
a "comment end marker", we prevent the paren from being counted by
forward-sexp, so it can't stop there.

> Emacs should treat `#;' as whitespace, nothing more.

Of course, that's also incorrect with its own set of problems (mostly
indentation), but admittedly, those problems are easier for the user to
understand/predict.

> As far as I can imagine, the only legitimate context for treating `#;'
> as more than whitespace is Font Lock, but since Font Lock works with
> regular expressions, it, too, will do the wrong thing.  (In the above

Font lock can work with arbitrary Elisp code (tho it mostly relies on
regexps).

Can you try the patch below?


        Stefan


=== modified file 'lisp/progmodes/scheme.el'
--- lisp/progmodes/scheme.el	2009-06-06 04:49:20 +0000
+++ lisp/progmodes/scheme.el	2009-07-14 19:50:40 +0000
@@ -99,7 +99,8 @@
     (modify-syntax-entry ?\( "()  " st)
     (modify-syntax-entry ?\) ")(  " st)
     ;; It's used for single-line comments as well as for #;(...) sexp-comments.
-    (modify-syntax-entry ?\; "< 2 " st)
+    ;; It's too difficult to make sexp-comment work right with syntax-tables.
+    (modify-syntax-entry ?\; "<   " st)
     (modify-syntax-entry ?\" "\"   " st)
     (modify-syntax-entry ?' "'   " st)
     (modify-syntax-entry ?` "'   " st)
@@ -170,8 +171,9 @@
          nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
          beginning-of-defun
          (font-lock-mark-block-function . mark-defun)
-         (font-lock-syntactic-face-function
-          . scheme-font-lock-syntactic-face-function)
+         (font-lock-syntactic-keywords . scheme-font-lock-syntactic-keywords)
+         ;; (font-lock-syntactic-face-function
+         ;;  . scheme-font-lock-syntactic-face-function)
          (parse-sexp-lookup-properties . t)
          (font-lock-extra-managed-props syntax-table)))
   (set (make-local-variable 'lisp-doc-string-elt-property)
@@ -290,6 +292,13 @@
      "^(declare\\(-\\sw+\\)+\\>\\s-+\\(\\sw+\\)" 2))
   "Imenu generic expression for DSSSL mode.  See `imenu-generic-expression'.")
 
+(defconst scheme-font-lock-syntactic-keywords
+  ;; Treat sexp-comment markers as "whitespace".
+  '(("#\\(;\\)"
+     (1 (if (nth 8 (save-excursion (syntax-ppss (match-beginning 0))))
+            ;; Check parser state to avoid problem with #|comment1|#;comment2
+            nil '(6))))))
+
 (defconst scheme-font-lock-keywords-1
   (eval-when-compile
     (list
@@ -317,8 +326,21 @@
      ))
   "Subdued expressions to highlight in Scheme modes.")
 
+(defun scheme-font-lock-sexp-comment (limit)
+  (when (search-forward "#;" limit t)
+    (let ((beg (match-beginning 0)))
+      (if (nth 8 (save-excursion (syntax-ppss beg)))
+          ;; Not a sexp-comment: keep looking.
+          (scheme-font-lock-sexp-comment limit)
+        (ignore-errors
+          (forward-sexp 1)
+          (set-match-data (list beg (point)))
+          (point))))))
+
 (defconst scheme-font-lock-keywords-2
-  (append scheme-font-lock-keywords-1
+  (append
+   '((scheme-font-lock-sexp-comment (0 font-lock-comment-face)))
+   scheme-font-lock-keywords-1
    (eval-when-compile
      (list
       ;;
@@ -388,6 +410,10 @@
         (when (< pos (- end 2))
           (put-text-property pos (- end 2)
                              'syntax-table scheme-sexp-comment-syntax-table))
+        ;; FIXME: This marks the closing paren as a comment-close
+        ;; instead, which means that forward-sexp won't stop here.
+        ;; I.e. it is fundamentally flawed: we should instead use a zero-width
+        ;; "comment-close" right after the closing paren.
         (put-text-property (- end 1) end 'syntax-table '(12)))))
   ;; Choose the face to use.
   (lisp-font-lock-syntactic-face-function state))




Acknowledgement sent to Stefan Monnier <monnier@HIDDEN>:
Extra info received and forwarded to list. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Information forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#3824; Package emacs. Full text available.

Message received at submit@HIDDEN:


Received: (at submit) by emacsbugs.donarmstrong.com; 14 Jul 2009 19:53:27 +0000
From monnier@HIDDEN Tue Jul 14 12:53:27 2009
X-Spam-Checker-Version: SpamAssassin 3.2.5-bugs.debian.org_2005_01_02
	(2008-06-10) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available.
	hammytokens:Tokens not available.
X-Spam-Status: No, score=-3.8 required=4.0 tests=AWL,HAS_BUG_NUMBER,
	MURPHY_DRUGS_REL8 autolearn=unavailable
	version=3.2.5-bugs.debian.org_2005_01_02
Received: from lists.gnu.org (lists.gnu.org [199.232.76.165])
	by rzlab.ucr.edu (8.14.3/8.14.3/Debian-5) with ESMTP id n6EJrHQd031096
	for <submit@HIDDEN>; Tue, 14 Jul 2009 12:53:18 -0700
Received: from mailman by lists.gnu.org with tmda-scanned (Exim 4.43)
	id 1MQo3x-0005zU-Ci
	for bug-gnu-emacs@HIDDEN; Tue, 14 Jul 2009 15:53:17 -0400
Received: from exim by lists.gnu.org with spam-scanned (Exim 4.43)
	id 1MQo3r-0005wc-E3
	for bug-gnu-emacs@HIDDEN; Tue, 14 Jul 2009 15:53:16 -0400
Received: from [199.232.76.173] (port=45822 helo=monty-python.gnu.org)
	by lists.gnu.org with esmtp (Exim 4.43)
	id 1MQo3r-0005wU-2J
	for bug-gnu-emacs@HIDDEN; Tue, 14 Jul 2009 15:53:11 -0400
Received: from ironport2-out.pppoe.ca ([206.248.154.182]:28453 helo=ironport2-out.teksavvy.com)
	by monty-python.gnu.org with esmtp (Exim 4.60)
	(envelope-from <monnier@HIDDEN>)
	id 1MQo3n-00083g-OU; Tue, 14 Jul 2009 15:53:08 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Au4EAHN/XEpMCpbp/2dsb2JhbACBUdBRhAgFhm4
X-IronPort-AV: E=Sophos;i="4.42,399,1243828800"; 
   d="scan'208";a="41588200"
Received: from 76-10-150-233.dsl.teksavvy.com (HELO pastel.home) ([76.10.150.233])
  by ironport2-out.teksavvy.com with ESMTP; 14 Jul 2009 15:52:38 -0400
Received: by pastel.home (Postfix, from userid 20848)
	id 8C2FB7EF7; Tue, 14 Jul 2009 15:52:52 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Taylor R Campbell <campbell@HIDDEN>
Cc: 3824 <at> debbugs.gnu.org, emacs-pretest-bug@HIDDEN,
        bug-gnu-emacs@HIDDEN
Subject: Re: bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems
Message-ID: <jwvljmr6q3e.fsf-monnier+emacsbugreports@HIDDEN>
References: <20090711163545.1FF939828B@HIDDEN>
Date: Tue, 14 Jul 2009 15:52:52 -0400
In-Reply-To: <20090711163545.1FF939828B@HIDDEN> (Taylor
	R. Campbell's message of "Sat, 11 Jul 2009 12:35:45 -0400")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.94 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized.

> on imbalanced parentheses.  This is because Emacs goes to excessive
> effort to handle S-expression comments in Scheme Mode, which causes
> more problems than it solves.

Thank you for appreciating my efforts.  The first part of the problem is
known (quoting from the code:)

                   ;; FIXME: this doesn't handle the case where the sexp
                   ;; itself contains a #; comment.

The second part is indeed new: by marking the closing-paren as
a "comment end marker", we prevent the paren from being counted by
forward-sexp, so it can't stop there.

> Emacs should treat `#;' as whitespace, nothing more.

Of course, that's also incorrect with its own set of problems (mostly
indentation), but admittedly, those problems are easier for the user to
understand/predict.

> As far as I can imagine, the only legitimate context for treating `#;'
> as more than whitespace is Font Lock, but since Font Lock works with
> regular expressions, it, too, will do the wrong thing.  (In the above

Font lock can work with arbitrary Elisp code (tho it mostly relies on
regexps).

Can you try the patch below?


        Stefan


=== modified file 'lisp/progmodes/scheme.el'
--- lisp/progmodes/scheme.el	2009-06-06 04:49:20 +0000
+++ lisp/progmodes/scheme.el	2009-07-14 19:50:40 +0000
@@ -99,7 +99,8 @@
     (modify-syntax-entry ?\( "()  " st)
     (modify-syntax-entry ?\) ")(  " st)
     ;; It's used for single-line comments as well as for #;(...) sexp-comments.
-    (modify-syntax-entry ?\; "< 2 " st)
+    ;; It's too difficult to make sexp-comment work right with syntax-tables.
+    (modify-syntax-entry ?\; "<   " st)
     (modify-syntax-entry ?\" "\"   " st)
     (modify-syntax-entry ?' "'   " st)
     (modify-syntax-entry ?` "'   " st)
@@ -170,8 +171,9 @@
          nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
          beginning-of-defun
          (font-lock-mark-block-function . mark-defun)
-         (font-lock-syntactic-face-function
-          . scheme-font-lock-syntactic-face-function)
+         (font-lock-syntactic-keywords . scheme-font-lock-syntactic-keywords)
+         ;; (font-lock-syntactic-face-function
+         ;;  . scheme-font-lock-syntactic-face-function)
          (parse-sexp-lookup-properties . t)
          (font-lock-extra-managed-props syntax-table)))
   (set (make-local-variable 'lisp-doc-string-elt-property)
@@ -290,6 +292,13 @@
      "^(declare\\(-\\sw+\\)+\\>\\s-+\\(\\sw+\\)" 2))
   "Imenu generic expression for DSSSL mode.  See `imenu-generic-expression'.")
 
+(defconst scheme-font-lock-syntactic-keywords
+  ;; Treat sexp-comment markers as "whitespace".
+  '(("#\\(;\\)"
+     (1 (if (nth 8 (save-excursion (syntax-ppss (match-beginning 0))))
+            ;; Check parser state to avoid problem with #|comment1|#;comment2
+            nil '(6))))))
+
 (defconst scheme-font-lock-keywords-1
   (eval-when-compile
     (list
@@ -317,8 +326,21 @@
      ))
   "Subdued expressions to highlight in Scheme modes.")
 
+(defun scheme-font-lock-sexp-comment (limit)
+  (when (search-forward "#;" limit t)
+    (let ((beg (match-beginning 0)))
+      (if (nth 8 (save-excursion (syntax-ppss beg)))
+          ;; Not a sexp-comment: keep looking.
+          (scheme-font-lock-sexp-comment limit)
+        (ignore-errors
+          (forward-sexp 1)
+          (set-match-data (list beg (point)))
+          (point))))))
+
 (defconst scheme-font-lock-keywords-2
-  (append scheme-font-lock-keywords-1
+  (append
+   '((scheme-font-lock-sexp-comment (0 font-lock-comment-face)))
+   scheme-font-lock-keywords-1
    (eval-when-compile
      (list
       ;;
@@ -388,6 +410,10 @@
         (when (< pos (- end 2))
           (put-text-property pos (- end 2)
                              'syntax-table scheme-sexp-comment-syntax-table))
+        ;; FIXME: This marks the closing paren as a comment-close
+        ;; instead, which means that forward-sexp won't stop here.
+        ;; I.e. it is fundamentally flawed: we should instead use a zero-width
+        ;; "comment-close" right after the closing paren.
         (put-text-property (- end 1) end 'syntax-table '(12)))))
   ;; Choose the face to use.
   (lisp-font-lock-syntactic-face-function state))





Acknowledgement sent to Stefan Monnier <monnier@HIDDEN>:
Extra info received and forwarded to list. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Information forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#3824; Package emacs. Full text available.

Message received at submit@HIDDEN:


Received: (at submit) by emacsbugs.donarmstrong.com; 14 Jul 2009 19:53:21 +0000
From monnier@HIDDEN Tue Jul 14 12:53:20 2009
X-Spam-Checker-Version: SpamAssassin 3.2.5-bugs.debian.org_2005_01_02
	(2008-06-10) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available.
	hammytokens:Tokens not available.
X-Spam-Status: No, score=-3.8 required=4.0 tests=AWL,HAS_BUG_NUMBER,
	MURPHY_DRUGS_REL8 autolearn=unavailable
	version=3.2.5-bugs.debian.org_2005_01_02
Received: from fencepost.gnu.org (fencepost.gnu.org [140.186.70.10])
	by rzlab.ucr.edu (8.14.3/8.14.3/Debian-5) with ESMTP id n6EJrBk3031082
	for <submit@HIDDEN>; Tue, 14 Jul 2009 12:53:12 -0700
Received: from mail.gnu.org ([199.232.76.166]:42080 helo=mx10.gnu.org)
	by fencepost.gnu.org with esmtp (Exim 4.67)
	(envelope-from <monnier@HIDDEN>)
	id 1MQo3q-0003cP-Cc
	for emacs-pretest-bug@HIDDEN; Tue, 14 Jul 2009 15:53:10 -0400
Received: from Debian-exim by monty-python.gnu.org with spam-scanned (Exim 4.60)
	(envelope-from <monnier@HIDDEN>)
	id 1MQo3o-00084Q-Fi
	for emacs-pretest-bug@HIDDEN; Tue, 14 Jul 2009 15:53:09 -0400
Received: from ironport2-out.pppoe.ca ([206.248.154.182]:28453 helo=ironport2-out.teksavvy.com)
	by monty-python.gnu.org with esmtp (Exim 4.60)
	(envelope-from <monnier@HIDDEN>)
	id 1MQo3n-00083g-OU; Tue, 14 Jul 2009 15:53:08 -0400
X-IronPort-Anti-Spam-Filtered: true
X-IronPort-Anti-Spam-Result: Au4EAHN/XEpMCpbp/2dsb2JhbACBUdBRhAgFhm4
X-IronPort-AV: E=Sophos;i="4.42,399,1243828800"; 
   d="scan'208";a="41588200"
Received: from 76-10-150-233.dsl.teksavvy.com (HELO pastel.home) ([76.10.150.233])
  by ironport2-out.teksavvy.com with ESMTP; 14 Jul 2009 15:52:38 -0400
Received: by pastel.home (Postfix, from userid 20848)
	id 8C2FB7EF7; Tue, 14 Jul 2009 15:52:52 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Taylor R Campbell <campbell@HIDDEN>
Cc: 3824 <at> debbugs.gnu.org, emacs-pretest-bug@HIDDEN,
        bug-gnu-emacs@HIDDEN
Subject: Re: bug#3824: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems
Message-ID: <jwvljmr6q3e.fsf-monnier+emacsbugreports@HIDDEN>
References: <20090711163545.1FF939828B@HIDDEN>
Date: Tue, 14 Jul 2009 15:52:52 -0400
In-Reply-To: <20090711163545.1FF939828B@HIDDEN> (Taylor
	R. Campbell's message of "Sat, 11 Jul 2009 12:35:45 -0400")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/23.0.94 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
X-detected-operating-system: by monty-python.gnu.org: Genre and OS details not recognized.

> on imbalanced parentheses.  This is because Emacs goes to excessive
> effort to handle S-expression comments in Scheme Mode, which causes
> more problems than it solves.

Thank you for appreciating my efforts.  The first part of the problem is
known (quoting from the code:)

                   ;; FIXME: this doesn't handle the case where the sexp
                   ;; itself contains a #; comment.

The second part is indeed new: by marking the closing-paren as
a "comment end marker", we prevent the paren from being counted by
forward-sexp, so it can't stop there.

> Emacs should treat `#;' as whitespace, nothing more.

Of course, that's also incorrect with its own set of problems (mostly
indentation), but admittedly, those problems are easier for the user to
understand/predict.

> As far as I can imagine, the only legitimate context for treating `#;'
> as more than whitespace is Font Lock, but since Font Lock works with
> regular expressions, it, too, will do the wrong thing.  (In the above

Font lock can work with arbitrary Elisp code (tho it mostly relies on
regexps).

Can you try the patch below?


        Stefan


=== modified file 'lisp/progmodes/scheme.el'
--- lisp/progmodes/scheme.el	2009-06-06 04:49:20 +0000
+++ lisp/progmodes/scheme.el	2009-07-14 19:50:40 +0000
@@ -99,7 +99,8 @@
     (modify-syntax-entry ?\( "()  " st)
     (modify-syntax-entry ?\) ")(  " st)
     ;; It's used for single-line comments as well as for #;(...) sexp-comments.
-    (modify-syntax-entry ?\; "< 2 " st)
+    ;; It's too difficult to make sexp-comment work right with syntax-tables.
+    (modify-syntax-entry ?\; "<   " st)
     (modify-syntax-entry ?\" "\"   " st)
     (modify-syntax-entry ?' "'   " st)
     (modify-syntax-entry ?` "'   " st)
@@ -170,8 +171,9 @@
          nil t (("+-*/.<>=!?$%_&~^:" . "w") (?#. "w 14"))
          beginning-of-defun
          (font-lock-mark-block-function . mark-defun)
-         (font-lock-syntactic-face-function
-          . scheme-font-lock-syntactic-face-function)
+         (font-lock-syntactic-keywords . scheme-font-lock-syntactic-keywords)
+         ;; (font-lock-syntactic-face-function
+         ;;  . scheme-font-lock-syntactic-face-function)
          (parse-sexp-lookup-properties . t)
          (font-lock-extra-managed-props syntax-table)))
   (set (make-local-variable 'lisp-doc-string-elt-property)
@@ -290,6 +292,13 @@
      "^(declare\\(-\\sw+\\)+\\>\\s-+\\(\\sw+\\)" 2))
   "Imenu generic expression for DSSSL mode.  See `imenu-generic-expression'.")
 
+(defconst scheme-font-lock-syntactic-keywords
+  ;; Treat sexp-comment markers as "whitespace".
+  '(("#\\(;\\)"
+     (1 (if (nth 8 (save-excursion (syntax-ppss (match-beginning 0))))
+            ;; Check parser state to avoid problem with #|comment1|#;comment2
+            nil '(6))))))
+
 (defconst scheme-font-lock-keywords-1
   (eval-when-compile
     (list
@@ -317,8 +326,21 @@
      ))
   "Subdued expressions to highlight in Scheme modes.")
 
+(defun scheme-font-lock-sexp-comment (limit)
+  (when (search-forward "#;" limit t)
+    (let ((beg (match-beginning 0)))
+      (if (nth 8 (save-excursion (syntax-ppss beg)))
+          ;; Not a sexp-comment: keep looking.
+          (scheme-font-lock-sexp-comment limit)
+        (ignore-errors
+          (forward-sexp 1)
+          (set-match-data (list beg (point)))
+          (point))))))
+
 (defconst scheme-font-lock-keywords-2
-  (append scheme-font-lock-keywords-1
+  (append
+   '((scheme-font-lock-sexp-comment (0 font-lock-comment-face)))
+   scheme-font-lock-keywords-1
    (eval-when-compile
      (list
       ;;
@@ -388,6 +410,10 @@
         (when (< pos (- end 2))
           (put-text-property pos (- end 2)
                              'syntax-table scheme-sexp-comment-syntax-table))
+        ;; FIXME: This marks the closing paren as a comment-close
+        ;; instead, which means that forward-sexp won't stop here.
+        ;; I.e. it is fundamentally flawed: we should instead use a zero-width
+        ;; "comment-close" right after the closing paren.
         (put-text-property (- end 1) end 'syntax-table '(12)))))
   ;; Choose the face to use.
   (lisp-font-lock-syntactic-face-function state))




Acknowledgement sent to Stefan Monnier <monnier@HIDDEN>:
Extra info received and forwarded to list. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Information forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#3824; Package emacs. Full text available.
Merged 3824 3825. Request was from Glenn Morris <rgm@HIDDEN> to control@HIDDEN. Full text available.

Message received at submit@HIDDEN:


Received: (at submit) by emacsbugs.donarmstrong.com; 11 Jul 2009 16:35:59 +0000
From campbell@HIDDEN Sat Jul 11 09:35:58 2009
X-Spam-Checker-Version: SpamAssassin 3.2.5-bugs.debian.org_2005_01_02
	(2008-06-10) on rzlab.ucr.edu
X-Spam-Level: 
X-Spam-Bayes: score:0.5 Bayes not run. spammytokens:Tokens not available.
	hammytokens:Tokens not available.
X-Spam-Status: No, score=0.6 required=4.0 tests=AWL,FOURLA autolearn=no
	version=3.2.5-bugs.debian.org_2005_01_02
Received: from fencepost.gnu.org (fencepost.gnu.org [140.186.70.10])
	by rzlab.ucr.edu (8.14.3/8.14.3/Debian-5) with ESMTP id n6BGZrJ5023823
	for <submit@HIDDEN>; Sat, 11 Jul 2009 09:35:54 -0700
Received: from mx10.gnu.org ([199.232.76.166]:38628)
	by fencepost.gnu.org with esmtp (Exim 4.67)
	(envelope-from <campbell@HIDDEN>)
	id 1MPfYG-0006sQ-TY
	for emacs-pretest-bug@HIDDEN; Sat, 11 Jul 2009 12:35:53 -0400
Received: from Debian-exim by monty-python.gnu.org with spam-scanned (Exim 4.60)
	(envelope-from <campbell@HIDDEN>)
	id 1MPfYD-0004L9-HA
	for emacs-pretest-bug@HIDDEN; Sat, 11 Jul 2009 12:35:52 -0400
Received: from pluto.mumble.net ([206.123.75.20]:39860)
	by monty-python.gnu.org with esmtp (Exim 4.60)
	(envelope-from <campbell@HIDDEN>)
	id 1MPfYC-0004Kl-Je; Sat, 11 Jul 2009 12:35:48 -0400
Received: from Oberon.local (localhost [127.0.0.1])
	by pluto.mumble.net (Postfix) with ESMTP id 1FF939828B;
	Sat, 11 Jul 2009 16:35:45 +0000 (UTC)
To: emacs-pretest-bug@HIDDEN, bug-gnu-emacs@HIDDEN
Subject: 23.1.50; too much effort is put into handling Scheme S-expression comments, causing problems
Date: Sat, 11 Jul 2009 12:35:45 -0400
From: Taylor R Campbell <campbell@HIDDEN>
User-Agent: IMAIL/1.21; Edwin/3.116; MIT-Scheme/7.7.90.+
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit
Message-Id: <20090711163545.1FF939828B@HIDDEN>
X-detected-operating-system: by monty-python.gnu.org: GNU/Linux 2.6, seldom 2.4 (older, 4)

(Apologies for duplicates: apparently sending mail from this machine
using Emacs is non-trivial.  Grmble.)

Consider the following line of Scheme code:

   (foo bar #;(baz (quux #;() zot) mumble) frotz)

If the point is at the beginning, hitting C-M-f causes Emacs to barf
on imbalanced parentheses.  This is because Emacs goes to excessive
effort to handle S-expression comments in Scheme Mode, which causes
more problems than it solves.

Emacs should treat `#;' as whitespace, nothing more.  The text
following `#;' must be a valid S-expression anyway, so treating it as
if it were a comment, which can contain any unstructured text except
for a comment ender, leads to trouble.  For example, because Emacs's
`parse-partial-sexp' says that the point is in a comment if it is in
an S-expression comment, paredit fails to preserve structure there.
This is an extreme example; Emacs's S-expression motion commands in
general should work inside S-expression comments, but they don't.  For
example, if `|' denotes the point in

   (foo bar #;(baz |(quux #;() zot) mumble) frotz),

then C-M-f should cause the point to turn up at

   (foo bar #;(baz (quux #;() zot)| mumble) frotz),

but instead it barfs on imbalanced parentheses.

As far as I can imagine, the only legitimate context for treating `#;'
as more than whitespace is Font Lock, but since Font Lock works with
regular expressions, it, too, will do the wrong thing.  (In the above
example, Emacs fontifies everything after the initial `#;' as a
comment, including the text `frotz)', which is outside the comment.)

If it is too hard to make Emacs treat `#;' as whitespace and nothing
more, then Emacs shouldn't treat the octothorpe specially, and simply
read the rest of the line as a line comment.  Scheme programmers can
then just break the line after the semicolon.  Emacs's current attempt
to be clever causes trouble even for this simple workaround.  For
example, if `|' denotes the point in

   (foo bar
	#;
       |(baz (quux #;
		   () zot) mumble) frotz),

then C-M-f causes the point to move to the very end of the whole text,
rather than to

   (foo bar
	#;
	(baz (quux #;
		   () zot) mumble)| frotz),

where it should go.

In GNU Emacs 23.1.50.1 (i386-apple-darwin9.7.0, GTK+ Version 2.12.9)
 of 2009-07-11 on Oberon.local
Windowing system distributor `The X.Org Foundation', version 11.0.10402000
Important settings:
  value of $LC_ALL: en_US.UTF-8
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: nil
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default-enable-multibyte-characters: t

Major mode: Scheme

Minor modes in effect:
  show-paren-mode: t
  tooltip-mode: t
  tool-bar-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  global-auto-composition-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
S-SPC f o o <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> # | f o o <backspace> <backspace> 
<backspace> SPC f o o SPC | # C-b C-b C-b C-b M-: M-p 
<return> <help-echo> <help-echo> C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b b <backspace> C-b 
C-b SPC ( f r o b b l e ) S-SPC C-M-u C-M-f C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b 
C-M-f C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b 
[ ] C-a C-M-f C-M-b C-M-f C-M-b C-M-f C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-b C-f SPC 
# ; C-f C-f SPC C-a C-M-f C-M-b C-M-f C-M-f C-M-f C-M-f 
C-M-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f 
C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-f C-d 
C-d ( ) C-a C-M-f C-M-f C-M-f C-M-b C-M-f C-M-b C-M-f 
C-M-b C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b 
C-M-f C-M-b C-M-f C-M-b C-M-f C-M-b <help-echo> C-a 
C-n C-p C-n C-p C-f C-e C-b C-b C-b C-b C-b C-b C-b 
C-b C-b C-b C-b C-b C-k C-n C-a C-p M-f M-f M-f z M-f 
<M-backspace> q u u x M-f <M-backspace> z o t M-f <M-backspace> 
m u m b l e M-f <M-backspace> f r o t z C-n C-x o C-x 
4 0 M-x r e p r o t SPC e m a <backspace> <backspace> 
<backspace> <backspace> <backspace> <backspace> o r 
t SPC e m a c s SPC b u g <help-echo> <help-echo> <help-echo> 
<return>

Recent messages:
(1 1 18 nil nil nil 0 nil nil (1))
(1 1 6 nil nil nil 0 nil nil (1))
(1 1 6 nil t nil 0 nil 10 (1))
(1 1 6 nil nil nil 0 nil nil (1))
let: End of file during parsing
(1 1 6 nil t nil 0 nil 10 (1))
Undo!
(0 nil 1 nil t nil 0 nil 26 nil)
(0 nil 1 nil 1 nil 0 t 26 nil)
forward-sexp: Scan error: "Unbalanced parentheses", 1, 53 [18 times]



Acknowledgement sent to Taylor R Campbell <campbell@HIDDEN>:
New bug report received and forwarded. Copy sent to Emacs Bugs <bug-gnu-emacs@HIDDEN>. Full text available.
Report forwarded to bug-submit-list@HIDDEN, Emacs Bugs <bug-gnu-emacs@HIDDEN>:
bug#3824; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Sun, 5 Jun 2016 03:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.