GNU bug report logs - #18380
24.3.93; sh-script multiline quoted subshell wreaks havoc with indentation

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Severity: minor; Reported by: Carlos Pita <carlosjosepita@HIDDEN>; dated Mon, 1 Sep 2014 14:31:01 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 18380 <at> debbugs.gnu.org:


Received: (at 18380) by debbugs.gnu.org; 3 Sep 2014 00:39:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Sep 02 20:39:42 2014
Received: from localhost ([127.0.0.1]:57239 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1XOybt-0007ZO-FC
	for submit <at> debbugs.gnu.org; Tue, 02 Sep 2014 20:39:41 -0400
Received: from chene.dit.umontreal.ca ([132.204.246.20]:48897)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <monnier@HIDDEN>) id 1XOybp-0007ZD-UT
 for 18380 <at> debbugs.gnu.org; Tue, 02 Sep 2014 20:39:38 -0400
Received: from pastel.home (lechon.iro.umontreal.ca [132.204.27.242])
 by chene.dit.umontreal.ca (8.14.1/8.14.1) with ESMTP id s830daPg023221;
 Tue, 2 Sep 2014 20:39:36 -0400
Received: by pastel.home (Postfix, from userid 20848)
 id 33DFA6308B; Tue,  2 Sep 2014 20:39:36 -0400 (EDT)
From: Stefan Monnier <monnier@HIDDEN>
To: Carlos Pita <carlosjosepita@HIDDEN>
Subject: Re: bug#18380: 24.3.93;
 sh-script multiline quoted subshell wreaks havoc with indentation
Message-ID: <jwv4mwp3626.fsf-monnier+emacsbugs@HIDDEN>
References: <877g1n1kjf.fsf@HIDDEN>
Date: Tue, 02 Sep 2014 20:39:36 -0400
In-Reply-To: <877g1n1kjf.fsf@HIDDEN> (Carlos Pita's message of "Mon, 01 Sep
 2014 11:29:24 -0300")
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.4.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-NAI-Spam-Flag: NO
X-NAI-Spam-Threshold: 5
X-NAI-Spam-Score: 0
X-NAI-Spam-Rules: 1 Rules triggered
	RV5052=0
X-NAI-Spam-Version: 2.3.0.9378 : core <5052> : inlines <1229> : streams
 <1279964> : uri <1809672>
X-Spam-Score: -3.0 (---)
X-Debbugs-Envelope-To: 18380
Cc: occitan@HIDDEN, 18380 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.0 (---)

> This will see the { as quoted:
> "$(echo  "{")"
>
> while this will not:
>
> "$(echo
>    "{")"

Actually, it depends: sometimes it will sometimes it won't :-(

> after this line indentation is broken as Emacs thinks there is an
> unclosed open brace.
>
> The problem with multiline quoted subshells is more general, as
> remarked by this comment in sh-script.el:
>
>   ;; FIXME: This can (and often does) match multiple lines, yet it makes no
>   ;; effort to handle multiline cases correctly, so it ends up being
>   ;; rather flaky.

Indeed, the problem you show is the direct result of the problem alluded
to in the above comment.  I installed the patch below which seems to help.


        Stefan


=== modified file 'lisp/progmodes/sh-script.el'
--- lisp/progmodes/sh-script.el	2014-07-21 01:41:59 +0000
+++ lisp/progmodes/sh-script.el	2014-09-03 00:37:50 +0000
@@ -1051,13 +1051,11 @@
   "Search for a subshell embedded in a string.
 Find all the unescaped \" characters within said subshell, remembering that
 subshells can nest."
-  ;; FIXME: This can (and often does) match multiple lines, yet it makes no
-  ;; effort to handle multiline cases correctly, so it ends up being
-  ;; rather flaky.
   (when (eq ?\" (nth 3 (syntax-ppss))) ; Check we matched an opening quote.
     ;; bingo we have a $( or a ` inside a ""
     (let (;; `state' can be: double-quote, backquote, code.
           (state (if (eq (char-before) ?`) 'backquote 'code))
+          (startpos (point))
           ;; Stacked states in the context.
           (states '(double-quote)))
       (while (and state (progn (skip-chars-forward "^'\\\\\"`$()" limit)
@@ -1088,7 +1086,12 @@
                  (`double-quote nil)
                  (_ (setq state (pop states)))))
           (_ (error "Internal error in sh-font-lock-quoted-subshell")))
-        (forward-char 1)))))
+        (forward-char 1))
+      (when (< startpos (line-beginning-position))
+        (put-text-property startpos (point) 'syntax-multiline t)
+        (add-hook 'syntax-propertize-extend-region-functions
+                  'syntax-propertize-multiline nil t))
+      )))
 
 
 (defun sh-is-quoted-p (pos)





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#18380; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 1 Sep 2014 14:30:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Sep 01 10:30:08 2014
Received: from localhost ([127.0.0.1]:55787 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.80)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1XOScN-000209-CO
	for submit <at> debbugs.gnu.org; Mon, 01 Sep 2014 10:30:08 -0400
Received: from eggs.gnu.org ([208.118.235.92]:46110)
 by debbugs.gnu.org with esmtp (Exim 4.80)
 (envelope-from <carlosjosepita@HIDDEN>) id 1XOScG-0001z9-9o
 for submit <at> debbugs.gnu.org; Mon, 01 Sep 2014 10:30:01 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <carlosjosepita@HIDDEN>) id 1XOSc3-0002BC-Uy
 for submit <at> debbugs.gnu.org; Mon, 01 Sep 2014 10:29:50 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,FREEMAIL_FROM,
 T_DKIM_INVALID autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:38938)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <carlosjosepita@HIDDEN>) id 1XOSc3-0002B5-Rl
 for submit <at> debbugs.gnu.org; Mon, 01 Sep 2014 10:29:43 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:47800)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <carlosjosepita@HIDDEN>) id 1XOSbw-0005nc-PX
 for bug-gnu-emacs@HIDDEN; Mon, 01 Sep 2014 10:29:43 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <carlosjosepita@HIDDEN>) id 1XOSbp-00027Q-LP
 for bug-gnu-emacs@HIDDEN; Mon, 01 Sep 2014 10:29:36 -0400
Received: from mail-vc0-x22f.google.com ([2607:f8b0:400c:c03::22f]:64857)
 by eggs.gnu.org with esmtp (Exim 4.71)
 (envelope-from <carlosjosepita@HIDDEN>) id 1XOSbp-00027D-7D
 for bug-gnu-emacs@HIDDEN; Mon, 01 Sep 2014 10:29:29 -0400
Received: by mail-vc0-f175.google.com with SMTP id lf12so5484096vcb.20
 for <bug-gnu-emacs@HIDDEN>; Mon, 01 Sep 2014 07:29:28 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113;
 h=from:to:cc:subject:date:message-id:mime-version:content-type;
 bh=QdO402W3xuWEh4c3YPTsBNG7LpB78GvSCOR8gQUPU4E=;
 b=mlridCnxrgHsJm8/lO4lCWbIeD7tgYFDhCYLp+RCJzoZOQG/L5XjRiOANC/HGwrYdB
 imp7W4zvtD6dO+z4j2hmK+vF/mhH084YB2/We0evktIsuz2cETbT+UmMwIyQ9sPc/xDr
 gCYmB9tfOtPt3kwJfLMFfMp9p8DyLzvdEFCB3rF2jfwBnHzNnfWHXHCF14n4IwMd5bWC
 dGkp8qk+pXkT4f8EDmdmK3M7T6qhBPUOUbN56ycTFrHw4K68NW8g61qeZNMPpd0FoZrl
 jr8WlF2ZfoSPsT2AVBoRAeMLQxo8J7/vMBcEFEbEmZEpNMO/9p0kkNnlb+j+c0hoa9sO
 D6rQ==
X-Received: by 10.52.52.136 with SMTP id t8mr21731731vdo.21.1409581768613;
 Mon, 01 Sep 2014 07:29:28 -0700 (PDT)
Received: from carlos ([181.228.217.164])
 by mx.google.com with ESMTPSA id jm8sm2349078vdb.27.2014.09.01.07.29.26
 for <multiple recipients>
 (version=TLSv1.2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Mon, 01 Sep 2014 07:29:27 -0700 (PDT)
From: Carlos Pita <carlosjosepita@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 24.3.93;
 sh-script multiline quoted subshell wreaks havoc with indentation
Date: Mon, 01 Sep 2014 11:29:24 -0300
Message-ID: <877g1n1kjf.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-detected-operating-system: by eggs.gnu.org: Error: Malformed IPv6 address
 (bad octet value).
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -4.0 (----)
X-Debbugs-Envelope-To: submit
Cc: occitan@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.15
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <http://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <http://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <http://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -4.0 (----)

This will see the { as quoted:

"$(echo  "{")"

while this will not:

"$(echo
   "{")"

after this line indentation is broken as emacs thinks there is an
unclosed open brace.

The problem with multiline quoted subshells is more general, as
remarked by this comment in sh-script.el:

  ;; FIXME: This can (and often does) match multiple lines, yet it makes no
  ;; effort to handle multiline cases correctly, so it ends up being
  ;; rather flaky.

Regards
--
Carlos

GNU Emacs 24.3.93.1 (x86_64-unknown-linux-gnu, GTK+ Version 3.12.2)
 of 2014-08-27 on carlos
Windowing system distributor `The X.Org Foundation', version 11.0.11600000
Configured using:
 `configure --prefix=/usr --sysconfdir=/etc --libexecdir=/usr/lib
 --localstatedir=/var --with-x-toolkit=gtk3 --with-xft
 'CFLAGS=-march=x86-64 -mtune=generic -O2 -pipe -fstack-protector-strong
 --param=ssp-buffer-size=4' CPPFLAGS=-D_FORTIFY_SOURCE=2
 LDFLAGS=-Wl,-O1,--sort-common,--as-needed,-z,relro'

Important settings:
  value of $LC_COLLATE: en_US.UTF-8
  value of $LC_CTYPE: en_US.UTF-8
  value of $LC_MESSAGES: en_US.UTF-8
  value of $LC_MONETARY: en_US.UTF-8
  value of $LC_NUMERIC: en_US.UTF-8
  value of $LC_TIME: en_US.UTF-8
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Shell-script

Minor modes in effect:
  shell-dirtrack-mode: t
  sh-electric-here-document-mode: t
  show-paren-mode: t
  eldoc-mode: t
  yas-global-mode: t
  yas-minor-mode: t
  ido-ubiquitous-mode: t
  global-auto-complete-mode: t
  auto-complete-mode: t
  ido-everywhere: t
  winner-mode: t
  tooltip-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
<down> <down> <down> <down> <down> C-w <up> <up> <up> 
<up> <up> SPC C-x C-s <backspace> <down> C-e <backspace> 
<backspace> " <left> d s a d s a C-x C-s <down> <up> 
<up> <left> SPC <down> C-x C-s <up> <backspace> SPC 
C-x C-s <backspace> <down> <right> <right> <right> 
<right> <right> <right> C-x C-s <right> { <up> <up> 
C-e C-d C-d C-d d <backspace> C-d C-d C-d <down> <down> 
<down> <down> <down> } <backspace> <up> <up> <up> <up> 
<up> C-e <C-left> <left> <left> <backspace> <return> 
<return> <tab> SPC SPC SPC SPC <up> <up> <left> C-k 
e c h o <return> <return> SPC SPC <backspace> <backspace> 
<backspace> <backspace> <return> SPC SPC SPC " { " 
) " <return> <up> <up> C-a C-SPC <down> <down> M-w 
<up> <up> <up> <up> <return> C-y <up> <up> C-e C-d 
C-d <down> <down> <backspace> <down> <down> <down> 
C-k C-k C-k C-k C-k C-k C-k C-k C-k C-k x C-s <backspace> 
<backspace> <backspace> <backspace> <up> <up> <down> 
<down> <backspace> <backspace> <return> <return> C-x 
C-s <backspace> <backspace> <backspace> <backspace> 
<backspace> <backspace> <return> C-x C-s <backspace> 
<backspace> <backspace> <backspace> <backspace> } <backspace> 
<up> <up> <up> <up> <return> } <backspace> <backspace> 
<down> <down> <down> x C-s <backspace> <backspace> 
<up> <up> <down> <down> <down> <up> <backspace> } <backspace> 
C-x C-s <up> <up> <return> <backspace> C-x C-s <down> 
<down> <down> } <backspace> <backspace> <backspace> 
<backspace> " <return> <backspace> <backspace> <backspace> 
<backspace> <backspace> <backspace> <return> C-a C-k 
<return> <backspace> C-a C-k <tab> <tab> <tab> <tab> 
<tab> s a } C-a C-k s a d s a d s a <tab> C-a C-k <up> 
<up> <up> <up> <up> SPC <down> <up> <backspace> C-SPC 
<tab> C-SPC <down> <down> <down> <down> M-q <down> 
C-/ <up> <up> <up> <up> C-SPC <down> <down> <down> 
<down> M-w <down> <down> M-x r e p o r t - e m <tab> 
<return>

Recent messages:
Saving file /tmp/test.sh...
Wrote /tmp/test.sh
Saving file /tmp/test.sh...
Wrote /tmp/test.sh
Saving file /tmp/test.sh...
Wrote /tmp/test.sh
Auto-saving...done
Mark activated [2 times]
Undo!
Mark activated

Load-path shadows:
~/.emacs.d/lisp/rmail hides /usr/share/emacs/24.3.93/lisp/mail/rmail

Features:
(eieio-opt speedbar sb-image ezimage dframe shadow emacsbug xterm
descr-text tramp-cache tramp tramp-compat tramp-loaddefs trampver shell
view woman man misearch multi-isearch cus-edit cus-start cus-load
sh-script smie executable gnus-draft mailalias smtpmail sendmail oauth2
warnings json plstore nnir url-http url-gw url-auth url-queue face-remap
org-colview diary-lib diary-loaddefs cal-iso org-agenda bookmark
thingatpt tabify image-file org-capture pp rect vc-git org-element
org-rmail org-mhe org-irc org-info org-gnus org-docview doc-view
jka-compr image-mode dired org-bibtex bibtex org-bbdb org-w3m mule-util
shr-color color shr browse-url gnus-dup sort smiley gnus-cite mm-archive
mail-extr gnus-async gnus-bcklg qp gnus-ml disp-table gnus-topic nndraft
nnmh nnfolder utf-7 epa-file epa derived epg gnutls network-stream
starttls nnimap parse-time tls utf7 netrc gnus-agent gnus-srvr
gnus-score score-mode nnvirtual gnus-msg gnus-art mm-uu mml2015
epg-config mm-view mml-smime smime dig nntp gnus-cache gnus-sum nnoo
gnus-group gnus-undo nnmail mail-source google-contacts-message
google-contacts xml url-cache url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf mailcap url-util url-parse
auth-source eieio eieio-core password-cache url-vars google-oauth
gnus-start gnus-spec gnus-int gnus-range message idna rfc822 mml mml-sec
mm-decode mm-bodies mm-encode mail-parse rfc2231 rfc2047 rfc2045
ietf-drums mailabbrev gmm-utils mailheader gnus-win gnus gnus-ems
nnheader gnus-util mail-utils mm-util mail-prsvr wid-edit server flymake
compile paredit paren eldoc cl-macs ob-python ob-R org byte-opt bytecomp
byte-compile cconv org-macro org-footnote org-pcomplete pcomplete
org-list org-faces org-entities time-date noutline outline easy-mmode
org-version ob-emacs-lisp ob ob-tangle org-src ob-ref ob-lob ob-table
ob-keys ob-exp ob-comint comint ansi-color ob-core ob-eval org-compat
org-macs org-loaddefs format-spec find-func cal-menu calendar
cal-loaddefs yasnippet help-mode ido-ubiquitous cl gv advice help-fns
auto-complete-config auto-complete popup imenu-anywhere imenu ido
windmove winner ring edmacro kmacro cl-loaddefs cl-lib info easymenu
package wombat-theme tooltip electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image regexp-opt
fringe tabulated-list newcomment lisp-mode prog-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer nadvice
loaddefs button faces cus-face macroexp files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
gfilenotify dynamic-setting system-font-setting font-render-setting
move-toolbar gtk x-toolkit x multi-tty emacs)

Memory information:
((conses 16 685711 94948)
 (symbols 48 57707 0)
 (miscs 40 4316 4531)
 (strings 32 245451 4719)
 (string-bytes 1 4862500)
 (vectors 16 51937)
 (vector-slots 8 1688568 35010)
 (floats 8 1010 1797)
 (intervals 56 37504 525)
 (buffers 960 93)
 (heap 1024 68950 5163))




Acknowledgement sent to Carlos Pita <carlosjosepita@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#18380; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Fri, 31 Oct 2014 17:00:04 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.