GNU bug report logs - #13131
24.1; Allow curly quotes to be found by searching for straight quotes?

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 13131 in the body.
You can then email your comments to 13131 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.1; Allow curly quotes to be found by searching for straight quotes?
Date: Mon, 10 Dec 2012 00:18:27 +0000

isearch kindly matches multiple spaces when you insert a space;
similarly, it'd be good if searching for a straight quote also matched
curly quotes (arguably only in modes where that really makes sense, i.e.
text modes).


In GNU Emacs 24.1.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.12)
 of 2012-09-22 on batsu, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11300000
Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--build' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.1/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/x86_64-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars'
 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall -O2'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: en_GB.UTF-8
  value of $LC_NUMERIC: en_GB.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: nil
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: LaTeX/P

Minor modes in effect:
  TeX-PDF-mode: t
  buffer-face-mode: t
  flyspell-mode: t
  smart-quotes-mode: t
  recentf-mode: t
  show-paren-mode: t
  server-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  iswitchb-mode: t
  icomplete-mode: t
  global-auto-revert-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  visual-line-mode: t
  transient-mark-mode: t
  abbrev-mode: t

Recent input:
C-s C-s C-s C-s C-s C-s C-s C-s C-s C-a C-c C-c <return> 
C-s s t i l l SPC o f C-a <down> <down> <down> <down> 
<left> <left> <left> <left> <right> C-s H e r e SPC 
a r e SPC <right> <left> <M-backspace> i s SPC <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <left> <left> <backspace> C-s 
w i C-a C-x C-s C-s w i t h SPC o n e SPC C-s <down> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<left> <left> " o n e SPC c c o <backspace> <backspace> 
<backspace> a c c o r d " M-b M-b M-c <right> <right> 
<right> <right> <right> <right> <right> <right> SPC 
h a s SPC m a n y SPC c o n c o r d s M-d M-d M-d M-d 
C-_ <backspace> <backspace> d s : C-d C-d C-x C-s C-d 
C-x C-s C-a <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <right> <right> <right> <right> <right> <right> 
<right> <tab> C-_ <left> <left> <left> <left> <left> 
<left> <left> <left> <left> <left> <left> <left> <left> 
<right> <right> <M-backspace> <backspace> , SPC a y 
e , SPC o n C-x C-s <down> <down> <down> <down> <down> 
<down> <down> <down> <down> <down> <down> C-s T h e 
r e ' <backspace> ' s <backspace> <backspace> M-x r 
e p o r t - e m c a s - <backspace> <backspace> <backspace> 
<backspace> a c s - b u g <return>

Recent messages:
Saving file /home/rrt/Writing/Poetry/youth/youth.tex...
Wrote /home/rrt/Writing/Poetry/youth/youth.tex
Saving file /home/rrt/Writing/Poetry/youth/youth.tex...
Wrote /home/rrt/Writing/Poetry/youth/youth.tex
Undo!
Auto-saving...done
Auto-saving...done
Saving file /home/rrt/Writing/Poetry/youth/youth.tex...
Wrote /home/rrt/Writing/Poetry/youth/youth.tex
Mark saved where search started

Load-path shadows:
/home/rrt/local/share/emacs/site-lisp/dict hides /usr/share/emacs24/site-lisp/emacs-goodies-el/dict
/home/rrt/local/share/emacs/site-lisp/browse-kill-ring hides /usr/share/emacs24/site-lisp/emacs-goodies-el/browse-kill-ring
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/link hides /usr/share/emacs24/site-lisp/dictionary-el/link
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/dictionary hides /usr/share/emacs24/site-lisp/dictionary-el/dictionary
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/dictionary-init hides /usr/share/emacs24/site-lisp/dictionary-el/dictionary-init
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/connection hides /usr/share/emacs24/site-lisp/dictionary-el/connection
/usr/share/emacs24/site-lisp/auctex/multi-prompt hides /usr/share/emacs/site-lisp/auctex/multi-prompt
/usr/share/emacs24/site-lisp/auctex/latex hides /usr/share/emacs/site-lisp/auctex/latex
/usr/share/emacs24/site-lisp/auctex/texmathp hides /usr/share/emacs/site-lisp/auctex/texmathp
/usr/share/emacs24/site-lisp/auctex/tex-fold hides /usr/share/emacs/site-lisp/auctex/tex-fold
/usr/share/emacs24/site-lisp/auctex/tex-jp hides /usr/share/emacs/site-lisp/auctex/tex-jp
/usr/share/emacs24/site-lisp/auctex/preview hides /usr/share/emacs/site-lisp/auctex/preview
/usr/share/emacs24/site-lisp/auctex/tex-bar hides /usr/share/emacs/site-lisp/auctex/tex-bar
/usr/share/emacs24/site-lisp/auctex/tex-mik hides /usr/share/emacs/site-lisp/auctex/tex-mik
/usr/share/emacs24/site-lisp/auctex/toolbar-x hides /usr/share/emacs/site-lisp/auctex/toolbar-x
/usr/share/emacs24/site-lisp/auctex/tex hides /usr/share/emacs/site-lisp/auctex/tex
/usr/share/emacs24/site-lisp/auctex/prv-emacs hides /usr/share/emacs/site-lisp/auctex/prv-emacs
/usr/share/emacs24/site-lisp/auctex/context hides /usr/share/emacs/site-lisp/auctex/context
/usr/share/emacs24/site-lisp/auctex/tex-info hides /usr/share/emacs/site-lisp/auctex/tex-info
/usr/share/emacs24/site-lisp/auctex/tex-font hides /usr/share/emacs/site-lisp/auctex/tex-font
/usr/share/emacs24/site-lisp/auctex/tex-style hides /usr/share/emacs/site-lisp/auctex/tex-style
/usr/share/emacs24/site-lisp/auctex/bib-cite hides /usr/share/emacs/site-lisp/auctex/bib-cite
/usr/share/emacs24/site-lisp/auctex/font-latex hides /usr/share/emacs/site-lisp/auctex/font-latex
/usr/share/emacs24/site-lisp/auctex/context-nl hides /usr/share/emacs/site-lisp/auctex/context-nl
/usr/share/emacs24/site-lisp/auctex/plain-tex hides /usr/share/emacs/site-lisp/auctex/plain-tex
/usr/share/emacs24/site-lisp/auctex/context-en hides /usr/share/emacs/site-lisp/auctex/context-en
/usr/share/emacs24/site-lisp/auctex/tex-buf hides /usr/share/emacs/site-lisp/auctex/tex-buf
/usr/share/emacs/site-lisp/golang-mode/go-mode-load hides /usr/share/emacs/24.1/site-lisp/golang-mode/go-mode-load
/usr/share/emacs/site-lisp/golang-mode/go-mode hides /usr/share/emacs/24.1/site-lisp/golang-mode/go-mode
/usr/share/emacs/24.1/site-lisp/cmake-data/cmake-mode hides /usr/share/emacs/site-lisp/cmake-mode
/usr/share/emacs24/site-lisp/dictionaries-common/ispell hides /usr/share/emacs/24.1/lisp/textmodes/ispell
/usr/share/emacs24/site-lisp/dictionaries-common/flyspell hides /usr/share/emacs/24.1/lisp/textmodes/flyspell
/usr/share/emacs/site-lisp/rst hides /usr/share/emacs/24.1/lisp/textmodes/rst

Features:
(plain-tex qp newcomment shadow sort gnus-util mail-extr emacsbug
message format-spec rfc822 mml mml-sec mm-decode mm-bodies mm-encode
mail-parse rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047
rfc2045 ietf-drums mm-util mail-prsvr mail-utils jka-compr find-func pp
help-mode view multi-isearch etags texmathp latexenc make-mode autoconf
autoconf-mode vc-git lua-mode cc-mode cc-fonts cc-guess cc-menus cc-cmds
cc-styles cc-align cc-engine cc-vars cc-defs flymake compile comint
ansi-color ring preview prv-emacs warnings tex-buf noutline outline
font-latex byte-opt bytecomp byte-compile cconv macroexp latex tex-style
tex dbus xml face-remap regexp-opt flyspell ispell smart-quotes
auto-dictionary-autoloads c-eldoc-autoloads dictionary-autoloads
diff-git-autoloads dired-isearch-autoloads full-ack-autoloads
guess-style-autoloads kill-ring-search-autoloads magit-autoloads
mv-shell-autoloads tumble-autoloads http-post-simple-autoloads package
tabulated-list completing-help recentf tree-widget wid-edit uniquify
paren server savehist minibuf-eldef iswitchb icomplete autorevert
desktop cus-start cus-load ropemacs pymacs user-site-loaddefs advice
advice-preload yasnippet help-fns derived edmacro kmacro easymenu assoc
cl go-mode-load emacs-goodies-el emacs-goodies-custom
emacs-goodies-loaddefs easy-mmode preview-latex tex-site auto-loads
time-date tooltip ediff-hook vc-hooks lisp-float-type mwheel x-win x-dnd
tool-bar dnd fontset image fringe lisp-mode register page menu-bar
rfn-eshadow timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang vietnamese
tibetan thai tai-viet lao korean japanese hebrew greek romanian slovak
czech european ethiopic indian cyrillic chinese case-table epa-hook
jka-cmpr-hook help simple abbrev minibuffer loaddefs button faces
cus-face files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable backquote
make-network-process dbusbind dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty emacs)

-- 
http://rrt.sc3d.org/

Message #8 received at 13131 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> jurta.org>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13131 <at> debbugs.gnu.org
Subject: Re: bug#13131: 24.1;
	Allow curly quotes to be found by searching for straight quotes?
Date: Mon, 10 Dec 2012 09:59:05 +0200

> isearch kindly matches multiple spaces when you insert a space;
> similarly, it'd be good if searching for a straight quote also matched
> curly quotes (arguably only in modes where that really makes sense, i.e.
> text modes).

I suppose under "straight quote" you mean Unicode char named APOSTROPHE, and
"curly quotes" are LEFT SINGLE QUOTATION MARK and RIGHT SINGLE QUOTATION MARK.

There is a mapping of LEFT DOUBLE QUOTATION MARK and RIGHT DOUBLE QUOTATION MARK
to "``" and "''" in `IT-unicode-translations' in lisp/term/internal.el, but
it's not suitable for this.  I can't find more mappings in the source code,
ucs-normalize.el says that Punctuation is excluded from Normalization:

;; HFS Normalization excludes following area for decomposition.
;;  U+02000 .. U+02FFF :: Punctuation, symbols, dingbats, arrows, etc.

But it seems Normalization can't help anyway because there is no
decomposition information for such quotes in UnicodeData.txt.

Maybe the Unicode standard has a separate document for equivalence mappings
for punctuation.  I can found only the following table.  Is this what is needed?

http://unicode.org/cldr/charts/supplemental/character_fallback_substitutions.html

Message #11 received at 13131 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Juri Linkov <juri <at> jurta.org>
Cc: 13131 <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#13131: 24.1;
	Allow curly quotes to be found by searching for straight quotes?
Date: Mon, 10 Dec 2012 10:28:30 +0200

> From: Juri Linkov <juri <at> jurta.org>
> Date: Mon, 10 Dec 2012 09:59:05 +0200
> Cc: 13131 <at> debbugs.gnu.org
> 
> There is a mapping of LEFT DOUBLE QUOTATION MARK and RIGHT DOUBLE QUOTATION MARK
> to "``" and "''" in `IT-unicode-translations' in lisp/term/internal.el

Irrelevant to the issue at hand.  That mapping (and internal.el in
general) is used by the MS-DOS build of Emacs to allow a legible
display of characters that the DOS codepage doesn't support.

> I can't find more mappings in the source code, ucs-normalize.el says
> that Punctuation is excluded from Normalization:
> 
> ;; HFS Normalization excludes following area for decomposition.
> ;;  U+02000 .. U+02FFF :: Punctuation, symbols, dingbats, arrows, etc.
> 
> But it seems Normalization can't help anyway because there is no
> decomposition information for such quotes in UnicodeData.txt.

This is not a normalization issue.  This is about an Emacs specific
feature, and should be treated as such.

We should begin by discussing whether we really want this feature, by
default or otherwise.  One of the aspects we need to consider, IMO, is
what do spell-checkers do with these characters: if they don't treat
them as word-constituents, that would be one argument against the
change; if they do, it's an argument for the change.

> Maybe the Unicode standard has a separate document for equivalence mappings
> for punctuation.  I can found only the following table.  Is this what is needed?
> 
> http://unicode.org/cldr/charts/supplemental/character_fallback_substitutions.html

This seems to be relevant to display, not to the issue at hand.  If
anything, it should tell us to treat ' as we treat ’, not the other
way around.

Message #14 received at 13131 <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> jurta.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 13131 <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#13131: 24.1;
	Allow curly quotes to be found by searching for straight quotes?
Date: Mon, 10 Dec 2012 10:49:57 +0200

>> http://unicode.org/cldr/charts/supplemental/character_fallback_substitutions.html
>
> This seems to be relevant to display, not to the issue at hand.  If
> anything, it should tell us to treat ' as we treat ’, not the other
> way around.

Most of the equivalent sequences in character_fallback_substitutions.html
are NFC and NFKC that means normalization using the decomposition property
whose equivalence we already argeed to implement in the search.

A few of the equivalent sequences are marked "Explicit" and
all of them are defined in a separate very small file

http://unicode.org/repos/cldr/trunk/common/supplemental/characters.xml

whose mappings we could also add to our equivalence tables explicitly.

Message #17 received at 13131 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: Juri Linkov <juri <at> jurta.org>, 13131 <at> debbugs.gnu.org
Subject: Re: bug#13131: 24.1; Allow curly quotes to be found by searching for
	straight quotes?
Date: Mon, 10 Dec 2012 11:20:32 +0000

[Message part 1 (text/plain, inline)]

On 10 December 2012 08:28, Eli Zaretskii <eliz <at> gnu.org> wrote:

>
> We should begin by discussing whether we really want this feature, by
> default or otherwise.  One of the aspects we need to consider, IMO, is
> what do spell-checkers do with these characters: if they don't treat
> them as word-constituents, that would be one argument against the
> change; if they do, it's an argument for the change.
>

At least aspell now seems to accept curly quotes by default in some locales.

However,  I think the rationale for the feature is independent of
spell-checkers. Emacs has had support for displaying curly quotes for some
time. There are various ways to enter them conveniently (the Emacs Wiki
lists smart-quotes.el, which I use, and typopunct.el. The sole author of
smart-quotes.el would be quite happy to assign copyright to GNU and have it
added to Emacs; I suggested he contact the maintainers about this in August
2011. (Incidentally, I'd be happy to facilitate the process if Emacs
maintainers were interested.)

Since curly quotes make perfect sense as part of words, in exactly the same
way as straight quotes, it seems reasonable to treat them the same for
movement, even if spelling support is not yet perfect; indeed, any
additional incentive to spelling program authors to improve Unicode
punctuation support is welcome.

For writing text, the introduction of Unicode has been a boon, and it would
be great if Emacs's "semantic" support would catch up with its excellent
display support. This is one aspect of that.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #22 received at 13131-done <at> debbugs.gnu.org (full text, mbox):

From: Juri Linkov <juri <at> linkov.net>
To: Reuben Thomas <rrt <at> sc3d.org>
Cc: 13131-done <at> debbugs.gnu.org
Subject: Re: bug#13131: 24.1; Allow curly quotes to be found by searching
 for straight quotes?
Date: Mon, 29 Jul 2019 01:58:52 +0300

> isearch kindly matches multiple spaces when you insert a space;
> similarly, it'd be good if searching for a straight quote also matched
> curly quotes (arguably only in modes where that really makes sense, i.e.
> text modes).

This is implemented now by char-fold feature.

Message #25 received at 13131-done <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Juri Linkov <juri <at> linkov.net>
Cc: 13131-done <at> debbugs.gnu.org
Subject: Re: bug#13131: 24.1; Allow curly quotes to be found by searching for
 straight quotes?
Date: Mon, 29 Jul 2019 11:02:07 +0100

[Message part 1 (text/plain, inline)]

On Sun, 28 Jul 2019 at 23:59, Juri Linkov <juri <at> linkov.net> wrote:

> > isearch kindly matches multiple spaces when you insert a space;
> > similarly, it'd be good if searching for a straight quote also matched
> > curly quotes (arguably only in modes where that really makes sense, i.e.
> > text modes).
>
> This is implemented now by char-fold feature.
>

Thanks, that's a great improvement!

-- 
https://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.