GNU bug report logs - #12768
24.1; flyspell highlights words which ispell accepts

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 12768 in the body.
You can then email your comments to 12768 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.1; flyspell highlights words which ispell accepts
Date: Tue, 30 Oct 2012 13:23:12 +0000

For example, "executable's" is highlighted by flyspell, but
ispell-buffer ignores it.


In GNU Emacs 24.1.1 (x86_64-pc-linux-gnu, GTK+ Version 2.24.12)
 of 2012-09-22 on batsu, modified by Debian
Windowing system distributor `The X.Org Foundation', version 11.0.11300000
Configured using:
 `configure '--build' 'x86_64-linux-gnu' '--build' 'x86_64-linux-gnu'
 '--prefix=/usr' '--sharedstatedir=/var/lib' '--libexecdir=/usr/lib'
 '--localstatedir=/var/lib' '--infodir=/usr/share/info'
 '--mandir=/usr/share/man' '--with-pop=yes'
 '--enable-locallisppath=/etc/emacs24:/etc/emacs:/usr/local/share/emacs/24.1/site-lisp:/usr/local/share/emacs/site-lisp:/usr/share/emacs/24.1/site-lisp:/usr/share/emacs/site-lisp'
 '--with-crt-dir=/usr/lib/x86_64-linux-gnu' '--with-x=yes'
 '--with-x-toolkit=gtk' '--with-toolkit-scroll-bars'
 'build_alias=x86_64-linux-gnu' 'CFLAGS=-g -O2 -fstack-protector
 --param=ssp-buffer-size=4 -Wformat -Werror=format-security -Wall -O2'
 'CPPFLAGS=-D_FORTIFY_SOURCE=2''

Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: en_GB.UTF-8
  value of $LC_NUMERIC: en_GB.UTF-8
  value of $LC_TIME: en_GB.UTF-8
  value of $LANG: en_GB.UTF-8
  value of $XMODIFIERS: @im=none
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Lua

Minor modes in effect:
  TeX-PDF-mode: t
  flyspell-mode: t
  recentf-mode: t
  show-paren-mode: t
  server-mode: t
  savehist-mode: t
  minibuffer-electric-default-mode: t
  iswitchb-mode: t
  icomplete-mode: t
  global-auto-revert-mode: t
  desktop-save-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
c o u n t <backspace> t e r - i n t u i i t i <backspace> 
<backspace> <backspace> t i v e l y SPC a n d SPC a 
n o <backspace> n o y i n g l y . M-q <return> <return> 
I S-SPC s u m p a <backspace> <backspace> <backspace> 
<backspace> y m p a t h i s e SPC w i t h SPC t h e 
SPC <backspace> <backspace> i s SPC f r o m SPC a SPC 
p r o g r a m m <C-backspace> <C-backspace> t h e SPC 
p o i n t SPC o f SPC v i e w SPC o f SPC t h e SPC 
u n d e r l y i n g SPC c o d e C-q ' s SPC s t r u 
c t u r e , SPC b u t SPC i t ' s <backspace> <backspace> 
C-q ' s SPC o d d SPC SPC <backspace> f r o m SPC a 
SPC u i s <backspace> <backspace> s <backspace> s e 
r - i n t e r a c e SPC <backspace> <backspace> <backspace> 
<backspace> f a c e SPC p o i o n t SPC o f <backspace> 
<backspace> <backspace> <backspace> <backspace> <backspace> 
n t SPC o f SPC v i e w . M-q C-c C-c y e s <return> 
M-x C-g <help-echo> <down-mouse-1> <mouse-1> <double-down-mouse-1> 
<double-mouse-1> <down-mouse-1> <mouse-1> M-x r e p 
[ o r t <backspace> <backspace> <backspace> <backspace> 
r e t - <backspace> <backspace> <backspace> <backspace> 
o r t - b <backspace> e m , a c s <backspace> <backspace> 
<backspace> <backspace> a c s - b u f <return> <backspace> 
g <return>

Recent messages:
Checking 24 files in /usr/share/emacs/24.1/lisp/cedet...
Checking 57 files in /usr/share/emacs/24.1/lisp/calendar...
Checking 87 files in /usr/share/emacs/24.1/lisp/calc...
Checking 65 files in /usr/share/emacs/24.1/lisp/obsolete...
Checking 1 files in /usr/share/emacs/24.1/leim...
Checking for load-path shadows...done
Auto-saving...done
Sending...
Sending via mail...
Sending...done
Quit

Load-path shadows:
/home/rrt/local/share/emacs/site-lisp/browse-kill-ring hides /usr/share/emacs24/site-lisp/emacs-goodies-el/browse-kill-ring
/home/rrt/local/share/emacs/site-lisp/dict hides /usr/share/emacs24/site-lisp/emacs-goodies-el/dict
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/dictionary-init hides /usr/share/emacs24/site-lisp/dictionary-el/dictionary-init
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/dictionary hides /usr/share/emacs24/site-lisp/dictionary-el/dictionary
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/link hides /usr/share/emacs24/site-lisp/dictionary-el/link
/home/rrt/.emacs.d/elpa/dictionary-1.8.7/connection hides /usr/share/emacs24/site-lisp/dictionary-el/connection
/usr/share/emacs/site-lisp/golang-mode/go-mode-load hides /usr/share/emacs/24.1/site-lisp/golang-mode/go-mode-load
/usr/share/emacs/site-lisp/golang-mode/go-mode hides /usr/share/emacs/24.1/site-lisp/golang-mode/go-mode
/usr/share/emacs/24.1/site-lisp/cmake-data/cmake-mode hides /usr/share/emacs/site-lisp/cmake-mode
/usr/share/emacs/24.1/site-lisp/cdargs hides /usr/share/emacs/site-lisp/cdargs
/usr/share/emacs/site-lisp/rst hides /usr/share/emacs/24.1/lisp/textmodes/rst
/usr/share/emacs24/site-lisp/dictionaries-common/ispell hides /usr/share/emacs/24.1/lisp/textmodes/ispell
/usr/share/emacs24/site-lisp/dictionaries-common/flyspell hides /usr/share/emacs/24.1/lisp/textmodes/flyspell

Features:
(newcomment shadow sort gnus-util mail-extr emacsbug message format-spec
rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045 ietf-drums
mm-util mail-prsvr mail-utils find-func help-mode view etags preview
prv-emacs info reporter tex-buf font-latex latex tex-style tex noutline
outline sh-script executable multi-isearch make-mode cperl-mode cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs autoconf autoconf-mode flymake compile vc-git face-remap
flyspell smart-quotes lua-mode comint ansi-color ring
auto-dictionary-autoloads c-eldoc-autoloads dictionary-autoloads
diff-git-autoloads dired-isearch-autoloads full-ack-autoloads
guess-style-autoloads kill-ring-search-autoloads magit-autoloads
mv-shell-autoloads tumble-autoloads http-post-simple-autoloads package
tabulated-list completing-help recentf tree-widget wid-edit uniquify
regexp-opt paren server savehist minibuf-eldef iswitchb icomplete
autorevert desktop cus-start cus-load ropemacs pymacs ispell
user-site-loaddefs advice advice-preload yasnippet help-fns derived
edmacro kmacro easymenu assoc cl muse-autoloads go-mode-load
emacs-goodies-el emacs-goodies-custom emacs-goodies-loaddefs easy-mmode
preview-latex tex-site auto-loads time-date tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset image fringe
lisp-mode register page menu-bar rfn-eshadow timer select scroll-bar
mouse jit-lock font-lock syntax facemenu font-core frame cham georgian
utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao korean
japanese hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help simple abbrev
minibuffer loaddefs button faces cus-face files text-properties overlay
sha1 md5 base64 format env code-pages mule custom widget
hashtable-print-readable backquote make-network-process dbusbind
dynamic-setting system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

-- 
http://rrt.sc3d.org/

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 13:26:03 +0100

On Tue, Oct 30, 2012 at 01:23:12PM +0000, Reuben Thomas wrote:
> For example, "executable's" is highlighted by flyspell, but
> ispell-buffer ignores it.

Cannot reproduce here, both ispell-buffer and flyspell reject the word witn
en_US aspell dict.

Tried with emacs-snapshot and normal Debian emacs24.

-- 
Agustin

Message #11 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: 12768 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 13:31:40 +0100

[Sorry, replied directly to bug-gnu-emacs <at> gnu.org instead of 12768 <at> debbugs.gnu.org]

On Tue, Oct 30, 2012 at 01:23:12PM +0000, Reuben Thomas wrote:
> For example, "executable's" is highlighted by flyspell, but
> ispell-buffer ignores it.

Cannot reproduce here, both ispell-buffer and flyspell reject the word with
en_US aspell dict.

Tried with emacs-snapshot and normal Debian emacs24.

-- 
Agustin

Message #14 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 12:35:20 +0000

[Message part 1 (text/plain, inline)]

In that case, it seems my customizations are relevant:

 '(ispell-local-dictionary "british+accs")
 '(ispell-program-name "/usr/bin/aspell")

and

(setq ispell-local-dictionary-alist
      (append ispell-local-dictionary-alist
              '(("british+accs"         ; British version
                 "[[:alpha:]]"
                 "[^[:alpha:]]"
                 "[-'‘’]"
                 nil
                 ("-B" "-d" "en_GB")
                 nil
                 utf-8))))

Note that the only point of these customizations is to allow the use of
accented letters and curly quotes in English words.


On 31 October 2012 12:31, Agustin Martin <agustin.martin <at> hispalinux.es>wrote:

> [Sorry, replied directly to bug-gnu-emacs <at> gnu.org instead of
> 12768 <at> debbugs.gnu.org]
>
> On Tue, Oct 30, 2012 at 01:23:12PM +0000, Reuben Thomas wrote:
> > For example, "executable's" is highlighted by flyspell, but
> > ispell-buffer ignores it.
>
> Cannot reproduce here, both ispell-buffer and flyspell reject the word with
> en_US aspell dict.
>
> Tried with emacs-snapshot and normal Debian emacs24.
>
> --
> Agustin
>



-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #17 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: 12768 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 15:53:18 +0100

[Message part 1 (text/plain, inline)]

On Wed, Oct 31, 2012 at 12:35:20PM +0000, Reuben Thomas wrote:
> In that case, it seems my customizations are relevant:
> 
>  '(ispell-local-dictionary "british+accs")
>  '(ispell-program-name "/usr/bin/aspell")
> 
> and
> 
> (setq ispell-local-dictionary-alist
>       (append ispell-local-dictionary-alist
>               '(("british+accs"         ; British version
>                  "[[:alpha:]]"
>                  "[^[:alpha:]]"
>                  "[-'??]"
>                  nil
>                  ("-B" "-d" "en_GB")
>                  nil
>                  utf-8))))
> 
> Note that the only point of these customizations is to allow the use of
> accented letters and curly quotes in English words.

You no longer need to make this kind of things to use accented letters for
standard entries under aspell or hunspell, [:alpha:] will be used as
default for them (but not for stuff in `ispell-local-dictionary-alist' where
user choice is honoured).

On the other hand, aspell en.dat only contains ['] as allowed wordchars, 
adding other separators here (in .emacs) may trigger some errors about
different character maps when invoking ispell-word.

aspell seems to internally deal with the explicit acute accent similarly to
', so adding it seems to be OK here.

Apart from that, seems I still get similar results for ispell-buffer and
flyspell with your entry. Please find attached the file I used for tests,

-- 
Agustin

[test-12768.txt (text/plain, attachment)]

Message #20 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 15:20:32 +0000

[Message part 1 (text/plain, inline)]

Thanks very much for the details. I shall investigate; in particular, I
believe that I was told that I didn't need the customization any longer
before and found that to be false, but it's certainly worth another look.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #23 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 12:01:45 -0400

Agustin Martin wrote:

> [Sorry, replied directly to bug-gnu-emacs <at> gnu.org instead of 12768 <at> debbugs.gnu.org]

Although the latter form of address is generally better, the former is
indistinguishable if you have a recognizable "Re: bug#123" in the
subject (which you did).

Message #26 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 20:17:32 +0000

[Message part 1 (text/plain, inline)]

On 31 October 2012 15:20, Reuben Thomas <rrt <at> sc3d.org> wrote:

> Thanks very much for the details. I shall investigate; in particular, I
> believe that I was told that I didn't need the customization any longer
> before and found that to be false, but it's certainly worth another look.
>

I have switched to the built-in "british" setting for
ispell-local-dictionary, and when I now try to ispell-buffer the file for
which I reported the bug I get an alignment error:

ispell-process-line: Ispell misalignment: word `dnl' point 1227; probably
incompatible versions

If I ispell-word the word that before seemed to pass the check, it is
queried as expected.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #29 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: 12768 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 22:31:44 +0100

2012/10/31 Reuben Thomas <rrt <at> sc3d.org>:
> On 31 October 2012 15:20, Reuben Thomas <rrt <at> sc3d.org> wrote:
>>
>> Thanks very much for the details. I shall investigate; in particular, I
>> believe that I was told that I didn't need the customization any longer
>> before and found that to be false, but it's certainly worth another look.
>
>
> I have switched to the built-in "british" setting for
> ispell-local-dictionary, and when I now try to ispell-buffer the file for
> which I reported the bug I get an alignment error:
>
> ispell-process-line: Ispell misalignment: word `dnl' point 1227; probably
> incompatible versions
>
> If I ispell-word the word that before seemed to pass the check, it is
> queried as expected.

ispell-word and ispell-buffer use different methods. For efficiency
reasons ispell-buffer is not a loop of ispell-word calls but uses
ispell-process-line and deals with lines.

For a long time I did not see misalignment errors for Emacs+aspell
(basically after communication was switched to utf-8) and I thought
this was really fixed. They happened when spellchecking utf-8 files
with a dictionary with unibyte encoding and using unibyte
communication, when the file has a char that cannot be represented
in the unibyte encoding.

Can you please provide a minimal example file showing this problem?

Also, please have a look at how the spellchecking process is started
(relevant entry in 'ps -aux' call from a console) in case there is
something strange there.

Thanks for your feedback,

-- 
Agustin

Message #32 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Wed, 31 Oct 2012 21:50:43 +0000

[Message part 1 (text/plain, inline)]

On 31 October 2012 21:31, Agustin Martin <agustin.martin <at> hispalinux.es>wrote:

>
> Can you please provide a minimal example file showing this problem?
>

Attached. I visit the file, switch to autoconf-mode, and run ispell-buffer.
Without switching to autoconf-mode, I don't get the error.


>
> Also, please have a look at how the spellchecking process is started
> (relevant entry in 'ps -aux' call from a console) in case there is
> something strange there.
>

/usr/bin/aspell -a -m -B --encoding=utf-8

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

[foo (application/octet-stream, attachment)]

Message #35 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: 12768 <at> debbugs.gnu.org, Reuben Thomas <rrt <at> sc3d.org>
Cc: agustin.martin <at> upm.es
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Thu, 1 Nov 2012 01:10:11 +0100

2012/10/31 Reuben Thomas <rrt <at> sc3d.org>:
>
> On 31 October 2012 21:31, Agustin Martin <agustin.martin <at> hispalinux.es>
> wrote:
>>
>>
>> Can you please provide a minimal example file showing this problem?
>
>
> Attached. I visit the file, switch to autoconf-mode, and run ispell-buffer.
> Without switching to autoconf-mode, I don't get the error.

Thanks for the info,

Really strange. With your steps I can even reproduce the problem with
something as simple as

dnl -- D

If I add something before dnl  I still get the problem.

sdfdsfsdnl -- D

IIf I either break the double dash (no double dashes at all) or the
dnl or remove the "D" after the dashes problem disappear

dfnl -- D
dnl - - D
dnl --

autotest-mode.el is a small file, but II  do not see any hint about
what is happening here. Seems to do something strange with double
dashes associated with dnl, but I am absolutely clueless, help
welcome.
>>
>> Also, please have a look at how the spellchecking process is started
>> (relevant entry in 'ps -aux' call from a console) in case there is
>> something strange there.
>
> /usr/bin/aspell -a -m -B --encoding=utf-8

While probably unrelated, this should have a "-d en_GB" part. I am
reproducing this problem with other dicts as well as with your
british+accs entry, so this seems not the main problem.

Dirty workaround in the meantime, avoid double dashes and try something like

dnl - - - - Hi, whatever I want to add here

But I really wonder what is happening here.

-- 
Agustin

Message #38 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Reuben Thomas <rrt <at> sc3d.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org, agustin.martin <at> upm.es
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Thu, 1 Nov 2012 00:55:17 +0000

[Message part 1 (text/plain, inline)]

On 1 November 2012 00:10, Agustin Martin <agustin.martin <at> hispalinux.es>wrote:

> >
> > /usr/bin/aspell -a -m -B --encoding=utf-8
>
> While probably unrelated, this should have a "-d en_GB" part.


I was running with emacs -Q, so unless emacs uses my locale to determine
the default dictionary, wouldn't it have omitted that argument?

Thanks also for the workaround.

-- 
http://rrt.sc3d.org

[Message part 2 (text/html, inline)]

Message #41 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Agustin Martin <agustin.martin <at> hispalinux.es>
Cc: 12768 <at> debbugs.gnu.org, agustin.martin <at> upm.es, rrt <at> sc3d.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Thu, 01 Nov 2012 16:44:01 +0200

> Date: Thu, 1 Nov 2012 01:10:11 +0100
> From: Agustin Martin <agustin.martin <at> hispalinux.es>
> Cc: agustin.martin <at> upm.es
> 
> Really strange. With your steps I can even reproduce the problem with
> something as simple as
> 
> dnl -- D
> 
> If I add something before dnl  I still get the problem.
> 
> sdfdsfsdnl -- D
> 
> IIf I either break the double dash (no double dashes at all) or the
> dnl or remove the "D" after the dashes problem disappear
> 
> dfnl -- D
> dnl - - D
> dnl --
> 
> autotest-mode.el is a small file, but II  do not see any hint about
> what is happening here. Seems to do something strange with double
> dashes associated with dnl, but I am absolutely clueless, help
> welcome.

If you step through ispell-region, you will see that it goes bananas
because ispell-skip-region-alist has "--+" as part of it, which
matches the dashes between "dnl" and the text of the comment.  What
ispell-region does is remove the dashes, and then _prepend_ "dnl"
(because it's a comment-start string).  The result is that we send
something like "^dnl FOO" to the speller, and the character positions
get screwed up, because the characters between "dnl" and "FOO" were
removed.

FWIW, I really don't understand why ispell.el removes patterns that
pertain to email messages from buffers that have nothing to do with
mail-mode or its derivatives.

> Dirty workaround in the meantime, avoid double dashes and try something like
> 
> dnl - - - - Hi, whatever I want to add here
> 
> But I really wonder what is happening here.

See above.  A single dash doesn't get removed, so the workaround works
around.

Btw, WIBNI ispell.el had a debug option, which, if set, would collect
in a special buffer everything we send to the speller and its
responses.  I find myself adding '(message "sent %s" strin)'
etc. every time I need to debug the communications between ispell.el
and the speller program.  WDYT?

Message #44 received at 12768 <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: 12768 <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Fri, 2 Nov 2012 02:19:51 +0100

2012/11/1 Eli Zaretskii <eliz <at> gnu.org>:
>> From: Agustin Martin <agustin.martin <at> hispalinux.es>
>>
>> Really strange. With your steps I can even reproduce the problem with
>> something as simple as
>>
>> dnl -- D
>>
>> If I add something before dnl  I still get the problem.
>>
>> sdfdsfsdnl -- D
>>
>> IIf I either break the double dash (no double dashes at all) or the
>> dnl or remove the "D" after the dashes problem disappear
>>
>> dfnl -- D
>> dnl - - D
>> dnl --
>>
>> autotest-mode.el is a small file, but II  do not see any hint about
>> what is happening here. Seems to do something strange with double
>> dashes associated with dnl, but I am absolutely clueless, help
>> welcome.
>
> If you step through ispell-region, you will see that it goes bananas
> because ispell-skip-region-alist has "--+" as part of it, which
> matches the dashes between "dnl" and the text of the comment.  What
> ispell-region does is remove the dashes, and then _prepend_ "dnl"
> (because it's a comment-start string).  The result is that we send
> something like "^dnl FOO" to the speller, and the character positions
> get screwed up, because the characters between "dnl" and "FOO" were
> removed.

Thanks a lot for debugging. I only had time to note that as you point
put more that one dash is removed causing the misalignment. Was
currently looking at the wrong site (autoconf.el).

> FWIW, I really don't understand why ispell.el removes patterns that
> pertain to email messages from buffers that have nothing to do with
> mail-mode or its derivatives.

No idea. I never looked before into ispell-region and friends for
anything than trying to have a fallback behavior on misalignments, and
that was beyond my skills.

>> Dirty workaround in the meantime, avoid double dashes and try something like
>>
>> dnl - - - - Hi, whatever I want to add here
>>
>> But I really wonder what is happening here.
>
> See above.  A single dash doesn't get removed, so the workaround works
> around.
>
> Btw, WIBNI ispell.el had a debug option, which, if set, would collect
> in a special buffer everything we send to the speller and its
> responses.  I find myself adding '(message "sent %s" strin)'
> etc. every time I need to debug the communications between ispell.el
> and the speller program.  WDYT?

I think you are right. I am also an extensive user of the (message
...) debugger and it is really useful, but needing to re-add things
for every new debug is a pity.

flyspell has some of such debug options sending things to a
flyspell-debug buffer, something in the same line would be useful.

Thanks again for the info. Will try to have a look at ispell-region to
try understanding things better

-- 
Agustin

Message #49 received at 12768-done <at> debbugs.gnu.org (full text, mbox):

From: Agustin Martin <agustin.martin <at> hispalinux.es>
To: 12768-done <at> debbugs.gnu.org, rrt <at> sc3d.org
Subject: Re: bug#12768: 24.1; flyspell highlights words which ispell accepts
Date: Tue, 6 Nov 2012 18:48:19 +0100

On Thu, Nov 01, 2012 at 04:44:01PM +0200, Eli Zaretskii wrote:
> > Date: Thu, 1 Nov 2012 01:10:11 +0100
> > From: Agustin Martin <agustin.martin <at> hispalinux.es>
> > Cc: agustin.martin <at> upm.es
> > 
> > Really strange. With your steps I can even reproduce the problem with
> > something as simple as
> > 
> > dnl -- D
> > 
> > If I add something before dnl  I still get the problem.
> > 
> > sdfdsfsdnl -- D
> > 
> > IIf I either break the double dash (no double dashes at all) or the
> > dnl or remove the "D" after the dashes problem disappear
> > 
> > dfnl -- D
> > dnl - - D
> > dnl --
> > 
> > autotest-mode.el is a small file, but II  do not see any hint about
> > what is happening here. Seems to do something strange with double
> > dashes associated with dnl, but I am absolutely clueless, help
> > welcome.
> 
> If you step through ispell-region, you will see that it goes bananas
> because ispell-skip-region-alist has "--+" as part of it, which
> matches the dashes between "dnl" and the text of the comment.  What
> ispell-region does is remove the dashes, and then _prepend_ "dnl"
> (because it's a comment-start string).  The result is that we send
> something like "^dnl FOO" to the speller, and the character positions
> get screwed up, because the characters between "dnl" and "FOO" were
> removed.

I have been dealing with this a bit more and seems I now I understand
things better.

(ispell-region) splits regions to spell in lines, and inside lines in
portions separated by skip-regions, which are sent to ispell pipe. 
When inside a comment, the comment string is added as string prefix
before sending to the ispell pipe  (I guess to make ispell know that
the string is part of a comment).

However ispell understand this prefix in few modes and of course not in
autoconf mode, where `dnl' is spell-checked as any other word (is the only
mode I remember now where comment string is an alphabetic string). For
other modes the comment string is just ignored by ispell for anything than
knowing that string is inside a comment (but properly shifted), and things
work with proper shifts.

However, for autoconf mode `dnl -- D' dnl is checked first and later is
`D' what is sent is `dnl  D' (with a dnl prefix), where `dnl' is checked
again messing up everything because ispell checks dnl where `D' position
is tracked (and expected by ispell.el instead of `dnl').

I have committed a change (r110817) that simply does not prefix the
strings when comment-start when in autoconf-mode (ispell does not support
that prefix anyway).

ispell still does not skip `dnl' as comment header and is shown as a
misspelling, but for everything else seems to work. However, please test
extensively.

> Btw, WIBNI ispell.el had a debug option, which, if set, would collect
> in a special buffer everything we send to the speller and its
> responses.  I find myself adding '(message "sent %s" strin)'
> etc. every time I need to debug the communications between ispell.el
> and the speller program.  WDYT?

I have also added a proof of concept for that (r110816), mostly intended to
be enabled through dedicated wrapper functions that when called create the
debug buffer and enable debugging messages sent through a common function. 
This is still very preliminary, with only `ispell-buffer-with-debug' using
it, but provides some info and as problems get debugged can be extended. 
Suggestions are welcome, even about a complete re-design if really needed.

-- 
Agustin

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.