GNU bug report logs - #21825
2015-10-04; utf8 and latin1 coding problem in GNU emacs, Xemacs is fine

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 21825 in the body.
You can then email your comments to 21825 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Uwe Brauer <oub <at> mat.ucm.es>
To: bug-auctex <at> gnu.org
Subject: 2015-10-04;
 utf8 and latin1 coding problem in GNU emacs, Xemacs is fine
Date: Wed, 04 Nov 2015 15:17:58 +0000

[Message part 1 (text/plain, inline)]

Hello

The following problem only occurs in GNU emacs 24.5 or 25.0.50.

I have two files:

The first is saved in latin-1 but its header state
\usepackage[utf8]{inputenc}

And the other is the other way around
saved in UTF8 header is 

\usepackage[latin1]{inputenc}


Both are displayed correctly in Xemacs and but not in GNU emacs.

Regards

Uwe Brauer 


Emacs  : GNU Emacs 25.0.50.1 (i686-pc-linux-gnu, GTK+ Version 2.20.1)
 of 2015-09-28
Package: 2015-10-04

current state:
==============
(setq
 AUCTeX-date "2015-10-04"
 window-system 'x
 LaTeX-version "2e"
 TeX-style-path '("/home/oub/ALLES/var/auctex"
		  "/home/oub/emacs/site-lisp/packages/auctex-git/auctex/style"
		  "/home/oub/.emacs.d/auctex/auto" "/home/oub/tex/sty/auto" "auto" "style")
 TeX-auto-save t
 TeX-parse-self t
 TeX-master t
 TeX-command-list '(("TeX" "%(PDF)%(tex) %(file-line-error) %(extraopts) %`%S%(PDFout)%(mode)%' %t"
		     TeX-run-TeX nil (plain-tex-mode ams-tex-mode texinfo-mode) :help "Run plain TeX")
		    ("LaTeX" "%`%l%(mode)%' %t" TeX-run-TeX nil (latex-mode doctex-mode) :help
		     "Run LaTeX")
		    ("Makeinfo" "makeinfo %(extraopts) %t" TeX-run-compile nil (texinfo-mode) :help
		     "Run Makeinfo with Info output")
		    ("Makeinfo HTML" "makeinfo %(extraopts) --html %t" TeX-run-compile nil
		     (texinfo-mode) :help "Run Makeinfo with HTML output")
		    ("AmSTeX" "amstex %(PDFout) %(extraopts) %`%S%(mode)%' %t" TeX-run-TeX nil
		     (ams-tex-mode) :help "Run AMSTeX")
		    ("ConTeXt" "%(cntxcom) --once --texutil %(extraopts) %(execopts)%t" TeX-run-TeX nil
		     (context-mode) :help "Run ConTeXt once")
		    ("ConTeXt Full" "%(cntxcom) %(extraopts) %(execopts)%t" TeX-run-TeX nil
		     (context-mode) :help "Run ConTeXt until completion")
		    ("BibTeX" "bibtex %s" TeX-run-BibTeX nil t :help "Run BibTeX")
		    ("Biber" "biber %s" TeX-run-Biber nil t :help "Run Biber")
		    ("View" "%V" TeX-run-discard-or-function t t :help "Run Viewer")
		    ("Print" "%p" TeX-run-command t t :help "Print the file")
		    ("Queue" "%q" TeX-run-background nil t :help "View the printer queue" :visible
		     TeX-queue-command)
		    ("File" "%(o?)dvips %d -o %f " TeX-run-dvips t t :help "Generate PostScript file")
		    ("Dvips" "%(o?)dvips %d -o %f " TeX-run-dvips nil t :help
		     "Convert DVI file to PostScript")
		    ("Ps2pdf" "ps2pdf %f" TeX-run-ps2pdf nil t :help "Convert PostScript file to PDF")
		    ("Index" "makeindex %s" TeX-run-index nil t :help
		     "Run makeindex to create index file")
		    ("Xindy" "texindy %s" TeX-run-command nil t :help "Run xindy to create index file")
		    ("Check" "lacheck %s" TeX-run-compile nil (latex-mode) :help
		     "Check LaTeX file for correctness")
		    ("ChkTeX" "chktex -v6 %s" TeX-run-compile nil (latex-mode) :help
		     "Check LaTeX file for common mistakes")
		    ("Spell" "(TeX-ispell-document \"\")" TeX-run-function nil t :help
		     "Spell-check the document")
		    ("Clean" "TeX-clean" TeX-run-function nil t :help
		     "Delete generated intermediate files")
		    ("Clean All" "(TeX-clean t)" TeX-run-function nil t :help
		     "Delete generated intermediate and output files")
		    ("Other" "" TeX-run-command t t :help "Run an arbitrary command"))
 )

[testcodutf-headerlatin1.tex (application/octet-stream, attachment)]

[testcodtlatin1-headerut8.tex (application/octet-stream, attachment)]

Message #8 received at 21825 <at> debbugs.gnu.org (full text, mbox):

From: David Kastrup <dak <at> gnu.org>
To: Uwe Brauer <oub <at> mat.ucm.es>
Cc: 21825 <at> debbugs.gnu.org
Subject: Re: bug#21825: 2015-10-04;
 utf8 and latin1 coding problem in GNU emacs, Xemacs is fine
Date: Wed, 04 Nov 2015 16:50:22 +0100

Uwe Brauer <oub <at> mat.ucm.es> writes:

> Hello
>
> The following problem only occurs in GNU emacs 24.5 or 25.0.50.
>
> I have two files:
>
> The first is saved in latin-1 but its header state
> \usepackage[utf8]{inputenc}
>
> And the other is the other way around
> saved in UTF8 header is 
>
> \usepackage[latin1]{inputenc}
>
>
> Both are displayed correctly in Xemacs and but not in GNU emacs.

I have a hard time understanding what you mean by "displayed correctly"
when the display does not correspond to what LaTeX would output.

Why are you lying to LaTeX/Emacs about the document encoding?  And what
do you hope to achieve by Emacs ignoring this?

Of course one can switch off inputenc encoding recognition, cf.

latex-inputenc-coding-alist is a variable defined in ‘latexenc.el’.
Its value is shown below.

Documentation:
Mapping from LaTeX encodings in "inputenc.sty" to Emacs coding systems.
LaTeX encodings are specified with "\usepackage[encoding]{inputenc}".
Used by the function ‘latexenc-find-file-coding-system’.

You can customize this variable.

Value: (("ansinew" . windows-1252)
 ("applemac" . mac-roman)
 ("ascii" . us-ascii)
 ("cp1250" . windows-1250)
 ("cp1252" . windows-1252)
 ("cp1257" . cp1257)
 ("cp437de" . cp437)
 ("cp437" . cp437)
 ("cp850" . cp850)
 ("cp852" . cp852)
 ("cp858" . cp858)
 ("cp865" . cp865)
 ("latin1" . iso-8859-1)
 ("latin2" . iso-8859-2)
 ("latin3" . iso-8859-3)
 ("latin4" . iso-8859-4)
 ("latin5" . iso-8859-5)
 ("latin9" . iso-8859-15)
 ("next" . next)
 ("utf8" . utf-8)
 ("utf8x" . utf-8))

[back]


but in particular for telling apart various different 8-bit encodings
there is not much of an alternative.

-- 
David Kastrup

Message #11 received at 21825 <at> debbugs.gnu.org (full text, mbox):

From: Mosè Giordano <mose <at> gnu.org>
To: Uwe Brauer <oub <at> mat.ucm.es>
Cc: 21825 <at> debbugs.gnu.org
Subject: Re: bug#21825: 2015-10-04; utf8 and latin1 coding problem in GNU
 emacs, Xemacs is fine
Date: Wed, 4 Nov 2015 16:54:50 +0100

Hi Uwe,

2015-11-04 16:17 GMT+01:00 Uwe Brauer <oub <at> mat.ucm.es>:
>
> Hello
>
> The following problem only occurs in GNU emacs 24.5 or 25.0.50.
>
> I have two files:
>
> The first is saved in latin-1 but its header state
> \usepackage[utf8]{inputenc}
>
> And the other is the other way around
> saved in UTF8 header is
>
> \usepackage[latin1]{inputenc}
>
>
> Both are displayed correctly in Xemacs and but not in GNU emacs.

GNU Emacs has "latexenc" package which opens a file with the encoding
specified by the inputenc package and I think this feature is
valuable, since the actual file encoding should match the inputenc
encoding anyway.  Then, I believe this is the classic case of "this
isn't a bug but a feature".

Cheers,
Mosè

Message #14 received at 21825 <at> debbugs.gnu.org (full text, mbox):

From: Uwe Brauer <oub <at> mat.ucm.es>
To: David Kastrup <dak <at> gnu.org>
Cc: Uwe Brauer <oub <at> mat.ucm.es>, 21825 <at> debbugs.gnu.org
Subject: Re: bug#21825: 2015-10-04;
 utf8 and latin1 coding problem in GNU emacs, Xemacs is fine
Date: Wed, 04 Nov 2015 16:36:09 +0000

[Message part 1 (text/plain, inline)]

>>> "David" == David Kastrup <dak <at> gnu.org> writes:

   > Uwe Brauer <oub <at> mat.ucm.es> writes:
   >> Hello
   >> 
   >> The following problem only occurs in GNU emacs 24.5 or 25.0.50.
   >> 
   >> I have two files:
   >> 
   >> The first is saved in latin-1 but its header state
   >> \usepackage[utf8]{inputenc}
   >> 
   >> And the other is the other way around
   >> saved in UTF8 header is 
   >> 
   >> \usepackage[latin1]{inputenc}
   >> 
   >> 
   >> Both are displayed correctly in Xemacs and but not in GNU emacs.

   > I have a hard time understanding what you mean by "displayed correctly"
   > when the display does not correspond to what LaTeX would output.


I thought, that sending the latex files would have been enough. I attach
the screenshots.


   > Why are you lying to LaTeX/Emacs about the document encoding?  And what
   > do you hope to achieve by Emacs ignoring this?

This is not on purpose of course, so this could occur if one has either
a very large header and forgets what coding has been  selected (this is lame I
know) or more realistically you have a multi file documentation and
don't recall its encoding.

However there is one thing I just have to add. Maybe I was «spoiled»
in the past by Xemacs/x-symbol behavior which displayed any coding
correctly but internally had the files saved in ascii mode. (Like
running iso-iso2tex on every save).

Of course thinking about it again, you are right. This is a bad habit.
So this is not a bug.

I could solve the issue of the wrongly displayed coding by just removing
the incorrect header and reopening the file.


BTW how else could this wrong coding be repaired? I am asking because I
have encountered similar problems in non latex files, which have been
modified by Xemacs.


   > Of course one can switch off inputenc encoding recognition, cf.

   > latex-inputenc-coding-alist is a variable defined in ‘latexenc.el’.
   > Its value is shown below.


I did not know about this. Thanks for pointing it out.

   > Documentation:

   > but in particular for telling apart various different 8-bit encodings
   > there is not much of an alternative.

[utf8header-latin1-code.png (image/png, attachment)]

[latin1-header-utf8-cod.png (image/png, attachment)]

Message #17 received at 21825 <at> debbugs.gnu.org (full text, mbox):

From: David Kastrup <dak <at> gnu.org>
To: Uwe Brauer <oub <at> mat.ucm.es>
Cc: 21825 <at> debbugs.gnu.org
Subject: Re: bug#21825: 2015-10-04;
 utf8 and latin1 coding problem in GNU emacs, Xemacs is fine
Date: Wed, 04 Nov 2015 17:53:52 +0100

Uwe Brauer <oub <at> mat.ucm.es> writes:

>>>> "David" == David Kastrup <dak <at> gnu.org> writes:
>
>    > Uwe Brauer <oub <at> mat.ucm.es> writes:
>    >> Hello
>    >> 
>    >> The following problem only occurs in GNU emacs 24.5 or 25.0.50.
>    >> 
>    >> I have two files:
>    >> 
>    >> The first is saved in latin-1 but its header state
>    >> \usepackage[utf8]{inputenc}
>    >> 
>    >> And the other is the other way around
>    >> saved in UTF8 header is 
>    >> 
>    >> \usepackage[latin1]{inputenc}
>    >> 
>    >> 
>    >> Both are displayed correctly in Xemacs and but not in GNU emacs.
>
>    > I have a hard time understanding what you mean by "displayed correctly"
>    > when the display does not correspond to what LaTeX would output.
>
>
> I thought, that sending the latex files would have been enough. I attach
> the screenshots.
>
>
>    > Why are you lying to LaTeX/Emacs about the document encoding?
>    > And what do you hope to achieve by Emacs ignoring this?
>
> This is not on purpose of course, so this could occur if one has
> either a very large header and forgets what coding has been selected
> (this is lame I know) or more realistically you have a multi file
> documentation and don't recall its encoding.
>
> However there is one thing I just have to add. Maybe I was «spoiled»
> in the past by Xemacs/x-symbol behavior which displayed any coding
> correctly but internally had the files saved in ascii mode. (Like
> running iso-iso2tex on every save).

So you have X-Symbol convert the buffers to pure ASCII input for LaTeX
but load the inputenc package in your document anyway, specifying some
completely random encoding that's a superset of ASCII and has nothing to
do with the buffer encoding.

> Of course thinking about it again, you are right. This is a bad habit.

I cannot rule out that X-Symbol's operation is incompatible with that of
latexenc.el.  I have a hard time considering that a bug of latexenc.el,
though.

> So this is not a bug.
>
> I could solve the issue of the wrongly displayed coding by just
> removing the incorrect header and reopening the file.
>
>
> BTW how else could this wrong coding be repaired? I am asking because
> I have encountered similar problems in non latex files, which have
> been modified by Xemacs.

You can load a file with a particular encoding by using
C-x RET c latin-1 RET C-x C-f filename RET
for example.

You can change the coding system for saving by using

C-x RET f latin-9 RET
for example.

-- 
David Kastrup

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.