GNU bug report logs - #31685
12.1.1; Preview with cached preamble fails with lualatex engine and filenames with spaces

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 31685 in the body.
You can then email your comments to 31685 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: jfbu <jfbu <at> free.fr>
To: bug-auctex <at> gnu.org
Subject: 12.1.1;
 Preview with cached preamble fails with lualatex engine and filenames
 with spaces
Date: Sat, 2 Jun 2018 14:12:20 +0200

Hi,

consider this file

\documentclass{article}
\def\abc{abc}
\begin{document}
$\abc$
\end{document}

and switch engine to luatex.

Then generate previews for document, answering yes to caching preamble.

This fails 

1. the cached format can not be generated

Running `Preview-LaTeX' on `abc def ghi' with ``lualatex -ini -interaction=nonstopmode "&lualatex" prv_abc_def_ghi.ini \ def\ ghi  -file-line-error   "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" "\input" \"abc\ def\ ghi.tex\"''

notice that the above is ill-formed (see \ def\ ghi -file-line-error) bit

This is LuaTeX, Version 1.07.0 (TeX Live 2018)  (INITEX)
 restricted system commands enabled.
(./prv_abc_def_ghi.ini
LaTeX2e <2018-04-01> patch level 5
(using write cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic)(using read cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic /Users/jfb/Library/texlive/2018/texmf-var/luatex-cache/generic)
luaotfload | main : initialization completed in 0.116 seconds (/usr/local/texlive/2018/texmf-dist/tex/latex/carlisle/mylatex.ltx))
! I can't find file `def'.
<*> &lualatex prv_abc_def_ghi.ini  def 
                                    ghi -file-line-error \nonstopmode\nofile...
(Press Enter to retry, or Control-D to exit)
Please type another input file name
! Emergency stop.
<*> &lualatex prv_abc_def_ghi.ini  def 
                                    ghi -file-line-error \nonstopmode\nofile...
 246 words of node memory still in use:
   1 dir, 35 glue_spec nodes
   avail lists: 2:12,3:1,4:1
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on prv_abc_def_ghi.log.


2. even the fallback of not using cached preamble fails  (see bug #31684)

Running `Preview-LaTeX' on `abc def ghi' with ``lualatex --jobname=abc\ def\ ghi  -file-line-error   "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" "\input" \"abc\ def\ ghi.tex\"''
This is LuaTeX, Version 1.07.0 (TeX Live 2018) 
 restricted system commands enabled.
LaTeX2e <2018-04-01> patch level 5
(using write cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic)(using read cache: /usr/local/texlive/2018/texmf-var/luatex-cache/generic /Users/jfb/Library/texlive/2018/texmf-var/luatex-cache/generic)
luaotfload | main : initialization completed in 0.108 seconds
No auxiliary output files.

("./abc def ghi.tex" (/usr/local/texlive/2018/texmf-dist/tex/latex/base/article.cls
Document Class: article 2014/09/29 v1.4h Standard LaTeX document class
(/usr/local/texlive/2018/texmf-dist/tex/latex/base/size10.clo(load luc: /usr/local/texlive/2018/texmf-var/luatex-cache/generic/fonts/otl/lmroman10-regular.luc)))
No file "abc def ghi".aux.
(/Users/jfb/.emacs.d/elpa/auctex-12.1.1/latex/preview.sty (/usr/local/texlive/2018/texmf-dist/tex/generic/luatex85/luatex85.sty) (/Users/jfb/.emacs.d/elpa/auctex-12.1.1/latex/prtightpage.def) (/Users/jfb/.emacs.d/elpa/auctex-12.1.1/latex/prauctex.def
No auxiliary output files.


(/Users/jfb/.emacs.d/elpa/auctex-12.1.1/latex/prauctex.cfg)) (/Users/jfb/.emacs.d/elpa/auctex-12.1.1/latex/prfootnotes.def)
Preview: Fontsize 10pt
Preview: PDFoutput 1
)
./abc def ghi.tex:4: Preview: Snippet 1 started.
<-><->
      
l.4 $
   \abc$
Preview: Tightpage -32891 -32891 32891 32891
./abc def ghi.tex:4: Preview: Snippet 1 ended.(455111+0x911285).
<-><->
      
l.4 $\abc$
        
[1{/usr/local/texlive/2018/texmf-var/fonts/map/pdftex/updmap/pdftex.map}])
(see the transcript file for additional information)
 362 words of node memory still in use:
   2 hlist, 1 vlist, 1 rule, 2 glue, 4 attribute, 45 glue_spec, 4 attribute_list, 2 write nodes
   avail lists: 2:14,3:3,4:1,5:11,6:1,7:7,8:3,9:2
</usr/local/texlive/2018/texmf-dist/fonts/type1/public/amsfonts/cm/cmmi10.pfb>
Output written on abc def ghi.pdf (1 page, 8584 bytes).
Transcript written on "abc def ghi.log".

TeX Output exited as expected with code 1 at Sat Jun  2 14:03:07
Running `Preview-DviPNG' with ``dvipng -picky -noghostscript abc\ def\ ghi.dvi -o "abc\ def\ ghi.prv/tmp14898GFt/prev%03d.png"  -D186 ''
Parser: End of Preview snippet 1 unexpected

Preview-DviPNG killed: 9 at Sat Jun  2 14:03:07


Related notes:

1. Actually caching the preamble when using lualatex is very fragile. It would break or
at least be in inconsistent state if any Lua code is executed in the preamble.

Nevertheless, for simple file above, if filename has no spaces, then preview generation
appears to work as expected (in PDF mode, luatex engine)

2. When using xelatex, the situation is even simpler, it is impossible currently
to use mylatex.ltx with it. https://github.com/davidcarlisle/dpctex/issues/15

 

This is with AUCTeX 12.1.1.

Jean-François

Message #8 received at 31685 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: 31685 <at> debbugs.gnu.org
Cc: jfbu <jfbu <at> free.fr>
Subject: Re: bug#31685: 12.1.1;
 Preview with cached preamble fails with lualatex engine and filenames
 with spaces
Date: Tue, 05 Jun 2018 03:01:39 +0900

[Message part 1 (text/plain, inline)]

Hi all,

>>>>> jfbu <jfbu <at> free.fr> writes:

> 1. the cached format can not be generated

> Running `Preview-LaTeX' on `abc def ghi' with ``lualatex -ini -interaction=nonstopmode "&lualatex" prv_abc_def_ghi.ini \ def\ ghi  -file-line-error   "\nonstopmode\nofiles\PassOptionsToPackage{active,tightpage,auctex}{preview}\AtBeginDocument{\ifx\ifPreview\undefined\RequirePackage[displaymath,floats,graphics,textmath,sections,footnotes]{preview}[2004/11/05]\fi}" "\input" \"abc\ def\ ghi.tex\"''

> notice that the above is ill-formed (see \ def\ ghi -file-line-error) bit

During the personal correspondence between Jean and me, we reach a
resolution.  First of all, Jean kindly provided a new .ini code which
enables preamble cache with unusual file name for not only lualatex, but
also platex and uplatex!  This makes platex and uplatex to be able to
use preamble cache even with file names with space.  (This case caused
error before; That's independent of my recent commits.)

Next, he found out that an option -file-line-error is necessary in
`preview-undump-replacements' for lualatex to use preamble cache on file
name with space.

In addition, it turned out that -kanji option of platex cannot be
omitted even when dumping format file if the preamble contains Japanese
characters.

I combined the due change to cope with these issues and made up the
attached patch.  It dismisses the additional post process of
the command line string in `TeX-inline-preview-internal' to the effect
of not leaving unnecessary fragments in the command line string when
file name with space is used with lualatex.

----------------------------------------------------------------------
However, I'm puzzled now by very weird emacs 26 behavior.  Currently, I
cannot sort out regression test yet.  On my environment, simple test
vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv
(require 'ert)

(ert-deftest weird-cond ()
  "check for `cond'"
  (should
   (equal (xyz t)
	  2)))

(defun xyz (arg)
  "dummy"
;    (cond ((eq arg nil) ; OK
;    (cond ((eq arg 'abc) ; OK
;    (cond ((eq arg 'def) ; OK
    (cond ((eq arg 'default) ; NG
;    (cond ((eq arg 'default1) ; OK
	   1)
	  (t
	   2)))

(byte-compile 'xyz)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
fails; i.e., after evaluating these forms, M-x ert RET RET fails on
emacs 26.1.  Does this reproduce with others?
0. Because of this behavior, `LaTeX-includegraphics-extensions' in
   tests/latex/latex-test.el doesn't pass "make check" with emacs 26.1.
1. As the commens in the above suggests, only the symbol `default'
   triggers this behavior.
2. Emacs 25.3 doesn't have this problem.
3. If `xyz' is not byte compiled, the test does pass.
4. It seems that the following entry in NEWS is related:
** Certain cond/pcase/cl-case forms are now compiled using a faster jump
table implementation.  This uses a new bytecode op 'switch', which
isn't compatible with previous Emacs versions.  This functionality can
be disabled by setting 'byte-compile-cond-use-jump-table' to nil.
   If I replace the last line in the above example with
(let ((byte-compile-cond-use-jump-table nil))
  (byte-compile 'xyz))
, the test does pass.
----------------------------------------------------------------------

> 2. even the fallback of not using cached preamble fails  (see bug #31684)

Though I haven't examined this issue yet, I guess that preview-latex
assumes that fallback run always produces dvi file.  In actual, pdf
file is produced in this case:
> Output written on abc def ghi.pdf (1 page, 8584 bytes).
> Transcript written on "abc def ghi.log".

Therefore dvipng fails:
> TeX Output exited as expected with code 1 at Sat Jun  2 14:03:07
> Running `Preview-DviPNG' with ``dvipng -picky -noghostscript abc\ def\ ghi.dvi -o "abc\ def\ ghi.prv/tmp14898GFt/prev%03d.png"  -D186 ''
> Parser: End of Preview snippet 1 unexpected

> Preview-DviPNG killed: 9 at Sat Jun  2 14:03:07

Regards,
Ikumi Keita

[luatex-preamble-cache.gz (application/x-gzip, attachment)]

Message #11 received at 31685 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: 31685 <at> debbugs.gnu.org
Cc: jfbu <jfbu <at> free.fr>
Subject: Re: bug#31685: 12.1.1;
 Preview with cached preamble fails with lualatex engine and filenames
 with spaces
Date: Thu, 07 Jun 2018 15:00:44 +0900

[Message part 1 (text/plain, inline)]

>>>>> Ikumi Keita <ikumi <at> ikumi.que.jp> writes:
>>>>> jfbu <jfbu <at> free.fr> writes:

>> 1. the cached format can not be generated

[...]

> I combined the due change to cope with these issues and made up the
> attached patch.

The previous patch doesn't play nice with non-ascii characters escaped
with backslash again, so I revised it further.  The attached patch is
for the current git repository.  The regexp in
`preview-dump-replacements' is changed from
	\\\.  (written as \\\\\\. in string literal)
to
	\\.   (written as \\\\. in string leteral)
this time.  Whereas the former matches a sequence of "a backslash
followed by a period", I can't think of a reason to distinguish period
from other chars in the context here.  So I changed it to the latter so
that any chars escaped with backslash are treated equally.  The same
regexp is used in `preview-undump-replacements', too.

I also revised the regression test since the previous one didn't test
what should be tested actually, and added another test.

If other problems aren't found, I'll commit this in several days.

> ----------------------------------------------------------------------
> However, I'm puzzled now by very weird emacs 26 behavior.  Currently, I
> cannot sort out regression test yet.

I filed a bug report at
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=31718
and a fix is proposed.  It isn't yet clear whether this fix is
incorportated in Emacs officially.  Anyway, without this fix,
`LaTeX-includegraphics-extensions(-list)' in style/graphicx.el doesn't
return the correct value when `TeX-engine' is neither `default',
`luatex' nor `xetex', when byte-compiled by emacs 26.
In addition, other codes that have a structure of
(cond ((eq TeX-engine 'default)
       ...)
       ...
      (t
       ...))
suffer similar failure, if there are any such codes.

Regards,
Ikumi Keita

[luatex-preamble-cache2.gz (application/x-gzip, attachment)]

Message #14 received at 31685 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
To: Tassilo Horn <tsdh <at> gnu.org>,
 Mosè Giordano <mose <at> gnu.org>,
 Arash Esbati <arash <at> gnu.org>, David Kastrup <dak <at> gnu.org>
Cc: auctex-devel <at> gnu.org, 31685 <at> debbugs.gnu.org, jfbu <jfbu <at> free.fr>
Subject: copyright assignment
Date: Sat, 09 Jun 2018 22:53:20 +0900

Hi Tassilo, Mosè, Arash and David,

> First of all, Jean kindly provided a new .ini code which enables
> preamble cache with unusual file name for not only lualatex, but also
> platex and uplatex!

> If other problems aren't found, I'll commit this in several days.

Alhough I didn't think well before, now I think Jean's contribution of
.ini code related to this topic is big enough to need to get copyright
assignment.  Here is the diff in preview.el.in with respect to his
contribution:

@@ -3315,10 +3331,13 @@
        (push format-cons preview-dumped-alist))
       ;; mylatex.ltx expects a file name to follow.  Bad. `.tex'
       ;; in the tools bundle is an empty file.
-      (write-region "\\ifx\\pdfoutput\\undefined\\else\
-\\let\\PREVIEWdump\\dump\\def\\dump{%
-\\edef\\next{{\\catcode`\\ 9 \\pdfoutput=\\the\\pdfoutput\\relax\
-\\the\\everyjob}}\\everyjob\\next\\catcode`\\ 10 \\let\\dump\\PREVIEWdump\\dump}\\fi\\input mylatex.ltx \\relax\n" nil dump-file)
+      (write-region "\\let\\PREVIEWdump\\dump\\def\\dump{%
+\\edef\\next{{\\ifx\\pdfoutput\\undefined\\else\
+\\pdfoutput=\\the\\pdfoutput\\relax\\fi\
+\\the\\everyjob}}\\everyjob\\next\\catcode`\\ 10 %
+\\catcode`/ 0 %
+\\def\\AUCTEXINPUT##1{\\catcode`/ 12\\relax\\catcode`\\ 9\\relax\\input\\detokenize{##1}\\relax}%
+\\let\\dump\\PREVIEWdump\\dump}\\input mylatex.ltx \\relax%\n" nil dump-file)
       (TeX-save-document master)
       (prog1
          (preview-generate-preview

I already asked him and he answered that he is fine with giving over his
copyright, so could you please provide him instructions of necessary
procedure?

Best regards,
Ikumi Keita

Message #17 received at 31685 <at> debbugs.gnu.org (full text, mbox):

From: Ikumi Keita <ikumi <at> ikumi.que.jp>
Cc: 31685 <at> debbugs.gnu.org, jfbu <jfbu <at> free.fr>
Subject: Re: bug#31685: 12.1.1;
 Preview with cached preamble fails with lualatex engine and filenames
 with spaces
Date: Tue, 12 Jun 2018 15:11:54 +0900

>>>>> Ikumi Keita <ikumi <at> ikumi.que.jp> writes:

> If other problems aren't found, I'll commit this in several days.

Done.  The copyright assignment from Jean is in due process.

Best,
Ikumi Keita

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.