GNU bug report logs - #51252
[core-updates-frozen] lualatex needs additional setup

Previous Next

Package: guix;

Reported by: John Kehayias <john.kehayias <at> protonmail.com>

Date: Sun, 17 Oct 2021 15:26:01 UTC

Severity: normal

Done: Ludovic Courtès <ludo <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 51252 in the body.
You can then email your comments to 51252 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Sun, 17 Oct 2021 15:26:01 GMT) Full text and rfc822 format available.

Acknowledgement sent to John Kehayias <john.kehayias <at> protonmail.com>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Sun, 17 Oct 2021 15:26:01 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: "bug-guix <at> gnu.org" <bug-guix <at> gnu.org>
Subject: [core-updates-frozen] lualatex needs additional setup
Date: Sun, 17 Oct 2021 15:25:11 +0000
Hello,

This is my first time trying out TeX on Guix, but ran into a problem trying to run lualatex on core-updates-frozen. Since I wasn't sure what I needed, I tried the huge full texlive package in case that solved the problem I was seeing. It did not.

Running lualatex on anything gives the output

This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
 restricted system commands enabled.

kpathsea: Running mktexfmt lualatex.fmt
/gnu/store/cpflshy9z2yzd8dybm0j9a7i0bscpxa2-texlive-bin-20210325/share/texmf-dist/scripts/texlive/fmtutil.pl: Unexpected non-option argument(s): lualatex.fmt
Try "fmtutil --help" for more information.
I can't find the format file `lualatex.fmt'!

xelatex on the same file and environment worked fine. After some digging, I could get the lualatex.fmt file by installing texlive-tex-ini-files and running

lualatex -ini /gnu/store/skznnf2n2f4dwg44gm8bvcn78157z69p-texlive-tex-ini-files-59745/share/texmf-dist/tex/generic/tex-ini-files/lualatex.ini

With this lualatex.fmt file present, lualatex runs and produces the correct output. I'm guessing this is not the expected procedure, though I haven't been able to test on master to compare yet. I wasn't sure if I missed a texlive- package, but xelatex working as is makes me think not.

John




Information forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Sun, 17 Oct 2021 16:06:01 GMT) Full text and rfc822 format available.

Message #8 received at 51252 <at> debbugs.gnu.org (full text, mbox):

From: John Kehayias <john.kehayias <at> protonmail.com>
To: "51252 <at> debbugs.gnu.org" <51252 <at> debbugs.gnu.org>
Subject: Re: [core-updates-frozen] lualatex needs additional setup
Date: Sun, 17 Oct 2021 16:05:02 +0000
With just texlive-base I get:

---! lualatex.fmt was written by luahbtex
(Fatal format file error; I'm stymied)%

On IRC it was reported lualatex works with texlive installed without additional setup, so this does seem like a core-updates-frozen bug. Unless I've done something I didn't realize that would do this?




Information forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Sun, 17 Oct 2021 16:06:02 GMT) Full text and rfc822 format available.

Message #11 received at 51252 <at> debbugs.gnu.org (full text, mbox):

From: Robin Templeton <robin <at> terpri.org>
To: John Kehayias <john.kehayias <at> protonmail.com>
Cc: 51252 <at> debbugs.gnu.org
Subject: Re: bug#51252: [core-updates-frozen] lualatex needs additional setup
Date: Sun, 17 Oct 2021 12:05:02 -0400
John Kehayias <john.kehayias <at> protonmail.com> writes:

> With this lualatex.fmt file present, lualatex runs and produces the
> correct output. I'm guessing this is not the expected procedure,
> though I haven't been able to test on master to compare yet. I wasn't
> sure if I missed a texlive- package, but xelatex working as is makes
> me think not.

I tested this on master, and lualatex (installed via texlive) does
indeed work without any additional setup.

Robin




Information forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Wed, 20 Oct 2021 21:49:02 GMT) Full text and rfc822 format available.

Message #14 received at 51252 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 51252 <at> debbugs.gnu.org
Subject: [core-updates-frozen] lualatex needs additional setup
Date: Wed, 20 Oct 2021 21:45:35 +0000
“texlive-latex-base” provides that file.  We disable a whole bunch 
of formats that we cannot build that early in the process, and 
then we run fmtutil-sys on the patched file.
The cause for trouble lies in texlive-kpathsea, which provides 
share/texmf-dist/web2c/fmtutil.cnf 
.  That file states how to build the various fmt files.

The file in the earlier version of Tex Live contains this line for the lualatex format:

   lualatex luatex language.dat,language.dat.lua lualatex.ini

the new file (on core-updates-frozen) says this:

   lualatex luahbtex language.dat,language.dat.lua lualatex.ini

i.e. it will try to build the lualatex fmt file with luahbtex 
instead of luatex.  I suppose at this point in the build we don’t 
actually have a working luahbtex, so fmtutil-sys doesn’t generate 
the correct lualatex.fmt.

I’m not sure how to fix this, but I guess we can avoid a world 
rebuild by adding an extra package that installs a working 
lualatex.fmt file.

-- 
Ricardo




Information forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Tue, 23 Nov 2021 01:41:02 GMT) Full text and rfc822 format available.

Message #17 received at 51252 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: 51252 <at> debbugs.gnu.org
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 John Kehayias <john.kehayias <at> protonmail.com>,
 Robin Templeton <robin <at> terpri.org>
Subject: Re: bug#51252: [core-updates-frozen] lualatex needs additional setup
Date: Mon, 22 Nov 2021 22:40:01 -0300
Hello,

I did some investigation about this problem today. Sorry for the trouble.

Em quarta-feira, 20 de outubro de 2021, às 18:45:35 -03, Ricardo Wurmus escreveu:
> “texlive-latex-base” provides that file.  We disable a whole bunch
> of formats that we cannot build that early in the process, and
> then we run fmtutil-sys on the patched file.
> The cause for trouble lies in texlive-kpathsea, which provides
> share/texmf-dist/web2c/fmtutil.cnf
> .  That file states how to build the various fmt files.
> 
> The file in the earlier version of Tex Live contains this line for the
> lualatex format:
> 
>     lualatex luatex language.dat,language.dat.lua lualatex.ini
> 
> the new file (on core-updates-frozen) says this:
> 
>     lualatex luahbtex language.dat,language.dat.lua lualatex.ini
> 
> i.e. it will try to build the lualatex fmt file with luahbtex
> instead of luatex.

Thank you for this analysis!

> I suppose at this point in the build we don’t
> actually have a working luahbtex, so fmtutil-sys doesn’t generate
> the correct lualatex.fmt.

Looking at the build log of ‘texlive-latex-base’ from the core-updates-
frozen branch, LuaHBTeX seems to be functional at the time it is used to 
generate ‘lualatex.fmt’. Comparing it to the build log of the same package 
on master, there are some differences though:

• core-updates-frozen shows the following messages:
  • “No file TS1lmr.fd.”
  • “No file latex2e-first-aid-for-external-files.ltx.”
• core-updates-frozen lists some fonts with strange names and sizes:
  • \font\c__fp_exp_intarray=cmr10 at 0.00002pt
  • \font\c__fp_trig_intarray=cmr10 at 0.00003pt
  • \font\g__regex_state_active_intarray=cmr10 at 0.00005pt
  • \font\g__regex_thread_info_intarray=cmr10 at 0.00006pt
  • \font\g__regex_submatch_prev_intarray=cmr10 at 0.00008pt
  • \font\g__regex_submatch_begin_intarray=cmr10 at 0.00009pt
  • \font\g__regex_submatch_end_intarray=cmr10 at 0.0001pt
  • \font\g__regex_balance_intarray=cmr10 at 0.00012pt
• master instals file at web2c/luatex/lualatex.fmt, while
  core-updates-frozen installs it at web2c/luahbtex/lualatex.fmt.

This last difference coupled with the following excerpt from the TexLive 
news section¹:

  “LuaTeX: Integration with HarfBuzz library, available as new engines 
   luahbtex (used for lualatex) and luajithbtex.”

suggests that the command ‘lualatex’ is supposed to invoke the LuaHBTeX
engine rather than the LuaTeX engine. Indeed, when using LuaHBTeX
explicitly, there’s no error about the format file. Unfortunately, there’s an
error about font loading:

--8<---------------cut here---------------start------------->8---
popigai ⸤env⸥: luahbtex '&lualatex' hello.tex
This is LuaHBTeX, Version 1.13.0 (TeX Live 2021/GNU Guix) 
 restricted system commands enabled.
(./hello.tex
LaTeX2e <2020-10-01> patch level 4
 L3 programming layer <2021-02-18> (/gnu/store/nx4jih5xnm6hzfgvi04w4wkp4pbma8bm-profile/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/gnu/store/nx4jih5xnm6hzfgvi04w4wkp4pbma8bm-profile/share/texmf-dist/tex/latex/base/size10.clo
luaotfload | db : Font names database not found, generating new one.
luaotfload | db : This can take several minutes; please be patient.
luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: File not found: "lmroman10-regular".
! Font \TU/lmr/m/n/10=[lmroman10-regular]:+tlig; at 10pt not loadable: metric data not found or bad.
<to be read again> 
relax 
l.54 \normalsize
              
? 
--8<---------------cut here---------------end--------------->8---

I tried running `luaotfload-tool --update` as suggested on the interwebs
for a similar problem, but I ran into a separate issue with that tool:

--8<---------------cut here---------------start------------->8---
popigai ⸤env⸥: luaotfload-tool --update
...ih5xnm6hzfgvi04w4wkp4pbma8bm-profile/bin/luaotfload-tool:183: module 'alt_getopt' not found:
        no field package.preload['alt_getopt']
        [kpse lua searcher] file not found: 'alt_getopt'
        [kpse C searcher] file not found: 'alt_getopt'
popigai ⸤env⸥: 
--8<---------------cut here---------------end--------------->8---

-- 
Thanks,
Thiago

¹ https://tug.org/texlive/doc/texlive-en/texlive-en.html#news






Information forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Sat, 04 Dec 2021 02:17:02 GMT) Full text and rfc822 format available.

Message #20 received at 51252 <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: guix-patches <at> gnu.org
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 John Kehayias <john.kehayias <at> protonmail.com>, 51252 <at> debbugs.gnu.org,
 Robin Templeton <robin <at> terpri.org>,
 Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Subject: [PATCH core-updates-frozen] gnu: Add texlive-latex-luatex.
Date: Fri,  3 Dec 2021 23:16:11 -0300
When TeX Live was updated to version 2021 the ‘lualatex’ format started
being generated with the LuaHBTeX engine, but the ‘lualatex’ command still
uses the LuaTeX engine. This causes the command to fail:

  user <at> popigai:~$ lualatex hello.tex
  This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
  restricted system commands enabled.

  ---! lualatex.fmt was written by luahbtex
  (Fatal format file error; I'm stymied)user <at> popigai:~$

The correct way to fix this problem would be either to change texlive-bin
to make ‘lualatex’ use the LuaHBTeX engine, or to change texlive-latex-base
to generate ‘lualatex.fmt’ with LuaTeX. Both options would rebuild large
parts of the world.

Ricardo Wurmus suggested creating a new package to provide ‘lualatex.fmt’
for LuaTeX as a workaround. This is what this package does.

Fixes https://issues.guix.gnu.org/51252.

* gnu/packages/tex.scm (texlive-latex-luatex): New variable.
---

Hello,

Strictly speaking, this patch solves the problem reported in issue 51252, but
I still cannot use ‘lualatex’ because of a problem with fonts:

--8<---------------cut here---------------start------------->8---
user <at> popigai:~$ lualatex hello.tex
This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
 restricted system commands enabled.
(./hello.tex
LaTeX2e <2020-10-01> patch level 4
 L3 programming layer <2021-02-18> (/home/user/.guix-profile/share/texmf-dist/tex/latex/base/article.cls
Document Class: article 2020/04/10 v1.4m Standard LaTeX document class
(/home/user/.guix-profile/share/texmf-dist/tex/latex/base/size10.clo
luaotfload | db : Font names database not found, generating new one.
luaotfload | db : This can take several minutes; please be patient.
luaotfload | db : Reload initiated (formats: otf,ttf,ttc); reason: File not found: "lmroman10-regular".
! Font \TU/lmr/m/n/10=[lmroman10-regular]:+tlig; at 10pt not loadable: metric data not found or bad.
<to be read again>
relax
l.54 \normalsize

?
! Emergency stop.
<to be read again>
relax
l.54 \normalsize

 307 words of node memory still in use:
   1 hlist, 1 dir, 3 kern, 1 glyph, 1 attribute, 39 glue_spec, 1 attribute_list, 3 if_stack nodes
   avail lists: 2:7,3:3,4:1,5:1
!  ==> Fatal error occurred, no output PDF file produced!
Transcript written on hello.log.
--8<---------------cut here---------------end--------------->8---

This happens even though I have the packages texlive-base, texlive-lm and
texlive-fonts-latex installed in the profile. I suspect this is a problem with
luaotfload. I’ll poke at it a bit to see if I can find something out.

The reporter of 51252 doesn’t seem to hit this font issue, so perhaps this patch
will be enough for them.

In the (hopefuly) near future, I’ll submit a patch for core-updates changing
‘lualatex’ to use the LuaHBTeX engine and deprecating this package.

Thanks,
Thiago


 gnu/packages/tex.scm | 65 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 65 insertions(+)

diff --git a/gnu/packages/tex.scm b/gnu/packages/tex.scm
index 910be212ac17..256fe7da90bc 100644
--- a/gnu/packages/tex.scm
+++ b/gnu/packages/tex.scm
@@ -3746,6 +3746,71 @@ (define-public texlive-luaotfload
 
 (define-deprecated-package texlive-luatex-luaotfload texlive-luaotfload)
 
+;; FIXME: This package is a temporary workaround to provide ‘lualatex.fmt’ for
+;; the LuaTeX engine. It is needed because it was discovered too late in the
+;; core-updates-frozen cycle that texlive-latex-base only provides it for
+;; LuaHBTeX. See https://issues.guix.gnu.org/51252.
+(define-public texlive-latex-luatex
+  (package
+    (name "texlive-latex-luatex")
+    (version (number->string %texlive-revision))
+    (source #f)
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules ((guix build gnu-build-system)
+                  (guix build utils)
+                  (ice-9 rdelim)
+                  (ice-9 string-fun))
+       #:phases
+       (modify-phases %standard-phases
+         (delete 'unpack)
+         (delete 'bootstrap)
+         (delete 'configure)
+         (delete 'check)
+         (replace 'build
+           (lambda* (#:key inputs #:allow-other-keys)
+             (mkdir "web2c")
+             (let ((fmtutil.cnf-in (open-file
+                                    (string-append
+                                     (assoc-ref inputs "texlive-kpathsea")
+                                     "/share/texmf-dist/web2c/fmtutil.cnf")
+                                    "r"))
+                   (fmtutil.cnf-out (open-file "web2c/fmtutil.cnf" "w")))
+
+               ;; Copy ‘lualatex’ format lines to the new fmtutil.cnf, changing
+               ;; the engine from ‘luahbtex’ to ‘luatex’.
+               (do ((line "" (read-line fmtutil.cnf-in 'concat)))
+                   ((eof-object? line))
+                 (when (string-prefix? "lualatex" line)
+                   (display (string-replace-substring line "luahbtex" "luatex")
+                            fmtutil.cnf-out)))
+               (close-port fmtutil.cnf-out)
+               (close-port fmtutil.cnf-in)
+
+               (invoke "fmtutil" "--sys" "--all" "--fmtdir=web2c"
+                       "--cnffile=web2c/fmtutil.cnf")
+
+               ;; Don't risk this file interfering with anything else.
+               (delete-file "web2c/fmtutil.cnf"))))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((web2c (string-append %output "/share/texmf-dist/web2c")))
+               (mkdir-p web2c)
+               (copy-recursively "web2c" web2c)))))))
+    (native-inputs
+     `(("texlive-bin" ,texlive-bin)
+       ("texlive-babel" ,texlive-babel)
+       ("texlive-cm" ,texlive-cm)
+       ("texlive-fonts-latex" ,texlive-fonts-latex)
+       ("texlive-kpathsea" ,texlive-kpathsea)
+       ("texlive-latex-base" ,texlive-latex-base)
+       ("texlive-lm" ,texlive-lm)
+       ("texlive-tex-ini-files" ,texlive-tex-ini-files)))
+    (home-page (package-home-page texlive-latex-base))
+    (synopsis "LuaLaTeX format files for LuaTeX")
+    (description "This package is necessary to use LaTeX with the LuaTeX engine.")
+    (license (package-license texlive-latex-base))))
+
 (define-public texlive-latex-amsmath
   (package
     (name "texlive-latex-amsmath")

base-commit: 6d4f260166eb499564fa31ce487ecf301f445e4e




Reply sent to Ludovic Courtès <ludo <at> gnu.org>:
You have taken responsibility. (Sun, 05 Dec 2021 18:09:13 GMT) Full text and rfc822 format available.

Notification sent to John Kehayias <john.kehayias <at> protonmail.com>:
bug acknowledged by developer. (Sun, 05 Dec 2021 18:09:13 GMT) Full text and rfc822 format available.

Message #25 received at 51252-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 John Kehayias <john.kehayias <at> protonmail.com>, 51252-done <at> debbugs.gnu.org,
 Robin Templeton <robin <at> terpri.org>, 52268-done <at> debbugs.gnu.org
Subject: Re: bug#52268: [PATCH core-updates-frozen] gnu: Add
 texlive-latex-luatex.
Date: Sun, 05 Dec 2021 16:40:33 +0100
Hello,

Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:

> When TeX Live was updated to version 2021 the ‘lualatex’ format started
> being generated with the LuaHBTeX engine, but the ‘lualatex’ command still
> uses the LuaTeX engine. This causes the command to fail:
>
>   user <at> popigai:~$ lualatex hello.tex
>   This is LuaTeX, Version 1.13.0 (TeX Live 2021/GNU Guix)
>   restricted system commands enabled.
>
>   ---! lualatex.fmt was written by luahbtex
>   (Fatal format file error; I'm stymied)user <at> popigai:~$
>
> The correct way to fix this problem would be either to change texlive-bin
> to make ‘lualatex’ use the LuaHBTeX engine, or to change texlive-latex-base
> to generate ‘lualatex.fmt’ with LuaTeX. Both options would rebuild large
> parts of the world.
>
> Ricardo Wurmus suggested creating a new package to provide ‘lualatex.fmt’
> for LuaTeX as a workaround. This is what this package does.
>
> Fixes https://issues.guix.gnu.org/51252.
>
> * gnu/packages/tex.scm (texlive-latex-luatex): New variable.

Applied, thanks!

I don’t fully understand the issue but it looks like a reasonable and
non-intrusive fix.

Ludo’.




Information forwarded to bug-guix <at> gnu.org:
bug#51252; Package guix. (Mon, 06 Dec 2021 22:55:02 GMT) Full text and rfc822 format available.

Message #28 received at 51252-done <at> debbugs.gnu.org (full text, mbox):

From: Thiago Jung Bauermann <bauermann <at> kolabnow.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 John Kehayias <john.kehayias <at> protonmail.com>, 51252-done <at> debbugs.gnu.org,
 Robin Templeton <robin <at> terpri.org>, 52268-done <at> debbugs.gnu.org
Subject: Re: bug#52268: [PATCH core-updates-frozen] gnu: Add
 texlive-latex-luatex.
Date: Mon, 06 Dec 2021 19:53:51 -0300
Hello Ludo,

Em domingo, 5 de dezembro de 2021, às 12:40:33 -03, Ludovic Courtès 
escreveu:
> Thiago Jung Bauermann <bauermann <at> kolabnow.com> skribis:
> > Ricardo Wurmus suggested creating a new package to provide
> > ‘lualatex.fmt’ for LuaTeX as a workaround. This is what this package
> > does.
> > 
> > Fixes https://issues.guix.gnu.org/51252.
> > 
> > * gnu/packages/tex.scm (texlive-latex-luatex): New variable.
> 
> Applied, thanks!
> 
> I don’t fully understand the issue but it looks like a reasonable and
> non-intrusive fix.

Thank you!

-- 
Thanks,
Thiago






bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Tue, 04 Jan 2022 12:24:06 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 74 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.