GNU bug report logs - #51122
[PATCH] maint: Factorize po xref translation.

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: Julien Lepiller <julien@HIDDEN>; Keywords: patch; dated Sun, 10 Oct 2021 21:34:01 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.

Message received at 51122 <at> debbugs.gnu.org:


Received: (at 51122) by debbugs.gnu.org; 11 Oct 2021 17:17:44 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 11 13:17:44 2021
Received: from localhost ([127.0.0.1]:60822 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mZyvo-0000l5-5k
	for submit <at> debbugs.gnu.org; Mon, 11 Oct 2021 13:17:44 -0400
Received: from pelzflorian.de ([5.45.111.108]:33078 helo=mail.pelzflorian.de)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pelzflorian@HIDDEN>) id 1mZyvi-0000g7-DT
 for 51122 <at> debbugs.gnu.org; Mon, 11 Oct 2021 13:17:42 -0400
Received: from pelzflorian.localdomain (unknown [5.45.111.108])
 by mail.pelzflorian.de (Postfix) with ESMTPSA id 1403236069D;
 Mon, 11 Oct 2021 19:17:37 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=pelzflorian.de;
 s=mail; t=1633972657;
 bh=Y+sSpwFDqupkQ4gZj6dhcznqOFfhtBaUuOROKtcJZLw=;
 h=Date:From:To:Cc:Subject:References:In-Reply-To;
 b=Ry+CGe6s3D/KIcUWlCyN9zkqb+eqPBs7Q2aaRNykHXfZW1Fn+UNHx+thqXIQWIhQy
 7ZkSWXGgm33P2B2NlWW41b7j+Buyy2yzYraVCQpXVh6jDw0TyXIbxUdxs0ks4vuRXj
 kJ79d+RduTlikR26ws0YnqqEUvIFthWx7IF6iEYE=
Date: Mon, 11 Oct 2021 19:17:28 +0200
From: "pelzflorian (Florian Pelz)" <pelzflorian@HIDDEN>
To: Julien Lepiller <julien@HIDDEN>
Subject: Re: [bug#51122] [PATCH v2] maint: Factorize po xref translation.
Message-ID: <20211011171603.3m7spi6bwlhpgjiv@HIDDEN>
References: <8a210e8ceaadfa600e60b0b97a1e7aba04b2bb02.1633901513.git.julien@HIDDEN>
 <20211010233711.23bb1e9e@HIDDEN>
 <20211011082415.j6wera4b5t4mtnxa@HIDDEN>
 <20211011144626.0049af43@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20211011144626.0049af43@HIDDEN>
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 51122
Cc: 51122 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Thanks Julien!

On Mon, Oct 11, 2021 at 02:46:26PM +0200, Julien Lepiller wrote:
> Thanks for the quick review! I've updated the commit message and
> updated the copyright headers in guix/build/po.scm and guix/self.scm.

In guix/self.scm you only removed code.  I don’t think a copyright
header is needed there.


> I had to make the texi files depend on make-go because otherwise, I get
> messages like "guix/build/po.scm is newer than compiled
> .../guix/build/po.go" and it takes forever to build that file, once for
> each texi. For some reason, it's almost instantaneous when building in
> make-go.

make-core-go instead of make-go is enough in my testing, but something
more fine-grained would save time.  Also POXREF is run twice for each
guix.??.texi file.  Maybe I did something wrong, maybe it is your
patch.  Not sure.  Testing takes long anyway.

Regards,
Florian




Information forwarded to guix-patches@HIDDEN:
bug#51122; Package guix-patches. Full text available.

Message received at 51122 <at> debbugs.gnu.org:


Received: (at 51122) by debbugs.gnu.org; 11 Oct 2021 12:46:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 11 08:46:47 2021
Received: from localhost ([127.0.0.1]:57025 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mZuhW-00007Y-Pa
	for submit <at> debbugs.gnu.org; Mon, 11 Oct 2021 08:46:47 -0400
Received: from lepiller.eu ([89.234.186.109]:35312)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <julien@HIDDEN>) id 1mZuhU-0008UJ-Oa
 for 51122 <at> debbugs.gnu.org; Mon, 11 Oct 2021 08:46:42 -0400
Received: from lepiller.eu (localhost [127.0.0.1])
 by lepiller.eu (OpenSMTPD) with ESMTP id 4b3fddac;
 Mon, 11 Oct 2021 12:46:35 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from
 :to:cc:subject:message-id:in-reply-to:references:mime-version
 :content-type; s=dkim; bh=5okbTg7Q0pXAHO9czDjAoZ9+BNJWfc5ZllMB6x
 J2Vds=; b=TcyagrwWStEsNFOJfO6bkgObZ3uh7gR7nbsbTgxttEuvxaOlQPC8WF
 vwnq1XAv/JrfL4Emvavglpk9oOEulQ+vckbDR3rKqc+Tyobmn81AOFGNK/qGFUhY
 lbp+nb5jA3V01UD479b43/55milaDWOZ7hy5hBEfMzZByGQl9oAJC3svRnNWutJ+
 4MLmy3eoiFU+O6/00YVPXHqWAu3QPC78NOKCNeZ4xHODjNpMHmqs6hP4tR0WaGv8
 Z9sSqeJCsN2UUyCVl/AlBirAWC2a8nmAAEZRirnf2awYjM11fuybKxNwwm+TgdAo
 YVq3OaIGVukiz4vKayCv4Tk6xug3LcgQ==
Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id d3b219e6
 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); 
 Mon, 11 Oct 2021 12:46:35 +0000 (UTC)
Date: Mon, 11 Oct 2021 14:46:26 +0200
From: Julien Lepiller <julien@HIDDEN>
To: "pelzflorian (Florian Pelz)" <pelzflorian@HIDDEN>
Subject: Re: [bug#51122] [PATCH v2] maint: Factorize po xref translation.
Message-ID: <20211011144626.0049af43@HIDDEN>
In-Reply-To: <20211011082415.j6wera4b5t4mtnxa@HIDDEN>
References: <8a210e8ceaadfa600e60b0b97a1e7aba04b2bb02.1633901513.git.julien@HIDDEN>
 <20211010233711.23bb1e9e@HIDDEN>
 <20211011082415.j6wera4b5t4mtnxa@HIDDEN>
X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="MP_/rT0vi8_QcOgtQ61Cv5jhSxv"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 51122
Cc: 51122 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

--MP_/rT0vi8_QcOgtQ61Cv5jhSxv
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Le Mon, 11 Oct 2021 10:45:45 +0200,
"pelzflorian (Florian Pelz)" <pelzflorian@HIDDEN> a =C3=A9crit :

> Hi Julien!  Thank you for replacing that hard-to-read sed script.
> 65;6003;1c
> On Sun, Oct 10, 2021 at 11:37:11PM +0200, Julien Lepiller wrote:
> > For info, I built "make" and "make as-derivation" successfully with
> > this patch. I need it because with recent changes in guix.texi, a
> > lot of text has become fuzzy, including node names, which are
> > translated by the current xref_command (in doc/local.mk), when they
> > shouldn't. I improved the method we used for guix pull, and used it
> > instead. =20
>=20
> I think you should add to the commit message that your patch Fixes
> fuzzy translations being inserted.
>=20
> Your changes to guix/build/po.scm do more than just move code (which
> you mentioned in the commit message): you added parsing of =E2=80=9C#, fu=
zzy=E2=80=9D
> flags.
>=20
> You should update the copyright header for guix/build/po.scm.
>=20
> Otherwise LGTM, but I haven=E2=80=99t tested yet.  (make now converts the=
 xref
> after scheme modules are compiled, I think(?), so testing PO files
> takes longer for me on a clean checkout.  But it is fine.)
>=20
> Regards,
> Florian

Thanks for the quick review! I've updated the commit message and
updated the copyright headers in guix/build/po.scm and guix/self.scm.

I had to make the texi files depend on make-go because otherwise, I get
messages like "guix/build/po.scm is newer than compiled
.../guix/build/po.go" and it takes forever to build that file, once for
each texi. For some reason, it's almost instantaneous when building in
make-go.

--MP_/rT0vi8_QcOgtQ61Cv5jhSxv
Content-Type: text/x-patch
Content-Transfer-Encoding: quoted-printable
Content-Disposition: attachment;
 filename=0001-maint-Factorize-po-xref-translation.patch

=46rom 75198f83fccf27b447e5eaaa6fb6caeed3b29408 Mon Sep 17 00:00:00 2001
Message-Id: <75198f83fccf27b447e5eaaa6fb6caeed3b29408.1633955976.git.julien=
@lepiller.eu>
From: Julien Lepiller <julien@HIDDEN>
Date: Sun, 10 Oct 2021 22:07:51 +0200
Subject: [PATCH] maint: Factorize po xref translation.

This ensures we use the same method in "make" as in "guix/self.scm".

* Makefile.am: Build guix/build/po.scm.
* build-aux/convert-xref.scm: New file.
* doc/local.mk (xref_command): Use it.
* guix/self.scm (translate-cross-references): Move it...
* guix/build/po.scm: Parse comments and flags separately to find fuzzy
flags.
(translate-cross-references): ...here.
(parse-tree->assoc): Ignore fuzzy entries.
---
 Makefile.am                |   3 +-
 build-aux/convert-xref.scm |  26 +++++++++
 doc/local.mk               |  29 +++------
 guix/build/po.scm          | 117 ++++++++++++++++++++++++++++++++++---
 guix/self.scm              |  81 ++++---------------------
 5 files changed, 157 insertions(+), 99 deletions(-)
 create mode 100644 build-aux/convert-xref.scm

diff --git a/Makefile.am b/Makefile.am
index 635147efc1..dc53b2c810 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -69,7 +69,6 @@ nodist_noinst_SCRIPTS =3D				\
 # Modules that are not compiled but are installed nonetheless, such as
 # build-side modules with unusual dependencies.
 MODULES_NOT_COMPILED =3D				\
-  guix/build/po.scm				\
   guix/man-db.scm
=20
 include gnu/local.mk
@@ -227,6 +226,7 @@ MODULES =3D					\
   guix/build/pack.scm				\
   guix/build/utils.scm				\
   guix/build/union.scm				\
+  guix/build/po.scm				\
   guix/build/profiles.scm			\
   guix/build/compile.scm			\
   guix/build/rpath.scm				\
@@ -640,6 +640,7 @@ EXTRA_DIST +=3D						\
   build-aux/check-final-inputs-self-contained.scm	\
   build-aux/check-channel-news.scm			\
   build-aux/compile-as-derivation.scm			\
+  build-aux/convert-xref.scm			\
   build-aux/generate-authors.scm			\
   build-aux/test-driver.scm				\
   build-aux/update-guix-package.scm			\
diff --git a/build-aux/convert-xref.scm b/build-aux/convert-xref.scm
new file mode 100644
index 0000000000..47c8828857
--- /dev/null
+++ b/build-aux/convert-xref.scm
@@ -0,0 +1,26 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright =C2=A9 2021 Julien Lepiller <julien@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+;; Translate cross-references in a translated .texi manual.
+
+(use-modules (guix build po)
+             (ice-9 match))
+
+(match (command-line)
+  ((program texi pofile)
+   (translate-cross-references texi pofile)))
diff --git a/doc/local.mk b/doc/local.mk
index 8340b75a87..88f7e67155 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -97,44 +97,29 @@ PO4A_PARAMS +=3D -k 0 # produce an output even if the t=
ranslation is not complete
 PO4A_PARAMS +=3D -f texinfo # texinfo format
=20
 # When a change to guix.texi occurs, it is not translated immediately.
-# Because @pxref and @xref commands are reference to a section by name, th=
ey
+# Because @pxref and @xref commands are references to sections by name, th=
ey
 # should be translated. If a modification adds a reference to a section, t=
his
 # reference is not translated, which means it references a section that do=
es not
 # exist.
-# This command loops through the translated files looking for references. =
For
-# each of these references, it tries to find the translation and replaces =
the
-# reference name, even in untranslated strings.
-# The last sed is a multiline sed because some references span multiple li=
nes.
 define xref_command
-cat "$@.tmp" | egrep '@p?x?ref' -A1 | sed 'N;s|--\n||g;P;D' | sed 's|^| |g=
' | \
-        tr -d '\012' | sed 's|\(@p\?x\?ref\)|\n\1|g' | egrep '@p?x?ref' | \
-        sed 's|^.*@p\?x\?ref{\([^,}]*\).*$$|\1|g' | sort | uniq | while re=
ad e; do \
-    if [ -n "$$e" ]; then \
-      line=3D$$(grep -n "^msgid \"$$e\"" "$<" | cut -f1 --delimiter=3D":")=
 ;\
-      ((line++)) ;\
-      if [ "$$line" !=3D "1" ]; then \
-	translation=3D$$(head -n "$$line" "$<" | tail -1 | grep msgstr | sed 's|m=
sgstr "\([^"]*\)"|\1|') ;\
-	if [ "$$translation" !=3D "" ]; then \
-	      sed "N;s@\(p\?x\?ref\){$$(echo $$e | sed 's| |[\\n ]|g')\(,\|}\)@\1=
{$$translation\2@g;P;D" -i "$@.tmp" ;\
-	fi ;\
-      fi ;\
-   fi ;\
-done
+$(top_srcdir)/pre-inst-env $(GUILE) --no-auto-compile	\
+  "$(top_srcdir)/build-aux/convert-xref.scm"		\
+  $@.tmp $<
 endef
=20
-$(srcdir)/%D%/guix.%.texi: po/doc/guix-manual.%.po $(srcdir)/%D%/contribut=
ing.%.texi
+$(srcdir)/%D%/guix.%.texi: po/doc/guix-manual.%.po $(srcdir)/%D%/contribut=
ing.%.texi make-go
 	-$(AM_V_PO4A)$(PO4A_TRANSLATE) $(PO4A_PARAMS) -m "%D%/guix.texi" -p "$<" =
-l "$@.tmp"
 	-sed -i "s|guix\.info|$$(basename "$@" | sed 's|texi$$|info|')|" "$@.tmp"
 	-$(AM_V_POXREF)$(xref_command)
 	-mv "$@.tmp" "$@"
=20
-$(srcdir)/%D%/guix-cookbook.%.texi: po/doc/guix-cookbook.%.po
+$(srcdir)/%D%/guix-cookbook.%.texi: po/doc/guix-cookbook.%.po make-go
 	-$(AM_V_PO4A)$(PO4A_TRANSLATE) $(PO4A_PARAMS) -m "%D%/guix-cookbook.texi"=
 -p "$<" -l "$@.tmp"
 	-sed -i "s|guix-cookbook\.info|$$(basename "$@" | sed 's|texi$$|info|')|"=
 "$@.tmp"
 	-$(AM_V_POXREF)$(xref_command)
 	-mv "$@.tmp" "$@"
=20
-$(srcdir)/%D%/contributing.%.texi: po/doc/guix-manual.%.po
+$(srcdir)/%D%/contributing.%.texi: po/doc/guix-manual.%.po make-go
 	-$(AM_V_PO4A)$(PO4A_TRANSLATE) $(PO4A_PARAMS) -m "%D%/contributing.texi" =
-p "$<" -l "$@.tmp"
 	-$(AM_V_POXREF)$(xref_command)
 	-mv "$@.tmp" "$@"
diff --git a/guix/build/po.scm b/guix/build/po.scm
index eb9690ad1a..7f88164cd8 100644
--- a/guix/build/po.scm
+++ b/guix/build/po.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright =C2=A9 2019 Julien Lepiller <julien@HIDDEN>
+;;; Copyright =C2=A9 2019, 2021 Julien Lepiller <julien@HIDDEN>
 ;;; Copyright =C2=A9 2020 Ludovic Court=C3=A8s <ludo@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -20,17 +20,23 @@
 (define-module (guix build po)
   #:use-module (ice-9 match)
   #:use-module (ice-9 peg)
+  #:use-module (ice-9 regex)
   #:use-module (ice-9 textual-ports)
-  #:export (read-po-file))
+  #:use-module (ice-9 vlist)
+  #:use-module (srfi srfi-1)
+  #:export (read-po-file
+            translate-cross-references))
=20
 ;; A small parser for po files
-(define-peg-pattern po-file body (* (or comment entry whitespace)))
+(define-peg-pattern po-file body (* (or entry whitespace)))
 (define-peg-pattern whitespace body (or " " "\t" "\n"))
 (define-peg-pattern comment-chr body (range #\space #\=F0=AF=A7=BF))
 (define-peg-pattern comment none (and "#" (* comment-chr) "\n"))
+(define-peg-pattern flags all (and (ignore "#, ") (* comment-chr) (ignore =
"\n")))
 (define-peg-pattern entry all
-  (and (ignore (* whitespace)) (ignore "msgid ") msgid
-       (ignore (* whitespace)) (ignore "msgstr ") msgstr))
+  (and (* (or flags comment (ignore (* whitespace))))
+       (ignore "msgid ") msgid (ignore (* whitespace))
+       (ignore "msgstr ") msgstr))
 (define-peg-pattern escape body (or "\\\\" "\\\"" "\\n"))
 (define-peg-pattern str-chr body (or " " "!" (and (ignore "\\") "\"")
                                      "\\n" (and (ignore "\\") "\\")
@@ -53,7 +59,24 @@
                (append (list "\n" prefix) result)))))))
=20
 (define (parse-tree->assoc parse-tree)
-  "Converts a po PARSE-TREE to an association list."
+  "Converts a po PARSE-TREE to an association list, where the key is the m=
sgid
+and the value is the msgstr.  The result only contains non fuzzy strings."
+  (define (comments->flags comments)
+    (match comments
+      (('flags flags)
+       (map (lambda (flag) (string->symbol (string-trim-both flag #\space)=
))
+            (string-split flags #\,)))
+      ((? list? comments)
+       (fold
+         (lambda (comment res)
+           (match comment
+             ((? string? _) res)
+             (flags
+              (append (comments->flags flags)
+                      res))))
+         '()
+         comments))))
+
   (match parse-tree
     (() '())
     ((entry . parse-tree)
@@ -66,10 +89,22 @@
        ;; empty msgstr
        (('entry ('msgid msgid) 'msgstr)
         (parse-tree->assoc parse-tree))
+       (('entry _ ('msgid msgid) 'msgstr)
+        (parse-tree->assoc parse-tree))
        (('entry ('msgid msgid) ('msgstr msgstr))
         (acons (interpret-newline-escape msgid)
                (interpret-newline-escape msgstr)
-               (parse-tree->assoc parse-tree)))))))
+               (parse-tree->assoc parse-tree)))
+       (('entry ('msgid msgid) ('msgstr msgstr))
+        (acons (interpret-newline-escape msgid)
+               (interpret-newline-escape msgstr)
+               (parse-tree->assoc parse-tree)))
+       (('entry comments ('msgid msgid) ('msgstr msgstr))
+        (if (member 'fuzzy (comments->flags comments))
+            (parse-tree->assoc parse-tree)
+            (acons (interpret-newline-escape msgid)
+                   (interpret-newline-escape msgstr)
+                   (parse-tree->assoc parse-tree))))))))
=20
 (define (read-po-file port)
   "Read a .po file from PORT and return an alist of msgid and msgstr."
@@ -77,3 +112,71 @@
                           po-file
                           (get-string-all port)))))
     (parse-tree->assoc tree)))
+
+(define (canonicalize-whitespace str)
+  "Change whitespace (newlines, etc.) in STR to @code{#\\space}."
+  (string-map (lambda (chr)
+                (if (char-set-contains? char-set:whitespace chr)
+                    #\space
+                    chr))
+              str))
+
+(define xref-regexp
+  ;; Texinfo cross-reference regexp.
+  (make-regexp "@(px|x)?ref\\{([^,}]+)"))
+
+(define (translate-cross-references texi pofile)
+  "Translate the cross-references that appear in @var{texi}, the initial
+translation of a Texinfo file, using the msgid/msgstr pairs from @var{pofi=
le}."
+  (define translations
+    (call-with-input-file pofile read-po-file))
+
+  (define content
+    (call-with-input-file texi get-string-all))
+
+  (define matches
+    (list-matches xref-regexp content))
+
+  (define translation-map
+    (fold (match-lambda*
+            (((msgid . str) result)
+             (vhash-cons msgid str result)))
+          vlist-null
+          translations))
+
+  (define translated
+    ;; Iterate over MATCHES and replace cross-references with their
+    ;; translation found in TRANSLATION-MAP.  (We can't use
+    ;; 'substitute*' because matches can span multiple lines.)
+    (let loop ((matches matches)
+               (offset 0)
+               (result '()))
+      (match matches
+        (()
+         (string-concatenate-reverse
+          (cons (string-drop content offset) result)))
+        ((head . tail)
+         (let ((prefix (match:substring head 1))
+               (ref    (canonicalize-whitespace (match:substring head 2))))
+           (define translated
+             (string-append "@" (or prefix "")
+                            "ref{"
+                            (match (vhash-assoc ref translation-map)
+                              (#f ref)
+                              ((_ . str) str))))
+
+           (loop tail
+                 (match:end head)
+                 (append (list translated
+                               (string-take
+                                (string-drop content offset)
+                                (- (match:start head) offset)))
+                         result)))))))
+
+  (format (current-error-port)
+          "translated ~a cross-references in '~a'~%"
+          (length matches) texi)
+
+  (call-with-output-file texi
+    (lambda (port)
+      (display translated port))))
diff --git a/guix/self.scm b/guix/self.scm
index 61ff423086..fc581becd4 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -1,6 +1,7 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright =C2=A9 2017, 2018, 2019, 2020, 2021 Ludovic Court=C3=A8s <lu=
do@HIDDEN>
 ;;; Copyright =C2=A9 2020 Martin Becze <mjbecze@HIDDEN>
+;;; Copyright =C2=A9 2021 Julien Lepiller <julien@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -316,81 +317,23 @@ the result to OUTPUT."
                               chr))
                         str))
=20
-          (define xref-regexp
-            ;; Texinfo cross-reference regexp.
-            (make-regexp "@(px|x)?ref\\{([^,}]+)"))
-
-          (define (translate-cross-references texi translations)
-            ;; Translate the cross-references that appear in TEXI, a Texin=
fo
-            ;; file, using the msgid/msgstr pairs from TRANSLATIONS.
-            (define content
-              (call-with-input-file texi get-string-all))
-
-            (define matches
-              (list-matches xref-regexp content))
-
-            (define translation-map
-              (fold (match-lambda*
-                      (((msgid . str) result)
-                       (vhash-cons msgid str result)))
-                    vlist-null
-                    translations))
-
-            (define translated
-              ;; Iterate over MATCHES and replace cross-references with th=
eir
-              ;; translation found in TRANSLATION-MAP.  (We can't use
-              ;; 'substitute*' because matches can span multiple lines.)
-              (let loop ((matches matches)
-                         (offset 0)
-                         (result '()))
-                (match matches
-                  (()
-                   (string-concatenate-reverse
-                    (cons (string-drop content offset) result)))
-                  ((head . tail)
-                   (let ((prefix (match:substring head 1))
-                         (ref    (canonicalize-whitespace (match:substring=
 head 2))))
-                     (define translated
-                       (string-append "@" (or prefix "")
-                                      "ref{"
-                                      (match (vhash-assoc ref translation-=
map)
-                                        (#f ref)
-                                        ((_ . str) str))))
-
-                     (loop tail
-                           (match:end head)
-                           (append (list translated
-                                         (string-take
-                                          (string-drop content offset)
-                                          (- (match:start head) offset)))
-                                   result)))))))
-
-            (format (current-error-port)
-                    "translated ~a cross-references in '~a'~%"
-                    (length matches) texi)
-            (call-with-output-file texi
-              (lambda (port)
-                (display translated port))))
-
           (define* (translate-texi prefix po lang
                                    #:key (extras '()))
             "Translate the manual for one language LANG using the PO file.
 PREFIX must be the prefix of the manual, 'guix' or 'guix-cookbook'.  EXTRA=
S is
 a list of extra files, such as '(\"contributing\")."
-            (let ((translations (call-with-input-file po read-po-file)))
-              (for-each (lambda (file)
-                          (translate-tmp-texi po (string-append file ".tex=
i")
-                                              (string-append file "." lang
-                                                             ".texi.tmp")))
-                        (cons prefix extras))
+            (for-each (lambda (file)
+                        (translate-tmp-texi po (string-append file ".texi")
+                                            (string-append file "." lang
+                                                           ".texi.tmp")))
+                      (cons prefix extras))
=20
-              (for-each (lambda (file)
-                          (let* ((texi (string-append file "." lang ".texi=
"))
-                                 (tmp  (string-append texi ".tmp")))
-                            (copy-file tmp texi)
-                            (translate-cross-references texi
-                                                        translations)))
-                        (cons prefix extras))))
+            (for-each (lambda (file)
+                        (let* ((texi (string-append file "." lang ".texi"))
+                               (tmp  (string-append texi ".tmp")))
+                          (copy-file tmp texi)
+                          (translate-cross-references texi po)))
+                      (cons prefix extras)))
=20
           (define (available-translations directory domain)
             ;; Return the list of available translations under DIRECTORY f=
or
--=20
2.33.0


--MP_/rT0vi8_QcOgtQ61Cv5jhSxv--




Information forwarded to guix-patches@HIDDEN:
bug#51122; Package guix-patches. Full text available.

Message received at 51122 <at> debbugs.gnu.org:


Received: (at 51122) by debbugs.gnu.org; 11 Oct 2021 08:46:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 11 04:46:00 2021
Received: from localhost ([127.0.0.1]:56544 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mZqwZ-00005Y-QT
	for submit <at> debbugs.gnu.org; Mon, 11 Oct 2021 04:45:59 -0400
Received: from pelzflorian.de ([5.45.111.108]:60858 helo=mail.pelzflorian.de)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <pelzflorian@HIDDEN>) id 1mZqwV-00005O-Vw
 for 51122 <at> debbugs.gnu.org; Mon, 11 Oct 2021 04:45:58 -0400
Received: from pelzflorian.localdomain (unknown [5.45.111.108])
 by mail.pelzflorian.de (Postfix) with ESMTPSA id D243636069D;
 Mon, 11 Oct 2021 10:45:53 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=pelzflorian.de;
 s=mail; t=1633941954;
 bh=Vaeq6cTaxWUQd17t7cBG5PPZa8jjFjJ0kvkrN2E5mCg=;
 h=Date:From:To:Cc:Subject:References:In-Reply-To;
 b=zzHZRNbEb6F+GI+5YzVuvTs94L+wOMppQvFDGxPvgx/t/nYsl9n/rg7grZeDBVDeb
 Vc2AobVzCVPDfE/t1iFwxn0X6HATC7FOlbIzPnrOJFAY177km0C3Wk0j0VvZAXmQU6
 6vz0JsdXN5LguWpMXPMrpMztZ1dwe6p9ZWTe4dkI=
Date: Mon, 11 Oct 2021 10:45:45 +0200
From: "pelzflorian (Florian Pelz)" <pelzflorian@HIDDEN>
To: Julien Lepiller <julien@HIDDEN>
Subject: Re: [bug#51122] [PATCH] maint: Factorize po xref translation.
Message-ID: <20211011082415.j6wera4b5t4mtnxa@HIDDEN>
References: <8a210e8ceaadfa600e60b0b97a1e7aba04b2bb02.1633901513.git.julien@HIDDEN>
 <20211010233711.23bb1e9e@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
In-Reply-To: <20211010233711.23bb1e9e@HIDDEN>
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 51122
Cc: 51122 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Julien!  Thank you for replacing that hard-to-read sed script.
65;6003;1c
On Sun, Oct 10, 2021 at 11:37:11PM +0200, Julien Lepiller wrote:
> For info, I built "make" and "make as-derivation" successfully with
> this patch. I need it because with recent changes in guix.texi, a lot
> of text has become fuzzy, including node names, which are translated by
> the current xref_command (in doc/local.mk), when they shouldn't. I
> improved the method we used for guix pull, and used it instead.

I think you should add to the commit message that your patch Fixes
fuzzy translations being inserted.

Your changes to guix/build/po.scm do more than just move code (which
you mentioned in the commit message): you added parsing of “#, fuzzy”
flags.

You should update the copyright header for guix/build/po.scm.

Otherwise LGTM, but I haven’t tested yet.  (make now converts the xref
after scheme modules are compiled, I think(?), so testing PO files
takes longer for me on a clean checkout.  But it is fine.)

Regards,
Florian




Information forwarded to guix-patches@HIDDEN:
bug#51122; Package guix-patches. Full text available.

Message received at 51122 <at> debbugs.gnu.org:


Received: (at 51122) by debbugs.gnu.org; 10 Oct 2021 21:37:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 10 17:37:28 2021
Received: from localhost ([127.0.0.1]:55934 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mZgVc-0001ds-4D
	for submit <at> debbugs.gnu.org; Sun, 10 Oct 2021 17:37:28 -0400
Received: from lepiller.eu ([89.234.186.109]:35264)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <julien@HIDDEN>) id 1mZgVa-0001dj-7c
 for 51122 <at> debbugs.gnu.org; Sun, 10 Oct 2021 17:37:26 -0400
Received: from lepiller.eu (localhost [127.0.0.1])
 by lepiller.eu (OpenSMTPD) with ESMTP id 393fea5a
 for <51122 <at> debbugs.gnu.org>; Sun, 10 Oct 2021 21:37:23 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=date:from
 :to:subject:message-id:in-reply-to:references:mime-version
 :content-type:content-transfer-encoding; s=dkim; bh=13p/gG5aUJSz
 Vazq7FgcEL+WJjgcD/L3833/kHU65us=; b=UN8YAsAX4nhahKfxktkcU+5uPTJ/
 BENfhLKSXQkS8/yFD16IuXhpfdTlMRkSRcKWAyVYeYGMPn8oPiYp/+gW3Aqf2cbz
 JrFDq3aaL1aRKNidABa5k/OVWmaDg45xu+BWZAvbwypL41vViyzzFzMIRYFXuq1i
 5TAutUZRPurYVcC7ejQeAoWIjDV3Mp0/JXfKaSFD3hDHUtMaSt4fWkc2su+nCfBr
 gU5KSUGzq6Y/w/+keTDaVUvQrM6kottmNxnetMkOC436OyALfiC3UF6f3TxYS2em
 rJWOJ+NQqcjXmfrmc2jMXcuZeFBRQUoNU0thC58e6ktqujBKdZY95aVmrQ==
Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id 139a3954
 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for <51122 <at> debbugs.gnu.org>;
 Sun, 10 Oct 2021 21:37:23 +0000 (UTC)
Date: Sun, 10 Oct 2021 23:37:11 +0200
From: Julien Lepiller <julien@HIDDEN>
To: 51122 <at> debbugs.gnu.org
Subject: Re: [bug#51122] [PATCH] maint: Factorize po xref translation.
Message-ID: <20211010233711.23bb1e9e@HIDDEN>
In-Reply-To: <8a210e8ceaadfa600e60b0b97a1e7aba04b2bb02.1633901513.git.julien@HIDDEN>
References: <8a210e8ceaadfa600e60b0b97a1e7aba04b2bb02.1633901513.git.julien@HIDDEN>
X-Mailer: Claws Mail 3.18.0 (GTK+ 2.24.32; x86_64-pc-linux-gnu)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 51122
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

For info, I built "make" and "make as-derivation" successfully with
this patch. I need it because with recent changes in guix.texi, a lot
of text has become fuzzy, including node names, which are translated by
the current xref_command (in doc/local.mk), when they shouldn't. I
improved the method we used for guix pull, and used it instead.




Information forwarded to guix-patches@HIDDEN:
bug#51122; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 10 Oct 2021 21:33:16 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 10 17:33:16 2021
Received: from localhost ([127.0.0.1]:55927 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mZgRX-0001Xm-VA
	for submit <at> debbugs.gnu.org; Sun, 10 Oct 2021 17:33:16 -0400
Received: from lists.gnu.org ([209.51.188.17]:48024)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <julien@HIDDEN>) id 1mZgRT-0001Xc-Bo
 for submit <at> debbugs.gnu.org; Sun, 10 Oct 2021 17:33:14 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:53190)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <julien@HIDDEN>)
 id 1mZgRS-00055P-NT
 for guix-patches@HIDDEN; Sun, 10 Oct 2021 17:33:11 -0400
Received: from lepiller.eu ([89.234.186.109]:33194)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <julien@HIDDEN>)
 id 1mZgRP-0007qP-Ja
 for guix-patches@HIDDEN; Sun, 10 Oct 2021 17:33:10 -0400
Received: from lepiller.eu (localhost [127.0.0.1])
 by lepiller.eu (OpenSMTPD) with ESMTP id 66f1dfe7
 for <guix-patches@HIDDEN>; Sun, 10 Oct 2021 21:32:56 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=lepiller.eu; h=from:to
 :subject:date:message-id:mime-version:content-type
 :content-transfer-encoding; s=dkim; bh=WIXRb5FE/3JUfiw96S0Atow+8
 NSdZvwizOnoanlYzTA=; b=QMzYKQDi1PYfKMq7Ynjidcs0i9mTIKtCcPC0qfNeN
 9yvJ01JavZcshEtQNbd3h+Vs7kuFp9PbLNMphwLd6s8QKqdnGuJCrPGd+gvFQJei
 fQK/RIRFWkkA34gMN/8wgcXhixZ7Z+OxV+k8bvgAcfsCDW2LfM/NZ7QGGpWF2Y+H
 1UxPoE2QFjCJTLvSqLLlQfSnzh+K60RNmMAwYN3Q/U8wOF/SKni5h5dQbYxWwJmV
 X4iUdn+wnWkCGoi4KIUs3dR/1tl+uf1bUNeTVyDQFlBvKeSBEbkXgxuS/Zs4/QU9
 m7nXp7xt9CfgkM9LsDqwgr71V/0TknHUAY7F5C9n7u3Jg==
Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id ed9c56a7
 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO) for <guix-patches@HIDDEN>;
 Sun, 10 Oct 2021 21:32:56 +0000 (UTC)
From: Julien Lepiller <julien@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] maint: Factorize po xref translation.
Date: Sun, 10 Oct 2021 23:32:50 +0200
Message-Id: <8a210e8ceaadfa600e60b0b97a1e7aba04b2bb02.1633901513.git.julien@HIDDEN>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=89.234.186.109; envelope-from=julien@HIDDEN;
 helo=lepiller.eu
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

This ensures we use the same method in "make" as in "guix/self.scm".

* Makefile.am: Build guix/build/po.scm.
* build-aux/convert-xref.scm: New file.
* doc/local.mk (xref_command): Use it.
* guix/self.scm (translate-cross-references): Move it...
* guix/build/po.scm (translate-cross-references): ...here.
---
 Makefile.am                |   3 +-
 build-aux/convert-xref.scm |  26 +++++++++
 doc/local.mk               |  29 +++-------
 guix/build/po.scm          | 115 +++++++++++++++++++++++++++++++++++--
 guix/self.scm              |  80 ++++----------------------
 5 files changed, 155 insertions(+), 98 deletions(-)
 create mode 100644 build-aux/convert-xref.scm

diff --git a/Makefile.am b/Makefile.am
index 635147efc1..dc53b2c810 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -69,7 +69,6 @@ nodist_noinst_SCRIPTS =				\
 # Modules that are not compiled but are installed nonetheless, such as
 # build-side modules with unusual dependencies.
 MODULES_NOT_COMPILED =				\
-  guix/build/po.scm				\
   guix/man-db.scm
 
 include gnu/local.mk
@@ -227,6 +226,7 @@ MODULES =					\
   guix/build/pack.scm				\
   guix/build/utils.scm				\
   guix/build/union.scm				\
+  guix/build/po.scm				\
   guix/build/profiles.scm			\
   guix/build/compile.scm			\
   guix/build/rpath.scm				\
@@ -640,6 +640,7 @@ EXTRA_DIST +=						\
   build-aux/check-final-inputs-self-contained.scm	\
   build-aux/check-channel-news.scm			\
   build-aux/compile-as-derivation.scm			\
+  build-aux/convert-xref.scm			\
   build-aux/generate-authors.scm			\
   build-aux/test-driver.scm				\
   build-aux/update-guix-package.scm			\
diff --git a/build-aux/convert-xref.scm b/build-aux/convert-xref.scm
new file mode 100644
index 0000000000..47c8828857
--- /dev/null
+++ b/build-aux/convert-xref.scm
@@ -0,0 +1,26 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Julien Lepiller <julien@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+;; Translate cross-references in a translated .texi manual.
+
+(use-modules (guix build po)
+             (ice-9 match))
+
+(match (command-line)
+  ((program texi pofile)
+   (translate-cross-references texi pofile)))
diff --git a/doc/local.mk b/doc/local.mk
index 8340b75a87..88f7e67155 100644
--- a/doc/local.mk
+++ b/doc/local.mk
@@ -97,44 +97,29 @@ PO4A_PARAMS += -k 0 # produce an output even if the translation is not complete
 PO4A_PARAMS += -f texinfo # texinfo format
 
 # When a change to guix.texi occurs, it is not translated immediately.
-# Because @pxref and @xref commands are reference to a section by name, they
+# Because @pxref and @xref commands are references to sections by name, they
 # should be translated. If a modification adds a reference to a section, this
 # reference is not translated, which means it references a section that does not
 # exist.
-# This command loops through the translated files looking for references. For
-# each of these references, it tries to find the translation and replaces the
-# reference name, even in untranslated strings.
-# The last sed is a multiline sed because some references span multiple lines.
 define xref_command
-cat "$@.tmp" | egrep '@p?x?ref' -A1 | sed 'N;s|--\n||g;P;D' | sed 's|^| |g' | \
-        tr -d '\012' | sed 's|\(@p\?x\?ref\)|\n\1|g' | egrep '@p?x?ref' | \
-        sed 's|^.*@p\?x\?ref{\([^,}]*\).*$$|\1|g' | sort | uniq | while read e; do \
-    if [ -n "$$e" ]; then \
-      line=$$(grep -n "^msgid \"$$e\"" "$<" | cut -f1 --delimiter=":") ;\
-      ((line++)) ;\
-      if [ "$$line" != "1" ]; then \
-	translation=$$(head -n "$$line" "$<" | tail -1 | grep msgstr | sed 's|msgstr "\([^"]*\)"|\1|') ;\
-	if [ "$$translation" != "" ]; then \
-	      sed "N;s@\(p\?x\?ref\){$$(echo $$e | sed 's| |[\\n ]|g')\(,\|}\)@\1{$$translation\2@g;P;D" -i "$@.tmp" ;\
-	fi ;\
-      fi ;\
-   fi ;\
-done
+$(top_srcdir)/pre-inst-env $(GUILE) --no-auto-compile	\
+  "$(top_srcdir)/build-aux/convert-xref.scm"		\
+  $@.tmp $<
 endef
 
-$(srcdir)/%D%/guix.%.texi: po/doc/guix-manual.%.po $(srcdir)/%D%/contributing.%.texi
+$(srcdir)/%D%/guix.%.texi: po/doc/guix-manual.%.po $(srcdir)/%D%/contributing.%.texi make-go
 	-$(AM_V_PO4A)$(PO4A_TRANSLATE) $(PO4A_PARAMS) -m "%D%/guix.texi" -p "$<" -l "$@.tmp"
 	-sed -i "s|guix\.info|$$(basename "$@" | sed 's|texi$$|info|')|" "$@.tmp"
 	-$(AM_V_POXREF)$(xref_command)
 	-mv "$@.tmp" "$@"
 
-$(srcdir)/%D%/guix-cookbook.%.texi: po/doc/guix-cookbook.%.po
+$(srcdir)/%D%/guix-cookbook.%.texi: po/doc/guix-cookbook.%.po make-go
 	-$(AM_V_PO4A)$(PO4A_TRANSLATE) $(PO4A_PARAMS) -m "%D%/guix-cookbook.texi" -p "$<" -l "$@.tmp"
 	-sed -i "s|guix-cookbook\.info|$$(basename "$@" | sed 's|texi$$|info|')|" "$@.tmp"
 	-$(AM_V_POXREF)$(xref_command)
 	-mv "$@.tmp" "$@"
 
-$(srcdir)/%D%/contributing.%.texi: po/doc/guix-manual.%.po
+$(srcdir)/%D%/contributing.%.texi: po/doc/guix-manual.%.po make-go
 	-$(AM_V_PO4A)$(PO4A_TRANSLATE) $(PO4A_PARAMS) -m "%D%/contributing.texi" -p "$<" -l "$@.tmp"
 	-$(AM_V_POXREF)$(xref_command)
 	-mv "$@.tmp" "$@"
diff --git a/guix/build/po.scm b/guix/build/po.scm
index eb9690ad1a..a167252244 100644
--- a/guix/build/po.scm
+++ b/guix/build/po.scm
@@ -20,17 +20,23 @@
 (define-module (guix build po)
   #:use-module (ice-9 match)
   #:use-module (ice-9 peg)
+  #:use-module (ice-9 regex)
   #:use-module (ice-9 textual-ports)
-  #:export (read-po-file))
+  #:use-module (ice-9 vlist)
+  #:use-module (srfi srfi-1)
+  #:export (read-po-file
+            translate-cross-references))
 
 ;; A small parser for po files
-(define-peg-pattern po-file body (* (or comment entry whitespace)))
+(define-peg-pattern po-file body (* (or entry whitespace)))
 (define-peg-pattern whitespace body (or " " "\t" "\n"))
 (define-peg-pattern comment-chr body (range #\space #\頋))
 (define-peg-pattern comment none (and "#" (* comment-chr) "\n"))
+(define-peg-pattern flags all (and (ignore "#, ") (* comment-chr) (ignore "\n")))
 (define-peg-pattern entry all
-  (and (ignore (* whitespace)) (ignore "msgid ") msgid
-       (ignore (* whitespace)) (ignore "msgstr ") msgstr))
+  (and (* (or flags comment (ignore (* whitespace))))
+       (ignore "msgid ") msgid (ignore (* whitespace))
+       (ignore "msgstr ") msgstr))
 (define-peg-pattern escape body (or "\\\\" "\\\"" "\\n"))
 (define-peg-pattern str-chr body (or " " "!" (and (ignore "\\") "\"")
                                      "\\n" (and (ignore "\\") "\\")
@@ -53,7 +59,24 @@
                (append (list "\n" prefix) result)))))))
 
 (define (parse-tree->assoc parse-tree)
-  "Converts a po PARSE-TREE to an association list."
+  "Converts a po PARSE-TREE to an association list, where the key is the msgid
+and the value is the msgstr.  The result only contains non fuzzy strings."
+  (define (comments->flags comments)
+    (match comments
+      (('flags flags)
+       (map (lambda (flag) (string->symbol (string-trim-both flag #\space)))
+            (string-split flags #\,)))
+      ((? list? comments)
+       (fold
+         (lambda (comment res)
+           (match comment
+             ((? string? _) res)
+             (flags
+              (append (comments->flags flags)
+                      res))))
+         '()
+         comments))))
+
   (match parse-tree
     (() '())
     ((entry . parse-tree)
@@ -66,10 +89,22 @@
        ;; empty msgstr
        (('entry ('msgid msgid) 'msgstr)
         (parse-tree->assoc parse-tree))
+       (('entry _ ('msgid msgid) 'msgstr)
+        (parse-tree->assoc parse-tree))
        (('entry ('msgid msgid) ('msgstr msgstr))
         (acons (interpret-newline-escape msgid)
                (interpret-newline-escape msgstr)
-               (parse-tree->assoc parse-tree)))))))
+               (parse-tree->assoc parse-tree)))
+       (('entry ('msgid msgid) ('msgstr msgstr))
+        (acons (interpret-newline-escape msgid)
+               (interpret-newline-escape msgstr)
+               (parse-tree->assoc parse-tree)))
+       (('entry comments ('msgid msgid) ('msgstr msgstr))
+        (if (member 'fuzzy (comments->flags comments))
+            (parse-tree->assoc parse-tree)
+            (acons (interpret-newline-escape msgid)
+                   (interpret-newline-escape msgstr)
+                   (parse-tree->assoc parse-tree))))))))
 
 (define (read-po-file port)
   "Read a .po file from PORT and return an alist of msgid and msgstr."
@@ -77,3 +112,71 @@
                           po-file
                           (get-string-all port)))))
     (parse-tree->assoc tree)))
+
+(define (canonicalize-whitespace str)
+  "Change whitespace (newlines, etc.) in STR to @code{#\\space}."
+  (string-map (lambda (chr)
+                (if (char-set-contains? char-set:whitespace chr)
+                    #\space
+                    chr))
+              str))
+
+(define xref-regexp
+  ;; Texinfo cross-reference regexp.
+  (make-regexp "@(px|x)?ref\\{([^,}]+)"))
+
+(define (translate-cross-references texi pofile)
+  "Translate the cross-references that appear in @var{texi}, the initial
+translation of a Texinfo file, using the msgid/msgstr pairs from @var{pofile}."
+  (define translations
+    (call-with-input-file pofile read-po-file))
+
+  (define content
+    (call-with-input-file texi get-string-all))
+
+  (define matches
+    (list-matches xref-regexp content))
+
+  (define translation-map
+    (fold (match-lambda*
+            (((msgid . str) result)
+             (vhash-cons msgid str result)))
+          vlist-null
+          translations))
+
+  (define translated
+    ;; Iterate over MATCHES and replace cross-references with their
+    ;; translation found in TRANSLATION-MAP.  (We can't use
+    ;; 'substitute*' because matches can span multiple lines.)
+    (let loop ((matches matches)
+               (offset 0)
+               (result '()))
+      (match matches
+        (()
+         (string-concatenate-reverse
+          (cons (string-drop content offset) result)))
+        ((head . tail)
+         (let ((prefix (match:substring head 1))
+               (ref    (canonicalize-whitespace (match:substring head 2))))
+           (define translated
+             (string-append "@" (or prefix "")
+                            "ref{"
+                            (match (vhash-assoc ref translation-map)
+                              (#f ref)
+                              ((_ . str) str))))
+
+           (loop tail
+                 (match:end head)
+                 (append (list translated
+                               (string-take
+                                (string-drop content offset)
+                                (- (match:start head) offset)))
+                         result)))))))
+
+  (format (current-error-port)
+          "translated ~a cross-references in '~a'~%"
+          (length matches) texi)
+
+  (call-with-output-file texi
+    (lambda (port)
+      (display translated port))))
diff --git a/guix/self.scm b/guix/self.scm
index 61ff423086..769e0e8a7c 100644
--- a/guix/self.scm
+++ b/guix/self.scm
@@ -316,81 +316,23 @@ the result to OUTPUT."
                               chr))
                         str))
 
-          (define xref-regexp
-            ;; Texinfo cross-reference regexp.
-            (make-regexp "@(px|x)?ref\\{([^,}]+)"))
-
-          (define (translate-cross-references texi translations)
-            ;; Translate the cross-references that appear in TEXI, a Texinfo
-            ;; file, using the msgid/msgstr pairs from TRANSLATIONS.
-            (define content
-              (call-with-input-file texi get-string-all))
-
-            (define matches
-              (list-matches xref-regexp content))
-
-            (define translation-map
-              (fold (match-lambda*
-                      (((msgid . str) result)
-                       (vhash-cons msgid str result)))
-                    vlist-null
-                    translations))
-
-            (define translated
-              ;; Iterate over MATCHES and replace cross-references with their
-              ;; translation found in TRANSLATION-MAP.  (We can't use
-              ;; 'substitute*' because matches can span multiple lines.)
-              (let loop ((matches matches)
-                         (offset 0)
-                         (result '()))
-                (match matches
-                  (()
-                   (string-concatenate-reverse
-                    (cons (string-drop content offset) result)))
-                  ((head . tail)
-                   (let ((prefix (match:substring head 1))
-                         (ref    (canonicalize-whitespace (match:substring head 2))))
-                     (define translated
-                       (string-append "@" (or prefix "")
-                                      "ref{"
-                                      (match (vhash-assoc ref translation-map)
-                                        (#f ref)
-                                        ((_ . str) str))))
-
-                     (loop tail
-                           (match:end head)
-                           (append (list translated
-                                         (string-take
-                                          (string-drop content offset)
-                                          (- (match:start head) offset)))
-                                   result)))))))
-
-            (format (current-error-port)
-                    "translated ~a cross-references in '~a'~%"
-                    (length matches) texi)
-            (call-with-output-file texi
-              (lambda (port)
-                (display translated port))))
-
           (define* (translate-texi prefix po lang
                                    #:key (extras '()))
             "Translate the manual for one language LANG using the PO file.
 PREFIX must be the prefix of the manual, 'guix' or 'guix-cookbook'.  EXTRAS is
 a list of extra files, such as '(\"contributing\")."
-            (let ((translations (call-with-input-file po read-po-file)))
-              (for-each (lambda (file)
-                          (translate-tmp-texi po (string-append file ".texi")
-                                              (string-append file "." lang
-                                                             ".texi.tmp")))
-                        (cons prefix extras))
+            (for-each (lambda (file)
+                        (translate-tmp-texi po (string-append file ".texi")
+                                            (string-append file "." lang
+                                                           ".texi.tmp")))
+                      (cons prefix extras))
 
-              (for-each (lambda (file)
-                          (let* ((texi (string-append file "." lang ".texi"))
-                                 (tmp  (string-append texi ".tmp")))
-                            (copy-file tmp texi)
-                            (translate-cross-references texi
-                                                        translations)))
-                        (cons prefix extras))))
+            (for-each (lambda (file)
+                        (let* ((texi (string-append file "." lang ".texi"))
+                               (tmp  (string-append texi ".tmp")))
+                          (copy-file tmp texi)
+                          (translate-cross-references texi po)))
+                      (cons prefix extras)))
 
           (define (available-translations directory domain)
             ;; Return the list of available translations under DIRECTORY for
-- 
2.33.0





Acknowledgement sent to Julien Lepiller <julien@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#51122; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Mon, 11 Oct 2021 17:30:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.