GNU bug report logs - #41668
Failing test: gui-installed-desktop-os-encrypted

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Reported by: Brice Waegeneire <brice@HIDDEN>; dated Tue, 2 Jun 2020 18:53:02 UTC; Maintainer for guix is bug-guix@HIDDEN.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 9 Jun 2020 11:18:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 09 07:18:37 2020
Received: from localhost ([127.0.0.1]:59080 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jicH7-0003P1-8q
	for submit <at> debbugs.gnu.org; Tue, 09 Jun 2020 07:18:37 -0400
Received: from eggs.gnu.org ([209.51.188.92]:55298)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1jicH4-0003Om-9T
 for 41668 <at> debbugs.gnu.org; Tue, 09 Jun 2020 07:18:35 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:39429)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>)
 id 1jicGx-000350-J6; Tue, 09 Jun 2020 07:18:27 -0400
Received: from [2a01:e0a:fa:a50:d939:1174:2a3f:75c2] (port=51158 helo=meru)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <othacehe@HIDDEN>)
 id 1jicGw-0004Np-Px; Tue, 09 Jun 2020 07:18:27 -0400
From: Mathieu Othacehe <othacehe@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
 <87mu5glcse.fsf@HIDDEN> <877dwidhf0.fsf@HIDDEN>
 <87blluslc8.fsf@HIDDEN> <87zh9eaqq9.fsf@HIDDEN>
 <878sgxbshs.fsf@HIDDEN>
Date: Tue, 09 Jun 2020 13:18:25 +0200
In-Reply-To: <878sgxbshs.fsf@HIDDEN> (Mathieu Othacehe's message of "Mon, 08
 Jun 2020 15:24:15 +0200")
Message-ID: <87v9k0o5by.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, Brice Waegeneire <brice@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

--=-=-=
Content-Type: text/plain


Hey,

> Instead I did reimplement the command in (gnu installer utils) in the
> attached patch :).

There were an issue with exception handling, here's a v2. Note that it
uses the &invoke-error constructor that should be made public I guess.

Thanks,

Mathieu

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-installer-utils-Dump-command-output-to-syslog-when-t.patch

From 18754c8c62eabb341e0f710d83ff435ef950ca8e Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@HIDDEN>
Date: Mon, 8 Jun 2020 15:14:49 +0200
Subject: [PATCH] installer: utils: Dump command output to syslog when testing.

When debugging the installation tests, it can be very handy to be able to read
"run-command" output, for instance when executing "guix system init".

Introduce a new "invoke-with-log" procedure that is able to log a command
standard and error outputs to the syslog. Use it, only when running the
installation tests, to dump "run-command" output.

* gnu/installer/utils.scm (open-pipe-with-stderr, invoke-with-log): New
procedures,
(invoke-log-port): new variable,
(run-command): move to the end of the file and use invoke-with-log when
running the installation tests.
---
 gnu/installer/utils.scm | 164 +++++++++++++++++++++++++++++-----------
 1 file changed, 120 insertions(+), 44 deletions(-)

diff --git a/gnu/installer/utils.scm b/gnu/installer/utils.scm
index 5f8fe8ca01..68b3dd5009 100644
--- a/gnu/installer/utils.scm
+++ b/gnu/installer/utils.scm
@@ -22,8 +22,13 @@
   #:use-module (guix build utils)
   #:use-module (guix i18n)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
+  #:use-module (ice-9 popen)
   #:use-module (ice-9 rdelim)
   #:use-module (ice-9 regex)
   #:use-module (ice-9 format)
@@ -68,50 +73,6 @@ number. If no percentage is found, return #f"
     (and result
          (string->number (match:substring result 1)))))
 
-(define* (run-command command #:key locale)
-  "Run COMMAND, a list of strings, in the given LOCALE.  Return true if
-COMMAND exited successfully, #f otherwise."
-  (define env (environ))
-
-  (define (pause)
-    (format #t (G_ "Press Enter to continue.~%"))
-    (send-to-clients '(pause))
-    (environ env)                               ;restore environment variables
-    (match (select (cons (current-input-port) (current-clients))
-             '() '())
-      (((port _ ...) _ _)
-       (read-line port))))
-
-  (setenv "PATH" "/run/current-system/profile/bin")
-
-  (when locale
-    (let ((supported? (false-if-exception
-                       (setlocale LC_ALL locale))))
-      ;; If LOCALE is not supported, then set LANGUAGE, which might at
-      ;; least give us translated messages.
-      (if supported?
-          (setenv "LC_ALL" locale)
-          (setenv "LANGUAGE"
-                  (string-take locale
-                               (or (string-index locale #\_)
-                                   (string-length locale)))))))
-
-  (guard (c ((invoke-error? c)
-             (newline)
-             (format (current-error-port)
-                     (G_ "Command failed with exit code ~a.~%")
-                     (invoke-error-exit-status c))
-             (syslog "command ~s failed with exit code ~a"
-                     command (invoke-error-exit-status c))
-             (pause)
-             #f))
-    (syslog "running command ~s~%" command)
-    (apply invoke command)
-    (syslog "command ~s succeeded~%" command)
-    (newline)
-    (pause)
-    #t))
-
 
 ;;;
 ;;; Logging.
@@ -219,3 +180,118 @@ accepting socket."
 
   (current-clients (reverse remainder))
   exp)
+
+
+;;;
+;;; Run commands.
+;;;
+
+;; XXX: This is taken from (guix build utils) and could be factorized.
+(define (open-pipe-with-stderr program . args)
+  "Run PROGRAM with ARGS in an input pipe, but, unlike 'open-pipe*', redirect
+both its standard output and standard error to the pipe.  Return two value:
+the pipe to read PROGRAM's data from, and the PID of the child process running
+PROGRAM."
+  ;; 'open-pipe*' doesn't attempt to capture stderr in any way, which is why
+  ;; we need to roll our own.
+  (match (pipe)
+    ((input .  output)
+     (match (primitive-fork)
+       (0
+        (dynamic-wind
+          (const #t)
+          (lambda ()
+            (close-port input)
+            (close-port (syslog-port))
+            (dup2 (fileno output) 1)
+            (dup2 (fileno output) 2)
+            (apply execlp program program args))
+          (lambda ()
+            (primitive-exit 127))))
+       (pid
+        (close-port output)
+        (values input pid))))))
+
+(define invoke-log-port
+  ;; Port used by INVOKE-WITH-LOG for logging.
+  (make-parameter #f))
+
+(define* (invoke-with-log program . args)
+  "Invoke PROGRAM with ARGS and log PROGRAM's standard output and standard
+error to INVOKE-LOG-PORT.  If PROGRAM succeeds, print nothing and return the
+unspecified value; otherwise, raise a '&message' error condition with the
+status code.  This procedure is very similar to INVOKE/QUIET with the
+noticeable difference that the program output, that can be quite heavy, is not
+stored but directly sent to INVOKE-LOG-PORT if defined."
+  (let-values (((pipe pid)
+                (apply open-pipe-with-stderr program args)))
+    (let loop ()
+      (match (read-line pipe)
+        ((? eof-object?)
+         (close-port pipe)
+         (match (waitpid pid)
+           ((_ . status)
+            (unless (zero? status)
+              (raise
+               (condition ((@@ (guix build utils) &invoke-error)
+                           (program program)
+                           (arguments args)
+                           (exit-status (status:exit-val status))
+                           (term-signal (status:term-sig status))
+                           (stop-signal (status:stop-sig status)))))))))
+        (line
+         (and=> (invoke-log-port) (cut format <> "~a~%" line))
+         (loop))))))
+
+(define* (run-command command #:key locale)
+  "Run COMMAND, a list of strings, in the given LOCALE.  Return true if
+COMMAND exited successfully, #f otherwise."
+  (define env (environ))
+
+  (define (pause)
+    (format #t (G_ "Press Enter to continue.~%"))
+    (send-to-clients '(pause))
+    (environ env)                               ;restore environment variables
+    (match (select (cons (current-input-port) (current-clients))
+             '() '())
+      (((port _ ...) _ _)
+       (read-line port))))
+
+  (setenv "PATH" "/run/current-system/profile/bin")
+
+  (when locale
+    (let ((supported? (false-if-exception
+                       (setlocale LC_ALL locale))))
+      ;; If LOCALE is not supported, then set LANGUAGE, which might at
+      ;; least give us translated messages.
+      (if supported?
+          (setenv "LC_ALL" locale)
+          (setenv "LANGUAGE"
+                  (string-take locale
+                               (or (string-index locale #\_)
+                                   (string-length locale)))))))
+
+  (guard (c ((invoke-error? c)
+             (newline)
+             (format (current-error-port)
+                     (G_ "Command failed with exit code ~a.~%")
+                     (invoke-error-exit-status c))
+             (syslog "command ~s failed with exit code ~a"
+                     command (invoke-error-exit-status c))
+             (pause)
+             #f))
+    (syslog "running command ~s~%" command)
+    ;; If there are any connected clients, assume that we are running
+    ;; installation tests. In that case, dump the standard and error outputs
+    ;; to syslog.
+    (let ((testing? (not (null? (current-clients)))))
+      (if testing?
+          (parameterize ((invoke-log-port (syslog-port)))
+            (apply invoke-with-log command))
+          (apply invoke command)))
+    (syslog "command ~s succeeded~%" command)
+    (newline)
+    (pause)
+    #t))
+
+;;; utils.scm ends here
-- 
2.26.2


--=-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 8 Jun 2020 14:26:09 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 08 10:26:09 2020
Received: from localhost ([127.0.0.1]:57596 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jiIj2-0003yt-Tf
	for submit <at> debbugs.gnu.org; Mon, 08 Jun 2020 10:26:09 -0400
Received: from dd26836.kasserver.com ([85.13.145.193]:48604)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dannym@HIDDEN>) id 1jiIj1-0003yl-Hy
 for 41668 <at> debbugs.gnu.org; Mon, 08 Jun 2020 10:26:08 -0400
Received: from localhost (80-110-127-207.cgn.dynamic.surfer.at
 [80.110.127.207])
 by dd26836.kasserver.com (Postfix) with ESMTPSA id E334D336023C;
 Mon,  8 Jun 2020 16:26:05 +0200 (CEST)
Date: Mon, 8 Jun 2020 16:26:02 +0200
From: Danny Milosavljevic <dannym@HIDDEN>
To: Ludovic =?ISO-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
Message-ID: <20200608162602.5bff8512@HIDDEN>
In-Reply-To: <87blluslc8.fsf@HIDDEN>
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
 <87mu5glcse.fsf@HIDDEN> <877dwidhf0.fsf@HIDDEN>
 <87blluslc8.fsf@HIDDEN>
X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/BI0ukv.v6ANM8RPK21_XhEW";
 protocol="application/pgp-signature"; micalg=pgp-sha512
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, Mathieu Othacehe <othacehe@HIDDEN>,
 Brice Waegeneire <brice@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

--Sig_/BI0ukv.v6ANM8RPK21_XhEW
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Hi,

my CI setup for work automatically mails me failures.

It would be really nice to have that for guix master eventually, too.
Maybe just mail failures to guix-devel as they happen.

I don't check https://ci.guix.gnu.org/ so often, and even when I do the job=
set
names are kinda weird there, and there's too few stuff on one page--and jus=
t in
general there's no good overview on there.  I mean I can search, but a serv=
er
can just automate that and just send me the results as they happen.

It could also automatically mail failures to the last commiters of the sour=
ce
files that are relevant--but that's probably difficult to implement.

Or even use guix data service in order to find out where it failed exactly
and whether it does so for all archs--but maybe later.

--Sig_/BI0ukv.v6ANM8RPK21_XhEW
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl7eSnoACgkQ5xo1VCww
uqWEAAgAlF03g3wh2NZJn4ZX59n1prZ/R2ESFOL6cOGkJKQ6e4GfQwJgThynjR3k
z/cH+bDJ9gARsTMBvBDK1SPg0sr1AgykfuCXd568zCShqN4gE8Hwxo8AAsaKkzZk
pnU+myQGzcCX9d8hJlibkO9L1IaHe0Q/3XWHJoa8MrRRtVo4CNDmOAb8ITDb9oyw
IJ8iIXsoPd0WyYKvP2uCcm1z9jo8o1jAYzONOWHiiEPJDVtOkLfg3nKUCiAZtlcm
1PG/PwlyBmQXaeYjYvQB3EQbtOQ1/WUi4z2YahpnQYF9zJqIsnlQlBUdnh6ufI/Z
U0h5n/CdeeqTr/kUkU76ZrtWEYfMYQ==
=3Dyx
-----END PGP SIGNATURE-----

--Sig_/BI0ukv.v6ANM8RPK21_XhEW--




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 8 Jun 2020 13:24:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 08 09:24:30 2020
Received: from localhost ([127.0.0.1]:56198 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jiHlO-00026B-Bm
	for submit <at> debbugs.gnu.org; Mon, 08 Jun 2020 09:24:30 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39884)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1jiHlM-00025z-OE
 for 41668 <at> debbugs.gnu.org; Mon, 08 Jun 2020 09:24:29 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:48223)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>)
 id 1jiHlG-0004OB-RE; Mon, 08 Jun 2020 09:24:22 -0400
Received: from lfbn-ann-1-136-86.w86-200.abo.wanadoo.fr ([86.200.104.86]:45334
 helo=meru)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <othacehe@HIDDEN>)
 id 1jiHlD-0003sr-IH; Mon, 08 Jun 2020 09:24:20 -0400
From: Mathieu Othacehe <othacehe@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
 <87mu5glcse.fsf@HIDDEN> <877dwidhf0.fsf@HIDDEN>
 <87blluslc8.fsf@HIDDEN> <87zh9eaqq9.fsf@HIDDEN>
Date: Mon, 08 Jun 2020 15:24:15 +0200
In-Reply-To: <87zh9eaqq9.fsf@HIDDEN> (Mathieu Othacehe's message of "Mon, 08
 Jun 2020 10:47:42 +0200")
Message-ID: <878sgxbshs.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, Brice Waegeneire <brice@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

--=-=-=
Content-Type: text/plain


Hey,

Turns out using "invoke/quiet" is not really convenient because it
stores all the command output, which can be pretty big, in memory.

Instead I did reimplement the command in (gnu installer utils) in the
attached patch :).

Thanks,

Mathieu

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-installer-utils-Dump-command-output-to-syslog-when-t.patch

From 7c1a03b03d5e62876a88d87aff36776c8ce4ddf2 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@HIDDEN>
Date: Mon, 8 Jun 2020 15:14:49 +0200
Subject: [PATCH] installer: utils: Dump command output to syslog when testing.

When debugging the installation tests, it can be very handy to be able to read
"run-command" output, for instance when executing "guix system init".

Introduce a new "invoke-with-log" procedure that is able to log a command
standard and error outputs to the syslog. Use it, only when running the
installation tests, to dump "run-command" output.

* gnu/installer/utils.scm (open-pipe-with-stderr, invoke-with-log): New
procedures,
(invoke-log-port): new variable,
(run-command): move to the end of the file and use invoke-with-log when
running the installation tests.
---
 gnu/installer/utils.scm | 166 +++++++++++++++++++++++++++++-----------
 1 file changed, 122 insertions(+), 44 deletions(-)

diff --git a/gnu/installer/utils.scm b/gnu/installer/utils.scm
index 5f8fe8ca01..b9d88e46d9 100644
--- a/gnu/installer/utils.scm
+++ b/gnu/installer/utils.scm
@@ -22,8 +22,13 @@
   #:use-module (guix build utils)
   #:use-module (guix i18n)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
+  #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
   #:use-module (ice-9 match)
+  #:use-module (ice-9 popen)
   #:use-module (ice-9 rdelim)
   #:use-module (ice-9 regex)
   #:use-module (ice-9 format)
@@ -68,50 +73,6 @@ number. If no percentage is found, return #f"
     (and result
          (string->number (match:substring result 1)))))
 
-(define* (run-command command #:key locale)
-  "Run COMMAND, a list of strings, in the given LOCALE.  Return true if
-COMMAND exited successfully, #f otherwise."
-  (define env (environ))
-
-  (define (pause)
-    (format #t (G_ "Press Enter to continue.~%"))
-    (send-to-clients '(pause))
-    (environ env)                               ;restore environment variables
-    (match (select (cons (current-input-port) (current-clients))
-             '() '())
-      (((port _ ...) _ _)
-       (read-line port))))
-
-  (setenv "PATH" "/run/current-system/profile/bin")
-
-  (when locale
-    (let ((supported? (false-if-exception
-                       (setlocale LC_ALL locale))))
-      ;; If LOCALE is not supported, then set LANGUAGE, which might at
-      ;; least give us translated messages.
-      (if supported?
-          (setenv "LC_ALL" locale)
-          (setenv "LANGUAGE"
-                  (string-take locale
-                               (or (string-index locale #\_)
-                                   (string-length locale)))))))
-
-  (guard (c ((invoke-error? c)
-             (newline)
-             (format (current-error-port)
-                     (G_ "Command failed with exit code ~a.~%")
-                     (invoke-error-exit-status c))
-             (syslog "command ~s failed with exit code ~a"
-                     command (invoke-error-exit-status c))
-             (pause)
-             #f))
-    (syslog "running command ~s~%" command)
-    (apply invoke command)
-    (syslog "command ~s succeeded~%" command)
-    (newline)
-    (pause)
-    #t))
-
 
 ;;;
 ;;; Logging.
@@ -219,3 +180,120 @@ accepting socket."
 
   (current-clients (reverse remainder))
   exp)
+
+
+;;;
+;;; Run commands.
+;;;
+
+;; XXX: This is taken from (guix build utils) and could be factorized.
+(define (open-pipe-with-stderr program . args)
+  "Run PROGRAM with ARGS in an input pipe, but, unlike 'open-pipe*', redirect
+both its standard output and standard error to the pipe.  Return two value:
+the pipe to read PROGRAM's data from, and the PID of the child process running
+PROGRAM."
+  ;; 'open-pipe*' doesn't attempt to capture stderr in any way, which is why
+  ;; we need to roll our own.
+  (match (pipe)
+    ((input .  output)
+     (match (primitive-fork)
+       (0
+        (dynamic-wind
+          (const #t)
+          (lambda ()
+            (close-port input)
+            (close-port (syslog-port))
+            (dup2 (fileno output) 1)
+            (dup2 (fileno output) 2)
+            (apply execlp program program args))
+          (lambda ()
+            (primitive-exit 127))))
+       (pid
+        (close-port output)
+        (values input pid))))))
+
+(define invoke-log-port
+  ;; Port used by INVOKE-WITH-LOG for logging.
+  (make-parameter #f))
+
+(define* (invoke-with-log program . args)
+  "Invoke PROGRAM with ARGS and log PROGRAM's standard output and standard
+error to INVOKE-LOG-PORT.  If PROGRAM succeeds, print nothing and return the
+unspecified value; otherwise, raise a '&message' error condition with the
+status code.  This procedure is very similar to INVOKE/QUIET with the
+noticeable difference that the program output, that can be quite heavy, is not
+stored but directly sent to INVOKE-LOG-PORT if defined."
+  (let-values (((pipe pid)
+                (apply open-pipe-with-stderr program args)))
+    (let loop ()
+      (match (read-line pipe)
+        ((? eof-object?)
+         (close-port pipe)
+         (match (waitpid pid)
+           ((_ . status)
+            (unless (zero? status)
+              (let-syntax ((G_ (syntax-rules ()   ;for xgettext
+                                 ((_ str) str))))
+                (raise (condition
+                        (&message
+                         (message (format #f (G_ "'~a~{ ~a~}' exited \
+with status ~a.")
+                                          program args
+                                          (or (status:exit-val status)
+                                              status)))))))))))
+        (line
+         (and=> (invoke-log-port) (cut format <> "~a~%" line))
+         (loop))))))
+
+(define* (run-command command #:key locale)
+  "Run COMMAND, a list of strings, in the given LOCALE.  Return true if
+COMMAND exited successfully, #f otherwise."
+  (define env (environ))
+
+  (define (pause)
+    (format #t (G_ "Press Enter to continue.~%"))
+    (send-to-clients '(pause))
+    (environ env)                               ;restore environment variables
+    (match (select (cons (current-input-port) (current-clients))
+             '() '())
+      (((port _ ...) _ _)
+       (read-line port))))
+
+  (setenv "PATH" "/run/current-system/profile/bin")
+
+  (when locale
+    (let ((supported? (false-if-exception
+                       (setlocale LC_ALL locale))))
+      ;; If LOCALE is not supported, then set LANGUAGE, which might at
+      ;; least give us translated messages.
+      (if supported?
+          (setenv "LC_ALL" locale)
+          (setenv "LANGUAGE"
+                  (string-take locale
+                               (or (string-index locale #\_)
+                                   (string-length locale)))))))
+
+  (guard (c ((invoke-error? c)
+             (newline)
+             (format (current-error-port)
+                     (G_ "Command failed with exit code ~a.~%")
+                     (invoke-error-exit-status c))
+             (syslog "command ~s failed with exit code ~a"
+                     command (invoke-error-exit-status c))
+             (pause)
+             #f))
+    (syslog "running command ~s~%" command)
+    ;; If there are any connected clients, assume that we are running
+    ;; installation tests. In that case, dump the standard and error outputs
+    ;; to syslog.
+    (let ((testing? (not (null? (current-clients)))))
+      (if testing?
+          (parameterize ((invoke-log-port (syslog-port)))
+            (apply invoke-with-log command))
+          (apply invoke command)))
+    (syslog "command ~s succeeded~%" command)
+    (newline)
+    (pause)
+    #t))
+
+;;; utils.scm ends here
-- 
2.26.2


--=-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 8 Jun 2020 08:48:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 08 04:47:59 2020
Received: from localhost ([127.0.0.1]:55897 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jiDRn-0001Wg-Jh
	for submit <at> debbugs.gnu.org; Mon, 08 Jun 2020 04:47:59 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35820)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1jiDRl-0001WR-Vp
 for 41668 <at> debbugs.gnu.org; Mon, 08 Jun 2020 04:47:58 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:44265)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>)
 id 1jiDRf-00020c-8o; Mon, 08 Jun 2020 04:47:51 -0400
Received: from lfbn-ann-1-136-86.w86-200.abo.wanadoo.fr ([86.200.104.86]:42334
 helo=meru)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <othacehe@HIDDEN>)
 id 1jiDRe-0001cv-Fr; Mon, 08 Jun 2020 04:47:51 -0400
From: Mathieu Othacehe <othacehe@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
 <87mu5glcse.fsf@HIDDEN> <877dwidhf0.fsf@HIDDEN>
 <87blluslc8.fsf@HIDDEN>
Date: Mon, 08 Jun 2020 10:47:42 +0200
In-Reply-To: <87blluslc8.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Sun, 07 Jun 2020 21:54:31 +0200")
Message-ID: <87zh9eaqq9.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, Brice Waegeneire <brice@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)


Hey Ludo,

> How did you do it in the example above?  Perhaps we can have a local
> hack specifically in the installer (otherwise we=E2=80=99d find ourselves
> duplicating syslog-related logic there.)

I used a custom re-implementation of "invoke/quiet" printing every line
on the syslog port. Maybe, we could enable such a mechanism only during
the tests.

> I don=E2=80=99t have good ideas for (1), although I think the speedups in=
duced
> by the new image API might help.

Yes, that's for sure making the debug more comfortable!

> As for (2), I think it=E2=80=99s kind of a problem of CI in general, conn=
ected
> with the fact that we=E2=80=99re all somewhat busy building stuff rather =
than
> waiting for CI notifications to act on.  :-)
>
> That=E2=80=99s my impression, but I=E2=80=99d really like to hear from pe=
ople with
> experience in CI and/or the system tests.

Totally agree. When submitting patches to Linux for instance, the CI is
able to reply to the patch mail, and warn about any failures. An other
difference is that patches are often kept in the maintainer 'staging'
branches until the CI is over.

I don't know if Guix is a big enough project to appoint some maintainers
that would periodically merge their branches to 'master' or if it would
just be a huge burden. Something we need to discuss in the future maybe.

> Perfect, thanks!

Anyway, thanks for your fast review :)

Mathieu




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 7 Jun 2020 19:54:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 07 15:54:42 2020
Received: from localhost ([127.0.0.1]:54987 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1ji1NS-0003Pj-0E
	for submit <at> debbugs.gnu.org; Sun, 07 Jun 2020 15:54:42 -0400
Received: from eggs.gnu.org ([209.51.188.92]:52350)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1ji1NQ-0003PV-1W
 for 41668 <at> debbugs.gnu.org; Sun, 07 Jun 2020 15:54:40 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:57234)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1ji1NK-0005wB-02; Sun, 07 Jun 2020 15:54:34 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=40092 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1ji1NI-0005mZ-Ik; Sun, 07 Jun 2020 15:54:32 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Mathieu Othacehe <othacehe@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
 <87mu5glcse.fsf@HIDDEN> <877dwidhf0.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 20 Prairial an 228 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Sun, 07 Jun 2020 21:54:31 +0200
In-Reply-To: <877dwidhf0.fsf@HIDDEN> (Mathieu Othacehe's message of "Sun, 07
 Jun 2020 17:28:19 +0200")
Message-ID: <87blluslc8.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, Brice Waegeneire <brice@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

Hi,

Mathieu Othacehe <othacehe@HIDDEN> skribis:

>> * A new issue I discovered, whereby 'guix system init' try to
>> build/download stuff, when it shouldn't:
>>
>> Jun  6 11:50:06 localhost shepherd[1]: Service cow-store has been starte=
d.=20
>> Jun  6 11:50:06 localhost installer[233]: running command ("guix" "syste=
m" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.sc=
m" "/mnt")=20
>> Jun  6 11:50:20 localhost installer[233]: 'guix system init --fallback -=
-no-grafts --no-substitutes /mnt/etc/config.scm /mnt' exited with status 1;=
 output follows:    The following derivations will be bu)
>>
>> Jun  6 11:50:20 localhost 7-gzip-mesboot-1.2.4.drv      /gnu/store/zrqjn=
3p0x5gkdnj6qb4b9xxl2dld4fvl-gzip-1.2.4.tar.drv      /gnu/store/98g8d622pqhz=
l0z92y4bjcqi9yl0327x-linux-libre-headers-bootstrap-0.drv2
>> Jun  6 11:50:20 localhost g76sla-bash-mesboot-4.4.drv      /gnu/store/8y=
ad2zhh91pnf27xkwh0kyx4ggnsn5li-bash-4.4.tar.gz.drv      /gnu/store/5vp07s0g=
dvrybbhcz43i58y5p03rrdsk-coreutils-mesboot-8.32.drv     k
>> Jun  6 11:50:20 localhost h6mfp7hisj118ndcp8d0wy3ca63-gmp-6.0.0a.tar.xz.=
drv      /gnu/store/iy7vn17sqbrgp2142px897ij0njsr5f2-gmp-6.0.0a.tar.xz.drv =
     /gnu/store/rxr1wc67y74bsfasln4jw8viawpa79g4-mpfr-4.0
>> Jun  6 11:50:20 localhost ilding /gnu/store/3pz8g76l2kkv36w1n7k25zlswgdy=
j554-Python-3.5.9.tar.xz.drv...   builder for `/gnu/store/3pz8g76l2kkv36w1n=
7k25zlswgdyj554-Python-3.5.9.tar.xz.drv' failed to produg
>> Jun  6 11:50:20 localhost=20=20
>
> Ok, I think I found what's going on here. As we are now using multiple
> glibc to build locales, we need to make sure that they are all available
> in the installation closure.

Good catch!

> The attached patch fixes the issue. I will also propose a patch,
> probably on core-update, patching "invoke/quiet" so that we can have the
> output of "guix system init" in the syslog.

How did you do it in the example above?  Perhaps we can have a local
hack specifically in the installer (otherwise we=E2=80=99d find ourselves
duplicating syslog-related logic there.)

> Debugging (gnu tests install) module always takes me a lot of effort,
> because of all the code layers involved. I don't have much to propose to
> improve it, maybe when the CI will be more robust, we will be able to
> detect those test failures as soon as they happen.

Yeah there are two issues: (1) the installer tests are tricky to debug,
and (2) we basically don=E2=80=99t notice when something break.

I don=E2=80=99t have good ideas for (1), although I think the speedups indu=
ced
by the new image API might help.

As for (2), I think it=E2=80=99s kind of a problem of CI in general, connec=
ted
with the fact that we=E2=80=99re all somewhat busy building stuff rather th=
an
waiting for CI notifications to act on.  :-)

That=E2=80=99s my impression, but I=E2=80=99d really like to hear from peop=
le with
experience in CI and/or the system tests.

> From 25950adfa4988c31beaeb334367337ee409868c4 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe@HIDDEN>
> Date: Sun, 7 Jun 2020 15:39:39 +0200
> Subject: [PATCH] system: install: Add default libcs to gc-roots.
>
> During the installation, it may be needed to build locales that are not
> covered by glibc-utf8-locales. Make sure that the libraries required to b=
uild
> locales are available.
>
> This is a follow-up of 0eed77127592323d89f56c215a15374a1aaae110.
>
> * gnu/system/install.scm (installation-os)[services]: Add
> %default-locale-libcs to gc-root-service-type.

Perfect, thanks!

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 7 Jun 2020 15:28:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Jun 07 11:28:32 2020
Received: from localhost ([127.0.0.1]:54640 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jhxDs-000760-Cv
	for submit <at> debbugs.gnu.org; Sun, 07 Jun 2020 11:28:32 -0400
Received: from eggs.gnu.org ([209.51.188.92]:58398)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1jhxDp-00075n-G7
 for 41668 <at> debbugs.gnu.org; Sun, 07 Jun 2020 11:28:30 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:53437)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>)
 id 1jhxDj-0007qx-6U; Sun, 07 Jun 2020 11:28:23 -0400
Received: from [2a01:e0a:fa:a50:d939:1174:2a3f:75c2] (port=55272 helo=meru)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <othacehe@HIDDEN>)
 id 1jhxDi-0000Hf-0L; Sun, 07 Jun 2020 11:28:22 -0400
From: Mathieu Othacehe <othacehe@HIDDEN>
To: Brice Waegeneire <brice@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
 <87mu5glcse.fsf@HIDDEN>
Date: Sun, 07 Jun 2020 17:28:19 +0200
In-Reply-To: <87mu5glcse.fsf@HIDDEN> (Mathieu Othacehe's message of "Sat, 06
 Jun 2020 12:16:33 +0200")
Message-ID: <877dwidhf0.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, ludo@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

--=-=-=
Content-Type: text/plain


Hey,

> * A new issue I discovered, whereby 'guix system init' try to
> build/download stuff, when it shouldn't:
>
> Jun  6 11:50:06 localhost shepherd[1]: Service cow-store has been started. 
> Jun  6 11:50:06 localhost installer[233]: running command ("guix" "system" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.scm" "/mnt") 
> Jun  6 11:50:20 localhost installer[233]: 'guix system init --fallback --no-grafts --no-substitutes /mnt/etc/config.scm /mnt' exited with status 1; output follows:    The following derivations will be bu)
>
> Jun  6 11:50:20 localhost 7-gzip-mesboot-1.2.4.drv      /gnu/store/zrqjn3p0x5gkdnj6qb4b9xxl2dld4fvl-gzip-1.2.4.tar.drv      /gnu/store/98g8d622pqhzl0z92y4bjcqi9yl0327x-linux-libre-headers-bootstrap-0.drv2
> Jun  6 11:50:20 localhost g76sla-bash-mesboot-4.4.drv      /gnu/store/8yad2zhh91pnf27xkwh0kyx4ggnsn5li-bash-4.4.tar.gz.drv      /gnu/store/5vp07s0gdvrybbhcz43i58y5p03rrdsk-coreutils-mesboot-8.32.drv     k
> Jun  6 11:50:20 localhost h6mfp7hisj118ndcp8d0wy3ca63-gmp-6.0.0a.tar.xz.drv      /gnu/store/iy7vn17sqbrgp2142px897ij0njsr5f2-gmp-6.0.0a.tar.xz.drv      /gnu/store/rxr1wc67y74bsfasln4jw8viawpa79g4-mpfr-4.0
> Jun  6 11:50:20 localhost ilding /gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv...   builder for `/gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv' failed to produg
> Jun  6 11:50:20 localhost  

Ok, I think I found what's going on here. As we are now using multiple
glibc to build locales, we need to make sure that they are all available
in the installation closure.

The attached patch fixes the issue. I will also propose a patch,
probably on core-update, patching "invoke/quiet" so that we can have the
output of "guix system init" in the syslog.

Debugging (gnu tests install) module always takes me a lot of effort,
because of all the code layers involved. I don't have much to propose to
improve it, maybe when the CI will be more robust, we will be able to
detect those test failures as soon as they happen.

Now, the cow-store issue!

Thanks,

Mathieu

--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-system-install-Add-default-libcs-to-gc-roots.patch

From 25950adfa4988c31beaeb334367337ee409868c4 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe@HIDDEN>
Date: Sun, 7 Jun 2020 15:39:39 +0200
Subject: [PATCH] system: install: Add default libcs to gc-roots.

During the installation, it may be needed to build locales that are not
covered by glibc-utf8-locales. Make sure that the libraries required to build
locales are available.

This is a follow-up of 0eed77127592323d89f56c215a15374a1aaae110.

* gnu/system/install.scm (installation-os)[services]: Add
%default-locale-libcs to gc-root-service-type.
---
 gnu/system/install.scm | 11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/gnu/system/install.scm b/gnu/system/install.scm
index fe49ffdb94..d0ff2e7c52 100644
--- a/gnu/system/install.scm
+++ b/gnu/system/install.scm
@@ -32,6 +32,7 @@
   #:use-module ((guix packages) #:select (package-version))
   #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (gnu installer)
+  #:use-module (gnu system locale)
   #:use-module (gnu services dbus)
   #:use-module (gnu services networking)
   #:use-module (gnu services shepherd)
@@ -439,10 +440,12 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m
           ;; things needed by 'profile-derivation' to minimize the amount of
           ;; download.
           (service gc-root-service-type
-                   (list bare-bones-os
-                         glibc-utf8-locales
-                         texinfo
-                         guile-3.0))
+                   (append
+                    (list bare-bones-os
+                          glibc-utf8-locales
+                          texinfo
+                          guile-3.0)
+                    %default-locale-libcs))
 
           ;; Machines without Kernel Mode Setting (those with many old and
           ;; current AMD GPUs, SiS GPUs, ...) need uvesafb to show the GUI
-- 
2.26.2


--=-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at 41668 <at> debbugs.gnu.org:


Received: (at 41668) by debbugs.gnu.org; 6 Jun 2020 10:16:55 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jun 06 06:16:55 2020
Received: from localhost ([127.0.0.1]:50779 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jhVsi-0002x3-1o
	for submit <at> debbugs.gnu.org; Sat, 06 Jun 2020 06:16:55 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39532)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <othacehe@HIDDEN>) id 1jhVsY-0002wl-TE
 for 41668 <at> debbugs.gnu.org; Sat, 06 Jun 2020 06:16:50 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:58731)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <othacehe@HIDDEN>)
 id 1jhVsR-0002vh-UU; Sat, 06 Jun 2020 06:16:36 -0400
Received: from [2a01:e0a:fa:a50:809b:e323:9e90:279b] (port=39358 helo=meru)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <othacehe@HIDDEN>)
 id 1jhVsR-0007Rq-Cy; Sat, 06 Jun 2020 06:16:35 -0400
From: Mathieu Othacehe <othacehe@HIDDEN>
To: Brice Waegeneire <brice@HIDDEN>
Subject: Re: bug#41668: Failing test: gui-installed-desktop-os-encrypted
References: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
Date: Sat, 06 Jun 2020 12:16:33 +0200
In-Reply-To: <3f66774219bb18c47b305b8760f58abe@HIDDEN> (Brice
 Waegeneire's message of "Tue, 02 Jun 2020 18:52:04 +0000")
Message-ID: <87mu5glcse.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 41668
Cc: 41668 <at> debbugs.gnu.org, ludo@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


Hello Brice,

> For some time now the system test 'gui-installed-desktop-os-encrypted'
> doesn't succeed, the last time it passed was 2020-04-11[0]. This is
> problematic since it exercise a lot of packages and thus can be used to
> test those packages. Ludovic attempted to fixed it[1] but it didn't fixed
> the test on my side.

The "gui-installed-*" tests are in a bad shape right now. The good news
is that the can reveal many issues, the bad news is that I can't keep
the distance fixing them.

They are affected by:

* Shepherd race condition: https://issues.guix.gnu.org/40981. But this
should be fixed with the recent 0.8.1 release.

* The cow-store issue here:
https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00043.html.

* A new issue I discovered, whereby 'guix system init' try to
build/download stuff, when it shouldn't:

--8<---------------cut here---------------start------------->8---
Jun  6 11:50:06 localhost shepherd[1]: Service cow-store has been started. 
Jun  6 11:50:06 localhost installer[233]: running command ("guix" "system" "init" "--fallback" "--no-grafts" "--no-substitutes" "/mnt/etc/config.scm" "/mnt") 
Jun  6 11:50:20 localhost installer[233]: 'guix system init --fallback --no-grafts --no-substitutes /mnt/etc/config.scm /mnt' exited with status 1; output follows:    The following derivations will be bu)

Jun  6 11:50:20 localhost 7-gzip-mesboot-1.2.4.drv      /gnu/store/zrqjn3p0x5gkdnj6qb4b9xxl2dld4fvl-gzip-1.2.4.tar.drv      /gnu/store/98g8d622pqhzl0z92y4bjcqi9yl0327x-linux-libre-headers-bootstrap-0.drv2
Jun  6 11:50:20 localhost g76sla-bash-mesboot-4.4.drv      /gnu/store/8yad2zhh91pnf27xkwh0kyx4ggnsn5li-bash-4.4.tar.gz.drv      /gnu/store/5vp07s0gdvrybbhcz43i58y5p03rrdsk-coreutils-mesboot-8.32.drv     k
Jun  6 11:50:20 localhost h6mfp7hisj118ndcp8d0wy3ca63-gmp-6.0.0a.tar.xz.drv      /gnu/store/iy7vn17sqbrgp2142px897ij0njsr5f2-gmp-6.0.0a.tar.xz.drv      /gnu/store/rxr1wc67y74bsfasln4jw8viawpa79g4-mpfr-4.0
Jun  6 11:50:20 localhost ilding /gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv...   builder for `/gnu/store/3pz8g76l2kkv36w1n7k25zlswgdyj554-Python-3.5.9.tar.xz.drv' failed to produg
Jun  6 11:50:20 localhost  
--8<---------------cut here---------------end--------------->8---

Thanks,

Mathieu

  




Information forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 2 Jun 2020 18:52:12 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jun 02 14:52:12 2020
Received: from localhost ([127.0.0.1]:40959 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1jgC1D-0007A7-UP
	for submit <at> debbugs.gnu.org; Tue, 02 Jun 2020 14:52:12 -0400
Received: from lists.gnu.org ([209.51.188.17]:42260)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <brice@HIDDEN>) id 1jgC1C-00079z-HJ
 for submit <at> debbugs.gnu.org; Tue, 02 Jun 2020 14:52:10 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:56238)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <brice@HIDDEN>) id 1jgC1C-00077B-90
 for bug-guix@HIDDEN; Tue, 02 Jun 2020 14:52:10 -0400
Received: from relay2-d.mail.gandi.net ([217.70.183.194]:50479)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <brice@HIDDEN>)
 id 1jgC1A-0005w9-H9; Tue, 02 Jun 2020 14:52:09 -0400
Received: from webmail.gandi.net (webmail18.sd4.0x35.net [10.200.201.18])
 (Authenticated sender: brice@HIDDEN)
 by relay2-d.mail.gandi.net (Postfix) with ESMTPA id 6560440002;
 Tue,  2 Jun 2020 18:52:04 +0000 (UTC)
MIME-Version: 1.0
Content-Type: text/plain; charset=US-ASCII;
 format=flowed
Content-Transfer-Encoding: 7bit
Date: Tue, 02 Jun 2020 18:52:04 +0000
From: Brice Waegeneire <brice@HIDDEN>
To: bug-guix@HIDDEN
Subject: Failing test: gui-installed-desktop-os-encrypted
Message-ID: <3f66774219bb18c47b305b8760f58abe@HIDDEN>
X-Sender: brice@HIDDEN
User-Agent: Roundcube Webmail/1.3.8
Received-SPF: pass client-ip=217.70.183.194; envelope-from=brice@HIDDEN;
 helo=relay2-d.mail.gandi.net
X-detected-operating-system: by eggs.gnu.org: First seen = 2020/06/02 14:52:04
X-ACL-Warn: Detected OS   = Linux 3.11 and newer
X-Spam_score_int: -25
X-Spam_score: -2.6
X-Spam_bar: --
X-Spam_report: (-2.6 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_LOW=-0.7,
 RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_PASS=-0.001,
 URIBL_BLOCKED=0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -1.6 (-)
X-Debbugs-Envelope-To: submit
Cc: ludo@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

Hello Guix,

For some time now the system test 'gui-installed-desktop-os-encrypted'
doesn't succeed, the last time it passed was 2020-04-11[0]. This is
problematic since it exercise a lot of packages and thus can be used to
test those packages. Ludovic attempted to fixed it[1] but it didn't 
fixed
the test on my side.

[0]: 
http://ci.guix.gnu.org/search?query=gui-installed-desktop-os-encrypted+system%3Ax86_64-linux+spec%3Aguix-master+status%3Asuccess
[1]: https://issues.guix.info/40790#7

- Brice




Acknowledgement sent to Brice Waegeneire <brice@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#41668; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 9 Jun 2020 11:30:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.