GNU bug report logs - #45023
[PATCH] install: Discover local substitute servers.

Previous Next

Package: guix-patches;

Reported by: Mathieu Othacehe <othacehe <at> gnu.org>

Date: Thu, 3 Dec 2020 14:03:02 UTC

Severity: normal

Tags: patch

Done: Mathieu Othacehe <othacehe <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45023 in the body.
You can then email your comments to 45023 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#45023; Package guix-patches. (Thu, 03 Dec 2020 14:03:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Mathieu Othacehe <othacehe <at> gnu.org>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Thu, 03 Dec 2020 14:03:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH] install: Discover local substitute servers.
Date: Thu,  3 Dec 2020 15:01:55 +0100
* gnu/system/install.scm (%installation-services): Add avahi-service-type and
enable substitute server discover in guix-service-type.
[<name-service-switch>]: Set it to %mdns-host-lookup-nss.
---

Hello,

This allows the installation image to use local substitute servers discovered
using Avahi.

It only increases the image closure by 10MiB.

Thanks,

Mathieu

 gnu/system/install.scm | 15 ++++++++++++---
 1 file changed, 12 insertions(+), 3 deletions(-)

diff --git a/gnu/system/install.scm b/gnu/system/install.scm
index 7701297411..7720a42c60 100644
--- a/gnu/system/install.scm
+++ b/gnu/system/install.scm
@@ -34,6 +34,7 @@
   #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (gnu installer)
   #:use-module (gnu system locale)
+  #:use-module (gnu services avahi)
   #:use-module (gnu services dbus)
   #:use-module (gnu services networking)
   #:use-module (gnu services shepherd)
@@ -335,11 +336,18 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m
           ;; The usual services.
           (syslog-service)
 
+          ;; Use the Avahi daemon to discover substitute servers on the local
+          ;; network.  It can be faster than fetching from remote servers.
+          (service avahi-service-type
+                   (avahi-configuration (debug? #t)))
+
           ;; The build daemon.  Register the default substitute server key(s)
-          ;; as trusted to allow the installation process to use substitutes by
-          ;; default.
+          ;; as trusted to allow the installation process to use substitutes
+          ;; by default.
           (service guix-service-type
-                   (guix-configuration (authorize-key? #t)))
+                   (guix-configuration
+                    (authorize-key? #t)
+                    (discover? #t))) ;see above.
 
           ;; Start udev so that useful device nodes are available.
           ;; Use device-mapper rules for cryptsetup & co; enable the CRDA for
@@ -435,6 +443,7 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m
     (host-name "gnu")
     (timezone "Europe/Paris")
     (locale "en_US.utf8")
+    (name-service-switch %mdns-host-lookup-nss)
     (bootloader (bootloader-configuration
                  (bootloader grub-bootloader)
                  (target "/dev/sda")))
-- 
2.29.2





Information forwarded to guix-patches <at> gnu.org:
bug#45023; Package guix-patches. (Fri, 04 Dec 2020 13:07:02 GMT) Full text and rfc822 format available.

Message #8 received at 45023 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: 45023 <at> debbugs.gnu.org
Subject: Re: [bug#45023] [PATCH] install: Discover local substitute servers.
Date: Fri, 04 Dec 2020 14:05:58 +0100
[Message part 1 (text/plain, inline)]
Hey,

> This allows the installation image to use local substitute servers discovered
> using Avahi.

Here's a v2, that adds a new installer page proposing to enable or not
local substitute servers discovery.

Thanks,

Mathieu
[0001-install-Discover-local-substitute-servers.patch (text/x-diff, inline)]
From 3e29ee9755ef0bb9917fc482528fdbdc864ae72e Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Fri, 4 Dec 2020 11:33:16 +0100
Subject: [PATCH v2] install: Discover local substitute servers.

* gnu/installer/substitutes.scm: New file.
* gnu/installer/newt/substitutes.scm: New file.
* gnu/local.mk (INSTALLER_MODULES): Add them.
* po/guix/POTFILES.in: Add gnu/installer/newt/substitutes.scm.
* gnu/installer/proxy.scm (with-silent-shepherd): Move to ...
* gnu/installer/utils.scm: ... here.
* gnu/installer/record.scm (<installer>)[substitutes-page]: New field.
* gnu/installer/newt.scm (substitutes-page): New procedure,
(newt-installer): register it.
* gnu/installer.scm (installer-steps): Add "substitutes-page" step.
* gnu/system/install.scm (%installation-services): Add avahi-service-type and
enable substitute server discover in guix-service-type.
[<name-service-switch>]: Set it to %mdns-host-lookup-nss.
---
 gnu/installer.scm                  |  7 +++++
 gnu/installer/newt.scm             |  5 ++++
 gnu/installer/newt/substitutes.scm | 40 +++++++++++++++++++++++++++++
 gnu/installer/proxy.scm            |  6 +----
 gnu/installer/record.scm           |  3 +++
 gnu/installer/substitutes.scm      | 41 ++++++++++++++++++++++++++++++
 gnu/installer/utils.scm            | 11 +++++++-
 gnu/local.mk                       |  2 ++
 gnu/system/install.scm             | 14 +++++++---
 po/guix/POTFILES.in                |  1 +
 10 files changed, 121 insertions(+), 9 deletions(-)
 create mode 100644 gnu/installer/newt/substitutes.scm
 create mode 100644 gnu/installer/substitutes.scm

diff --git a/gnu/installer.scm b/gnu/installer.scm
index f401b242f8..25734cdcb7 100644
--- a/gnu/installer.scm
+++ b/gnu/installer.scm
@@ -266,6 +266,13 @@ selected keymap."
           (compute (lambda _
                      ((installer-network-page current-installer)))))
 
+         ;; Ask whether to enable substitute servers discovery.
+         (installer-step
+          (id 'substitutes)
+          (description (G_ "Substitutes server discovery"))
+          (compute (lambda _
+                     ((installer-substitutes-page current-installer)))))
+
          ;; Prompt for users (name, group and home directory).
          (installer-step
           (id 'user)
diff --git a/gnu/installer/newt.scm b/gnu/installer/newt.scm
index a1cbeca49a..4f7fc6f4dc 100644
--- a/gnu/installer/newt.scm
+++ b/gnu/installer/newt.scm
@@ -30,6 +30,7 @@
   #:use-module (gnu installer newt page)
   #:use-module (gnu installer newt partition)
   #:use-module (gnu installer newt services)
+  #:use-module (gnu installer newt substitutes)
   #:use-module (gnu installer newt timezone)
   #:use-module (gnu installer newt user)
   #:use-module (gnu installer newt utils)
@@ -101,6 +102,9 @@ problem. The backtrace is displayed below. Please report it by email to \
 (define (network-page)
   (run-network-page))
 
+(define (substitutes-page)
+  (run-substitutes-page))
+
 (define (hostname-page)
   (run-hostname-page))
 
@@ -130,6 +134,7 @@ problem. The backtrace is displayed below. Please report it by email to \
    (locale-page locale-page)
    (menu-page menu-page)
    (network-page network-page)
+   (substitutes-page substitutes-page)
    (timezone-page timezone-page)
    (hostname-page hostname-page)
    (user-page user-page)
diff --git a/gnu/installer/newt/substitutes.scm b/gnu/installer/newt/substitutes.scm
new file mode 100644
index 0000000000..8ffe7d3cb9
--- /dev/null
+++ b/gnu/installer/newt/substitutes.scm
@@ -0,0 +1,40 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Mathieu Othacehe <othacehe <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu installer newt substitutes)
+  #:use-module (gnu installer substitutes)
+  #:use-module (gnu installer utils)
+  #:use-module (guix i18n)
+  #:use-module (newt)
+  #:use-module (ice-9 match)
+  #:export (run-substitutes-page))
+
+(define* (run-substitutes-page)
+  (match (current-clients)
+    (()
+     (case (choice-window
+            (G_ "Substitute servers discovery.")
+            (G_ "Enable") (G_ "Disable")
+            (G_ "If you have substitute servers on your local network, you can \
+enable the following option. It might be faster than fetching from remote \
+servers. There are no security risks, only genuine substitutes will be used \
+but as servers may serve substitute over HTTP, unencrypted, anyone on the LAN \
+can see what software you’re installing."))
+       ((1) (enable-discover))
+       ((2) (disable-discover))))
+    (_ #f)))
diff --git a/gnu/installer/proxy.scm b/gnu/installer/proxy.scm
index befaf3ab0a..86c827294e 100644
--- a/gnu/installer/proxy.scm
+++ b/gnu/installer/proxy.scm
@@ -17,15 +17,11 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu installer proxy)
+  #:use-module (gnu installer utils)
   #:use-module (gnu services herd)
   #:export (set-http-proxy
             clear-http-proxy))
 
-(define-syntax-rule (with-silent-shepherd exp ...)
-  (parameterize ((shepherd-message-port
-                  (%make-void-port "w")))
-    exp ...))
-
 (define (set-http-proxy proxy)
   (with-silent-shepherd
     (with-shepherd-action 'guix-daemon
diff --git a/gnu/installer/record.scm b/gnu/installer/record.scm
index 6ebd87f6a6..0b34318c45 100644
--- a/gnu/installer/record.scm
+++ b/gnu/installer/record.scm
@@ -33,6 +33,7 @@
             installer-locale-page
             installer-menu-page
             installer-network-page
+            installer-substitutes-page
             installer-timezone-page
             installer-hostname-page
             installer-user-page
@@ -73,6 +74,8 @@
   (menu-page installer-menu-page)
   ;; procedure void -> void
   (network-page installer-network-page)
+  ;; procedure void -> void
+  (substitutes-page installer-substitutes-page)
   ;; procedure (zonetab) -> posix-timezone
   (timezone-page installer-timezone-page)
   ;; procedure void -> void
diff --git a/gnu/installer/substitutes.scm b/gnu/installer/substitutes.scm
new file mode 100644
index 0000000000..211a7ea5bf
--- /dev/null
+++ b/gnu/installer/substitutes.scm
@@ -0,0 +1,41 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2020 Mathieu Othacehe <m.othacehe <at> gmail.com>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu installer substitutes)
+  #:use-module (gnu installer utils)
+  #:use-module (gnu services herd)
+  #:export (enable-discover
+            disable-discover))
+
+(define (enable-discover)
+  (with-silent-shepherd
+    (with-shepherd-action 'guix-daemon
+        ('discover "on")
+        result
+      result)))
+
+(define (disable-discover)
+  (with-silent-shepherd
+    (with-shepherd-action 'guix-daemon
+        ('discover "off")
+        result
+      result)))
+
+;; Local Variables:
+;; eval: (put 'with-silent-shepherd 'scheme-indent-function 0)
+;; End:
diff --git a/gnu/installer/utils.scm b/gnu/installer/utils.scm
index a7fa66a199..bb97bc5560 100644
--- a/gnu/installer/utils.scm
+++ b/gnu/installer/utils.scm
@@ -18,6 +18,7 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu installer utils)
+  #:use-module (gnu services herd)
   #:use-module (guix utils)
   #:use-module (guix build utils)
   #:use-module (guix i18n)
@@ -43,7 +44,9 @@
             with-server-socket
             current-server-socket
             current-clients
-            send-to-clients))
+            send-to-clients
+
+            with-silent-shepherd))
 
 (define* (read-lines #:optional (port (current-input-port)))
   "Read lines from PORT and return them as a list."
@@ -233,3 +236,9 @@ accepting socket."
 
   (current-clients (reverse remainder))
   exp)
+
+(define-syntax-rule (with-silent-shepherd exp ...)
+  "Evaluate EXP while discarding shepherd messages."
+  (parameterize ((shepherd-message-port
+                  (%make-void-port "w")))
+    exp ...))
diff --git a/gnu/local.mk b/gnu/local.mk
index 28ad119846..cffd736742 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -716,6 +716,7 @@ INSTALLER_MODULES =                             \
   %D%/installer/record.scm		        \
   %D%/installer/services.scm			\
   %D%/installer/steps.scm			\
+  %D%/installer/substitutes.scm			\
   %D%/installer/tests.scm			\
   %D%/installer/timezone.scm			\
   %D%/installer/user.scm			\
@@ -732,6 +733,7 @@ INSTALLER_MODULES =                             \
   %D%/installer/newt/page.scm			\
   %D%/installer/newt/partition.scm		\
   %D%/installer/newt/services.scm		\
+  %D%/installer/newt/substitutes.scm		\
   %D%/installer/newt/timezone.scm		\
   %D%/installer/newt/user.scm			\
   %D%/installer/newt/utils.scm			\
diff --git a/gnu/system/install.scm b/gnu/system/install.scm
index 7701297411..6765fc0a04 100644
--- a/gnu/system/install.scm
+++ b/gnu/system/install.scm
@@ -34,6 +34,7 @@
   #:use-module ((guix store) #:select (%store-prefix))
   #:use-module (gnu installer)
   #:use-module (gnu system locale)
+  #:use-module (gnu services avahi)
   #:use-module (gnu services dbus)
   #:use-module (gnu services networking)
   #:use-module (gnu services shepherd)
@@ -335,11 +336,17 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m
           ;; The usual services.
           (syslog-service)
 
+          ;; Use the Avahi daemon to discover substitute servers on the local
+          ;; network.  It can be faster than fetching from remote servers.
+          (service avahi-service-type
+                   (avahi-configuration (debug? #t)))
+
           ;; The build daemon.  Register the default substitute server key(s)
-          ;; as trusted to allow the installation process to use substitutes by
-          ;; default.
+          ;; as trusted to allow the installation process to use substitutes
+          ;; by default.
           (service guix-service-type
-                   (guix-configuration (authorize-key? #t)))
+                   (guix-configuration
+                    (authorize-key? #t)))
 
           ;; Start udev so that useful device nodes are available.
           ;; Use device-mapper rules for cryptsetup & co; enable the CRDA for
@@ -435,6 +442,7 @@ Access documentation at any time by pressing Alt-F2.\x1b[0m
     (host-name "gnu")
     (timezone "Europe/Paris")
     (locale "en_US.utf8")
+    (name-service-switch %mdns-host-lookup-nss)
     (bootloader (bootloader-configuration
                  (bootloader grub-bootloader)
                  (target "/dev/sda")))
diff --git a/po/guix/POTFILES.in b/po/guix/POTFILES.in
index 5afb13ffdb..1aec3bef3c 100644
--- a/po/guix/POTFILES.in
+++ b/po/guix/POTFILES.in
@@ -25,6 +25,7 @@ gnu/installer/newt/network.scm
 gnu/installer/newt/page.scm
 gnu/installer/newt/partition.scm
 gnu/installer/newt/services.scm
+gnu/installer/newt/substitutes.scm
 gnu/installer/newt/timezone.scm
 gnu/installer/newt/user.scm
 gnu/installer/newt/utils.scm
-- 
2.29.2


Information forwarded to guix-patches <at> gnu.org:
bug#45023; Package guix-patches. (Fri, 11 Dec 2020 16:46:03 GMT) Full text and rfc822 format available.

Message #11 received at 45023 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 45023 <at> debbugs.gnu.org
Subject: Re: [bug#45023] [PATCH] install: Discover local substitute servers.
Date: Fri, 11 Dec 2020 17:45:33 +0100
Hi!

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>>From 3e29ee9755ef0bb9917fc482528fdbdc864ae72e Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe <at> gnu.org>
> Date: Fri, 4 Dec 2020 11:33:16 +0100
> Subject: [PATCH v2] install: Discover local substitute servers.
>
> * gnu/installer/substitutes.scm: New file.
> * gnu/installer/newt/substitutes.scm: New file.
> * gnu/local.mk (INSTALLER_MODULES): Add them.
> * po/guix/POTFILES.in: Add gnu/installer/newt/substitutes.scm.
> * gnu/installer/proxy.scm (with-silent-shepherd): Move to ...
> * gnu/installer/utils.scm: ... here.
> * gnu/installer/record.scm (<installer>)[substitutes-page]: New field.
> * gnu/installer/newt.scm (substitutes-page): New procedure,
> (newt-installer): register it.
> * gnu/installer.scm (installer-steps): Add "substitutes-page" step.
> * gnu/system/install.scm (%installation-services): Add avahi-service-type and
> enable substitute server discover in guix-service-type.
> [<name-service-switch>]: Set it to %mdns-host-lookup-nss.

Neat!  Overall LGTM; some suggestions below.

> +         ;; Ask whether to enable substitute servers discovery.
> +         (installer-step
> +          (id 'substitutes)
> +          (description (G_ "Substitutes server discovery"))

Maybe “Substitute” (singular)

> +     (case (choice-window
> +            (G_ "Substitute servers discovery.")

“server” (singular)

> +            (G_ "If you have substitute servers on your local network, you can \
> +enable the following option. It might be faster than fetching from remote \
> +servers. There are no security risks, only genuine substitutes will be used \
> +but as servers may serve substitute over HTTP, unencrypted, anyone on the LAN \
> +can see what software you’re installing."))

Slightly reworked:

  By turning this option on, you allow Guix to fetch ``substitutes''
  (pre-built binaries) during installation from servers discovered on
  your local area network (LAN) in addition to the official server.
  This can increase download throughput.

  There are no security risks: only genuine substitutes may be retrieved
  from those servers.  However, eavesdroppers on your LAN may be able to
  see what software you are installing.

> +(define (enable-discover)
> +  (with-silent-shepherd
> +    (with-shepherd-action 'guix-daemon
> +        ('discover "on")
> +        result
> +      result)))
> +
> +(define (disable-discover)

‘{enable,disable}-discovery’ would seem more natural to me.

> +          ;; Use the Avahi daemon to discover substitute servers on the local
> +          ;; network.  It can be faster than fetching from remote servers.
> +          (service avahi-service-type
> +                   (avahi-configuration (debug? #t)))

Maybe we can remove the ‘debug?’ flag?

How bigger is the resulting ISO?

>            ;; The build daemon.  Register the default substitute server key(s)
> -          ;; as trusted to allow the installation process to use substitutes by
> -          ;; default.
> +          ;; as trusted to allow the installation process to use substitutes
> +          ;; by default.
>            (service guix-service-type
> -                   (guix-configuration (authorize-key? #t)))
> +                   (guix-configuration
> +                    (authorize-key? #t)))

Unneeded.  :-)

Thanks!

Ludo’.




Reply sent to Mathieu Othacehe <othacehe <at> gnu.org>:
You have taken responsibility. (Fri, 11 Dec 2020 20:44:02 GMT) Full text and rfc822 format available.

Notification sent to Mathieu Othacehe <othacehe <at> gnu.org>:
bug acknowledged by developer. (Fri, 11 Dec 2020 20:44:02 GMT) Full text and rfc822 format available.

Message #16 received at 45023-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 45023-done <at> debbugs.gnu.org
Subject: Re: [bug#45023] [PATCH] install: Discover local substitute servers.
Date: Fri, 11 Dec 2020 21:43:23 +0100
Hey,

> How bigger is the resulting ISO?

It's 10MiB bigger! I took your remarks into account and pushed as
8361817bf693742757b096468198626f297bb09e.

Thanks for reviewing,

Mathieu




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sat, 09 Jan 2021 12:24:06 GMT) Full text and rfc822 format available.

This bug report was last modified 3 years and 78 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.