GNU bug report logs - #35697
[PATCH 0/8] Make 'guix system docker-image' readily usable

Previous Next

Package: guix-patches;

Reported by: Ludovic Courtès <ludo <at> gnu.org>

Date: Sun, 12 May 2019 10:37:02 UTC

Severity: normal

Tags: patch

Done: Ludovic Courtès <ludo <at> gnu.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 35697 in the body.
You can then email your comments to 35697 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:37:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ludovic Courtès <ludo <at> gnu.org>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Sun, 12 May 2019 10:37:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 0/8] Make 'guix system docker-image' readily usable
Date: Sun, 12 May 2019 12:30:55 +0200
Hello Guix,

On current master, ‘guix system docker-image’ produces an image without
an entry point, so one has to carefully follow the “GUIX_NEW_SYSTEM hack”
described in the manual.

Furthermore, due to other issues, the resulting image doesn’t properly
boot because it tries to mount file systems that it cannot mount, such as
/dev/shm and /dev/pts.

These patches fix both issues, such that one can just do ‘docker create’
and ‘docker start’ to get Guix System up and running in the container.
I think that’s a nice improvement.  :-)

We discussed this Friday on IRC and people said that it’s quite unusual
to provide a “full OS” (with PID 1) as a Docker image; instead, people
would rather do one image per (micro)service.  But anyway, that’s
the purpose of ‘guix system docker-image’, and I can imagine it has
use cases too.  For example, it’s a simple way to get Guix set up in
a container, for people who want to perform Guix builds in a container.

Thoughts?

Ludo’.

Ludovic Courtès (8):
  system: Export 'operating-system-default-essential-services'.
  linux-container: Improve filtering of unnecessary file systems.
  services: 'gc-root-service-type' now has a default value.
  linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image
    OSes.
  linux-container: Compute essential services for THIS-OPERATING-SYSTEM.
  system: Add 'operating-system-with-gc-roots'.
  docker: 'build-docker-image' accepts an optional #:entry-point.
  vm: 'system-docker-image' provides an entry point.

 doc/guix.texi                  |  18 +++--
 gnu/services.scm               |   5 +-
 gnu/system.scm                 |  18 ++++-
 gnu/system/linux-container.scm |  30 ++++++---
 gnu/system/vm.scm              |  18 ++++-
 gnu/tests/docker.scm           | 118 ++++++++++++++++++++++++++++++++-
 gnu/tests/install.scm          |  11 ---
 guix/docker.scm                |  15 +++--
 8 files changed, 195 insertions(+), 38 deletions(-)

-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:02 GMT) Full text and rfc822 format available.

Message #8 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 1/8] system: Export
 'operating-system-default-essential-services'.
Date: Sun, 12 May 2019 12:37:55 +0200
* gnu/system.scm (essential-services): Rename to...
(operating-system-default-essential-services): ... this.
(<operating-system>)[essential-services]: Adjust accordingly.
---
 gnu/system.scm | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/gnu/system.scm b/gnu/system.scm
index 0489b9720d..2c4ca55ffc 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -72,6 +72,7 @@
             operating-system-bootloader
             operating-system-services
             operating-system-essential-services
+            operating-system-default-essential-services
             operating-system-user-services
             operating-system-packages
             operating-system-host-name
@@ -213,7 +214,8 @@
 
   (essential-services operating-system-essential-services ; list of services
                       (thunked)
-                      (default (essential-services this-operating-system)))
+                      (default (operating-system-default-essential-services
+                                this-operating-system)))
   (services operating-system-user-services        ; list of services
             (default %base-services))
 
@@ -463,7 +465,7 @@ value of the SYSTEM-SERVICE-TYPE service."
                 ("initrd" ,initrd)
                 ("locale" ,locale))))))   ;used by libc
 
-(define* (essential-services os)
+(define (operating-system-default-essential-services os)
   "Return the list of essential services for OS.  These are special services
 that implement part of what's declared in OS are responsible for low-level
 bookkeeping."
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:02 GMT) Full text and rfc822 format available.

Message #11 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 2/8] linux-container: Improve filtering of unnecessary file
 systems.
Date: Sun, 12 May 2019 12:37:56 +0200
* gnu/system/linux-container.scm (containerized-operating-system)[user-file-systems]:
Add trailing slash for the "/dev/" and "/sys/" prefixes.
---
 gnu/system/linux-container.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm
index 149c3d08a3..ded5f279fe 100644
--- a/gnu/system/linux-container.scm
+++ b/gnu/system/linux-container.scm
@@ -65,8 +65,8 @@ containerized OS."
                     (string=? target "/")
                     (and (string? source)
                          (string-prefix? "/dev/" source))
-                    (string-prefix? "/dev" target)
-                    (string-prefix? "/sys" target))))
+                    (string-prefix? "/dev/" target)
+                    (string-prefix? "/sys/" target))))
             (operating-system-file-systems os)))
 
   (define (mapping->fs fs)
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:03 GMT) Full text and rfc822 format available.

Message #14 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 3/8] services: 'gc-root-service-type' now has a default value.
Date: Sun, 12 May 2019 12:37:57 +0200
* gnu/services.scm (gc-root-service-type)[default-value]: New field.
---
 gnu/services.scm | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/gnu/services.scm b/gnu/services.scm
index f151bbaa9d..7de78105ff 100644
--- a/gnu/services.scm
+++ b/gnu/services.scm
@@ -1,5 +1,5 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright © 2015, 2016, 2017, 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2015, 2016, 2017, 2018, 2019 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2016 Chris Marusich <cmmarusich <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -649,7 +649,8 @@ as Wifi cards.")))
                 (extend append)
                 (description
                  "Register garbage-collector roots---i.e., store items that
-will not be reclaimed by the garbage collector.")))
+will not be reclaimed by the garbage collector.")
+                (default-value '())))
 
 
 ;;;
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:03 GMT) Full text and rfc822 format available.

Message #17 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 4/8] linux-container: Do not add %CONTAINER-FILE-SYSTEMS to
 Docker image OSes.
Date: Sun, 12 May 2019 12:37:58 +0200
Previously, 'guix system docker-image' would end up providing an OS that
would try to mount all of %CONTAINER-FILE-SYSTEMS as well as /gnu/store,
which is bound to fail in unprivileged Docker.

This patch makes it so that 'guix system container' still gets those
file systems, but 'guix system docker-image' doesn't.

* gnu/system/linux-container.scm (containerized-operating-system): Add
 #:extra-file-systems parameter and honor it.  Do not import
 %STORE-MAPPING.
(container-script): Add %STORE-MAPPING to MAPPINGS and pass
 #:extra-file-systems.
---
 gnu/system/linux-container.scm | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm
index ded5f279fe..5adec064f7 100644
--- a/gnu/system/linux-container.scm
+++ b/gnu/system/linux-container.scm
@@ -53,10 +53,12 @@ from OS that are needed on the bare metal and not in a container."
                      (return `(("locale" ,locale))))))
         base))
 
-(define (containerized-operating-system os mappings)
+(define* (containerized-operating-system os mappings
+                                         #:key
+                                         (extra-file-systems '()))
   "Return an operating system based on OS for use in a Linux container
 environment.  MAPPINGS is a list of <file-system-mapping> to realize in the
-containerized OS."
+containerized OS.  EXTRA-FILE-SYSTEMS is a list of file systems to add to OS."
   (define user-file-systems
     (remove (lambda (fs)
               (let ((target (file-system-mount-point fs))
@@ -88,15 +90,17 @@ containerized OS."
                         (memq (service-kind service)
                               useless-services))
                       (operating-system-user-services os)))
-    (file-systems (append (map mapping->fs (cons %store-mapping mappings))
-                          %container-file-systems
+    (file-systems (append (map mapping->fs mappings)
+                          extra-file-systems
                           user-file-systems))))
 
 (define* (container-script os #:key (mappings '()))
   "Return a derivation of a script that runs OS as a Linux container.
 MAPPINGS is a list of <file-system> objects that specify the files/directories
 that will be shared with the host system."
-  (let* ((os           (containerized-operating-system os mappings))
+  (let* ((os           (containerized-operating-system
+                        os (cons %store-mapping mappings)
+                        #:extra-file-systems %container-file-systems))
          (file-systems (filter file-system-needed-for-boot?
                                (operating-system-file-systems os)))
          (specs        (map file-system->spec file-systems)))
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:03 GMT) Full text and rfc822 format available.

Message #20 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 5/8] linux-container: Compute essential services for
 THIS-OPERATING-SYSTEM.
Date: Sun, 12 May 2019 12:37:59 +0200
Previously, the 'essential-services' would correspond to the initial,
non-containerized OS.  Thus, all the file systems removed in
'container-essential-services' would actually still be there because the
essential services would be computed on the non-containerized OS.

This is a followup to 69cae3d3356a69b7fe69481338f760545995485e.

* gnu/system/linux-container.scm (container-essential-services): Call
'operating-system-default-essential-services' to get the baseline
services.
(containerized-operating-system): Pass THIS-OPERATING-SYSTEM, not OS, to
'container-essential-services'.
Add a dummy root file system to 'file-systems'.
---
 gnu/system/linux-container.scm | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/gnu/system/linux-container.scm b/gnu/system/linux-container.scm
index 5adec064f7..fc2e05a5bc 100644
--- a/gnu/system/linux-container.scm
+++ b/gnu/system/linux-container.scm
@@ -45,7 +45,7 @@ from OS that are needed on the bare metal and not in a container."
                     (list (service-kind %linux-bare-metal-service)
                           firmware-service-type
                           system-service-type)))
-            (operating-system-essential-services os)))
+            (operating-system-default-essential-services os)))
 
   (cons (service system-service-type
                  (let ((locale (operating-system-locale-directory os)))
@@ -85,14 +85,20 @@ containerized OS.  EXTRA-FILE-SYSTEMS is a list of file systems to add to OS."
   (operating-system
     (inherit os)
     (swap-devices '()) ; disable swap
-    (essential-services (container-essential-services os))
+    (essential-services (container-essential-services this-operating-system))
     (services (remove (lambda (service)
                         (memq (service-kind service)
                               useless-services))
                       (operating-system-user-services os)))
     (file-systems (append (map mapping->fs mappings)
                           extra-file-systems
-                          user-file-systems))))
+                          user-file-systems
+
+                          ;; Provide a dummy root file system.
+                          (list (file-system
+                                  (mount-point "/")
+                                  (device "none")
+                                  (type "none")))))))
 
 (define* (container-script os #:key (mappings '()))
   "Return a derivation of a script that runs OS as a Linux container.
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:04 GMT) Full text and rfc822 format available.

Message #23 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 6/8] system: Add 'operating-system-with-gc-roots'.
Date: Sun, 12 May 2019 12:38:00 +0200
* gnu/tests/install.scm (operating-system-with-gc-roots): Move to...
* gnu/system.scm (operating-system-with-gc-roots): ... here.  New
procedure.
---
 gnu/system.scm        | 12 ++++++++++++
 gnu/tests/install.scm | 11 -----------
 2 files changed, 12 insertions(+), 11 deletions(-)

diff --git a/gnu/system.scm b/gnu/system.scm
index 2c4ca55ffc..01be1243fe 100644
--- a/gnu/system.scm
+++ b/gnu/system.scm
@@ -109,6 +109,7 @@
             operating-system-boot-script
 
             system-linux-image-file-name
+            operating-system-with-gc-roots
 
             boot-parameters
             boot-parameters?
@@ -519,6 +520,17 @@ bookkeeping."
    (append (operating-system-user-services os)
            (operating-system-essential-services os))))
 
+(define (operating-system-with-gc-roots os roots)
+  "Return a variant of OS where ROOTS are registered as GC roots."
+  (operating-system
+    (inherit os)
+
+    ;; We use this procedure for the installation OS, which already defines GC
+    ;; roots.  Add ROOTS to those.
+    (services (cons (simple-service 'extra-root
+                                    gc-root-service-type roots)
+                    (operating-system-user-services os)))))
+
 
 ;;;
 ;;; /etc.
diff --git a/gnu/tests/install.scm b/gnu/tests/install.scm
index 430a102378..7b5ee18505 100644
--- a/gnu/tests/install.scm
+++ b/gnu/tests/install.scm
@@ -123,17 +123,6 @@
                                     (inherit config)
                                     (guix (current-guix))))))))
 
-(define (operating-system-with-gc-roots os roots)
-  "Return a variant of OS where ROOTS are registered as GC roots."
-  (operating-system
-    (inherit os)
-
-    ;; We use this procedure for the installation OS, which already defines GC
-    ;; roots.  Add ROOTS to those.
-    (services (cons (simple-service 'extra-root
-                                    gc-root-service-type roots)
-                    (operating-system-user-services os)))))
-
 
 (define MiB (expt 2 20))
 
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:04 GMT) Full text and rfc822 format available.

Message #26 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 7/8] docker: 'build-docker-image' accepts an optional
 #:entry-point.
Date: Sun, 12 May 2019 12:38:01 +0200
* guix/docker.scm (config): Add #:entry-point and honor it.
(build-docker-image): Likewise.
---
 guix/docker.scm | 15 +++++++++++----
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/guix/docker.scm b/guix/docker.scm
index c6e9c6fee5..7fe83d9797 100644
--- a/guix/docker.scm
+++ b/guix/docker.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017 Ricardo Wurmus <rekado <at> elephly.net>
-;;; Copyright © 2017, 2018 Ludovic Courtès <ludo <at> gnu.org>
+;;; Copyright © 2017, 2018, 2019 Ludovic Courtès <ludo <at> gnu.org>
 ;;; Copyright © 2018 Chris Marusich <cmmarusich <at> gmail.com>
 ;;;
 ;;; This file is part of GNU Guix.
@@ -73,7 +73,7 @@
   `((,(generate-tag path) . ((latest . ,id)))))
 
 ;; See https://github.com/opencontainers/image-spec/blob/master/config.md
-(define (config layer time arch)
+(define* (config layer time arch #:key entry-point)
   "Generate a minimal image configuration for the given LAYER file."
   ;; "architecture" must be values matching "platform.arch" in the
   ;; runtime-spec at
@@ -81,7 +81,9 @@
   `((architecture . ,arch)
     (comment . "Generated by GNU Guix")
     (created . ,time)
-    (config . #nil)
+    (config . ,(if entry-point
+                   `((entrypoint . ,entry-point))
+                   #nil))
     (container_config . #nil)
     (os . "linux")
     (rootfs . ((type . "layers")
@@ -110,6 +112,7 @@ return \"a\"."
                              (transformations '())
                              (system (utsname:machine (uname)))
                              database
+                             entry-point
                              compressor
                              (creation-time (current-time time-utc)))
   "Write to IMAGE a Docker image archive containing the given PATHS.  PREFIX
@@ -118,6 +121,9 @@ must be a store path that is a prefix of any store paths in PATHS.
 When DATABASE is true, copy it to /var/guix/db in the image and create
 /var/guix/gcroots and friends.
 
+When ENTRY-POINT is true, it must be a list of strings; it is stored as the
+entry point in the Docker image JSON structure.
+
 SYMLINKS must be a list of (SOURCE -> TARGET) tuples describing symlinks to be
 created in the image, where each TARGET is relative to PREFIX.
 TRANSFORMATIONS must be a list of (OLD -> NEW) tuples describing how to
@@ -227,7 +233,8 @@ SRFI-19 time-utc object, as the creation time in metadata."
       (with-output-to-file "config.json"
         (lambda ()
           (scm->json (config (string-append id "/layer.tar")
-                             time arch))))
+                             time arch
+                             #:entry-point entry-point))))
       (with-output-to-file "manifest.json"
         (lambda ()
           (scm->json (manifest prefix id))))
-- 
2.21.0





Information forwarded to guix-patches <at> gnu.org:
bug#35697; Package guix-patches. (Sun, 12 May 2019 10:39:05 GMT) Full text and rfc822 format available.

Message #29 received at 35697 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>,
 Chris Marusich <cmmarusich <at> gmail.com>
Subject: [PATCH 8/8] vm: 'system-docker-image' provides an entry point.
Date: Sun, 12 May 2019 12:38:02 +0200
This simplifies use of images created with 'guix system docker-image'.

* gnu/system/vm.scm (system-docker-image)[boot-program]: New variable.
[os]: Add it to the GC roots.
[build]: Pass #:entry-point to 'build-docker-image'.
* gnu/tests/docker.scm (run-docker-system-test): New procedure.
(%test-docker-system): New variable.
* doc/guix.texi (Invoking guix system): Remove GUIX_NEW_SYSTEM hack and
'--entrypoint' from the example.  Mention 'docker create', 'docker
start', and 'docker exec'.
---
 doc/guix.texi        |  18 ++++---
 gnu/system/vm.scm    |  18 ++++++-
 gnu/tests/docker.scm | 118 ++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 145 insertions(+), 9 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index df7208229c..da65fd8a4e 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -24497,20 +24497,26 @@ system configuration file.  You can then load the image and launch a
 Docker container using commands like the following:
 
 @example
-image_id="$(docker load < guix-system-docker-image.tar.gz)"
-docker run -e GUIX_NEW_SYSTEM=/var/guix/profiles/system \\
-    --entrypoint /var/guix/profiles/system/profile/bin/guile \\
-    $image_id /var/guix/profiles/system/boot
+image_id="`docker load < guix-system-docker-image.tar.gz`"
+container_id="`docker create $image_id`"
+docker start $container_id
 @end example
 
 This command starts a new Docker container from the specified image.  It
 will boot the Guix system in the usual manner, which means it will
 start any services you have defined in the operating system
-configuration.  Depending on what you run in the Docker container, it
+configuration.  You can get an interactive shell running in the container
+using @command{docker exec}:
+
+@example
+docker exec -ti $container_id /run/current-system/profile/bin/bash --login
+@end example
+
+Depending on what you run in the Docker container, it
 may be necessary to give the container additional permissions.  For
 example, if you intend to build software using Guix inside of the Docker
 container, you may need to pass the @option{--privileged} option to
-@code{docker run}.
+@code{docker create}.
 
 @item container
 Return a script to run the operating system declared in @var{file}
diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 124abd0fc9..f3027cd4ca 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -473,7 +473,7 @@ the image."
 
 (define* (system-docker-image os
                               #:key
-                              (name "guixsd-docker-image")
+                              (name "guix-docker-image")
                               register-closures?)
   "Build a docker image.  OS is the desired <operating-system>.  NAME is the
 base name to use for the output file.  When REGISTER-CLOSURES? is not #f,
@@ -487,7 +487,19 @@ should set REGISTER-CLOSURES? to #f."
          (local-file (search-path %load-path
                                   "guix/store/schema.sql"))))
 
-  (let ((os    (containerized-operating-system os '()))
+  (define boot-program
+    ;; Program that runs the boot script of OS, which in turn starts shepherd.
+    (program-file "boot-program"
+                  #~(let ((system (cadr (command-line))))
+                      (setenv "GUIX_NEW_SYSTEM" system)
+                      (execl #$(file-append guile-2.2 "/bin/guile")
+                             "guile" "--no-auto-compile"
+                             (string-append system "/boot")))))
+
+
+  (let ((os    (operating-system-with-gc-roots
+                (containerized-operating-system os '())
+                (list boot-program)))
         (name  (string-append name ".tar.gz"))
         (graph "system-graph"))
     (define build
@@ -538,9 +550,11 @@ should set REGISTER-CLOSURES? to #f."
                                  (string-append "/xchg/" #$graph)
                                read-reference-graph)))
                  #$os
+                 #:entry-point '(#$boot-program #$os)
                  #:compressor '(#+(file-append gzip "/bin/gzip") "-9n")
                  #:creation-time (make-time time-utc 0 1)
                  #:transformations `((,root-directory -> ""))))))))
+
     (expression->derivation-in-linux-vm
      name build
      #:make-disk-image? #f
diff --git a/gnu/tests/docker.scm b/gnu/tests/docker.scm
index 25e172efae..3cd3a27884 100644
--- a/gnu/tests/docker.scm
+++ b/gnu/tests/docker.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2019 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;; Copyright © 2019 Ludovic Courtès <ludo <at> gnu.org>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -28,6 +29,7 @@
   #:use-module (gnu services desktop)
   #:use-module (gnu packages bootstrap) ; %bootstrap-guile
   #:use-module (gnu packages docker)
+  #:use-module (gnu packages guile)
   #:use-module (guix gexp)
   #:use-module (guix grafts)
   #:use-module (guix monads)
@@ -38,7 +40,8 @@
   #:use-module (guix tests)
   #:use-module (guix build-system trivial)
   #:use-module ((guix licenses) #:prefix license:)
-  #:export (%test-docker))
+  #:export (%test-docker
+            %test-docker-system))
 
 (define %docker-os
   (simple-operating-system
@@ -166,3 +169,116 @@ standard output device and then enters a new line.")
    (name "docker")
    (description "Test Docker container of Guix.")
    (value (build-tarball&run-docker-test))))
+
+
+(define (run-docker-system-test tarball)
+  "Load DOCKER-TARBALL as Docker image and run it in a Docker container,
+inside %DOCKER-OS."
+  (define os
+    (marionette-operating-system
+     %docker-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     ;; FIXME: Because we're using the volatile-root setup where the root file
+     ;; system is a tmpfs overlaid over a small root file system, 'docker
+     ;; load' must be able to store the whole image into memory, hence the
+     ;; huge memory requirements.  We should avoid the volatile-root setup
+     ;; instead.
+     (memory-size 3000)
+     (port-forwardings '())))
+
+  (define test
+    (with-imported-modules '((gnu build marionette)
+                             (guix build utils))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (gnu build marionette)
+                       (guix build utils))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "docker")
+
+          (test-assert "service running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (match (start-service 'dockerd)
+                  (#f #f)
+                  (('service response-parts ...)
+                   (match (assq-ref response-parts 'running)
+                     ((pid) (number? pid))))))
+             marionette))
+
+          (test-assert "load system image and run it"
+            (marionette-eval
+             `(begin
+                (define (slurp command . args)
+                  ;; Return the output from COMMAND.
+                  (let* ((port (apply open-pipe* OPEN_READ command args))
+                         (output (read-line port))
+                         (status (close-pipe port)))
+                    output))
+
+                (define (docker-cli command . args)
+                  ;; Run the given Docker COMMAND.
+                  (apply invoke #$(file-append docker-cli "/bin/docker")
+                         command args))
+
+                (define (wait-for-container-file container file)
+                  ;; Wait for FILE to show up in CONTAINER.
+                  (docker-cli "exec" container
+                              #$(file-append guile-2.2 "/bin/guile")
+                              "-c"
+                              (object->string
+                               `(let loop ((n 15))
+                                  (when (zero? n)
+                                    (error "file didn't show up" ,file))
+                                  (unless (file-exists? ,file)
+                                    (sleep 1)
+                                    (loop (- n 1)))))))
+
+                (let* ((line (slurp #$(file-append docker-cli "/bin/docker")
+                                    "load" "-i" #$tarball))
+                       (repository&tag (string-drop line
+                                                    (string-length
+                                                     "Loaded image: ")))
+                       (container (slurp
+                                   #$(file-append docker-cli "/bin/docker")
+                                   "create" repository&tag)))
+                  (docker-cli "start" container)
+
+                  ;; Wait for shepherd to be ready.
+                  (wait-for-container-file container
+                                           "/var/run/shepherd/socket")
+
+                  (docker-cli "exec" container
+                              "/run/current-system/profile/bin/herd"
+                              "status")
+                  (slurp #$(file-append docker-cli "/bin/docker")
+                         "exec" container
+                         "/run/current-system/profile/bin/herd"
+                         "status" "guix-daemon")))
+             marionette))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "docker-system-test" test))
+
+(define %test-docker-system
+  (system-test
+   (name "docker-system")
+   (description "Run a system image as produced by @command{guix system
+docker-image} inside Docker.")
+   (value (with-monad %store-monad
+            (>>= (system-docker-image (simple-operating-system))
+                 run-docker-system-test)))))
-- 
2.21.0





Reply sent to Ludovic Courtès <ludo <at> gnu.org>:
You have taken responsibility. (Wed, 15 May 2019 14:39:01 GMT) Full text and rfc822 format available.

Notification sent to Ludovic Courtès <ludo <at> gnu.org>:
bug acknowledged by developer. (Wed, 15 May 2019 14:39:01 GMT) Full text and rfc822 format available.

Message #34 received at 35697-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 35697-done <at> debbugs.gnu.org
Cc: Chris Marusich <cmmarusich <at> gmail.com>
Subject: Re: [bug#35697] [PATCH 0/8] Make 'guix system docker-image' readily
 usable
Date: Wed, 15 May 2019 16:37:51 +0200
Hi,

Ludovic Courtès <ludo <at> gnu.org> skribis:

> Ludovic Courtès (8):
>   system: Export 'operating-system-default-essential-services'.
>   linux-container: Improve filtering of unnecessary file systems.
>   services: 'gc-root-service-type' now has a default value.
>   linux-container: Do not add %CONTAINER-FILE-SYSTEMS to Docker image
>     OSes.
>   linux-container: Compute essential services for THIS-OPERATING-SYSTEM.
>   system: Add 'operating-system-with-gc-roots'.
>   docker: 'build-docker-image' accepts an optional #:entry-point.
>   vm: 'system-docker-image' provides an entry point.

Pushed!

I had to rebase and adjust to changes made over the last couple of days
in that area of the code.

Feedback welcome!

Ludo’.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Thu, 13 Jun 2019 11:24:06 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 290 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.