GNU bug report logs - #33893
[PATCH 0/2] Add docker.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 33893 in the body.
You can then email your comments to 33893 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: guix-patches <at> gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH 0/2] Add docker.
Date: Fri, 28 Dec 2018 11:13:11 +0100

Danny Milosavljevic (2):
  gnu: Add docker-engine.
  gnu: Add docker-cli.

 gnu/packages/docker.scm | 144 ++++++++++++++++++++++++++++++++++++++++
 1 file changed, 144 insertions(+)

Message #8 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH 1/2] gnu: Add docker-engine.
Date: Fri, 28 Dec 2018 11:17:50 +0100

* gnu/packages/docker.scm (%docker-version): New variable.
(docker-engine): New variable.  Export it.
---
 gnu/packages/docker.scm | 83 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 83 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..3d1a90fc7 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,12 +23,18 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
+(define %docker-version "18.09.0")
+
 (define-public python-docker-py
   (package
     (name "python-docker-py")
@@ -142,3 +148,80 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             ;(substitute* "hack/make/install-binary"
+             ; (("/binary-daemon") "/dynbinary-daemon"))
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("lvm2" ,lvm2)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))

Message #11 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH 2/2] gnu: Add docker-cli.
Date: Fri, 28 Dec 2018 11:17:51 +0100

* gnu/packages/docker.scm (docker-cli): New variable.  Export it.
---
 gnu/packages/docker.scm | 61 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 61 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3d1a90fc7..caf70cbc9 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -24,8 +24,10 @@
   #:use-module (guix download)
   #:use-module (guix git-download)
   #:use-module (guix build-system gnu)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
@@ -225,3 +227,62 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))

Message #14 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v2 0/3] Add docker.
Date: Sat, 29 Dec 2018 02:32:42 +0100

Danny Milosavljevic (3):
  gnu: Add containerd.
  gnu: Add docker-engine.
  services: Add docker.

 gnu/local.mk            |   1 +
 gnu/packages/docker.scm | 201 +++++++++++++++++++++++++++++++++++++++-
 gnu/services/docker.scm |  90 ++++++++++++++++++
 3 files changed, 291 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/docker.scm

Message #17 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v2 1/3] gnu: Add containerd.
Date: Sat, 29 Dec 2018 02:32:43 +0100

* gnu/packages/docker.scm (containerd): New variable.
---
 gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..f4e676a9f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,48 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))

Message #20 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v2 2/3] gnu: Add docker-engine.
Date: Sat, 29 Dec 2018 02:32:44 +0100

* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
 gnu/packages/docker.scm | 152 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 151 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index f4e676a9f..3ca2fadfd 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -191,3 +196,148 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (substitute* (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "\\.go$"))
+              (("\"ps\"")
+               (string-append "\"" (assoc-ref inputs "procps") "/bin/ps\""))
+              ; TODO: zfs ?
+;getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO --init-path for docker-init
+; ./cmd/dockerd/config_unix.go InitPath
+;./daemon/config/config.go DefaultInitBinary
+              (("exec\\.LookPath\\(\"mkfs\\.xfs\"\\)")
+               (string-append "\"" (assoc-ref inputs "xfsprogs")
+                              "/bin/mkfs.xfs\""))
+              (("exec\\.LookPath\\(\"lvmdiskscan\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/lvmdiskscan\""))
+              (("exec\\.LookPath\\(\"pvdisplay\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/pvdisplay\""))
+              (("exec\\.LookPath\\(\"blkid\"\\)")
+               (string-append "\"" (assoc-ref inputs "util-linux")
+                              "/sbin/blkid\""))
+              (("exec\\.LookPath\\(\"unpigz\"\\)")
+               (string-append "\"" (assoc-ref inputs "pigz")
+                              "/bin/unpigz\""))
+              (("exec\\.LookPath\\(\"iptables\"\\)")
+               (string-append "\"" (assoc-ref inputs "iptables")
+                              "/sbin/iptables\""))
+              (("exec\\.LookPath\\(\"ip\"\\)")
+               (string-append "\"" (assoc-ref inputs "iproute2")
+                              "/sbin/ip\""))
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             )
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             ;(substitute* "hack/make/install-binary"
+             ; (("/binary-daemon") "/dynbinary-daemon"))
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))

Message #23 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v2 3/3] services: Add docker.
Date: Sat, 29 Dec 2018 02:32:45 +0100

* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 gnu/local.mk            |  1 +
 gnu/services/docker.scm | 90 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)
 create mode 100644 gnu/services/docker.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..e592185f8
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,90 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))

Message #26 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 0/4] Add docker.
Date: Sat, 29 Dec 2018 02:39:02 +0100

Includes docker-cli now.

Danny Milosavljevic (4):
  gnu: Add containerd.
  gnu: Add docker-engine.
  services: Add docker.
  gnu: Add docker-cli.

 gnu/local.mk            |   1 +
 gnu/packages/docker.scm | 264 +++++++++++++++++++++++++++++++++++++++-
 gnu/services/docker.scm |  90 ++++++++++++++
 3 files changed, 354 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/docker.scm

Message #29 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 1/4] gnu: Add containerd.
Date: Sat, 29 Dec 2018 02:39:03 +0100

* gnu/packages/docker.scm (containerd): New variable.
---
 gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..f4e676a9f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,48 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))

Message #32 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 2/4] gnu: Add docker-engine.
Date: Sat, 29 Dec 2018 02:39:04 +0100

* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
 gnu/packages/docker.scm | 152 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 151 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index f4e676a9f..3ca2fadfd 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -191,3 +196,148 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (substitute* (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "\\.go$"))
+              (("\"ps\"")
+               (string-append "\"" (assoc-ref inputs "procps") "/bin/ps\""))
+              ; TODO: zfs ?
+;getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO --init-path for docker-init
+; ./cmd/dockerd/config_unix.go InitPath
+;./daemon/config/config.go DefaultInitBinary
+              (("exec\\.LookPath\\(\"mkfs\\.xfs\"\\)")
+               (string-append "\"" (assoc-ref inputs "xfsprogs")
+                              "/bin/mkfs.xfs\""))
+              (("exec\\.LookPath\\(\"lvmdiskscan\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/lvmdiskscan\""))
+              (("exec\\.LookPath\\(\"pvdisplay\"\\)")
+               (string-append "\"" (assoc-ref inputs "lvm2")
+                              "/sbin/pvdisplay\""))
+              (("exec\\.LookPath\\(\"blkid\"\\)")
+               (string-append "\"" (assoc-ref inputs "util-linux")
+                              "/sbin/blkid\""))
+              (("exec\\.LookPath\\(\"unpigz\"\\)")
+               (string-append "\"" (assoc-ref inputs "pigz")
+                              "/bin/unpigz\""))
+              (("exec\\.LookPath\\(\"iptables\"\\)")
+               (string-append "\"" (assoc-ref inputs "iptables")
+                              "/sbin/iptables\""))
+              (("exec\\.LookPath\\(\"ip\"\\)")
+               (string-append "\"" (assoc-ref inputs "iproute2")
+                              "/sbin/ip\""))
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             )
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             ;(substitute* "hack/make/install-binary"
+             ; (("/binary-daemon") "/dynbinary-daemon"))
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))

Message #35 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 3/4] services: Add docker.
Date: Sat, 29 Dec 2018 02:39:05 +0100

* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
---
 gnu/local.mk            |  1 +
 gnu/services/docker.scm | 90 +++++++++++++++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)
 create mode 100644 gnu/services/docker.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..e592185f8
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,90 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))

Message #38 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v3 4/4] gnu: Add docker-cli.
Date: Sat, 29 Dec 2018 02:39:06 +0100

* gnu/packages/docker.scm (docker-cli): New variable.
---
 gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3ca2fadfd..cbf84aecf 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -27,6 +27,7 @@
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
@@ -341,3 +342,65 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+
+             ;; Make build reproducible.
+             (setenv "BUILDTIME" "1970-01-01 00:00:01.000000000+00:00")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))

Message #41 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Subject: Re: [PATCH v3 3/4] services: Add docker.
Date: Sun, 30 Dec 2018 10:50:12 +0100

[Message part 1 (text/plain, inline)]

Better with this additional patch:

diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
index e592185f8..19d7e598f 100644
--- a/gnu/services/docker.scm
+++ b/gnu/services/docker.scm
@@ -72,7 +72,10 @@
            (provision '(dockerd))
            (requirement '(containerd))
            (start #~(make-forkexec-constructor
-                     (list (string-append #$docker "/bin/dockerd"))))
+                     (list (string-append #$docker "/bin/dockerd")
+                           "-p" "/var/run/docker.pid")
+                     #:pid-file "/var/run/docker.pid"
+                     #:log-file "/var/log/docker.log"))
            (stop #~(make-kill-destructor)))))
 
 (define docker-service-type

[Message part 2 (application/pgp-signature, inline)]

Message #44 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 0/4] Add docker.
Date: Sun, 30 Dec 2018 13:17:50 +0100

Now with macro.

Danny Milosavljevic (4):
  gnu: Add containerd.
  gnu: Add docker-engine.
  services: Add docker.
  gnu: Add docker-cli.

 doc/guix.texi           |  10 ++
 gnu/local.mk            |   1 +
 gnu/packages/docker.scm | 261 +++++++++++++++++++++++++++++++++++++++-
 gnu/services/docker.scm |  93 ++++++++++++++
 4 files changed, 364 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/docker.scm

Message #47 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 1/4] gnu: Add containerd.
Date: Sun, 30 Dec 2018 13:17:51 +0100

* gnu/packages/docker.scm (containerd): New variable.
---
 gnu/packages/docker.scm | 49 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..f4e676a9f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,48 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))

Message #50 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 2/4] gnu: Add docker-engine.
Date: Sun, 30 Dec 2018 13:17:52 +0100

* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
 gnu/packages/docker.scm | 149 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 148 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index f4e676a9f..3b6f00834 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -191,3 +196,145 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (let ((source-files (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "." "\\.go$"))))
+               (let-syntax ((substitute-LookPath
+                             (lambda (x)
+                               (syntax-case x ()
+                                 ((substitute-LookPath source-text package
+                                                       relative-path)
+                                  #`(substitute* source-files
+                                     ((#,(string-append "exec\\.LookPath\\(\""
+                                                        (syntax->datum
+                                                         #'source-text)
+                                                        "\")"))
+                                      (string-append "\""
+                                                     (assoc-ref inputs package)
+                                                     relative-path
+                                                     "\", error(nil)"))))))))
+                 (substitute-LookPath "ps" "procps" "/bin/ps")
+                 (substitute-LookPath "mkfs.xfs" "xfsprogs" "/bin/mkfs.xfs")
+                 (substitute-LookPath "lvmdiskscan" "lvm2" "/sbin/lvmdiskscan")
+                 (substitute-LookPath "pvdisplay" "lvm2" "/sbin/pvdisplay")
+                 (substitute-LookPath "blkid" "util-linux" "/sbin/blkid")
+                 (substitute-LookPath "unpigz" "pigz" "/bin/unpigz")
+                 (substitute-LookPath "iptables" "iptables" "/sbin/iptables")
+                 (substitute-LookPath "ip" "iproute2" "/sbin/ip")
+                 ; TODO: zfs ?
+; TODO: getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO: --init-path for docker-init [./cmd/dockerd/config_unix.go InitPath];
+; ./daemon/config/config.go DefaultInitBinary
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             ))
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)
+       ("xfsprogs" ,xfsprogs)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))

Message #53 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 3/4] services: Add docker.
Date: Sun, 30 Dec 2018 13:17:53 +0100

* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* doc/guix.texi (Miscellaneous Services): Document the service.
---
 doc/guix.texi           | 10 +++++
 gnu/local.mk            |  1 +
 gnu/services/docker.scm | 93 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 104 insertions(+)
 create mode 100644 gnu/services/docker.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index fcb5b8c08..b129b1bd1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -22115,6 +22115,16 @@ The following is an example @code{dicod-service} configuration.
                     %dicod-database:gcide))))
 @end example
 
+@cindex docker
+@subsubheading Docker Service
+
+The @code{(gnu services docker)} module provides the following service.
+
+@defvr {Scheme Variable} docker-service-type
+
+This is a service that runs @url{http://www.docker.com,Docker}, a daemon that
+provides container functionality.
+
 @node Setuid Programs
 @subsection Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..19d7e598f
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,93 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd")
+                           "-p" "/var/run/docker.pid")
+                     #:pid-file "/var/run/docker.pid"
+                     #:log-file "/var/log/docker.log"))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))

Message #56 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v4 4/4] gnu: Add docker-cli.
Date: Sun, 30 Dec 2018 13:17:54 +0100

* gnu/packages/docker.scm (docker-cli): New variable.
---
 gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 3b6f00834..81e79c42d 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -27,6 +27,7 @@
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
@@ -338,3 +339,65 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+
+             ;; Make build reproducible.
+             (setenv "BUILDTIME" "1970-01-01 00:00:01.000000000+00:00")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))

Message #59 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v5 0/4] Add docker.
Date: Mon, 31 Dec 2018 00:38:59 +0100

Danny Milosavljevic (4):
  gnu: Add containerd.
  gnu: Add docker-engine.
  services: Add docker.
  gnu: Add docker-cli.

 doc/guix.texi           |  10 ++
 gnu/local.mk            |   1 +
 gnu/packages/docker.scm | 299 +++++++++++++++++++++++++++++++++++++++-
 gnu/services/docker.scm |  93 +++++++++++++
 4 files changed, 402 insertions(+), 1 deletion(-)
 create mode 100644 gnu/services/docker.scm

Message #62 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v5 1/4] gnu: Add containerd.
Date: Mon, 31 Dec 2018 00:39:00 +0100

* gnu/packages/docker.scm (containerd): New variable.
---
 gnu/packages/docker.scm | 68 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 68 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index c58f3f3ca..877800042 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,9 +23,13 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages golang)
+  #:use-module (gnu packages linux)
+  #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
   #:use-module (gnu packages python-web))
 
@@ -142,3 +146,67 @@ created and all the services are started as specified in the configuration.")
 store API.  It allows programmers to interact with a Docker registry using
 Python without keeping their credentials in a Docker configuration file.")
     (license license:asl2.0)))
+
+(define-public containerd
+  (package
+    (name "containerd")
+    (version "1.2.1")
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/containerd/containerd.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "16zn6p1ky3yrgn53z8h9wza53ch91fj47wj5xgz6w4c57j30f66p"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/containerd/containerd"
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'chdir
+           (lambda _
+             (chdir "src/github.com/containerd/containerd")
+             #t))
+         (add-after 'chdir 'patch-paths
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             ;; TODO: Patch "socat", "unpigz".
+             (substitute* "./runtime/v1/linux/runtime.go"
+              (("defaultRuntime[ \t]*=.*")
+               (string-append "defaultRuntime = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n"))
+              (("defaultShim[ \t]*=.*")
+               (string-append "defaultShim = \""
+                              (assoc-ref outputs "out")
+                              "/bin/containerd-shim\"\n")))
+            (substitute* "./vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand[ \t]*=.*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             #t))
+         (replace 'build
+           (lambda* (#:key (make-flags '()) #:allow-other-keys)
+             (apply invoke "make" make-flags)))
+         (replace 'install
+           (lambda* (#:key outputs (make-flags '()) #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out")))
+               (apply invoke "make" (string-append "DESTDIR=" out) "install"
+                      make-flags)))))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("libseccomp" ,libseccomp)
+       ("runc" ,runc)))
+    (native-inputs
+     `(("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Container runtime")
+    (description "This package provides the container daemon for Docker.
+It includes image transfer and storage, container execution and supervision,
+network attachments.")
+    (home-page "http://containerd.io/")
+    (license license:asl2.0)))

Message #65 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v5 2/4] gnu: Add docker-engine.
Date: Mon, 31 Dec 2018 00:39:01 +0100

* gnu/packages/docker.scm (docker-engine): New variable.
(%docker-version): New variable.
---
 gnu/packages/docker.scm | 168 +++++++++++++++++++++++++++++++++++++++-
 1 file changed, 167 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index 877800042..a3510529a 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -23,15 +23,20 @@
   #:use-module (guix packages)
   #:use-module (guix download)
   #:use-module (guix git-download)
+  #:use-module (guix build-system gnu)
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
   #:use-module (gnu packages check)
+  #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
   #:use-module (gnu packages linux)
   #:use-module (gnu packages pkg-config)
   #:use-module (gnu packages python)
-  #:use-module (gnu packages python-web))
+  #:use-module (gnu packages python-web)
+  #:use-module (gnu packages virtualization))
+
+(define %docker-version "18.09.0")
 
 (define-public python-docker-py
   (package
@@ -210,3 +215,164 @@ It includes image transfer and storage, container execution and supervision,
 network attachments.")
     (home-page "http://containerd.io/")
     (license license:asl2.0)))
+
+(define-public docker-engine
+  (package
+    (name "docker-engine")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/engine.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1liqbx58grqih6m8hz9y20y5waflv19pv15l3wl64skap2bsn21c"))))
+    (build-system gnu-build-system)
+    (arguments
+     `(#:modules
+       ((guix build gnu-build-system)
+        ((guix build go-build-system) #:prefix go:)
+        (guix build utils))
+       #:imported-modules
+       (,@%gnu-build-system-modules
+        (guix build go-build-system))
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch-paths
+           (lambda* (#:key inputs #:allow-other-keys)
+             ;(substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+             ; (("") ""))
+             (substitute* "builder/builder-next/executor_unix.go"
+              (("CommandCandidates:.*runc.*")
+               (string-append "CommandCandidates: []string{\""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"},\n")))
+             (substitute* "vendor/github.com/containerd/go-runc/runc.go"
+              (("DefaultCommand = .*")
+               (string-append "DefaultCommand = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "vendor/github.com/containerd/containerd/runtime/v1/linux/runtime.go"
+              (("defaultRuntime[ \t]*=.*")
+               (string-append "defaultRuntime = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n"))
+              (("defaultShim[ \t]*=.*")
+               (string-append "defaultShim = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n")))
+             (substitute* "daemon/daemon_unix.go"
+              (("DefaultShimBinary = .*")
+               (string-append "DefaultShimBinary = \""
+                              (assoc-ref inputs "containerd")
+                              "/bin/containerd-shim\"\n"))
+              (("DefaultRuntimeBinary = .*")
+               (string-append "DefaultRuntimeBinary = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n"))
+              (("DefaultRuntimeName = .*")
+               (string-append "DefaultRuntimeName = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             (substitute* "daemon/config/config.go"
+              (("StockRuntimeName = .*")
+               (string-append "StockRuntimeName = \""
+                              (assoc-ref inputs "runc")
+                              "/sbin/runc\"\n")))
+             ; TODO DefaultInitBinary
+
+             (substitute* "vendor/github.com/moby/buildkit/executor/runcexecutor/executor.go"
+              (("var defaultCommandCandidates = .*")
+               (string-append "var defaultCommandCandidates = []string{\""
+                              (assoc-ref inputs "runc") "/sbin/runc\"}")))
+             (let ((source-files (filter (lambda (name)
+                                    (not (string-contains name "test")))
+                                  (find-files "." "\\.go$"))))
+               (let-syntax ((substitute-LookPath
+                             (lambda (x)
+                               (syntax-case x ()
+                                 ((substitute-LookPath source-text package
+                                                       relative-path)
+                                  #`(substitute* source-files
+                                     ((#,(string-append "exec\\.LookPath\\(\""
+                                                        (syntax->datum
+                                                         #'source-text)
+                                                        "\")"))
+                                      (string-append "\""
+                                                     (assoc-ref inputs package)
+                                                     relative-path
+                                                     "\", error(nil)"))))))))
+                 (substitute-LookPath "ps" "procps" "/bin/ps")
+                 (substitute-LookPath "mkfs.xfs" "xfsprogs" "/bin/mkfs.xfs")
+                 (substitute-LookPath "lvmdiskscan" "lvm2" "/sbin/lvmdiskscan")
+                 (substitute-LookPath "pvdisplay" "lvm2" "/sbin/pvdisplay")
+                 (substitute-LookPath "blkid" "util-linux" "/sbin/blkid")
+                 (substitute-LookPath "unpigz" "pigz" "/bin/unpigz")
+                 (substitute-LookPath "iptables" "iptables" "/sbin/iptables")
+                 (substitute-LookPath "ip" "iproute2" "/sbin/ip")
+                 ; TODO: zfs ?
+; TODO: getPlatformContainerdDaemonOpts ./cmd/dockerd/daemon_unix.go
+; TODO: --init-path for docker-init [./cmd/dockerd/config_unix.go InitPath];
+              ;(("LookPath") "Guix_doesnt_want_LookPath")
+             ))
+             #t))
+         (replace 'configure
+           (lambda _
+             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
+             (setenv "AUTO_GOPATH" "1")
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+             #t))
+         (add-before 'build 'setup-environment
+           (assoc-ref go:%standard-phases 'setup-environment))
+         (replace 'build
+           (lambda _
+             ;(invoke "hack/make.sh" "binary")
+             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'
+             ;; Our LD doesn't like the statically linked relocatable things
+             ;; that go produces, so install the dynamic version of
+             ;; dockerd instead.
+             (invoke "hack/make.sh" "dynbinary")))
+         (replace 'check
+           (lambda _
+             ; FIXME: Those don't find any of the go packages
+             ; needed.  Probably GOPATH/GOROOT related.
+             ;(invoke "hack/test/unit")
+             #t))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
+               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
+             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
+             ; TODO: KEEPBUNDLE=1
+             ;./source/bundles/dynbinary-daemon/dockerd
+             ;(invoke "hack/make.sh" "install-binary")
+             #t)))))
+    (inputs
+     `(("btrfs-progs" ,btrfs-progs)
+       ("containerd" ,containerd) ; for containerd-shim
+       ("runc" ,runc)
+       ("iproute2" ,iproute)
+       ("iptables" ,iptables)
+       ("pigz" ,pigz)
+       ("procps" ,procps)
+       ("util-linux" ,util-linux)
+       ("lvm2" ,lvm2)
+       ("xfsprogs" ,xfsprogs)))
+    (native-inputs
+     `(("eudev" ,eudev) ; TODO: Should be propagated by lvm2 (.pc -> .pc)
+       ("go" ,go)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Docker container component library")
+    (description "This package provides a framework to assemble specialized
+container systems.  It includes components for orchestration, image
+management, secret management, configuration management, networking,
+provisioning etc.")
+    (home-page "https://mobyproject.org/")
+    (license license:asl2.0)))

Message #68 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v5 3/4] services: Add docker.
Date: Mon, 31 Dec 2018 00:39:02 +0100

* gnu/services/docker.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
* doc/guix.texi (Miscellaneous Services): Document the service.
---
 doc/guix.texi           | 10 +++++
 gnu/local.mk            |  1 +
 gnu/services/docker.scm | 93 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 104 insertions(+)
 create mode 100644 gnu/services/docker.scm

diff --git a/doc/guix.texi b/doc/guix.texi
index fcb5b8c08..b129b1bd1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -22115,6 +22115,16 @@ The following is an example @code{dicod-service} configuration.
                     %dicod-database:gcide))))
 @end example
 
+@cindex docker
+@subsubheading Docker Service
+
+The @code{(gnu services docker)} module provides the following service.
+
+@defvr {Scheme Variable} docker-service-type
+
+This is a service that runs @url{http://www.docker.com,Docker}, a daemon that
+provides container functionality.
+
 @node Setuid Programs
 @subsection Setuid Programs
 
diff --git a/gnu/local.mk b/gnu/local.mk
index 925d955a6..f6c91dcc7 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -482,6 +482,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/services/desktop.scm			\
   %D%/services/dict.scm				\
   %D%/services/dns.scm				\
+  %D%/services/docker.scm			\
   %D%/services/authentication.scm		\
   %D%/services/games.scm			\
   %D%/services/kerberos.scm			\
diff --git a/gnu/services/docker.scm b/gnu/services/docker.scm
new file mode 100644
index 000000000..19d7e598f
--- /dev/null
+++ b/gnu/services/docker.scm
@@ -0,0 +1,93 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2018 Danny Milosavljevic <dannym <at> scratchpost.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services docker)
+  #:use-module (gnu services)
+  #:use-module (gnu services configuration)
+  #:use-module (gnu services base)
+  #:use-module (gnu services dbus)
+  #:use-module (gnu services shepherd)
+  #:use-module (gnu system shadow)
+  #:use-module (gnu packages docker)
+  #:use-module (guix records)
+  #:use-module (guix gexp)
+  #:use-module (guix packages)
+
+  #:export (docker-configuration
+            docker-service-type))
+
+(define-configuration docker-configuration
+  (docker
+   (package docker-engine)
+   "Docker daemon package.")
+  (containerd
+   (package containerd)
+   "containerd package."))
+
+(define %docker-accounts
+  (list (user-group (name "docker") (system? #t))))
+
+(define (%containerd-activation config)
+  (let ((state-dir "/var/lib/containerd"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+(define (%docker-activation config)
+  (%containerd-activation config)
+  (let ((state-dir "/var/lib/docker"))
+    #~(begin
+        (use-modules (guix build utils))
+        (mkdir-p #$state-dir))))
+
+;; TODO: Refactor out into its own module?  How to depend on it then?
+(define (containerd-shepherd-service config)
+  (let* ((package (docker-configuration-containerd config)))
+    (shepherd-service
+           (documentation "containerd daemon.")
+           (provision '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$package "/bin/containerd"))))
+           (stop #~(make-kill-destructor)))))
+
+(define (docker-shepherd-service config)
+  (let* ((docker (docker-configuration-docker config)))
+    (shepherd-service
+           (documentation "Docker daemon.")
+           (provision '(dockerd))
+           (requirement '(containerd))
+           (start #~(make-forkexec-constructor
+                     (list (string-append #$docker "/bin/dockerd")
+                           "-p" "/var/run/docker.pid")
+                     #:pid-file "/var/run/docker.pid"
+                     #:log-file "/var/log/docker.log"))
+           (stop #~(make-kill-destructor)))))
+
+(define docker-service-type
+  (service-type (name 'docker)
+		(extensions
+                 (list
+                  (service-extension activation-service-type
+                                     %docker-activation)
+                  (service-extension shepherd-root-service-type
+                                     (lambda args
+                                       (list (apply containerd-shepherd-service args)
+                                             (apply docker-shepherd-service args))))
+                  (service-extension account-service-type
+                                     (const %docker-accounts))))
+                (default-value (docker-configuration))))

Message #71 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: 33893 <at> debbugs.gnu.org
Cc: Danny Milosavljevic <dannym <at> scratchpost.org>
Subject: [PATCH v5 4/4] gnu: Add docker-cli.
Date: Mon, 31 Dec 2018 00:39:03 +0100

* gnu/packages/docker.scm (docker-cli): New variable.
---
 gnu/packages/docker.scm | 63 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 63 insertions(+)

diff --git a/gnu/packages/docker.scm b/gnu/packages/docker.scm
index a3510529a..19b4d504f 100644
--- a/gnu/packages/docker.scm
+++ b/gnu/packages/docker.scm
@@ -27,6 +27,7 @@
   #:use-module (guix build-system go)
   #:use-module (guix build-system python)
   #:use-module (guix utils)
+  #:use-module (gnu packages autotools)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
   #:use-module (gnu packages golang)
@@ -376,3 +377,65 @@ management, secret management, configuration management, networking,
 provisioning etc.")
     (home-page "https://mobyproject.org/")
     (license license:asl2.0)))
+
+(define-public docker-cli
+  (package
+    (name "docker-cli")
+    (version %docker-version)
+    (source
+     (origin
+      (method git-fetch)
+      (uri (git-reference
+            (url "https://github.com/docker/cli.git")
+            (commit (string-append "v" version))))
+      (file-name (git-file-name name version))
+      (sha256
+       (base32
+        "1ivisys20kphvbqlazc3bsg7pk0ykj9gjx5d4yg439x4n13jxwvb"))))
+    (build-system go-build-system)
+    (arguments
+     `(#:import-path "github.com/docker/cli"
+       ;; TODO: Tests require a running Docker daemon.
+       #:tests? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-before 'build 'setup-environment-2
+           (lambda _
+             ;; Respectively, strip the symbol table and debug
+             ;; information, and the DWARF symbol table.
+             (setenv "LDFLAGS" "-s -w")
+
+             ;; Make build reproducible.
+             (setenv "BUILDTIME" "1970-01-01 00:00:01.000000000+00:00")
+             (symlink "src/github.com/docker/cli/scripts" "./scripts")
+             (symlink "src/github.com/docker/cli/docker.Makefile" "./docker.Makefile")
+             #t))
+         (replace 'build
+           (lambda _
+             (invoke "./scripts/build/dynbinary")))
+         (replace 'check
+           (lambda* (#:key make-flags tests? #:allow-other-keys)
+             (setenv "PATH" (string-append (getcwd) "/build:" (getenv "PATH")))
+             (if tests?
+                 ;; Use the newly-built docker client for the tests.
+                 (with-directory-excursion "src/github.com/docker/cli"
+                   ;; TODO: Run test-e2e as well?
+                   (apply invoke "make" "-f" "docker.Makefile" "test-unit"
+                          (or make-flags '())))
+                 #t)))
+         (replace 'install
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (out-bin (string-append out "/bin")))
+               (chdir "build")
+               (install-file (readlink "docker") out-bin)
+               (install-file "docker" out-bin)
+               #t))))))
+    (native-inputs
+     `(("go" ,go)
+       ("libltdl" ,libltdl)
+       ("pkg-config" ,pkg-config)))
+    (synopsis "Command line interface to Docker")
+    (description "This package provides a command line interface to Docker.")
+    (home-page "http://www.docker.com/")
+    (license license:asl2.0)))

Message #74 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 1/4] gnu: Add containerd.
Date: Sun, 06 Jan 2019 21:14:59 +0100

Hello,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/docker.scm (containerd): New variable.

LGTM, thanks!

Ludo’.

Message #77 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Sun, 06 Jan 2019 21:20:35 +0100

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/docker.scm (docker-engine): New variable.
> (%docker-version): New variable.

[...]

> +              ;(("LookPath") "Guix_doesnt_want_LookPath")

No longer needed?

> +         (replace 'configure
> +           (lambda _
> +             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
> +             (setenv "AUTO_GOPATH" "1")

Could you add a comment saying what AUTO_GOPATH does?

> +         (replace 'build
> +           (lambda _
> +             ;(invoke "hack/make.sh" "binary")
> +             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'

It’s not clear to me what should be fixed; perhaps a leftover?

> +         (replace 'check
> +           (lambda _
> +             ; FIXME: Those don't find any of the go packages
> +             ; needed.  Probably GOPATH/GOROOT related.
> +             ;(invoke "hack/test/unit")
> +             #t))

That’s potentially problematic.  :-)  Any idea how difficult it would be
to run these tests?

> +         (replace 'install
> +           (lambda* (#:key outputs #:allow-other-keys)
> +             (let* ((out (assoc-ref outputs "out"))
> +                    (out-bin (string-append out "/bin")))
> +               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
> +               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
> +             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
> +             ; TODO: KEEPBUNDLE=1
> +             ;./source/bundles/dynbinary-daemon/dockerd
> +             ;(invoke "hack/make.sh" "install-binary")

Comments can be removed?

Otherwise LGTM, thanks!

Ludo’.

Message #80 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 3/4] services: Add docker.
Date: Sun, 06 Jan 2019 21:31:00 +0100

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/services/docker.scm: New file.
> * gnu/local.mk (GNU_SYSTEM_MODULES): Add it.
> * doc/guix.texi (Miscellaneous Services): Document the service.

Nice!

> +@cindex docker

“Docker” with a capital.

> +@subsubheading Docker Service
> +
> +The @code{(gnu services docker)} module provides the following service.
> +
> +@defvr {Scheme Variable} docker-service-type
> +
> +This is a service that runs @url{http://www.docker.com,Docker}, a daemon that
> +provides container functionality.
> +

We’re missing “@end defvr” I guess.

I think we shouldn’t propagate the narrative that Docker = container.
So what about something like:

  This is the type of the service that runs @url{…, Docker}, a daemon
  that can execute application bundles (sometimes referred to as
  ``containers'') in isolated environments.

?

Also could you document ‘docker-configuration’ as well?


[...]

> +;; TODO: Refactor out into its own module?  How to depend on it then?
> +(define (containerd-shepherd-service config)
> +  (let* ((package (docker-configuration-containerd config)))
> +    (shepherd-service
> +           (documentation "containerd daemon.")
> +           (provision '(containerd))
> +           (start #~(make-forkexec-constructor
> +                     (list (string-append #$package "/bin/containerd"))))
> +           (stop #~(make-kill-destructor)))))

I suppose there could be a separate ‘containerd-service-type’ if it’s
useful; if it’s not, it’s OK to keep it this way.

As for the dependency, users would have to add both docker and
containerd to their service list, or docker-service-type could extend
containerd-service-type, which would ensure containerd-service-type is
automatically instantiated if it’s not already in the user’s service
list.

> +(define docker-service-type
> +  (service-type (name 'docker)
> +		(extensions
> +                 (list
> +                  (service-extension activation-service-type
> +                                     %docker-activation)
> +                  (service-extension shepherd-root-service-type
> +                                     (lambda args
> +                                       (list (apply containerd-shepherd-service args)
> +                                             (apply docker-shepherd-service args))))

You can make the above (lambda (config) …) instead of (lambda (args) …).

> +                  (service-extension account-service-type
> +                                     (const %docker-accounts))))
> +                (default-value (docker-configuration))))

Please add a ‘description’ field here, and please remove tabs from the
file.  :-)

Could you consider adding a system test for docker/containerd?  Perhaps
we could go as far as using ‘docker-image’ in (guix scripts pack) to
generate an image and make sure ‘docker load’ works, but maybe that’s
too much work.

Thank you,
Ludo’.

Message #83 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 4/4] gnu: Add docker-cli.
Date: Sun, 06 Jan 2019 21:33:12 +0100

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> * gnu/packages/docker.scm (docker-cli): New variable.

[...]

> +    (arguments
> +     `(#:import-path "github.com/docker/cli"
> +       ;; TODO: Tests require a running Docker daemon.
> +       #:tests? #f

I suppose we cannot run the daemon in the build environment, can we?

Or is it possible to use some of the tests?

> +    (native-inputs
> +     `(("go" ,go)
> +       ("libltdl" ,libltdl)

Shouldn’t libltdl be an input?

Otherwise LGTM, thanks!

Ludo’.

Message #86 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 33893 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Mon, 7 Jan 2019 19:44:55 +0100

[Message part 1 (text/plain, inline)]

Hi Ludo,

On Sun, 06 Jan 2019 21:20:35 +0100
Ludovic Courtès <ludo <at> gnu.org> wrote:

> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> 
> > * gnu/packages/docker.scm (docker-engine): New variable.
> > (%docker-version): New variable.  
> 
> [...]
> 
> > +              ;(("LookPath") "Guix_doesnt_want_LookPath")  
> 
> No longer needed?

It was meant as a detector in order to make compilation fail when, in future
versions, docker wants to invok new stuff that we didn't patch yet.
Should we do that?

> > +         (replace 'configure
> > +           (lambda _
> > +             (setenv "DOCKER_GITCOMMIT" (string-append "v" ,%docker-version))
> > +             (setenv "AUTO_GOPATH" "1")  
> 
> Could you add a comment saying what AUTO_GOPATH does?

Yes, I'll add one.

> > +         (replace 'build
> > +           (lambda _
> > +             ;(invoke "hack/make.sh" "binary")
> > +             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'  
> 
> It’s not clear to me what should be fixed; perhaps a leftover?

Yeah, I meant to check what hack/validate/default does and it seems to do
developer-specific tests (commit message formatted the right way etc), so
I guess we can just not invoke it.

> > +         (replace 'check
> > +           (lambda _
> > +             ; FIXME: Those don't find any of the go packages
> > +             ; needed.  Probably GOPATH/GOROOT related.
> > +             ;(invoke "hack/test/unit")
> > +             #t))  
> 
> That’s potentially problematic.  :-)  Any idea how difficult it would be
> to run these tests?

Go has peculiar ideas of how the directory layout is supposed to be set up.
I could probably figure it out - but if someone with more Go knowledge could
step forward it would be much faster.

> > +         (replace 'install
> > +           (lambda* (#:key outputs #:allow-other-keys)
> > +             (let* ((out (assoc-ref outputs "out"))
> > +                    (out-bin (string-append out "/bin")))
> > +               (install-file "bundles/dynbinary-daemon/dockerd" out-bin)
> > +               (install-file "bundles/dynbinary-daemon/dockerd-dev" out-bin))
> > +             ;(setenv "DOCKER_MAKE_INSTALL_PREFIX" (assoc-ref outputs "out"))
> > +             ; TODO: KEEPBUNDLE=1
> > +             ;./source/bundles/dynbinary-daemon/dockerd
> > +             ;(invoke "hack/make.sh" "install-binary")  
> 
> Comments can be removed?

Yeah.

Thanks!

[Message part 2 (application/pgp-signature, inline)]

Message #89 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Tue, 08 Jan 2019 09:42:14 +0100

Hello,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Sun, 06 Jan 2019 21:20:35 +0100
> Ludovic Courtès <ludo <at> gnu.org> wrote:
>
>> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
>> 
>> > * gnu/packages/docker.scm (docker-engine): New variable.
>> > (%docker-version): New variable.  
>> 
>> [...]
>> 
>> > +              ;(("LookPath") "Guix_doesnt_want_LookPath")  
>> 
>> No longer needed?
>
> It was meant as a detector in order to make compilation fail when, in future
> versions, docker wants to invok new stuff that we didn't patch yet.
> Should we do that?

I see, it sounds like a good idea.  Also add a comment explaining the
rationale.

>> > +         (replace 'build
>> > +           (lambda _
>> > +             ;(invoke "hack/make.sh" "binary")
>> > +             ; FIXME: bash -c 'hack/validate/default && hack/make.sh'  
>> 
>> It’s not clear to me what should be fixed; perhaps a leftover?
>
> Yeah, I meant to check what hack/validate/default does and it seems to do
> developer-specific tests (commit message formatted the right way etc), so
> I guess we can just not invoke it.

OK.

>> > +         (replace 'check
>> > +           (lambda _
>> > +             ; FIXME: Those don't find any of the go packages
>> > +             ; needed.  Probably GOPATH/GOROOT related.
>> > +             ;(invoke "hack/test/unit")
>> > +             #t))  
>> 
>> That’s potentially problematic.  :-)  Any idea how difficult it would be
>> to run these tests?
>
> Go has peculiar ideas of how the directory layout is supposed to be set up.
> I could probably figure it out - but if someone with more Go knowledge could
> step forward it would be much faster.

I see Leo is Cc’d so we’ll see.  :-)

Thank you,
Ludo’.

Message #92 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Ludovic Courtès <ludo <at> gnu.org>, Leo Famulari
 <leo <at> famulari.name>
Cc: 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Thu, 10 Jan 2019 03:22:10 +0100

[Message part 1 (text/plain, inline)]

Hi Ludo,
Hi Leo,

On Tue, 08 Jan 2019 09:42:14 +0100
Ludovic Courtès <ludo <at> gnu.org> wrote:

> > Go has peculiar ideas of how the directory layout is supposed to be set up.
> > I could probably figure it out - but if someone with more Go knowledge could
> > step forward it would be much faster.  
> 
> I see Leo is Cc’d so we’ll see.  :-)

Nevermind, I've fixed it and learned something in the process:

Linux doesn't actually know the current working directory as a string.
It only knows the inode, so if you call getcwd, what libc actually does is
it opendirs "..", then finds the entry with the same inode number as
the current directory, and then returns the name of that entry.

Now, gopath uses symlinks to set up their preferred directory hierarchy
in such a way:

ln -s ../../../.. .gopath/src/github.com/docker/docker

Now if you chdir into ".gopath/src/github.com/docker/docker" and then Go later
does getcwd, it will appear as if the chdir did not succeed (because it will
just use the old working directory because it has the same inode).

So Go was erroring out because the directory structure there was *still* wrong.

Solution: Set environment variable PWD to the correct name of the directory.

I've pushed this patchset to master.

I'll try to add a system test next - let's see.

[Message part 2 (application/pgp-signature, inline)]

Message #97 received at 33893-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893-done <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Thu, 10 Jan 2019 09:50:49 +0100

Howdy!

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> On Tue, 08 Jan 2019 09:42:14 +0100
> Ludovic Courtès <ludo <at> gnu.org> wrote:
>
>> > Go has peculiar ideas of how the directory layout is supposed to be set up.
>> > I could probably figure it out - but if someone with more Go knowledge could
>> > step forward it would be much faster.  
>> 
>> I see Leo is Cc’d so we’ll see.  :-)
>
> Nevermind, I've fixed it and learned something in the process:
>
> Linux doesn't actually know the current working directory as a string.
> It only knows the inode, so if you call getcwd, what libc actually does is
> it opendirs "..", then finds the entry with the same inode number as
> the current directory, and then returns the name of that entry.

Are you sure?  In the Linux port of glibc I see this:

--8<---------------cut here---------------start------------->8---
char *
__getcwd (char *buf, size_t size)
{
  char *path;
  char *result;

  // […]

  retval = INLINE_SYSCALL (getcwd, 2, path, alloc_size);
--8<---------------cut here---------------end--------------->8---

And indeed, there’s a ‘getcwd’ syscall:

--8<---------------cut here---------------start------------->8---
$ strace -e getcwd guile -c '(getcwd)'
getcwd("/home/ludo", 100)               = 11
+++ exited with 0 +++
--8<---------------cut here---------------end--------------->8---

> Now, gopath uses symlinks to set up their preferred directory hierarchy
> in such a way:
>
> ln -s ../../../.. .gopath/src/github.com/docker/docker
>
> Now if you chdir into ".gopath/src/github.com/docker/docker" and then Go later
> does getcwd, it will appear as if the chdir did not succeed (because it will
> just use the old working directory because it has the same inode).
>
> So Go was erroring out because the directory structure there was *still* wrong.
>
> Solution: Set environment variable PWD to the correct name of the directory.

Great that you found a solution.

Thanks for taking the time to address this!

Ludo’.

Message #100 received at 33893-done <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 33893-done <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Thu, 10 Jan 2019 14:15:42 +0100

[Message part 1 (text/plain, inline)]

Hi Ludo,

On Thu, 10 Jan 2019 09:50:49 +0100
Ludovic Courtès <ludo <at> gnu.org> wrote:

> Howdy!
> 
> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
> 
> > On Tue, 08 Jan 2019 09:42:14 +0100
> > Ludovic Courtès <ludo <at> gnu.org> wrote:
> >  
> >> > Go has peculiar ideas of how the directory layout is supposed to be set up.
> >> > I could probably figure it out - but if someone with more Go knowledge could
> >> > step forward it would be much faster.    
> >> 
> >> I see Leo is Cc’d so we’ll see.  :-)  
> >
> > Nevermind, I've fixed it and learned something in the process:
> >
> > Linux doesn't actually know the current working directory as a string.
> > It only knows the inode, so if you call getcwd, what libc actually does is
> > it opendirs "..", then finds the entry with the same inode number as
> > the current directory, and then returns the name of that entry.  

According to the POSIX standard ;)

> Are you sure?  In the Linux port of glibc I see this:
> 
> --8<---------------cut here---------------start------------->8---
> char *
> __getcwd (char *buf, size_t size)
> {
>   char *path;
>   char *result;
> 
>   // […]
> 
>   retval = INLINE_SYSCALL (getcwd, 2, path, alloc_size);
> --8<---------------cut here---------------end--------------->8---
> 
> And indeed, there’s a ‘getcwd’ syscall:
> 
> --8<---------------cut here---------------start------------->8---
> $ strace -e getcwd guile -c '(getcwd)'
> getcwd("/home/ludo", 100)               = 11
> +++ exited with 0 +++
> --8<---------------cut here---------------end--------------->8---

Huh.  I guess it boils down to whether the Linux "process" structure
has the cwd in it as a string or as an inode.

In Linux sources:

static inline void get_fs_pwd(struct fs_struct *fs, struct path *pwd)
{
        spin_lock(&fs->lock);
        *pwd = fs->pwd;
        path_get(pwd);
        spin_unlock(&fs->lock);
}

static void get_fs_root_and_pwd_rcu(struct fs_struct *fs, struct path *root,
                                    struct path *pwd)
{
        unsigned seq;

        do {
                seq = read_seqcount_begin(&fs->seq);
                *root = fs->root;
                *pwd = fs->pwd;
        } while (read_seqcount_retry(&fs->seq, seq));
}

struct path {
        struct vfsmount *mnt;
        struct dentry *dentry;
} __randomize_layout;

SYSCALL_DEFINE2(getcwd, char __user *, buf, unsigned long, size)
{
        int error;
        struct path pwd, root;
        char *page = __getname();

        if (!page)
                return -ENOMEM;

        rcu_read_lock();
        get_fs_root_and_pwd_rcu(current->fs, &root, &pwd);

        error = -ENOENT;
        if (!d_unlinked(pwd.dentry)) {
                unsigned long len;
                char *cwd = page + PATH_MAX;
                int buflen = PATH_MAX;

                prepend(&cwd, &buflen, "\0", 1);
                error = prepend_path(&pwd, &root, &cwd, &buflen);
                rcu_read_unlock();

                if (error < 0)
                        goto out;

                /* Unreachable from current root */
                if (error > 0) {
                        error = prepend_unreachable(&cwd, &buflen);
                        if (error)
                                goto out;
                }

                error = -ERANGE;
                len = PATH_MAX + page - cwd;
                if (len <= size) {
                        error = len;
                        if (copy_to_user(buf, cwd, len))
                                error = -EFAULT;
                }
        } else {
                rcu_read_unlock();
        }

out:
        __putname(page);
        return error;
}

/*
 * Replace the fs->{pwdmnt,pwd} with {mnt,dentry}. Put the old values.
 * It can block.
 */
void set_fs_pwd(struct fs_struct *fs, const struct path *path)
{
        struct path old_pwd;

        path_get(path);
        spin_lock(&fs->lock);
        write_seqcount_begin(&fs->seq);
        old_pwd = fs->pwd;
        fs->pwd = *path; <----------------- !!!!
        write_seqcount_end(&fs->seq);
        spin_unlock(&fs->lock);

        if (old_pwd.dentry)
                path_put(&old_pwd);
}

int ksys_chdir(const char __user *filename)
{
        struct path path;
        int error;
        unsigned int lookup_flags = LOOKUP_FOLLOW | LOOKUP_DIRECTORY;
retry:
        error = user_path_at(AT_FDCWD, filename, lookup_flags, &path);
        if (error)
                goto out;

        error = inode_permission(path.dentry->d_inode, MAY_EXEC | MAY_CHDIR);
        if (error)
                goto dput_and_out;

        set_fs_pwd(current->fs, &path); <----------------- !!!

dput_and_out:
        path_put(&path);
        if (retry_estale(error, lookup_flags)) {
                lookup_flags |= LOOKUP_REVAL;
                goto retry;
        }
out:
        return error;
}

SYSCALL_DEFINE1(chdir, const char __user *, filename)
{
        return ksys_chdir(filename);
}


SYSCALL_DEFINE1(fchdir, unsigned int, fd)
{
        struct fd f = fdget_raw(fd);
        int error;

        error = -EBADF;
        if (!f.file)
                goto out;

        error = -ENOTDIR;
        if (!d_can_lookup(f.file->f_path.dentry))
                goto out_putf;

        error = inode_permission(file_inode(f.file), MAY_EXEC | MAY_CHDIR);
        if (!error)
                set_fs_pwd(current->fs, &f.file->f_path);
out_putf:
        fdput(f);
out:
        return error;
}

Interesting!

> Thanks for taking the time to address this!

No problem :)

[Message part 2 (application/pgp-signature, inline)]

Message #103 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 2/4] gnu: Add docker-engine.
Date: Thu, 10 Jan 2019 15:31:07 -0500

[Message part 1 (text/plain, inline)]

On Thu, Jan 10, 2019 at 03:22:10AM +0100, Danny Milosavljevic wrote:
> Hi Ludo,
> Hi Leo,
> 
> On Tue, 08 Jan 2019 09:42:14 +0100
> Ludovic Courtès <ludo <at> gnu.org> wrote:
> 
> > > Go has peculiar ideas of how the directory layout is supposed to be set up.
> > > I could probably figure it out - but if someone with more Go knowledge could
> > > step forward it would be much faster.  
> > 
> > I see Leo is Cc’d so we’ll see.  :-)

Indeed, Go is very particular about this...

> Nevermind, I've fixed it and learned something in the process:

Okay, good :) Let me know if you have more Go questions.

[signature.asc (application/pgp-signature, inline)]

Message #106 received at 33893 <at> debbugs.gnu.org (full text, mbox):

From: Meiyo Peng <meiyo.peng <at> gmail.com>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 33893 <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 4/4] gnu: Add docker-cli.
Date: Mon, 14 Jan 2019 11:20:24 +0800

Hi Danny,

docker-cli provides two identical commands in
"/gnu/store/*docker-cli*/bin/".

#+begin_SRC sh
   ~  ll /gnu/store/*docker-cli*/bin/*
  -r-xr-xr-x 3 root root 64M Jan  1  1970 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker*
  -r-xr-xr-x 3 root root 64M Jan  1  1970 /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker-linux-amd64*

   ~  sha256sum /gnu/store/*docker-cli*/bin/*
  62bc8199fd11f37129d6e8183865df698f495faf90a86bdbe5ee4891b201cbc8  /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker
  62bc8199fd11f37129d6e8183865df698f495faf90a86bdbe5ee4891b201cbc8  /gnu/store/hr7h12q3gvs98pr832b66479cp8wlzhk-docker-cli-18.09.0/bin/docker-linux-amd64
#+end_SRC

It wastes 64MB disk space.  Can we remove "docker-linux-amd64"?


--
Meiyo Peng
https://www.pengmeiyu.com/

Message #109 received at 33893-done <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Meiyo Peng <meiyo.peng <at> gmail.com>
Cc: 33893-done <at> debbugs.gnu.org
Subject: Re: [bug#33893] [PATCH v5 4/4] gnu: Add docker-cli.
Date: Tue, 15 Jan 2019 13:34:31 +0100

[Message part 1 (text/plain, inline)]

Hi,

> It wastes 64MB disk space.  Can we remove "docker-linux-amd64"?

Done in commit f3705090965c2470a0ccc2c045edbc5f5fb7bb8d.

Thanks!

[Message part 2 (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.