Package: guix-patches;
Reported by: Christopher Baines <mail <at> cbaines.net>
Date: Sun, 15 Sep 2019 18:19:02 UTC
Severity: normal
Tags: patch
Done: Christopher Baines <mail <at> cbaines.net>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37412 in the body.
You can then email your comments to 37412 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 15 Sep 2019 18:19:02 GMT) Full text and rfc822 format available.Christopher Baines <mail <at> cbaines.net>:guix-patches <at> gnu.org.
(Sun, 15 Sep 2019 18:19:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: guix-patches <at> gnu.org Subject: [PATCH 0/2] Add package and service for the Guix Data Service. Date: Sun, 15 Sep 2019 19:18:06 +0100
[Message part 1 (text/plain, inline)]
These patches add a package definition, and service type for the Guix Data Service, as well as a system test. Christopher Baines (2): gnu: Add guix-data-service. services: Add the Guix Data Service. gnu/local.mk | 2 + gnu/packages/web.scm | 83 +++++++++++++++++ gnu/services/guix.scm | 204 ++++++++++++++++++++++++++++++++++++++++++ gnu/tests/guix.scm | 173 +++++++++++++++++++++++++++++++++++ 4 files changed, 462 insertions(+) create mode 100644 gnu/services/guix.scm create mode 100644 gnu/tests/guix.scm
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 15 Sep 2019 18:22:01 GMT) Full text and rfc822 format available.Message #8 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: 37412 <at> debbugs.gnu.org Subject: [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 15 Sep 2019 19:21:26 +0100
* gnu/packages/web.scm (guix-data-service): New variable.
---
gnu/packages/web.scm | 83 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 83 insertions(+)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 39475bd6a8..30adc75aa1 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -3973,6 +3973,89 @@ CDF, Atom 0.3, and Atom 1.0 feeds.")
(define-public python2-feedparser
(package-with-python2 python-feedparser))
+(define-public guix-data-service
+ (let ((commit "6a9f11fb94fbfd969396dd4d9f768a0c9ec67734")
+ (revision "1"))
+ (package
+ (name "guix-data-service")
+ (version (string-append "0.0.1-" revision "." (string-take commit 7)))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://git.savannah.gnu.org/git/guix/data-service.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "08kfi74dp8h1i03gfmjqfv07ff7m93psnh6xmrgwzr0ja5cdqjri"))))
+ (build-system gnu-build-system)
+ (arguments
+ '(#:tests? #f ; TODO Tests require PostgreSQL
+ #:modules ((guix build utils)
+ (guix build gnu-build-system)
+ (ice-9 rdelim)
+ (ice-9 popen))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'install 'wrap-executable
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (guile (assoc-ref inputs "guile"))
+ (guile-effective-version
+ (read-line
+ (open-pipe* OPEN_READ
+ (string-append guile "/bin/guile")
+ "-c" "(display (effective-version))")))
+ (scm (string-append out "/share/guile/site/"
+ guile-effective-version))
+ (go (string-append out "/lib/guile/"
+ guile-effective-version
+ "/site-ccache")))
+ (for-each
+ (lambda (file)
+ (wrap-program (string-append bin "/" file)
+ `("PATH" ":" prefix
+ (,bin))
+ `("GUILE_LOAD_PATH" ":" prefix
+ (,scm ,(getenv "GUILE_LOAD_PATH")))
+ `("GUILE_LOAD_COMPILED_PATH" ":" prefix
+ (,go ,(getenv "GUILE_LOAD_COMPILED_PATH")))))
+ '("guix-data-service"
+ "guix-data-service-process-branch-updated-email"
+ "guix-data-service-process-job"
+ "guix-data-service-process-jobs"
+ "guix-data-service-query-build-servers"))
+ #t))))))
+ (inputs
+ `(("guile" ,guile-2.2)
+ ("guix" ,guix)
+ ("guile-fibers" ,guile-fibers)
+ ("guile-json" ,guile-json-3)
+ ("guile-email" ,guile-email)
+ ("guile-squee" ,guile-squee)
+ ("postgresql" ,postgresql)
+ ("sqitch" ,sqitch)))
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("pkg-config" ,pkg-config)))
+ (native-search-paths
+ ;; guile-git requires this to be set
+ (list (search-path-specification
+ (variable "GIT_SSL_CAINFO")
+ (file-type 'regular)
+ (separator #f)
+ (files '("etc/ssl/certs/ca-certificates.crt")))))
+ (synopsis "Store and provide data about GNU Guix")
+ (description
+ "The Guix Data Service stores data about GNU Guix, and provides this
+through a web interface. It supports listening to the guix-commits mailing
+list to find out about new revisions, then loads the data from these in to a
+PostgreSQL database.")
+ (home-page "https://data.guix.gnu.org/")
+ (license license:agpl3+))))
+
(define-public gumbo-parser
(package
(name "gumbo-parser")
--
2.23.0
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 15 Sep 2019 18:22:02 GMT) Full text and rfc822 format available.Message #11 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: 37412 <at> debbugs.gnu.org Subject: [PATCH 2/2] services: Add the Guix Data Service. Date: Sun, 15 Sep 2019 19:21:27 +0100
* gnu/services/guix.scm: New file.
* gnu/tests/guix.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add both new files.
---
gnu/local.mk | 2 +
gnu/services/guix.scm | 204 ++++++++++++++++++++++++++++++++++++++++++
gnu/tests/guix.scm | 173 +++++++++++++++++++++++++++++++++++
3 files changed, 379 insertions(+)
create mode 100644 gnu/services/guix.scm
create mode 100644 gnu/tests/guix.scm
diff --git a/gnu/local.mk b/gnu/local.mk
index 3f32b9cbf2..b30afdd585 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -532,6 +532,7 @@ GNU_SYSTEM_MODULES = \
%D%/services/authentication.scm \
%D%/services/games.scm \
%D%/services/getmail.scm \
+ %D%/services/guix.scm \
%D%/services/kerberos.scm \
%D%/services/lirc.scm \
%D%/services/virtualization.scm \
@@ -596,6 +597,7 @@ GNU_SYSTEM_MODULES = \
%D%/tests/desktop.scm \
%D%/tests/dict.scm \
%D%/tests/docker.scm \
+ %D%/tests/guix.scm \
%D%/tests/monitoring.scm \
%D%/tests/nfs.scm \
%D%/tests/install.scm \
diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm
new file mode 100644
index 0000000000..65cffce2c0
--- /dev/null
+++ b/gnu/services/guix.scm
@@ -0,0 +1,204 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019 Christopher Baines <mail <at> cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify
+;;; it under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation, either version 3 of the License, or
+;;; (at your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services guix)
+ #:use-module (ice-9 match)
+ #:use-module (guix gexp)
+ #:use-module (guix records)
+ #:use-module ((gnu packages base)
+ #:select (glibc-utf8-locales))
+ #:use-module (gnu packages admin)
+ #:use-module (gnu packages web)
+ #:use-module (gnu services)
+ #:use-module (gnu services base)
+ #:use-module (gnu services admin)
+ #:use-module (gnu services shepherd)
+ #:use-module (gnu services getmail)
+ #:use-module (gnu system shadow)
+ #:export (<guix-data-service-configuration>
+ guix-data-service-configuration
+ guix-data-service-configuration?
+
+ guix-data-service-type))
+
+;;;; Commentary:
+;;;
+;;; This module implements a service that to run instances of the Guix Data
+;;; Service, which provides data about Guix over time.
+;;;
+;;;; Code:
+
+(define-record-type* <guix-data-service-configuration>
+ guix-data-service-configuration make-guix-data-service-configuration
+ guix-data-service-configuration?
+ (package guix-data-service-package
+ (default guix-data-service))
+ (user guix-data-service-configuration-user
+ (default "guix-data-service"))
+ (group guix-data-service-configuration-group
+ (default "guix-data-service"))
+ (port guix-data-service-port
+ (default 8765))
+ (host guix-data-service-host
+ (default "127.0.0.1"))
+ (getmail-idle-mailboxes
+ guix-data-service-getmail-idle-mailboxes
+ (default #f))
+ (commits-getmail-retriever-configuration
+ guix-data-service-commits-getmail-retriever-configuration
+ (default #f)))
+
+(define (guix-data-service-profile-packages config)
+ "Return the guix-data-service package, this will populate the
+ca-certificates.crt file in the system profile."
+ (list
+ (guix-data-service-package config)))
+
+(define (guix-data-service-shepherd-services config)
+ (match config
+ (($ <guix-data-service-configuration> package user group
+ port host)
+ (list
+ (shepherd-service
+ (documentation "Guix Data Service web server")
+ (provision '(guix-data-service))
+ (requirement '(postgres networking))
+ (start #~(make-forkexec-constructor
+ (list #$(file-append package
+ "/bin/guix-data-service")
+ "--pid-file=/var/run/guix-data-service/pid"
+ #$(string-append "--port=" (number->string port))
+ #$(string-append "--host=" host)
+ ;; Perform any database migrations when the
+ ;; service is started
+ "--update-database")
+
+ #:user #$user
+ #:group #$group
+ #:pid-file "/var/run/guix-data-service/pid"
+ ;; Allow time for migrations to run
+ #:pid-file-timeout 60
+ #:environment-variables
+ `(,(string-append
+ "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
+ "LC_ALL=en_US.utf8")
+ #:log-file "/var/log/guix-data-service/web.log"))
+ (stop #~(make-kill-destructor)))
+
+ (shepherd-service
+ (documentation "Guix Data Service process jobs")
+ (provision '(guix-data-service-process-jobs))
+ (requirement '(postgres
+ networking
+ ;; Require guix-data-service, as that the database
+ ;; migrations are handled through this service
+ guix-data-service))
+ (start #~(make-forkexec-constructor
+ (list
+ #$(file-append package
+ "/bin/guix-data-service-process-jobs"))
+ #:user #$user
+ #:group #$group
+ #:environment-variables
+ `("HOME=/var/lib/guix-data-service"
+ "GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt"
+ ,(string-append
+ "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
+ "LC_ALL=en_US.utf8")
+ #:log-file "/var/log/guix-data-service/process-jobs.log"))
+ (stop #~(make-kill-destructor)))))))
+
+(define (guix-data-service-activation config)
+ #~(begin
+ (use-modules (guix build utils))
+
+ (define %user (getpw "guix-data-service"))
+
+ (mkdir-p "/var/log/guix-data-service")
+
+ ;; Allow writing the PID file
+ (mkdir-p "/var/run/guix-data-service")
+ (chown "/var/run/guix-data-service"
+ (passwd:uid %user)
+ (passwd:gid %user))))
+
+(define (guix-data-service-account config)
+ (match config
+ (($ <guix-data-service-configuration> package user group)
+ (list (user-group
+ (name group)
+ (system? #t))
+ (user-account
+ (name user)
+ (group group)
+ (system? #t)
+ (comment "Guix Data Service user")
+ (home-directory "/var/lib/guix-data-service")
+ (shell (file-append shadow "/sbin/nologin")))))))
+
+(define (guix-data-service-getmail-configuration config)
+ (match config
+ (($ <guix-data-service-configuration> package user group
+ port host
+ #f #f)
+ '())
+ (($ <guix-data-service-configuration> package user group
+ port host
+ getmail-idle-mailboxes
+ commits-getmail-retriever-configuration)
+ (list
+ (getmail-configuration
+ (name 'guix-data-service)
+ (user user)
+ (group group)
+ (directory "/var/lib/getmail/guix-data-service")
+ (rcfile
+ (getmail-configuration-file
+ (retriever commits-getmail-retriever-configuration)
+ (destination
+ (getmail-destination-configuration
+ (type "MDA_external")
+ (path (file-append
+ package
+ "/bin/guix-data-service-process-branch-updated-email"))))
+ (options
+ (getmail-options-configuration
+ (read-all #f)
+ (delivered-to #f)
+ (received #f)))))
+ (idle getmail-idle-mailboxes))))))
+
+(define guix-data-service-type
+ (service-type
+ (name 'guix-data-service)
+ (extensions
+ (list
+ (service-extension profile-service-type
+ guix-data-service-profile-packages)
+ (service-extension shepherd-root-service-type
+ guix-data-service-shepherd-services)
+ (service-extension activation-service-type
+ guix-data-service-activation)
+ (service-extension account-service-type
+ guix-data-service-account)
+ (service-extension getmail-service-type
+ guix-data-service-getmail-configuration)))
+ (default-value
+ (guix-data-service-configuration))
+ (description
+ "Run an instance of the Guix Data Service.")))
diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm
new file mode 100644
index 0000000000..6139e31cf0
--- /dev/null
+++ b/gnu/tests/guix.scm
@@ -0,0 +1,173 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019 Christopher Baines <mail <at> cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests guix)
+ #:use-module (gnu tests)
+ #:use-module (gnu system)
+ #:use-module (gnu system file-systems)
+ #:use-module (gnu system shadow)
+ #:use-module (gnu system vm)
+ #:use-module (gnu services)
+ #:use-module (gnu services guix)
+ #:use-module (gnu services databases)
+ #:use-module (gnu services shepherd)
+ #:use-module (gnu services networking)
+ #:use-module (gnu packages databases)
+ #:use-module (guix packages)
+ #:use-module (guix modules)
+ #:use-module (guix records)
+ #:use-module (guix gexp)
+ #:use-module (guix store)
+ #:use-module (guix utils)
+ #:use-module (ice-9 match)
+ #:export (%test-guix-data-service))
+
+
+;;;
+;;; Guix Data Service
+;;;
+
+(define guix-data-service-initial-database-setup-service
+ (let ((user "guix_data_service")
+ (name "guix_data_service"))
+ (define start-gexp
+ #~(lambda ()
+ (let ((pid (primitive-fork))
+ (postgres (getpwnam "postgres")))
+ (if (eq? pid 0)
+ (dynamic-wind
+ (const #t)
+ (lambda ()
+ (setgid (passwd:gid postgres))
+ (setuid (passwd:uid postgres))
+ (primitive-exit
+ (if (and
+ (zero?
+ (system* #$(file-append postgresql "/bin/createuser")
+ #$user))
+ (zero?
+ (system* #$(file-append postgresql "/bin/createdb")
+ "-O" #$user #$name)))
+ 0
+ 1)))
+ (lambda ()
+ (primitive-exit 1)))
+ (zero? (cdr (waitpid pid)))))))
+
+ (shepherd-service
+ (requirement '(postgres))
+ (provision '(guix-data-service-initial-database-setup))
+ (start start-gexp)
+ (stop #~(const #f))
+ (respawn? #f)
+ (one-shot? #t)
+ (documentation "Setup Guix Data Service database."))))
+
+(define %guix-data-service-os
+ (simple-operating-system
+ (service dhcp-client-service-type)
+ (service postgresql-service-type
+ (postgresql-configuration
+ (config-file
+ (postgresql-config-file
+ (hba-file
+ (plain-file "pg_hba.conf"
+ "
+local all all trust
+host all all 127.0.0.1/32 trust
+host all all ::1/128 trust"))))))
+ (service guix-data-service-type
+ (guix-data-service-configuration
+ (host "0.0.0.0")))
+ (simple-service 'guix-data-service-database-setup
+ shepherd-root-service-type
+ (list guix-data-service-initial-database-setup-service))))
+
+(define (run-guix-data-service-test)
+ (define os
+ (marionette-operating-system
+ %guix-data-service-os
+ #:imported-modules '((gnu services herd)
+ (guix combinators))))
+
+ (define forwarded-port 8080)
+
+ (define vm
+ (virtual-machine
+ (operating-system os)
+ (memory-size 1024)
+ (port-forwardings `((,forwarded-port . 8765)))))
+
+ (define test
+ (with-imported-modules '((gnu build marionette))
+ #~(begin
+ (use-modules (srfi srfi-11) (srfi srfi-64)
+ (gnu build marionette)
+ (web uri)
+ (web client)
+ (web response))
+
+ (define marionette
+ (make-marionette (list #$vm)))
+
+ (mkdir #$output)
+ (chdir #$output)
+
+ (test-begin "guix-data-service")
+
+ (test-assert "service running"
+ (marionette-eval
+ '(begin
+ (use-modules (gnu services herd))
+ (match (start-service 'guix-data-service)
+ (#f #f)
+ (('service response-parts ...)
+ (match (assq-ref response-parts 'running)
+ ((pid) (number? pid))))))
+ marionette))
+
+ (test-assert "process jobs service running"
+ (marionette-eval
+ '(begin
+ (use-modules (gnu services herd))
+ (match (start-service 'guix-data-service-process-jobs)
+ (#f #f)
+ (('service response-parts ...)
+ (match (assq-ref response-parts 'running)
+ ((pid) (number? pid))))))
+ marionette))
+
+ (test-equal "http-get"
+ 200
+ (let-values
+ (((response text)
+ (http-get #$(simple-format
+ #f "http://localhost:~A/healthcheck" forwarded-port)
+ #:decode-body? #t)))
+ (response-code response)))
+
+ (test-end)
+ (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+ (gexp->derivation "guix-data-service-test" test))
+
+(define %test-guix-data-service
+ (system-test
+ (name "guix-data-service")
+ (description "Connect to a running Guix Data Service.")
+ (value (run-guix-data-service-test))))
--
2.23.0
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 15 Sep 2019 20:46:02 GMT) Full text and rfc822 format available.Message #14 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Alex ter Weele <alex.ter.weele <at> gmail.com> To: Christopher Baines <mail <at> cbaines.net> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 15 Sep 2019 15:45:06 -0500
Christopher Baines <mail <at> cbaines.net> writes:
> * gnu/packages/web.scm (guix-data-service): New variable.
> ---
> gnu/packages/web.scm | 83 ++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 83 insertions(+)
>
> diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
> index 39475bd6a8..30adc75aa1 100644
> --- a/gnu/packages/web.scm
> +++ b/gnu/packages/web.scm
> @@ -3973,6 +3973,89 @@ CDF, Atom 0.3, and Atom 1.0 feeds.")
> (define-public python2-feedparser
> (package-with-python2 python-feedparser))
>
> +(define-public guix-data-service
> + (let ((commit "6a9f11fb94fbfd969396dd4d9f768a0c9ec67734")
> + (revision "1"))
> + (package
> + (name "guix-data-service")
> + (version (string-append "0.0.1-" revision "." (string-take commit 7)))
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://git.savannah.gnu.org/git/guix/data-service.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "08kfi74dp8h1i03gfmjqfv07ff7m93psnh6xmrgwzr0ja5cdqjri"))))
> + (build-system gnu-build-system)
> + (arguments
> + '(#:tests? #f ; TODO Tests require PostgreSQL
> + #:modules ((guix build utils)
> + (guix build gnu-build-system)
> + (ice-9 rdelim)
> + (ice-9 popen))
> + #:phases
> + (modify-phases %standard-phases
> + (add-after 'install 'wrap-executable
> + (lambda* (#:key inputs outputs #:allow-other-keys)
> + (let* ((out (assoc-ref outputs "out"))
> + (bin (string-append out "/bin"))
> + (guile (assoc-ref inputs "guile"))
> + (guile-effective-version
> + (read-line
> + (open-pipe* OPEN_READ
> + (string-append guile "/bin/guile")
> + "-c" "(display (effective-version))")))
> + (scm (string-append out "/share/guile/site/"
> + guile-effective-version))
> + (go (string-append out "/lib/guile/"
> + guile-effective-version
> + "/site-ccache")))
> + (for-each
> + (lambda (file)
> + (wrap-program (string-append bin "/" file)
> + `("PATH" ":" prefix
> + (,bin))
> + `("GUILE_LOAD_PATH" ":" prefix
> + (,scm ,(getenv "GUILE_LOAD_PATH")))
> + `("GUILE_LOAD_COMPILED_PATH" ":" prefix
> + (,go ,(getenv "GUILE_LOAD_COMPILED_PATH")))))
> + '("guix-data-service"
> + "guix-data-service-process-branch-updated-email"
> + "guix-data-service-process-job"
> + "guix-data-service-process-jobs"
> + "guix-data-service-query-build-servers"))
> + #t))))))
> + (inputs
> + `(("guile" ,guile-2.2)
> + ("guix" ,guix)
> + ("guile-fibers" ,guile-fibers)
> + ("guile-json" ,guile-json-3)
> + ("guile-email" ,guile-email)
> + ("guile-squee" ,guile-squee)
> + ("postgresql" ,postgresql)
> + ("sqitch" ,sqitch)))
> + (native-inputs
> + `(("autoconf" ,autoconf)
> + ("automake" ,automake)
> + ("pkg-config" ,pkg-config)))
> + (native-search-paths
> + ;; guile-git requires this to be set
> + (list (search-path-specification
> + (variable "GIT_SSL_CAINFO")
> + (file-type 'regular)
> + (separator #f)
> + (files '("etc/ssl/certs/ca-certificates.crt")))))
> + (synopsis "Store and provide data about GNU Guix")
> + (description
> + "The Guix Data Service stores data about GNU Guix, and provides this
> +through a web interface. It supports listening to the guix-commits mailing
> +list to find out about new revisions, then loads the data from these in to a
> +PostgreSQL database.")
> + (home-page "https://data.guix.gnu.org/")
I found that only http://data.guix.gnu.org/ works.
>
> + (license license:agpl3+)))) + (define-public gumbo-parser (package
> (name "gumbo-parser")
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 15 Sep 2019 21:30:02 GMT) Full text and rfc822 format available.Message #17 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: Alex ter Weele <alex.ter.weele <at> gmail.com> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 15 Sep 2019 22:29:50 +0100
[Message part 1 (text/plain, inline)]
Alex ter Weele <alex.ter.weele <at> gmail.com> writes: >> + (home-page "https://data.guix.gnu.org/") > > I found that only http://data.guix.gnu.org/ works. Good spot. When I wrote this, I was planning on setting up HTTPS for data.guix.gnu.org, but when I came to do it, I didn't quite know how to configure it. It's still on my list of things to do, but I've now changed the patch to use http.
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Mon, 16 Sep 2019 07:49:01 GMT) Full text and rfc822 format available.Message #20 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Christopher Baines <mail <at> cbaines.net> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Mon, 16 Sep 2019 09:48:29 +0200
Good morning, Chris!
Christopher Baines <mail <at> cbaines.net> skribis:
> * gnu/packages/web.scm (guix-data-service): New variable.
Woohoo! \o/
> + (inputs
> + `(("guile" ,guile-2.2)
Guile should be ‘native-inputs’ because we use it for its compiler
(which can act as a “cross-compiler” with ‘--target’).
> + (native-search-paths
> + ;; guile-git requires this to be set
> + (list (search-path-specification
> + (variable "GIT_SSL_CAINFO")
> + (file-type 'regular)
> + (separator #f)
> + (files '("etc/ssl/certs/ca-certificates.crt")))))
It’s a bit of a workaround for
<https://issues.guix.gnu.org/issue/22138>, isn’t it? :-)
What about either removing it, or adding an “XXX” to the comment?
Otherwise LGTM!
Ludo’.
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Mon, 16 Sep 2019 07:54:02 GMT) Full text and rfc822 format available.Message #23 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Christopher Baines <mail <at> cbaines.net> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 2/2] services: Add the Guix Data Service. Date: Mon, 16 Sep 2019 09:53:39 +0200
Christopher Baines <mail <at> cbaines.net> skribis: > * gnu/services/guix.scm: New file. > * gnu/tests/guix.scm: New file. > * gnu/local.mk (GNU_SYSTEM_MODULES): Add both new files. Yay, nice! > + (commits-getmail-retriever-configuration > + guix-data-service-commits-getmail-retriever-configuration New world record: longer than ‘call-with-current-continuation’. :-) > +(define (guix-data-service-shepherd-services config) > + (match config > + (($ <guix-data-service-configuration> package user group > + port host) Perhaps use ‘match-record’ here and elsewhere, to avoid mistakes when the record type changes. > +(define %test-guix-data-service > + (system-test > + (name "guix-data-service") > + (description "Connect to a running Guix Data Service.") > + (value (run-guix-data-service-test)))) Nice. Could you add a bit to guix.texi, maybe under “Continuous Integration”? You don’t have to go into too much detail if you think some of the configuration elements are subject to change. Thank you! Ludo’.
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Mon, 16 Sep 2019 07:56:01 GMT) Full text and rfc822 format available.Message #26 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Christopher Baines <mail <at> cbaines.net> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Mon, 16 Sep 2019 09:55:41 +0200
Christopher Baines <mail <at> cbaines.net> skribis: > + (home-page "https://data.guix.gnu.org/") It would be nice to add a reference to data.guix.gnu.org at <https://guix.gnu.org/contribute/> or in the manual. WDYT? Ludo’.
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 10:11:01 GMT) Full text and rfc822 format available.Message #29 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 22 Sep 2019 11:10:53 +0100
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes: > Christopher Baines <mail <at> cbaines.net> skribis: > >> + (home-page "https://data.guix.gnu.org/") > > It would be nice to add a reference to data.guix.gnu.org at > <https://guix.gnu.org/contribute/> or in the manual. WDYT? Yeah, maybe there could be a "Services" section, and that could mention Cuirass, Mumi, and the Guix Data Service.
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 10:19:02 GMT) Full text and rfc822 format available.Message #32 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 22 Sep 2019 11:18:35 +0100
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes:
> Good morning, Chris!
>
> Christopher Baines <mail <at> cbaines.net> skribis:
>
>> * gnu/packages/web.scm (guix-data-service): New variable.
>
> Woohoo! \o/
>
>> + (inputs
>> + `(("guile" ,guile-2.2)
>
> Guile should be ‘native-inputs’ because we use it for its compiler
> (which can act as a “cross-compiler” with ‘--target’).
This makes sense, but the scripts will still refer to the guile used to
build the package, so building for a different architecture won't work
right? Does the package somehow need to take as input two Guiles, one
for build time, and one for runtime?
>> + (native-search-paths
>> + ;; guile-git requires this to be set
>> + (list (search-path-specification
>> + (variable "GIT_SSL_CAINFO")
>> + (file-type 'regular)
>> + (separator #f)
>> + (files '("etc/ssl/certs/ca-certificates.crt")))))
>
> It’s a bit of a workaround for
> <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-)
>
> What about either removing it, or adding an “XXX” to the comment?
I can't quite remember, I think I added that a long while ago when
trying to get the Guix service working. I'm not sure it's a great
approach. I'll add XXX to the comment though.
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 12:04:02 GMT) Full text and rfc822 format available.Message #35 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: 37412 <at> debbugs.gnu.org Subject: [PATCH 2/2] services: Add the Guix Data Service. Date: Sun, 22 Sep 2019 13:03:37 +0100
* gnu/services/guix.scm: New file.
* gnu/tests/guix.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add both new files.
* doc/guix.texi (Guix Services): New section documenting the Guix Data
Service.
---
doc/guix.texi | 52 +++++++++++
gnu/local.mk | 2 +
gnu/services/guix.scm | 212 ++++++++++++++++++++++++++++++++++++++++++
gnu/tests/guix.scm | 173 ++++++++++++++++++++++++++++++++++
4 files changed, 439 insertions(+)
create mode 100644 gnu/services/guix.scm
create mode 100644 gnu/tests/guix.scm
diff --git a/doc/guix.texi b/doc/guix.texi
index 39d4b865f6..24da18ea13 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -11657,6 +11657,7 @@ declaration.
* Virtualization Services:: Virtualization services.
* Version Control Services:: Providing remote access to Git repositories.
* Game Services:: Game servers.
+* Guix Services:: Services relating specifically to Guix.
* Miscellaneous Services:: Other services.
@end menu
@@ -24167,6 +24168,57 @@ The port to bind the server to.
@end table
@end deftp
+
+@node Guix Services
+@subsection Guix Services
+
+@subsubheading Guix Data Service
+The @uref{http://data.guix.gnu.org,Guix Data Service} processes, stores
+and provides data about GNU Guix. This includes information about
+packages, derivations and lint warnings.
+
+The data is stored in a PostgreSQL database, and available through a web
+interface.
+
+@defvar {Scheme Variable} guix-data-service-type
+Service type for the Guix Data Service. Its value must be a
+@code{guix-data-service-configuration} object. The service optionally
+extends the getmail service, as the guix-commits mailing list is used to
+find out about changes in the Guix git repository.
+@end defvar
+
+@deftp {Data Type} guix-data-service-configuration
+Data type representing the configuration of the Guix Data Service.
+
+@table @asis
+@item @code{package} (default: @code{guix-data-service})
+The Guix Data Service package to use.
+
+@item @code{user} (default: @code{"guix-data-service"})
+The system user to run the service as.
+
+@item @code{group} (default: @code{"guix-data-service"})
+The system group to run the service as.
+
+@item @code{port} (default: @code{8765})
+The port to bind the web service to.
+
+@item @code{host} (default: @code{"127.0.0.1"})
+The host to bind the web service to.
+
+@item @code{getmail-idle-mailboxes} (default: @code{#f})
+If set, this is the list of mailboxes that the getmail service will be
+configured to listen to.
+
+@item @code{commits-getmail-retriever-configuration} (default: @code{#f})
+If set, this is the @code{getmail-retriever-configuration} object with
+which to configure getmail to fetch mail from the guix-commits mailing
+list.
+
+@end table
+@end deftp
+
+
@node Miscellaneous Services
@subsection Miscellaneous Services
diff --git a/gnu/local.mk b/gnu/local.mk
index 3f32b9cbf2..b30afdd585 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -532,6 +532,7 @@ GNU_SYSTEM_MODULES = \
%D%/services/authentication.scm \
%D%/services/games.scm \
%D%/services/getmail.scm \
+ %D%/services/guix.scm \
%D%/services/kerberos.scm \
%D%/services/lirc.scm \
%D%/services/virtualization.scm \
@@ -596,6 +597,7 @@ GNU_SYSTEM_MODULES = \
%D%/tests/desktop.scm \
%D%/tests/dict.scm \
%D%/tests/docker.scm \
+ %D%/tests/guix.scm \
%D%/tests/monitoring.scm \
%D%/tests/nfs.scm \
%D%/tests/install.scm \
diff --git a/gnu/services/guix.scm b/gnu/services/guix.scm
new file mode 100644
index 0000000000..0f0fad39b0
--- /dev/null
+++ b/gnu/services/guix.scm
@@ -0,0 +1,212 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019 Christopher Baines <mail <at> cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify
+;;; it under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation, either version 3 of the License, or
+;;; (at your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful,
+;;; but WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu services guix)
+ #:use-module (ice-9 match)
+ #:use-module (guix gexp)
+ #:use-module (guix records)
+ #:use-module ((gnu packages base)
+ #:select (glibc-utf8-locales))
+ #:use-module (gnu packages admin)
+ #:use-module (gnu packages web)
+ #:use-module (gnu services)
+ #:use-module (gnu services base)
+ #:use-module (gnu services admin)
+ #:use-module (gnu services shepherd)
+ #:use-module (gnu services getmail)
+ #:use-module (gnu system shadow)
+ #:export (<guix-data-service-configuration>
+ guix-data-service-configuration
+ guix-data-service-configuration?
+ guix-data-service-package
+ guix-data-service-user
+ guix-data-service-group
+ guix-data-service-port
+ guix-data-service-host
+ guix-data-service-getmail-idle-mailboxes
+ guix-data-service-commits-getmail-retriever-configuration
+
+ guix-data-service-type))
+
+;;;; Commentary:
+;;;
+;;; This module implements a service that to run instances of the Guix Data
+;;; Service, which provides data about Guix over time.
+;;;
+;;;; Code:
+
+(define-record-type* <guix-data-service-configuration>
+ guix-data-service-configuration make-guix-data-service-configuration
+ guix-data-service-configuration?
+ (package guix-data-service-package
+ (default guix-data-service))
+ (user guix-data-service-configuration-user
+ (default "guix-data-service"))
+ (group guix-data-service-configuration-group
+ (default "guix-data-service"))
+ (port guix-data-service-port
+ (default 8765))
+ (host guix-data-service-host
+ (default "127.0.0.1"))
+ (getmail-idle-mailboxes
+ guix-data-service-getmail-idle-mailboxes
+ (default #f))
+ (commits-getmail-retriever-configuration
+ guix-data-service-commits-getmail-retriever-configuration
+ (default #f)))
+
+(define (guix-data-service-profile-packages config)
+ "Return the guix-data-service package, this will populate the
+ca-certificates.crt file in the system profile."
+ (list
+ (guix-data-service-package config)))
+
+(define (guix-data-service-shepherd-services config)
+ (match-record config <guix-data-service-configuration>
+ (package user group port host)
+ (list
+ (shepherd-service
+ (documentation "Guix Data Service web server")
+ (provision '(guix-data-service))
+ (requirement '(postgres networking))
+ (start #~(make-forkexec-constructor
+ (list #$(file-append package
+ "/bin/guix-data-service")
+ "--pid-file=/var/run/guix-data-service/pid"
+ #$(string-append "--port=" (number->string port))
+ #$(string-append "--host=" host)
+ ;; Perform any database migrations when the
+ ;; service is started
+ "--update-database")
+
+ #:user #$user
+ #:group #$group
+ #:pid-file "/var/run/guix-data-service/pid"
+ ;; Allow time for migrations to run
+ #:pid-file-timeout 60
+ #:environment-variables
+ `(,(string-append
+ "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
+ "LC_ALL=en_US.utf8")
+ #:log-file "/var/log/guix-data-service/web.log"))
+ (stop #~(make-kill-destructor)))
+
+ (shepherd-service
+ (documentation "Guix Data Service process jobs")
+ (provision '(guix-data-service-process-jobs))
+ (requirement '(postgres
+ networking
+ ;; Require guix-data-service, as that the database
+ ;; migrations are handled through this service
+ guix-data-service))
+ (start #~(make-forkexec-constructor
+ (list
+ #$(file-append package
+ "/bin/guix-data-service-process-jobs"))
+ #:user #$user
+ #:group #$group
+ #:environment-variables
+ `("HOME=/var/lib/guix-data-service"
+ "GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt"
+ ,(string-append
+ "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
+ "LC_ALL=en_US.utf8")
+ #:log-file "/var/log/guix-data-service/process-jobs.log"))
+ (stop #~(make-kill-destructor))))))
+
+(define (guix-data-service-activation config)
+ #~(begin
+ (use-modules (guix build utils))
+
+ (define %user (getpw "guix-data-service"))
+
+ (chmod "/var/lib/guix-data-service" #o755)
+
+ (mkdir-p "/var/log/guix-data-service")
+
+ ;; Allow writing the PID file
+ (mkdir-p "/var/run/guix-data-service")
+ (chown "/var/run/guix-data-service"
+ (passwd:uid %user)
+ (passwd:gid %user))))
+
+(define (guix-data-service-account config)
+ (match-record config <guix-data-service-configuration>
+ (user group)
+ (list (user-group
+ (name group)
+ (system? #t))
+ (user-account
+ (name user)
+ (group group)
+ (system? #t)
+ (comment "Guix Data Service user")
+ (home-directory "/var/lib/guix-data-service")
+ (shell (file-append shadow "/sbin/nologin"))))))
+
+(define (guix-data-service-getmail-configuration config)
+ (match config
+ (($ <guix-data-service-configuration> package user group
+ port host
+ #f #f)
+ '())
+ (($ <guix-data-service-configuration> package user group
+ port host
+ getmail-idle-mailboxes
+ commits-getmail-retriever-configuration)
+ (list
+ (getmail-configuration
+ (name 'guix-data-service)
+ (user user)
+ (group group)
+ (directory "/var/lib/getmail/guix-data-service")
+ (rcfile
+ (getmail-configuration-file
+ (retriever commits-getmail-retriever-configuration)
+ (destination
+ (getmail-destination-configuration
+ (type "MDA_external")
+ (path (file-append
+ package
+ "/bin/guix-data-service-process-branch-updated-email"))))
+ (options
+ (getmail-options-configuration
+ (read-all #f)
+ (delivered-to #f)
+ (received #f)))))
+ (idle getmail-idle-mailboxes))))))
+
+(define guix-data-service-type
+ (service-type
+ (name 'guix-data-service)
+ (extensions
+ (list
+ (service-extension profile-service-type
+ guix-data-service-profile-packages)
+ (service-extension shepherd-root-service-type
+ guix-data-service-shepherd-services)
+ (service-extension activation-service-type
+ guix-data-service-activation)
+ (service-extension account-service-type
+ guix-data-service-account)
+ (service-extension getmail-service-type
+ guix-data-service-getmail-configuration)))
+ (default-value
+ (guix-data-service-configuration))
+ (description
+ "Run an instance of the Guix Data Service.")))
diff --git a/gnu/tests/guix.scm b/gnu/tests/guix.scm
new file mode 100644
index 0000000000..6139e31cf0
--- /dev/null
+++ b/gnu/tests/guix.scm
@@ -0,0 +1,173 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2019 Christopher Baines <mail <at> cbaines.net>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu tests guix)
+ #:use-module (gnu tests)
+ #:use-module (gnu system)
+ #:use-module (gnu system file-systems)
+ #:use-module (gnu system shadow)
+ #:use-module (gnu system vm)
+ #:use-module (gnu services)
+ #:use-module (gnu services guix)
+ #:use-module (gnu services databases)
+ #:use-module (gnu services shepherd)
+ #:use-module (gnu services networking)
+ #:use-module (gnu packages databases)
+ #:use-module (guix packages)
+ #:use-module (guix modules)
+ #:use-module (guix records)
+ #:use-module (guix gexp)
+ #:use-module (guix store)
+ #:use-module (guix utils)
+ #:use-module (ice-9 match)
+ #:export (%test-guix-data-service))
+
+
+;;;
+;;; Guix Data Service
+;;;
+
+(define guix-data-service-initial-database-setup-service
+ (let ((user "guix_data_service")
+ (name "guix_data_service"))
+ (define start-gexp
+ #~(lambda ()
+ (let ((pid (primitive-fork))
+ (postgres (getpwnam "postgres")))
+ (if (eq? pid 0)
+ (dynamic-wind
+ (const #t)
+ (lambda ()
+ (setgid (passwd:gid postgres))
+ (setuid (passwd:uid postgres))
+ (primitive-exit
+ (if (and
+ (zero?
+ (system* #$(file-append postgresql "/bin/createuser")
+ #$user))
+ (zero?
+ (system* #$(file-append postgresql "/bin/createdb")
+ "-O" #$user #$name)))
+ 0
+ 1)))
+ (lambda ()
+ (primitive-exit 1)))
+ (zero? (cdr (waitpid pid)))))))
+
+ (shepherd-service
+ (requirement '(postgres))
+ (provision '(guix-data-service-initial-database-setup))
+ (start start-gexp)
+ (stop #~(const #f))
+ (respawn? #f)
+ (one-shot? #t)
+ (documentation "Setup Guix Data Service database."))))
+
+(define %guix-data-service-os
+ (simple-operating-system
+ (service dhcp-client-service-type)
+ (service postgresql-service-type
+ (postgresql-configuration
+ (config-file
+ (postgresql-config-file
+ (hba-file
+ (plain-file "pg_hba.conf"
+ "
+local all all trust
+host all all 127.0.0.1/32 trust
+host all all ::1/128 trust"))))))
+ (service guix-data-service-type
+ (guix-data-service-configuration
+ (host "0.0.0.0")))
+ (simple-service 'guix-data-service-database-setup
+ shepherd-root-service-type
+ (list guix-data-service-initial-database-setup-service))))
+
+(define (run-guix-data-service-test)
+ (define os
+ (marionette-operating-system
+ %guix-data-service-os
+ #:imported-modules '((gnu services herd)
+ (guix combinators))))
+
+ (define forwarded-port 8080)
+
+ (define vm
+ (virtual-machine
+ (operating-system os)
+ (memory-size 1024)
+ (port-forwardings `((,forwarded-port . 8765)))))
+
+ (define test
+ (with-imported-modules '((gnu build marionette))
+ #~(begin
+ (use-modules (srfi srfi-11) (srfi srfi-64)
+ (gnu build marionette)
+ (web uri)
+ (web client)
+ (web response))
+
+ (define marionette
+ (make-marionette (list #$vm)))
+
+ (mkdir #$output)
+ (chdir #$output)
+
+ (test-begin "guix-data-service")
+
+ (test-assert "service running"
+ (marionette-eval
+ '(begin
+ (use-modules (gnu services herd))
+ (match (start-service 'guix-data-service)
+ (#f #f)
+ (('service response-parts ...)
+ (match (assq-ref response-parts 'running)
+ ((pid) (number? pid))))))
+ marionette))
+
+ (test-assert "process jobs service running"
+ (marionette-eval
+ '(begin
+ (use-modules (gnu services herd))
+ (match (start-service 'guix-data-service-process-jobs)
+ (#f #f)
+ (('service response-parts ...)
+ (match (assq-ref response-parts 'running)
+ ((pid) (number? pid))))))
+ marionette))
+
+ (test-equal "http-get"
+ 200
+ (let-values
+ (((response text)
+ (http-get #$(simple-format
+ #f "http://localhost:~A/healthcheck" forwarded-port)
+ #:decode-body? #t)))
+ (response-code response)))
+
+ (test-end)
+ (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+ (gexp->derivation "guix-data-service-test" test))
+
+(define %test-guix-data-service
+ (system-test
+ (name "guix-data-service")
+ (description "Connect to a running Guix Data Service.")
+ (value (run-guix-data-service-test))))
--
2.23.0
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 12:04:02 GMT) Full text and rfc822 format available.Message #38 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: 37412 <at> debbugs.gnu.org Subject: [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 22 Sep 2019 13:03:36 +0100
* gnu/packages/web.scm (guix-data-service): New variable.
---
gnu/packages/web.scm | 102 +++++++++++++++++++++++++++++++++++++++++++
1 file changed, 102 insertions(+)
diff --git a/gnu/packages/web.scm b/gnu/packages/web.scm
index 39475bd6a8..065979f959 100644
--- a/gnu/packages/web.scm
+++ b/gnu/packages/web.scm
@@ -56,6 +56,7 @@
#:use-module ((guix licenses) #:prefix license:)
#:use-module (guix packages)
#:use-module (guix download)
+ #:use-module (guix gexp)
#:use-module (guix git-download)
#:use-module (guix cvs-download)
#:use-module (guix hg-download)
@@ -77,6 +78,8 @@
#:use-module (gnu packages check)
#:use-module (gnu packages documentation)
#:use-module (gnu packages docbook)
+ #:use-module (gnu packages emacs)
+ #:use-module (gnu packages emacs-xyz)
#:use-module (gnu packages autotools)
#:use-module (gnu packages compression)
#:use-module (gnu packages curl)
@@ -3973,6 +3976,105 @@ CDF, Atom 0.3, and Atom 1.0 feeds.")
(define-public python2-feedparser
(package-with-python2 python-feedparser))
+(define-public guix-data-service
+ (let ((commit "970bcb31cf994b0e3b67e7f912734e5b53355d8d")
+ (revision "1"))
+ (package
+ (name "guix-data-service")
+ (version (string-append "0.0.1-" revision "." (string-take commit 7)))
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://git.savannah.gnu.org/git/guix/data-service.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0wiw1m4g30sx155ldjrg4060ydqwgxv93day89pf6w515n5fv08j"))))
+ (build-system gnu-build-system)
+ (arguments
+ '(#:tests? #f ; TODO Tests require PostgreSQL
+ #:modules ((guix build utils)
+ (guix build gnu-build-system)
+ (ice-9 rdelim)
+ (ice-9 popen))
+ #:phases
+ (modify-phases %standard-phases
+ (add-after 'set-paths 'set-GUIX_ENVIRONMENT
+ (lambda* (#:key inputs #:allow-other-keys)
+ ;; This means guix.el finds the Emacs modules
+ (setenv "GUIX_ENVIRONMENT"
+ (assoc-ref inputs "emacs-with-modules"))
+ #t))
+ (add-before 'build 'set-GUILE_AUTO_COMPILE
+ (lambda _
+ ;; To avoid errors relating to guild
+ (setenv "GUILE_AUTO_COMPILE" "0")
+ #t))
+ (add-after 'install 'wrap-executable
+ (lambda* (#:key inputs outputs #:allow-other-keys)
+ (let* ((out (assoc-ref outputs "out"))
+ (bin (string-append out "/bin"))
+ (guile (assoc-ref inputs "guile"))
+ (guile-effective-version
+ (read-line
+ (open-pipe* OPEN_READ
+ (string-append guile "/bin/guile")
+ "-c" "(display (effective-version))")))
+ (scm (string-append out "/share/guile/site/"
+ guile-effective-version))
+ (go (string-append out "/lib/guile/"
+ guile-effective-version
+ "/site-ccache")))
+ (for-each
+ (lambda (file)
+ (wrap-program (string-append bin "/" file)
+ `("PATH" ":" prefix
+ (,bin))
+ `("GUILE_LOAD_PATH" ":" prefix
+ (,scm ,(getenv "GUILE_LOAD_PATH")))
+ `("GUILE_LOAD_COMPILED_PATH" ":" prefix
+ (,go ,(getenv "GUILE_LOAD_COMPILED_PATH")))))
+ '("guix-data-service"
+ "guix-data-service-process-branch-updated-email"
+ "guix-data-service-process-job"
+ "guix-data-service-process-jobs"
+ "guix-data-service-query-build-servers"))
+ #t)))
+ (delete 'strip)))) ; As the .go files aren't compatible
+ (inputs
+ `(("guile" ,guile-2.2)
+ ("guix" ,guix)
+ ("guile-fibers" ,guile-fibers)
+ ("guile-json" ,guile-json-3)
+ ("guile-email" ,guile-email)
+ ("guile-squee" ,guile-squee)
+ ("postgresql" ,postgresql)
+ ("sqitch" ,sqitch)))
+ (native-inputs
+ `(("autoconf" ,autoconf)
+ ("automake" ,automake)
+ ("emacs-with-modules" ,(directory-union
+ "emacs-union"
+ (list emacs-no-x
+ emacs-htmlize)))
+ ("pkg-config" ,pkg-config)))
+ (native-search-paths
+ ;; XXX guile-git requires this to be set, maybe there's a better way
+ (list (search-path-specification
+ (variable "GIT_SSL_CAINFO")
+ (file-type 'regular)
+ (separator #f)
+ (files '("etc/ssl/certs/ca-certificates.crt")))))
+ (synopsis "Store and provide data about GNU Guix")
+ (description
+ "The Guix Data Service stores data about GNU Guix, and provides this
+through a web interface. It supports listening to the guix-commits mailing
+list to find out about new revisions, then loads the data from these in to a
+PostgreSQL database.")
+ (home-page "http://data.guix.gnu.org/")
+ (license license:agpl3+))))
+
(define-public gumbo-parser
(package
(name "gumbo-parser")
--
2.23.0
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 12:16:01 GMT) Full text and rfc822 format available.Message #41 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 2/2] services: Add the Guix Data Service. Date: Sun, 22 Sep 2019 13:15:42 +0100
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes: > Christopher Baines <mail <at> cbaines.net> skribis: > >> +(define (guix-data-service-shepherd-services config) >> + (match config >> + (($ <guix-data-service-configuration> package user group >> + port host) > > Perhaps use ‘match-record’ here and elsewhere, to avoid mistakes when > the record type changes. Yeah, I've switched a couple of the match statements across to match-record, but I've left one, as it has a couple of clauses. >> +(define %test-guix-data-service >> + (system-test >> + (name "guix-data-service") >> + (description "Connect to a running Guix Data Service.") >> + (value (run-guix-data-service-test)))) > > Nice. > > Could you add a bit to guix.texi, maybe under “Continuous Integration”? > You don’t have to go into too much detail if you think some of the > configuration elements are subject to change. I forgot you mentioned the "Continuous Integration" section, so I actually added something under a new "Guix" section, but regardless, I've now sent some new patches, which includes some documentation.
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 21:05:01 GMT) Full text and rfc822 format available.Message #44 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Christopher Baines <mail <at> cbaines.net> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 22 Sep 2019 23:04:24 +0200
Hi Chris,
Christopher Baines <mail <at> cbaines.net> skribis:
> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Good morning, Chris!
>>
>> Christopher Baines <mail <at> cbaines.net> skribis:
>>
>>> * gnu/packages/web.scm (guix-data-service): New variable.
>>
>> Woohoo! \o/
>>
>>> + (inputs
>>> + `(("guile" ,guile-2.2)
>>
>> Guile should be ‘native-inputs’ because we use it for its compiler
>> (which can act as a “cross-compiler” with ‘--target’).
>
> This makes sense, but the scripts will still refer to the guile used to
> build the package, so building for a different architecture won't work
> right? Does the package somehow need to take as input two Guiles, one
> for build time, and one for runtime?
Yes, we probably need to have Guile both as input and native input.
Well don’t lose your hair on it anyway.
>>> + (native-search-paths
>>> + ;; guile-git requires this to be set
>>> + (list (search-path-specification
>>> + (variable "GIT_SSL_CAINFO")
>>> + (file-type 'regular)
>>> + (separator #f)
>>> + (files '("etc/ssl/certs/ca-certificates.crt")))))
>>
>> It’s a bit of a workaround for
>> <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-)
>>
>> What about either removing it, or adding an “XXX” to the comment?
>
> I can't quite remember, I think I added that a long while ago when
> trying to get the Guix service working. I'm not sure it's a great
> approach. I'll add XXX to the comment though.
What about removing it altogether and instead setting GIT_SSL_CAINFO in
the service itself?
Thanks,
Ludo’.
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Sun, 22 Sep 2019 21:12:01 GMT) Full text and rfc822 format available.Message #47 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Christopher Baines <mail <at> cbaines.net> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Sun, 22 Sep 2019 23:11:16 +0200
Christopher Baines <mail <at> cbaines.net> skribis:
> * gnu/packages/web.scm (guix-data-service): New variable.
[...]
> + (native-inputs
> + `(("autoconf" ,autoconf)
> + ("automake" ,automake)
> + ("emacs-with-modules" ,(directory-union
> + "emacs-union"
> + (list emacs-no-x
> + emacs-htmlize)))
Should it be ‘emacs-minimal’?
> + (native-search-paths
> + ;; XXX guile-git requires this to be set, maybe there's a better way
> + (list (search-path-specification
> + (variable "GIT_SSL_CAINFO")
> + (file-type 'regular)
> + (separator #f)
> + (files '("etc/ssl/certs/ca-certificates.crt")))))
Like I wrote, I’d rather remove the search path and add the environment
variable…
> + (start #~(make-forkexec-constructor
> + (list #$(file-append package
> + "/bin/guix-data-service")
> + "--pid-file=/var/run/guix-data-service/pid"
> + #$(string-append "--port=" (number->string port))
> + #$(string-append "--host=" host)
> + ;; Perform any database migrations when the
> + ;; service is started
> + "--update-database")
> +
> + #:user #$user
> + #:group #$group
> + #:pid-file "/var/run/guix-data-service/pid"
> + ;; Allow time for migrations to run
> + #:pid-file-timeout 60
> + #:environment-variables
> + `(,(string-append
> + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
> + "LC_ALL=en_US.utf8")
> + #:log-file "/var/log/guix-data-service/web.log"))
… here. ↑
But anyway, these are details so I think you can go ahead and push as
you see fit.
Thanks!
Ludo’.
guix-patches <at> gnu.org:bug#37412; Package guix-patches.
(Tue, 24 Sep 2019 19:36:02 GMT) Full text and rfc822 format available.Message #50 received at 37412 <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 37412 <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Tue, 24 Sep 2019 20:34:55 +0100
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes:
> Christopher Baines <mail <at> cbaines.net> skribis:
>
>> * gnu/packages/web.scm (guix-data-service): New variable.
>
> [...]
>
>> + (native-inputs
>> + `(("autoconf" ,autoconf)
>> + ("automake" ,automake)
>> + ("emacs-with-modules" ,(directory-union
>> + "emacs-union"
>> + (list emacs-no-x
>> + emacs-htmlize)))
>
> Should it be ‘emacs-minimal’?
It probably could be, but at the moment this doesn't work with the Emacs
Lisp code that's run at package build time to convert the README to HTML
[1].
1: https://git.savannah.gnu.org/cgit/guix/data-service.git/tree/Makefile.am#n35
Do you know if there's a way to remove the need for the autoloading
behaviour from the guix-emacs.el file? The error I get when using
emacs-minimal suggests htmlize isn't being found.
>> + (native-search-paths
>> + ;; XXX guile-git requires this to be set, maybe there's a better way
>> + (list (search-path-specification
>> + (variable "GIT_SSL_CAINFO")
>> + (file-type 'regular)
>> + (separator #f)
>> + (files '("etc/ssl/certs/ca-certificates.crt")))))
>
> Like I wrote, I’d rather remove the search path and add the environment
> variable…
Yeah, I'm actually unsure if this is doing anything. I'll remove it.
>> + (start #~(make-forkexec-constructor
>> + (list #$(file-append package
>> + "/bin/guix-data-service")
>> + "--pid-file=/var/run/guix-data-service/pid"
>> + #$(string-append "--port=" (number->string port))
>> + #$(string-append "--host=" host)
>> + ;; Perform any database migrations when the
>> + ;; service is started
>> + "--update-database")
>> +
>> + #:user #$user
>> + #:group #$group
>> + #:pid-file "/var/run/guix-data-service/pid"
>> + ;; Allow time for migrations to run
>> + #:pid-file-timeout 60
>> + #:environment-variables
>> + `(,(string-append
>> + "GUIX_LOCPATH=" #$glibc-utf8-locales "/lib/locale")
>> + "LC_ALL=en_US.utf8")
>> + #:log-file "/var/log/guix-data-service/web.log"))
>
> … here. ↑
So, this shepherd service which serves requests doesn't do any Git
access, and the process jobs shepherd service defined below actually
already sets it:
"GIT_SSL_CAINFO=/etc/ssl/certs/ca-certificates.crt"
So yeah, I think I can just remove the native-search-paths bit from the
package definition.
> But anyway, these are details so I think you can go ahead and push as
> you see fit.
Great, thanks for taking a look Ludo!
Chris
[signature.asc (application/pgp-signature, inline)]
Christopher Baines <mail <at> cbaines.net>:Christopher Baines <mail <at> cbaines.net>:Message #55 received at 37412-done <at> debbugs.gnu.org (full text, mbox):
From: Christopher Baines <mail <at> cbaines.net> To: Ludovic Courtès <ludo <at> gnu.org> Cc: 37412-done <at> debbugs.gnu.org Subject: Re: [bug#37412] [PATCH 1/2] gnu: Add guix-data-service. Date: Wed, 25 Sep 2019 08:46:20 +0100
[Message part 1 (text/plain, inline)]
Ludovic Courtès <ludo <at> gnu.org> writes:
> Hi Chris,
>
> Christopher Baines <mail <at> cbaines.net> skribis:
>
>> Ludovic Courtès <ludo <at> gnu.org> writes:
>>
>>> Good morning, Chris!
>>>
>>> Christopher Baines <mail <at> cbaines.net> skribis:
>>>
>>>> * gnu/packages/web.scm (guix-data-service): New variable.
>>>
>>> Woohoo! \o/
>>>
>>>> + (inputs
>>>> + `(("guile" ,guile-2.2)
>>>
>>> Guile should be ‘native-inputs’ because we use it for its compiler
>>> (which can act as a “cross-compiler” with ‘--target’).
>>
>> This makes sense, but the scripts will still refer to the guile used to
>> build the package, so building for a different architecture won't work
>> right? Does the package somehow need to take as input two Guiles, one
>> for build time, and one for runtime?
>
> Yes, we probably need to have Guile both as input and native input.
> Well don’t lose your hair on it anyway.
I've just gone with making it a native input for now.
>>>> + (native-search-paths
>>>> + ;; guile-git requires this to be set
>>>> + (list (search-path-specification
>>>> + (variable "GIT_SSL_CAINFO")
>>>> + (file-type 'regular)
>>>> + (separator #f)
>>>> + (files '("etc/ssl/certs/ca-certificates.crt")))))
>>>
>>> It’s a bit of a workaround for
>>> <https://issues.guix.gnu.org/issue/22138>, isn’t it? :-)
>>>
>>> What about either removing it, or adding an “XXX” to the comment?
>>
>> I can't quite remember, I think I added that a long while ago when
>> trying to get the Guix service working. I'm not sure it's a great
>> approach. I'll add XXX to the comment though.
>
> What about removing it altogether and instead setting GIT_SSL_CAINFO in
> the service itself?
Turns out I was already setting GIT_SSL_CAINFO in the service, so I've
removed the native-search-paths from the package definition.
I've now pushed both patches, thanks for taking a look!
Chris
[signature.asc (application/pgp-signature, inline)]
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Wed, 23 Oct 2019 11:24:04 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.