GNU bug report logs - #40738
Prometheus (and Alertmanager)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40738 in the body.
You can then email your comments to 40738 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: Prometheus (and Alertmanager)
Date: Mon, 20 Apr 2020 22:11:03 +0100

[Message part 1 (text/plain, inline)]

I've had a go at packaging Prometheus and Alertmanager, as well as
writing Guix services for both of them.

The packages build, but still need quite a bit of work. They're both
written in Go, so the Git repositories not only include the relevant
source code, but the source code of the entire dependency tree (hence
the packages have no inputs). That'll need addressing.

I've tested the Prometheus service, and it seems to work, although there
isn't any record types for the Prometheus configuration yet. I haven't
yet tested the Alertmanager service and the system test doesn't pass, I
believe more configuration is required than Prometheus.

Having a Prometheus service would be a good complement for the existing
Prometheus Node exporter service in Guix.

I'll send the patches following this email.

Thanks,

Chris

[signature.asc (application/pgp-signature, inline)]

Message #10 received at 40738 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 40738 <at> debbugs.gnu.org
Subject: [PATCH 1/4] gnu: Add prometheus.
Date: Mon, 20 Apr 2020 22:17:40 +0100

* gnu/packages/monitoring.scm (prometheus): New variable.
---
 gnu/packages/monitoring.scm | 49 +++++++++++++++++++++++++++++++++++++
 1 file changed, 49 insertions(+)

diff --git a/gnu/packages/monitoring.scm b/gnu/packages/monitoring.scm
index 8da31d6a84..345c2c16c5 100644
--- a/gnu/packages/monitoring.scm
+++ b/gnu/packages/monitoring.scm
@@ -422,6 +422,55 @@ written in Go with pluggable metric collectors.")
     (home-page "https://github.com/prometheus/node_exporter")
     (license license:asl2.0)))
 
+(define-public prometheus
+  (package
+    (name "prometheus")
+    (version "2.17.1")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/prometheus/prometheus.git")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1r7zpq6647lrm7cmid6nnf2xnljqh1i9g0fxvs0qrfd2sxxgj0c7"))))
+    (build-system go-build-system)
+    (arguments
+     '(#:unpack-path "github.com/prometheus/prometheus"
+       #:import-path "github.com/prometheus/prometheus/cmd/prometheus"
+       #:install-source? #f
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'patch
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (assets-prefix
+                     (string-append out "/var/lib/prometheus/assets")))
+               (substitute* "src/github.com/prometheus/prometheus/web/ui/ui.go"
+                 (("var assetsPrefix string")
+                  (string-append "var assetsPrefix string = \""
+                                 assets-prefix
+                                 "\""))))
+             #t))
+         (add-after 'install 'install-assets
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let* ((out (assoc-ref outputs "out"))
+                    (assets-prefix
+                     (string-append out "/var/lib/prometheus/assets")))
+               (for-each (lambda (directory)
+                           (copy-recursively
+                            (string-append "src/github.com/prometheus/prometheus"
+                                           "/web/ui/" directory)
+                            (string-append assets-prefix
+                                           "/" directory)))
+                         '("static" "templates")))
+             #t)))))
+    (home-page "https://prometheus.io/")
+    (synopsis "")
+    (description "")
+    (license "")))
+
 (define-public fswatch
   (package
     (name "fswatch")
-- 
2.26.0

Message #13 received at 40738 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 40738 <at> debbugs.gnu.org
Subject: [PATCH 2/4] services: Add a Prometheus service.
Date: Mon, 20 Apr 2020 22:17:41 +0100

---
 gnu/services/monitoring.scm | 82 +++++++++++++++++++++++++++++++++++++
 gnu/tests/monitoring.scm    | 73 ++++++++++++++++++++++++++++++++-
 2 files changed, 154 insertions(+), 1 deletion(-)

diff --git a/gnu/services/monitoring.scm b/gnu/services/monitoring.scm
index 511f4fb2fe..a37dfd80d8 100644
--- a/gnu/services/monitoring.scm
+++ b/gnu/services/monitoring.scm
@@ -40,6 +40,17 @@
             darkstat-service-type
             prometheus-node-exporter-service-type
 
+            prometheus-service-type
+            <prometheus-configuration>
+            prometheus-configuration
+            prometheus-configuration-package
+            prometheus-configuration-user
+            prometheus-configuration-group
+            prometheus-configuration-config-file
+            prometheus-configuration-web-listen-address
+            prometheus-configuration-storage-tsdb-path
+            prometheus-configuration-extra-options
+
             zabbix-server-configuration
             zabbix-server-service-type
             zabbix-agent-configuration
@@ -110,6 +121,77 @@ HTTP.")
           (service-extension shepherd-root-service-type
                              (compose list darkstat-shepherd-service))))))
 
+
+;;;
+;;; Prometheus
+;;;
+
+(define-record-type* <prometheus-configuration>
+  prometheus-configuration
+  make-prometheus-configuration
+  prometheus-configuration?
+  (package            prometheus-configuration-package
+                      (default prometheus))
+  (user               prometheus-configuration-user
+                      (default "prometheus"))
+  (group              prometheusconfiguration-group
+                      (default "prometheus"))
+  (config-file        prometheus-config-file
+                      (default (plain-file "prometheus.yml" "")))
+  (web-listen-address prometheus-web-listen-address
+                      (default "0.0.0.0:9090"))
+  (storage-tsdb-path  prometheus-storage-tsdb-path
+                      (default "/var/lib/prometheus/data/"))
+  (extra-options      prometheus-configuration-extra-options
+                      (default '())))
+
+(define prometheus-shepherd-service
+  (match-lambda
+    (($ <prometheus-configuration> package user group
+                                   config-file web-listen-address
+                                   storage-tsdb-path extra-options)
+     (shepherd-service
+      (documentation "Prometheus monitoring system and time series database.")
+      (provision '(prometheus))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor
+                (list #$(file-append package "/bin/prometheus")
+                      "--config.file" #$config-file
+                      "--web.listen-address" #$web-listen-address
+                      "--storage.tsdb.path" #$storage-tsdb-path
+                      #$@extra-options)
+                #:user #$user
+                #:group #$group
+                #:log-file "/var/log/prometheus.log"))
+      (stop #~(make-kill-destructor))))))
+
+(define (prometheus-account config)
+  (match-record config <prometheus-configuration>
+    (user group)
+    (list (user-group
+           (name group)
+           (system? #t))
+          (user-account
+           (name user)
+           (group group)
+           (system? #t)
+           (comment "Prometheus user")
+           (home-directory "/var/lib/prometheus")
+           (shell (file-append shadow "/sbin/nologin"))))))
+
+(define prometheus-service-type
+  (service-type
+   (name 'prometheus)
+   (description
+    "Run @command{prometheus} to scrape targets for mertrics and provide the
+web interface.")
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             (compose list prometheus-shepherd-service))
+          (service-extension account-service-type
+                             prometheus-account)))
+   (default-value (prometheus-configuration))))
+
 (define-record-type* <prometheus-node-exporter-configuration>
   prometheus-node-exporter-configuration
   make-prometheus-node-exporter-configuration
diff --git a/gnu/tests/monitoring.scm b/gnu/tests/monitoring.scm
index 732fbc54d7..e8c0847229 100644
--- a/gnu/tests/monitoring.scm
+++ b/gnu/tests/monitoring.scm
@@ -31,9 +31,80 @@
   #:use-module (gnu system)
   #:use-module (gnu tests)
   #:use-module (guix gexp)
-  #:export (%test-prometheus-node-exporter
+  #:export (%test-prometheus
+            %test-prometheus-node-exporter
             %test-zabbix))
 
+
+;;;
+;;; Prometheus
+;;;
+
+(define* (run-prometheus-test name test-os)
+  "Run tests in %TEST-OS, which has Prometheus running."
+  (define os
+    (marionette-operating-system
+     test-os
+     #:imported-modules '((gnu services herd))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (port-forwardings '((8080 . 9090)))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11)
+                       (srfi srfi-64)
+                       (gnu build marionette)
+                       (web client)
+                       (web response))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin #$name)
+
+          (test-assert "prometheus running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (match (start-service 'prometheus)
+                  (#f #f)
+                  (('service response-parts ...)
+                   (match (assq-ref response-parts 'running)
+                     ((pid) (number? pid))))))
+             marionette))
+
+          (test-equal "prometheus healthy"
+            200
+            (begin
+              (wait-for-tcp-port 9090 marionette)
+              (let-values (((response text)
+                            (http-get "http://localhost:8080/-/healthy")))
+                (response-code response))))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation (string-append name "-test") test))
+
+(define %prometheus-test-os
+  (simple-operating-system
+   (service dhcp-client-service-type)
+   (service prometheus-service-type)))
+
+(define %test-prometheus
+  (system-test
+   (name "prometheus")
+   (description "Connect to a running Prometheus service.")
+   (value (run-prometheus-test name
+                               %prometheus-test-os))))
+
 
 ;;;
 ;;; Prometheus Node Exporter
-- 
2.26.0

Message #16 received at 40738 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 40738 <at> debbugs.gnu.org
Subject: [PATCH 3/4] gnu: Add alertmanager.
Date: Mon, 20 Apr 2020 22:17:42 +0100

* gnu/packages/monitoring.scm (alertmanager): New variable.
---
 gnu/packages/monitoring.scm | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)

diff --git a/gnu/packages/monitoring.scm b/gnu/packages/monitoring.scm
index 345c2c16c5..e0e345d259 100644
--- a/gnu/packages/monitoring.scm
+++ b/gnu/packages/monitoring.scm
@@ -471,6 +471,29 @@ written in Go with pluggable metric collectors.")
     (description "")
     (license "")))
 
+(define-public alertmanager
+  (package
+    (name "alertmanager")
+    (version "0.20.0")
+    (source (origin
+              (method git-fetch)
+              (uri (git-reference
+                    (url "https://github.com/prometheus/alertmanager.git")
+                    (commit (string-append "v" version))))
+              (file-name (git-file-name name version))
+              (sha256
+               (base32
+                "1bq6vbpy25k7apvs2ga3fbp1cbnv9j0y1g1khvz2qgz6a2zvhgg3"))))
+    (build-system go-build-system)
+    (arguments
+     '(#:unpack-path "github.com/prometheus/alertmanager"
+       #:import-path "github.com/prometheus/alertmanager/cmd/alertmanager"
+       #:install-source? #f))
+    (home-page "https://prometheus.io/")
+    (synopsis "")
+    (description "")
+    (license "")))
+
 (define-public fswatch
   (package
     (name "fswatch")
-- 
2.26.0

Message #19 received at 40738 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 40738 <at> debbugs.gnu.org
Subject: [PATCH 4/4] services: Add a service for Alertmanager.
Date: Mon, 20 Apr 2020 22:17:43 +0100

---
 gnu/services/monitoring.scm | 82 +++++++++++++++++++++++++++++++++++++
 gnu/tests/monitoring.scm    | 71 ++++++++++++++++++++++++++++++++
 2 files changed, 153 insertions(+)

diff --git a/gnu/services/monitoring.scm b/gnu/services/monitoring.scm
index a37dfd80d8..50a4b7302c 100644
--- a/gnu/services/monitoring.scm
+++ b/gnu/services/monitoring.scm
@@ -51,6 +51,17 @@
             prometheus-configuration-storage-tsdb-path
             prometheus-configuration-extra-options
 
+            alertmanager-service-type
+            <alertmanager-configuration>
+            alertmanager-configuration
+            alertmanager-configuration-package
+            alertmanager-configuration-user
+            alertmanager-configuration-group
+            alertmanager-configuration-config-file
+            alertmanager-configuration-web-listen-address
+            alertmanager-configuration-storage-path
+            alertmanager-configuration-extra-options
+
             zabbix-server-configuration
             zabbix-server-service-type
             zabbix-agent-configuration
@@ -226,6 +237,77 @@ prometheus.")
            (compose list prometheus-node-exporter-shepherd-service))))
    (default-value (prometheus-node-exporter-configuration))))
 
+
+;;;
+;;; Alertmanager
+;;;
+
+(define-record-type* <alertmanager-configuration>
+  alertmanager-configuration
+  make-alertmanager-configuration
+  alertmanager-configuration?
+  (package            alertmanager-configuration-package
+                      (default alertmanager))
+  (user               alertmanager-configuration-user
+                      (default "alertmanager"))
+  (group              alertmanagerconfiguration-group
+                      (default "alertmanager"))
+  (config-file        alertmanager-config-file
+                      (default (plain-file "alertmanager.yml" "")))
+  (web-listen-address alertmanager-web-listen-address
+                      (default ":9093"))
+  (storage-tsdb-path  alertmanager-storage-tsdb-path
+                      (default "/var/lib/alertmanager/data/"))
+  (extra-options      alertmanager-configuration-extra-options
+                      (default '())))
+
+(define alertmanager-shepherd-service
+  (match-lambda
+    (($ <alertmanager-configuration> package user group
+                                   config-file web-listen-address
+                                   storage-tsdb-path extra-options)
+     (shepherd-service
+      (documentation "Alertmanager monitoring system and time series database.")
+      (provision '(alertmanager))
+      (requirement '(networking))
+      (start #~(make-forkexec-constructor
+                (list #$(file-append package "/bin/alertmanager")
+                      "--config.file" #$config-file
+                      "--web.listen-address" #$web-listen-address
+                      "--storage.path" #$storage-tsdb-path
+                      #$@extra-options)
+                #:user #$user
+                #:group #$group
+                #:log-file "/var/log/alertmanager.log"))
+      (stop #~(make-kill-destructor))))))
+
+(define (alertmanager-account config)
+  (match-record config <alertmanager-configuration>
+    (user group)
+    (list (user-group
+           (name group)
+           (system? #t))
+          (user-account
+           (name user)
+           (group group)
+           (system? #t)
+           (comment "Alertmanager user")
+           (home-directory "/var/lib/alertmanager")
+           (shell (file-append shadow "/sbin/nologin"))))))
+
+(define alertmanager-service-type
+  (service-type
+   (name 'alertmanager)
+   (description
+    "Run @command{alertmanager} to scrape targets for mertrics and provide the
+web interface.")
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             (compose list alertmanager-shepherd-service))
+          (service-extension account-service-type
+                             alertmanager-account)))
+   (default-value (alertmanager-configuration))))
+
 
 ;;;
 ;;; Zabbix server
diff --git a/gnu/tests/monitoring.scm b/gnu/tests/monitoring.scm
index e8c0847229..b77b654abc 100644
--- a/gnu/tests/monitoring.scm
+++ b/gnu/tests/monitoring.scm
@@ -33,6 +33,7 @@
   #:use-module (guix gexp)
   #:export (%test-prometheus
             %test-prometheus-node-exporter
+            %test-alertmanager
             %test-zabbix))
 
 
@@ -176,6 +177,76 @@
    (value (run-prometheus-node-exporter-server-test
            name %prometheus-node-exporter-os))))
 
+
+;;;
+;;; Alertmanager
+;;;
+
+(define* (run-alertmanager-test name test-os)
+  "Run tests in %TEST-OS, which has Alertmanager running."
+  (define os
+    (marionette-operating-system
+     test-os
+     #:imported-modules '((gnu services herd))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (port-forwardings '((8080 . 9093)))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11)
+                       (srfi srfi-64)
+                       (gnu build marionette)
+                       (web client)
+                       (web response))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin #$name)
+
+          (test-assert "alertmanager running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (match (start-service 'alertmanager)
+                  (#f #f)
+                  (('service response-parts ...)
+                   (match (assq-ref response-parts 'running)
+                     ((pid) (number? pid))))))
+             marionette))
+
+          (test-equal "alertmanager healthy"
+            200
+            (begin
+              (wait-for-tcp-port 9090 marionette)
+              (let-values (((response text)
+                            (http-get "http://localhost:8080/-/healthy")))
+                (response-code response))))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation (string-append name "-test") test))
+
+(define %alertmanager-test-os
+  (simple-operating-system
+   (service dhcp-client-service-type)
+   (service alertmanager-service-type)))
+
+(define %test-alertmanager
+  (system-test
+   (name "alertmanager")
+   (description "Connect to a running Alertmanager service.")
+   (value (run-alertmanager-test name
+                               %alertmanager-test-os))))
+
 
 ;;;
 ;;; Zabbix
-- 
2.26.0

Message #22 received at 40738 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 40738 <at> debbugs.gnu.org
Subject: Re: [bug#40738] Prometheus (and Alertmanager)
Date: Sun, 03 May 2020 12:56:19 +0200

Hello!

Christopher Baines <mail <at> cbaines.net> skribis:

> I've had a go at packaging Prometheus and Alertmanager, as well as
> writing Guix services for both of them.

Nice!  I’m not an expert, but it sounds like it could advantageously
replace Zabbix on berlin.

> The packages build, but still need quite a bit of work. They're both
> written in Go, so the Git repositories not only include the relevant
> source code, but the source code of the entire dependency tree (hence
> the packages have no inputs). That'll need addressing.

Would the gopkg importer at
<https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00310.html> be
of any help?  (I think I posted an updated version later on but I can’t
find it.)

> I've tested the Prometheus service, and it seems to work, although there
> isn't any record types for the Prometheus configuration yet. I haven't
> yet tested the Alertmanager service and the system test doesn't pass, I
> believe more configuration is required than Prometheus.
>
> Having a Prometheus service would be a good complement for the existing
> Prometheus Node exporter service in Guix.

Yup!

Thanks,
Ludo’.

Message #25 received at 40738 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 40738 <at> debbugs.gnu.org
Subject: Re: [bug#40738] Prometheus (and Alertmanager)
Date: Sun, 03 May 2020 12:17:55 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

>> The packages build, but still need quite a bit of work. They're both
>> written in Go, so the Git repositories not only include the relevant
>> source code, but the source code of the entire dependency tree (hence
>> the packages have no inputs). That'll need addressing.
>
> Would the gopkg importer at
> <https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00310.html> be
> of any help?  (I think I posted an updated version later on but I can’t
> find it.)

Yeah, I think getting a working importer is going to be the way to
address this.

[signature.asc (application/pgp-signature, inline)]

Message #30 received at 40738-done <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Christopher Baines <mail <at> cbaines.net>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 40738-done <at> debbugs.gnu.org
Subject: Re: bug#40738: Prometheus (and Alertmanager)
Date: Fri, 21 Feb 2025 23:09:42 +0100

Hi Chris,

For reference: https://issues.guix.gnu.org/issue/40738

On Sun, 03 May 2020 at 12:17, Christopher Baines <mail <at> cbaines.net> wrote:
> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>>> The packages build, but still need quite a bit of work. They're both
>>> written in Go, so the Git repositories not only include the relevant
>>> source code, but the source code of the entire dependency tree (hence
>>> the packages have no inputs). That'll need addressing.
>>
>> Would the gopkg importer at
>> <https://lists.gnu.org/archive/html/guix-devel/2018-04/msg00310.html> be
>> of any help?  (I think I posted an updated version later on but I can’t
>> find it.)
>
> Yeah, I think getting a working importer is going to be the way to
> address this.

This old report had been marked as moreinfo 4 years, 43 weeks, 6 days
ago.  With no more activity.

Therefore, I’m closing.

Feel free to reopen if I’m missing something.

Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.