GNU bug report logs -
#77585
[PATCH] services: mumi: Add Debbugs rsync as shepherd timer.
Previous Next
Reported by: Arun Isaac <arunisaac <at> systemreboot.net>
Date: Sun, 6 Apr 2025 19:32:01 UTC
Severity: normal
Tags: patch
Done: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 77585 in the body.
You can then email your comments to 77585 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org
:
bug#77585
; Package
guix-patches
.
(Sun, 06 Apr 2025 19:32:01 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Arun Isaac <arunisaac <at> systemreboot.net>
:
New bug report received and forwarded. Copy sent to
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org
.
(Sun, 06 Apr 2025 19:32:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
* gnu/services/web.scm (<mumi-configuration>)[data-directory, rsync-remote,
rsync-flags]: New fields.
(%mumi-worker-log): Delete variable.
(%mumi-rsync-and-index-log): New variable.
(mumi-rsync-and-index-gexp): New function.
(mumi-shepherd-services): Remove mumi-worker service. Add mumi-rsync-and-index
service.
(mumi-service-type): Remove default value.
* doc/guix.texi (Web Services)[mumi]: Document data-directory, rsync-remote
and rsync-flags fields.
Co-authored-by: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
---
doc/guix.texi | 9 +++++++
gnu/services/web.scm | 57 ++++++++++++++++++++++++++++++++------------
2 files changed, 51 insertions(+), 15 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 12f155e912..b6c0e64a53 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -33999,6 +33999,15 @@ Web Services
@item @code{mumi} (default: @code{mumi})
The Mumi package to use.
+@item @code{data-directory} (default: @file{"/var/mumi/data"})
+Data directory to store Debbugs data from upstream Debbugs instance.
+
+@item @code{rsync-remote}
+Remote rsync path from which to download Debbugs data.
+
+@item @code{rsync-flags} (default: @code{'()})
+Additional flags to pass to @command{rsync}.
+
@item @code{mailer?} (default: @code{#true})
Whether to enable or disable the mailer component.
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index f8cf06fb48..eea9a19426 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -18,6 +18,7 @@
;;; Copyright © 2023 Bruno Victal <mirai <at> makinata.eu>
;;; Copyright © 2023 Miguel Ángel Moreno <mail <at> migalmoreno.com>
;;; Copyright © 2024 Leo Nikkilä <hello <at> lnikki.la>
+;;; Copyright © 2025 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -51,6 +52,7 @@ (define-module (gnu services web)
#:use-module (gnu packages php)
#:use-module (gnu packages python)
#:use-module (gnu packages python-web)
+ #:use-module (gnu packages rsync)
#:use-module (gnu packages gnupg)
#:use-module (gnu packages guile)
#:use-module (gnu packages logging)
@@ -295,6 +297,9 @@ (define-module (gnu services web)
mumi-configuration
mumi-configuration?
mumi-configuration-mumi
+ mumi-configuration-data-directory
+ mumi-configuration-rsync-remote
+ mumi-configuration-rsync-flags
mumi-configuration-mailer?
mumi-configuration-sender
mumi-configuration-smtp
@@ -2033,6 +2038,11 @@ (define-record-type* <mumi-configuration>
mumi-configuration make-mumi-configuration
mumi-configuration?
(mumi mumi-configuration-mumi (default mumi))
+ (data-directory mumi-configuration-data-directory
+ (default "/var/mumi/data"))
+ (rsync-remote mumi-configuration-rsync-remote)
+ (rsync-flags mumi-configuration-rsync-flags
+ (default '()))
(mailer? mumi-configuration-mailer? (default #t))
(sender mumi-configuration-sender (default #f))
(smtp mumi-configuration-smtp (default #f))
@@ -2080,7 +2090,7 @@ (define %mumi-log "/var/log/mumi.log")
(define %mumi-mailer-log "/var/log/mumi.mailer.log")
-(define %mumi-worker-log "/var/log/mumi.worker.log")
+(define %mumi-rsync-and-index-log "/var/log/mumi.rsync-and-index.log")
(define mumi-package-configuration->alist
(match-record-lambda <mumi-package-configuration>
@@ -2118,6 +2128,23 @@ (define mumi-config-file
packages)))
<>))))))
+(define (mumi-rsync-and-index-gexp config)
+ (match-record config <mumi-configuration>
+ (data-directory rsync-remote rsync-flags)
+ (with-imported-modules '((guix build utils))
+ #~(begin
+ (use-modules (guix build utils))
+
+ (invoke #$(file-append rsync "/bin/rsync")
+ "--delete" "--archive" "--verbose"
+ "--timeout" "120"
+ #$@rsync-flags
+ #$rsync-remote
+ #$data-directory)
+ (invoke #$(file-append mumi "/bin/mumi") "fetch"
+ (string-append "--config="
+ #$(mumi-config-file config)))))))
+
(define (mumi-shepherd-services config)
(define environment
#~(list "LC_ALL=en_US.utf8"
@@ -2140,17 +2167,19 @@ (define (mumi-shepherd-services config)
#:log-file #$%mumi-log))
(stop #~(make-kill-destructor)))
(shepherd-service
- (provision '(mumi-worker))
- (documentation "Mumi bug-tracking web interface database worker.")
- (requirement '(user-processes networking))
- (start #~(make-forkexec-constructor
- `(#$(file-append mumi "/bin/mumi") "worker"
- ,(string-append "--config="
- #$(mumi-config-file config)))
- #:environment-variables #$environment
- #:user "mumi" #:group "mumi"
- #:log-file #$%mumi-worker-log))
- (stop #~(make-kill-destructor)))
+ (provision '(mumi-rsync-and-index))
+ (modules '((shepherd service timer)))
+ (start #~(make-timer-constructor
+ (calendar-event)
+ (command
+ (program-file "mumi-rsync-and-index"
+ (mumi-rsync-and-index-gexp config)))
+ #:log-file #$%mumi-rsync-and-index-log
+ #:max-duration (* 60 60)
+ #:wait-for-termination? #t))
+ (stop #~(make-timer-destructor))
+ (actions (list shepherd-trigger-action))
+ (documentation "Rsync and index the GNU Debbugs data"))
(shepherd-service
(provision '(mumi-mailer))
(documentation "Mumi bug-tracking web interface mailer.")
@@ -2181,9 +2210,7 @@ (define mumi-service-type
(service-extension shepherd-root-service-type
mumi-shepherd-services)))
(description
- "Run Mumi, a Web interface to the Debbugs bug-tracking server.")
- (default-value
- (mumi-configuration))))
+ "Run Mumi, a Web interface to the Debbugs bug-tracking server.")))
(define %default-gmnisrv-config-file
(plain-file "gmnisrv.ini" "
base-commit: ada14197fb465c1c90efbc450308c14f077ff167
--
2.49.0
Information forwarded
to
guix-patches <at> gnu.org
:
bug#77585
; Package
guix-patches
.
(Sun, 06 Apr 2025 20:24:02 GMT)
Full text and
rfc822 format available.
Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):
Hi Arun,
Arun Isaac <arunisaac <at> systemreboot.net> skribis:
> * gnu/services/web.scm (<mumi-configuration>)[data-directory, rsync-remote,
> rsync-flags]: New fields.
> (%mumi-worker-log): Delete variable.
> (%mumi-rsync-and-index-log): New variable.
> (mumi-rsync-and-index-gexp): New function.
> (mumi-shepherd-services): Remove mumi-worker service. Add mumi-rsync-and-index
> service.
> (mumi-service-type): Remove default value.
> * doc/guix.texi (Web Services)[mumi]: Document data-directory, rsync-remote
> and rsync-flags fields.
>
> Co-authored-by: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
[...]
> +(define (mumi-rsync-and-index-gexp config)
As a matter of style, I would return a program
(‘mumi-rsync-and-index-program’) rather than a gexp; I find it clearer
and more robust (the gexp leaves it up to the caller to insert it in the
right context).
> + (start #~(make-timer-constructor
> + (calendar-event)
You can add a margin comment say “every minute”, for clarity.
> + (command
> + (program-file "mumi-rsync-and-index"
> + (mumi-rsync-and-index-gexp config)))
I think you’re missing #$ before (mumi-rsync-and-index-gexp config).
> + #:log-file #$%mumi-rsync-and-index-log
> + #:max-duration (* 60 60)
> + #:wait-for-termination? #t))
Every minute is a lot, especially since it may take several seconds to
establish the rsync-over-ssh connection. It’s mitigated by the use of
#:wait-for-termination? but still sounds fairly aggressive to me.
(It would be ideal if the Debbugs instance could somehow notify mumi.)
Perhaps mention the polling frequency in the manual?
Thanks,
Ludo’.
Information forwarded
to
arunisaac <at> systemreboot.net, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org
:
bug#77585
; Package
guix-patches
.
(Sun, 06 Apr 2025 21:53:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 77585 <at> debbugs.gnu.org (full text, mbox):
* gnu/services/web.scm (<mumi-configuration>)[data-directory, rsync-remote,
rsync-flags]: New fields.
(%mumi-worker-log): Delete variable.
(%mumi-rsync-and-index-log): New variable.
(mumi-rsync-and-index): New function.
(mumi-shepherd-services): Remove mumi-worker service. Add mumi-rsync-and-index
service.
(mumi-service-type): Remove default value.
* doc/guix.texi (Web Services)[mumi]: Document data-directory, rsync-remote
and rsync-flags fields.
Co-authored-by: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
---
doc/guix.texi | 11 +++++++++
gnu/services/web.scm | 58 ++++++++++++++++++++++++++++++++------------
2 files changed, 54 insertions(+), 15 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 12f155e912..8aa0b5e3ba 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -33999,6 +33999,17 @@ Web Services
@item @code{mumi} (default: @code{mumi})
The Mumi package to use.
+@item @code{data-directory} (default: @file{"/var/mumi/data"})
+Data directory to store Debbugs data from upstream Debbugs instance.
+
+@item @code{rsync-remote}
+Remote rsync path from which to download Debbugs data. The remote is
+polled every 2 minutes, unless a previous instance of rsync is already
+running.
+
+@item @code{rsync-flags} (default: @code{'()})
+Additional flags to pass to @command{rsync}.
+
@item @code{mailer?} (default: @code{#true})
Whether to enable or disable the mailer component.
diff --git a/gnu/services/web.scm b/gnu/services/web.scm
index f8cf06fb48..3f0eb8780a 100644
--- a/gnu/services/web.scm
+++ b/gnu/services/web.scm
@@ -18,6 +18,7 @@
;;; Copyright © 2023 Bruno Victal <mirai <at> makinata.eu>
;;; Copyright © 2023 Miguel Ángel Moreno <mail <at> migalmoreno.com>
;;; Copyright © 2024 Leo Nikkilä <hello <at> lnikki.la>
+;;; Copyright © 2025 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -51,6 +52,7 @@ (define-module (gnu services web)
#:use-module (gnu packages php)
#:use-module (gnu packages python)
#:use-module (gnu packages python-web)
+ #:use-module (gnu packages rsync)
#:use-module (gnu packages gnupg)
#:use-module (gnu packages guile)
#:use-module (gnu packages logging)
@@ -295,6 +297,9 @@ (define-module (gnu services web)
mumi-configuration
mumi-configuration?
mumi-configuration-mumi
+ mumi-configuration-data-directory
+ mumi-configuration-rsync-remote
+ mumi-configuration-rsync-flags
mumi-configuration-mailer?
mumi-configuration-sender
mumi-configuration-smtp
@@ -2033,6 +2038,11 @@ (define-record-type* <mumi-configuration>
mumi-configuration make-mumi-configuration
mumi-configuration?
(mumi mumi-configuration-mumi (default mumi))
+ (data-directory mumi-configuration-data-directory
+ (default "/var/mumi/data"))
+ (rsync-remote mumi-configuration-rsync-remote)
+ (rsync-flags mumi-configuration-rsync-flags
+ (default '()))
(mailer? mumi-configuration-mailer? (default #t))
(sender mumi-configuration-sender (default #f))
(smtp mumi-configuration-smtp (default #f))
@@ -2080,7 +2090,7 @@ (define %mumi-log "/var/log/mumi.log")
(define %mumi-mailer-log "/var/log/mumi.mailer.log")
-(define %mumi-worker-log "/var/log/mumi.worker.log")
+(define %mumi-rsync-and-index-log "/var/log/mumi.rsync-and-index.log")
(define mumi-package-configuration->alist
(match-record-lambda <mumi-package-configuration>
@@ -2118,6 +2128,24 @@ (define mumi-config-file
packages)))
<>))))))
+(define (mumi-rsync-and-index config)
+ (match-record config <mumi-configuration>
+ (data-directory rsync-remote rsync-flags)
+ (program-file "mumi-rsync-and-index"
+ (with-imported-modules '((guix build utils))
+ #~(begin
+ (use-modules (guix build utils))
+
+ (invoke #$(file-append rsync "/bin/rsync")
+ "--delete" "--archive" "--verbose"
+ "--timeout" "120"
+ #$@rsync-flags
+ #$rsync-remote
+ #$data-directory)
+ (invoke #$(file-append mumi "/bin/mumi") "fetch"
+ (string-append "--config="
+ #$(mumi-config-file config))))))))
+
(define (mumi-shepherd-services config)
(define environment
#~(list "LC_ALL=en_US.utf8"
@@ -2140,17 +2168,19 @@ (define (mumi-shepherd-services config)
#:log-file #$%mumi-log))
(stop #~(make-kill-destructor)))
(shepherd-service
- (provision '(mumi-worker))
- (documentation "Mumi bug-tracking web interface database worker.")
- (requirement '(user-processes networking))
- (start #~(make-forkexec-constructor
- `(#$(file-append mumi "/bin/mumi") "worker"
- ,(string-append "--config="
- #$(mumi-config-file config)))
- #:environment-variables #$environment
- #:user "mumi" #:group "mumi"
- #:log-file #$%mumi-worker-log))
- (stop #~(make-kill-destructor)))
+ (provision '(mumi-rsync-and-index))
+ (modules '((shepherd service timer)))
+ (start #~(make-timer-constructor
+ ;; Run every 2 minutes, unless an instance of this job
+ ;; is already running.
+ (calendar-event #:minutes (iota 30 0 2))
+ (command #$(mumi-rsync-and-index config))
+ #:log-file #$%mumi-rsync-and-index-log
+ #:max-duration (* 60 60)
+ #:wait-for-termination? #t))
+ (stop #~(make-timer-destructor))
+ (actions (list shepherd-trigger-action))
+ (documentation "Rsync and index the GNU Debbugs data"))
(shepherd-service
(provision '(mumi-mailer))
(documentation "Mumi bug-tracking web interface mailer.")
@@ -2181,9 +2211,7 @@ (define mumi-service-type
(service-extension shepherd-root-service-type
mumi-shepherd-services)))
(description
- "Run Mumi, a Web interface to the Debbugs bug-tracking server.")
- (default-value
- (mumi-configuration))))
+ "Run Mumi, a Web interface to the Debbugs bug-tracking server.")))
(define %default-gmnisrv-config-file
(plain-file "gmnisrv.ini" "
base-commit: ada14197fb465c1c90efbc450308c14f077ff167
--
2.49.0
Information forwarded
to
guix-patches <at> gnu.org
:
bug#77585
; Package
guix-patches
.
(Sun, 06 Apr 2025 21:59:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 77585 <at> debbugs.gnu.org (full text, mbox):
Hi Ludo,
> As a matter of style, I would return a program
> (‘mumi-rsync-and-index-program’) rather than a gexp; I find it clearer
> and more robust (the gexp leaves it up to the caller to insert it in the
> right context).
Good point, thanks! I was merely trying to save whitespace in the left.
Maybe we should indent program-file differently in the future?
But, fixed for now.
>> + (command
>> + (program-file "mumi-rsync-and-index"
>> + (mumi-rsync-and-index-gexp config)))
>
> I think you’re missing #$ before (mumi-rsync-and-index-gexp config).
Good catch, thanks!
>> + #:log-file #$%mumi-rsync-and-index-log
>> + #:max-duration (* 60 60)
>> + #:wait-for-termination? #t))
>
> Every minute is a lot, especially since it may take several seconds to
> establish the rsync-over-ssh connection. It’s mitigated by the use of
> #:wait-for-termination? but still sounds fairly aggressive to me.
> (It would be ideal if the Debbugs instance could somehow notify mumi.)
I have switched to a polling frequency of 2 minutes now. Still quite
high, but we can cut it down to 5 minutes if it's too much.
> Perhaps mention the polling frequency in the manual?
Done now.
When I deploy this on berlin, how do I get shepherd to pick up the new
timer service. Is there a `herd reload' or similar? Or, do I have to
reboot?
Thanks,
Arun
Information forwarded
to
guix-patches <at> gnu.org
:
bug#77585
; Package
guix-patches
.
(Tue, 08 Apr 2025 01:58:02 GMT)
Full text and
rfc822 format available.
Message #17 received at 77585 <at> debbugs.gnu.org (full text, mbox):
Hi Arun,
Arun Isaac <arunisaac <at> systemreboot.net> writes:
> * gnu/services/web.scm (<mumi-configuration>)[data-directory, rsync-remote,
> rsync-flags]: New fields.
> (%mumi-worker-log): Delete variable.
> (%mumi-rsync-and-index-log): New variable.
> (mumi-rsync-and-index): New function.
> (mumi-shepherd-services): Remove mumi-worker service. Add mumi-rsync-and-index
> service.
> (mumi-service-type): Remove default value.
> * doc/guix.texi (Web Services)[mumi]: Document data-directory, rsync-remote
> and rsync-flags fields.
>
> Co-authored-by: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Reviewed-by: Maxim Cournoyer <maxim.cournoyer <at> gmail>
--
Thanks,
Maxim
Reply sent
to
Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
:
You have taken responsibility.
(Wed, 23 Apr 2025 02:20:01 GMT)
Full text and
rfc822 format available.
Notification sent
to
Arun Isaac <arunisaac <at> systemreboot.net>
:
bug acknowledged by developer.
(Wed, 23 Apr 2025 02:20:01 GMT)
Full text and
rfc822 format available.
Message #22 received at 77585-done <at> debbugs.gnu.org (full text, mbox):
Hi,
Arun Isaac <arunisaac <at> systemreboot.net> writes:
[...]
I think you'll see the new timer in 'herd status', e.g. on my machine:
Running timers:
+ log-cleanup
+ log-rotation
I assume they should be started automatically.
I'll close this issue as I'm assuming it's been taken care of already o
berlin. Feel free to reopen if there are issues.
--
Thanks,
Maxim
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Wed, 21 May 2025 11:24:11 GMT)
Full text and
rfc822 format available.
This bug report was last modified 15 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.