Package: guix-patches;
Reported by: Aljosha Papsch <ep <at> stern-data.com>
Date: Mon, 21 Jun 2021 14:23:01 UTC
Severity: normal
Tags: patch
Done: Marius Bakke <marius <at> gnu.org>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 49155 in the body.
You can then email your comments to 49155 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
guix-patches <at> gnu.org:bug#49155; Package guix-patches.
(Mon, 21 Jun 2021 14:23:01 GMT) Full text and rfc822 format available.Aljosha Papsch <ep <at> stern-data.com>:guix-patches <at> gnu.org.
(Mon, 21 Jun 2021 14:23:01 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Aljosha Papsch <ep <at> stern-data.com> To: guix-patches <at> gnu.org Cc: Ellen Papsch <ep <at> stern-data.com> Subject: [PATCH] gnu: mysql: Support custom data dir. Date: Mon, 21 Jun 2021 15:27:52 +0200
From: Ellen Papsch <ep <at> stern-data.com>
* gnu/services/databases.scm (mysql-configuration): Add datadir property.
* gnu/services/databases.scm (mysql-configuration-file): Replace hard coded
data dir with property from config.
* gnu/services/databases.scm (%mysql-activation): Remove activation, it runs
before PID 1. The data dir may reside on a file system not mounted at this
time.
* gnu/services/databases.scm (mysql-install-shepherd-service): Create service
which replaces the activation. Provide mysql-install.
* gnu/services/databases.scm (mysql-shepherd-service): Move invocation of
mysqld to mysql-start program-file, because the invocation gotten more
complex. Require mysql-install.
* gnu/services/databases.scm (mysql-start): Invoke mysqld only if a lock file
appears.
* gnu/services/databases.scm (mysql-shepherd-services): Prepend the install
service before the normal service.
* gnu/services/databases.scm (mysql-upgrade-wrapper): Increase timeout to
20s to let the mysql install procedure finish.
---
gnu/services/databases.scm | 187 +++++++++++++++++++++++--------------
1 file changed, 119 insertions(+), 68 deletions(-)
diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index eba88cdb68..f8f78619d3 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -8,6 +8,7 @@
;;; Copyright © 2019 Robert Vollmert <rob <at> vllmrt.net>
;;; Copyright © 2020 Marius Bakke <marius <at> gnu.org>
;;; Copyright © 2021 David Larsson <david.larsson <at> selfhosted.xyz>
+;;; Copyright © 2021 Aljosha Papsch <ep <at> stern-data.com>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -527,6 +528,7 @@ created after the PostgreSQL database is started.")))
(bind-address mysql-configuration-bind-address (default "127.0.0.1"))
(port mysql-configuration-port (default 3306))
(socket mysql-configuration-socket (default "/run/mysqld/mysqld.sock"))
+ (datadir mysql-configuration-datadir (default "/var/lib/mysql"))
(extra-content mysql-configuration-extra-content (default ""))
(extra-environment mysql-configuration-extra-environment (default #~'()))
(auto-upgrade? mysql-configuration-auto-upgrade? (default #t)))
@@ -544,85 +546,133 @@ created after the PostgreSQL database is started.")))
(define mysql-configuration-file
(match-lambda
- (($ <mysql-configuration> mysql bind-address port socket extra-content)
+ (($ <mysql-configuration> mysql bind-address port socket datadir extra-content)
(mixed-text-file "my.cnf" "[mysqld]
-datadir=/var/lib/mysql
+datadir=" datadir "
socket=" socket "
bind-address=" bind-address "
port=" (number->string port) "
" extra-content "
"))))
-(define (%mysql-activation config)
- "Return an activation gexp for the MySQL or MariaDB database server."
- (let ((mysql (mysql-configuration-mysql config))
- (my.cnf (mysql-configuration-file config)))
- #~(begin
- (use-modules (ice-9 popen)
- (guix build utils))
- (let* ((mysqld (string-append #$mysql "/bin/mysqld"))
- (user (getpwnam "mysql"))
- (uid (passwd:uid user))
- (gid (passwd:gid user))
- (datadir "/var/lib/mysql")
- (rundir "/run/mysqld"))
- (mkdir-p datadir)
- (chown datadir uid gid)
- (mkdir-p rundir)
- (chown rundir uid gid)
- ;; Initialize the database when it doesn't exist.
- (when (not (file-exists? (string-append datadir "/mysql")))
- (if (string-prefix? "mysql-" (strip-store-file-name #$mysql))
- ;; For MySQL.
- (system* mysqld
- (string-append "--defaults-file=" #$my.cnf)
- "--initialize"
- "--user=mysql")
- ;; For MariaDB.
- ;; XXX: The 'mysql_install_db' script doesn't work directly
- ;; due to missing 'mkdir' in PATH.
- (let ((p (open-pipe* OPEN_WRITE mysqld
- (string-append
- "--defaults-file=" #$my.cnf)
- "--bootstrap"
- "--user=mysql")))
- ;; Create the system database, as does by 'mysql_install_db'.
- (display "create database mysql;\n" p)
- (display "use mysql;\n" p)
- (for-each
- (lambda (sql)
- (call-with-input-file
- (string-append #$mysql:lib "/share/mysql/" sql)
- (lambda (in) (dump-port in p))))
- '("mysql_system_tables.sql"
- "mysql_performance_tables.sql"
- "mysql_system_tables_data.sql"
- "fill_help_tables.sql"))
- ;; Remove the anonymous user and disable root access from
- ;; remote machines, as does by 'mysql_secure_installation'.
- (display "
-DELETE FROM user WHERE User='';
-DELETE FROM user WHERE User='root' AND
- Host NOT IN ('localhost', '127.0.0.1', '::1');
-FLUSH PRIVILEGES;
-" p)
- (close-pipe p))))))))
+(define (mysql-with-install-lock)
+ "Return a loop function which evals thunk when the install is locked."
+ #~(lambda (thunk)
+ (let loop ((i 0))
+ (let ((timeout 10)
+ (lock-stat (stat "/var/lib/mysql.lock" #f)))
+ (if (and (not (eq? lock-stat #f))
+ (eq? (stat:type lock-stat) 'regular))
+ (apply thunk '())
+ (if (< i timeout)
+ (begin
+ (sleep 1)
+ (loop (+ 1 i)))
+ (throw 'timeout-error
+ "MySQL installation not locked in time!")))))))
+
+(define (mysql-start config)
+ "Start mysqld if install lock file appears"
+ (program-file
+ "mysql-start"
+ (let ((mysql (mysql-configuration-mysql config))
+ (my.cnf (mysql-configuration-file config)))
+ #~(let ((mysqld (string-append #$mysql "/bin/mysqld"))
+ (with-lock #$(mysql-with-install-lock)))
+ (with-lock (lambda ()
+ (execl mysqld mysqld
+ (string-append "--defaults-file=" #$my.cnf))))))))
(define (mysql-shepherd-service config)
(list (shepherd-service
(provision '(mysql))
+ (requirement '(mysql-install))
(documentation "Run the MySQL server.")
- (start (let ((mysql (mysql-configuration-mysql config))
+ (start (let ((mysql (mysql-configuration-mysql config))
(extra-env (mysql-configuration-extra-environment config))
(my.cnf (mysql-configuration-file config)))
#~(make-forkexec-constructor
- (list (string-append #$mysql "/bin/mysqld")
- (string-append "--defaults-file=" #$my.cnf))
- #:user "mysql" #:group "mysql"
- #:log-file "/var/log/mysqld.log"
- #:environment-variables #$extra-env)))
+ (list #$(mysql-start config))
+ #:user "mysql" #:group "mysql"
+ #:log-file "/var/log/mysqld.log"
+ #:environment-variables #$extra-env)))
(stop #~(make-kill-destructor)))))
+(define (mysql-install config)
+ "Install MySQL system database and secure the installation."
+ (let ((mysql (mysql-configuration-mysql config))
+ (my.cnf (mysql-configuration-file config))
+ (datadir (mysql-configuration-datadir config))
+ (extra-env (mysql-configuration-extra-environment config)))
+ (program-file
+ "mysql-install"
+ (with-imported-modules (source-module-closure
+ '((ice-9 popen)
+ (guix build utils)))
+ #~(begin
+ (use-modules (ice-9 popen)
+ (guix build utils))
+ (let* ((mysqld (string-append #$mysql "/bin/mysqld"))
+ (user (getpwnam "mysql"))
+ (uid (passwd:uid user))
+ (gid (passwd:gid user))
+ (datadir #$datadir)
+ (rundir "/run/mysqld"))
+ (mkdir-p datadir)
+ (chown datadir uid gid)
+ (mkdir-p rundir)
+ (chown rundir uid gid)
+ ;; Initialize the database when it doesn't exist.
+ (when (not (file-exists? (string-append datadir "/mysql")))
+ (if (string-prefix? "mysql-" (strip-store-file-name #$mysql))
+ ;; For MySQL.
+ (system* mysqld
+ (string-append "--defaults-file=" #$my.cnf)
+ "--initialize"
+ "--user=mysql")
+ ;; For MariaDB.
+ ;; XXX: The 'mysql_install_db' script doesn't work directly
+ ;; due to missing 'mkdir' in PATH.
+ (let ((p (open-pipe* OPEN_WRITE mysqld
+ (string-append
+ "--defaults-file=" #$my.cnf)
+ "--bootstrap"
+ "--user=mysql")))
+ ;; Create the system database, as does by 'mysql_install_db'.
+ (display "create database mysql;\n" p)
+ (display "use mysql;\n" p)
+ (for-each
+ (lambda (sql)
+ (call-with-input-file
+ (string-append #$mysql:lib "/share/mysql/" sql)
+ (lambda (in) (dump-port in p))))
+ '("mysql_system_tables.sql"
+ "mysql_performance_tables.sql"
+ "mysql_system_tables_data.sql"
+ "fill_help_tables.sql"))
+ ;; Remove the anonymous user and disable root access from
+ ;; remote machines, as does by 'mysql_secure_installation'.
+ (display "
+DELETE FROM user WHERE User='';
+DELETE FROM user WHERE User='root' AND
+ Host NOT IN ('localhost', '127.0.0.1', '::1');
+FLUSH PRIVILEGES;
+" p)
+ (close-pipe p))))
+ (call-with-output-file "/var/lib/mysql.lock"
+ (lambda (p)
+ (write #t p)))))))))
+
+(define (mysql-install-shepherd-service config)
+ (list (shepherd-service
+ (provision '(mysql-install))
+ (requirement '(file-systems))
+ (one-shot? #t)
+ (documentation "Install MySQL system database and secure installation.")
+ (start #~(make-forkexec-constructor
+ (list #$(mysql-install config))
+ #:log-file "/var/log/mysqld-install.log")))))
+
(define (mysql-upgrade-wrapper mysql socket-file)
;; The MySQL socket and PID file may appear before the server is ready to
;; accept connections. Ensure the socket is responsive before attempting
@@ -631,7 +681,7 @@ FLUSH PRIVILEGES;
"mysql-upgrade-wrapper"
#~(begin
(let ((mysql-upgrade #$(file-append mysql "/bin/mysql_upgrade"))
- (timeout 10))
+ (timeout 20))
(begin
(let loop ((i 0))
(catch 'system-error
@@ -663,11 +713,14 @@ FLUSH PRIVILEGES;
(list #$(mysql-upgrade-wrapper mysql socket))
#:user "mysql" #:group "mysql"))))))
+
(define (mysql-shepherd-services config)
- (if (mysql-configuration-auto-upgrade? config)
- (append (mysql-shepherd-service config)
+ (let ((min-services (append (mysql-install-shepherd-service config)
+ (mysql-shepherd-service config))))
+ (if (mysql-configuration-auto-upgrade? config)
+ (append min-services
(mysql-upgrade-shepherd-service config))
- (mysql-shepherd-service config)))
+ min-services)))
(define mysql-service-type
(service-type
@@ -675,8 +728,6 @@ FLUSH PRIVILEGES;
(extensions
(list (service-extension account-service-type
(const %mysql-accounts))
- (service-extension activation-service-type
- %mysql-activation)
(service-extension shepherd-root-service-type
mysql-shepherd-services)))
(default-value (mysql-configuration))))
--
2.32.0
Marius Bakke <marius <at> gnu.org>:Aljosha Papsch <ep <at> stern-data.com>:Message #10 received at 49155-done <at> debbugs.gnu.org (full text, mbox):
From: Marius Bakke <marius <at> gnu.org> To: Aljosha Papsch <ep <at> stern-data.com>, 49155-done <at> debbugs.gnu.org Cc: Ellen Papsch <ep <at> stern-data.com> Subject: Re: [bug#49155] [PATCH] gnu: mysql: Support custom data dir. Date: Sun, 20 Nov 2022 17:56:12 +0100
[Message part 1 (text/plain, inline)]
Hi! Thanks for the patch, and sorry for the slooow response... Aljosha Papsch <ep <at> stern-data.com> skriver: > From: Ellen Papsch <ep <at> stern-data.com> > > * gnu/services/databases.scm (mysql-configuration): Add datadir property. > * gnu/services/databases.scm (mysql-configuration-file): Replace hard coded > data dir with property from config. > * gnu/services/databases.scm (%mysql-activation): Remove activation, it runs > before PID 1. The data dir may reside on a file system not mounted at this > time. > * gnu/services/databases.scm (mysql-install-shepherd-service): Create service > which replaces the activation. Provide mysql-install. > * gnu/services/databases.scm (mysql-shepherd-service): Move invocation of > mysqld to mysql-start program-file, because the invocation gotten more > complex. Require mysql-install. > * gnu/services/databases.scm (mysql-start): Invoke mysqld only if a lock file > appears. > * gnu/services/databases.scm (mysql-shepherd-services): Prepend the install > service before the normal service. > * gnu/services/databases.scm (mysql-upgrade-wrapper): Increase timeout to > 20s to let the mysql install procedure finish. I have finally applied this patch, along with a follow-up commit to make the mysql-upgrade service also work with a custom datadir. I also ended up dropping the 'mysql-install' service in favor of a mysqld wrapper script that calls the mysql-install script at startup when the "mysql" DB is missing. Pushed in c7b266fdf2754c139803c156677bf2828c78d072.
[signature.asc (application/pgp-signature, inline)]
Debbugs Internal Request <help-debbugs <at> gnu.org>
to internal_control <at> debbugs.gnu.org.
(Mon, 19 Dec 2022 12:24:10 GMT) Full text and rfc822 format available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.