GNU bug report logs - #28198
[PATCH] Add MongoDB package and service.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 28198 in the body.
You can then email your comments to 28198 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: [PATCH] Add MongoDB package and service.
Date: Wed, 23 Aug 2017 10:18:31 +0100

[Message part 1 (text/plain, inline)]

This is a work in progress issue for supporting MongoDB in Guix.

Back in March 2016, Roel Janssen sent a working package patch to
guix-devel, but there was concern from those reviewing about the
security implications of the amount of third party code that is
included with the MongoDB source.

I've taken that original patch, and started to try and get MongoDB to
build using libraries and tools from Guix. In addition, I've added a
very basic service definition and system test.

This isn't ready to be merged yet, but I think it's good to work on in
public. Patches to follow...

[Message part 2 (application/pgp-signature, inline)]

Message #8 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 1/4] vm: Add disk-image-size to <virtual-machine>.
Date: Wed, 23 Aug 2017 10:25:13 +0100

* gnu/system/vm.scm (<virtual-machine>): Add
  disk-image-size.
  (port-forwardings->qemu-options): Use disk-image-size from
  <virtual-machine>.
---
 gnu/system/vm.scm | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 4494af003..45a14ef67 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -653,6 +653,8 @@ it is mostly useful when FULL-BOOT?  is true."
                     (default #f))
   (memory-size      virtual-machine-memory-size   ;integer (MiB)
                     (default 256))
+  (disk-image-size  virtual-machine-disk-image-size   ;integer (bytes)
+                    (default (* 70 (expt 2 20))))
   (port-forwardings virtual-machine-port-forwardings ;list of integer pairs
                     (default '())))
 
@@ -681,12 +683,15 @@ FORWARDINGS is a list of host-port/guest-port pairs."
                                                 system target)
   ;; XXX: SYSTEM and TARGET are ignored.
   (match vm
-    (($ <virtual-machine> os qemu graphic? memory-size ())
+    (($ <virtual-machine> os qemu graphic? disk-image-size memory-size ())
      (system-qemu-image/shared-store-script os
                                             #:qemu qemu
                                             #:graphic? graphic?
-                                            #:memory-size memory-size))
-    (($ <virtual-machine> os qemu graphic? memory-size forwardings)
+                                            #:memory-size memory-size
+                                            #:disk-image-size
+                                            disk-image-size))
+    (($ <virtual-machine> os qemu graphic? memory-size disk-image-size
+                          forwardings)
      (let ((options
             `("-net" ,(string-append
                        "user,"
@@ -695,6 +700,8 @@ FORWARDINGS is a list of host-port/guest-port pairs."
                                               #:qemu qemu
                                               #:graphic? graphic?
                                               #:memory-size memory-size
+                                              #:disk-image-size
+                                              disk-image-size
                                               #:options options)))))
 
 ;;; vm.scm ends here
-- 
2.14.1

Message #11 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 3/4] services: Add MongoDB.
Date: Wed, 23 Aug 2017 10:25:15 +0100

* ...
---
 gnu/services/databases.scm | 87 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 87 insertions(+)

diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index de1f6b841..107bb2d09 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -44,6 +44,10 @@
             memcached-configuration-udp-port
             memcached-configuration-additional-options
 
+            mongodb-configuration
+            mongodb-configuration?
+            mongodb-service-type
+
             mysql-service
             mysql-service-type
             mysql-configuration
@@ -263,6 +267,89 @@ and stores the database cluster in @var{data-directory}."
                 (default-value (memcached-configuration))))
 
 
+;;;
+;;; MongoDB
+;;;
+
+(define %default-mongodb-configuration-file
+  (plain-file
+   "mongodb.yaml"
+   "# GNU Guix: MongoDB default configuration file
+processManagement:
+  pidFilePath: /var/run/mongodb/pid
+storage:
+  dbPath: /var/lib/mongodb
+"))
+
+
+(define-record-type* <mongodb-configuration>
+  mongodb-configuration make-mongodb-configuration
+  mongodb-configuration?
+  (mongodb             mongodb-configuration-mongodb
+                       (default mongodb))
+  (port                mongodb-congiguration-port
+                       (default 27017))
+  (configuration-file  mongodb-configuration-file
+                       (default %default-mongodb-configuration-file))
+  (data-directory      mongodb-configuration-data-directory
+                       (default "/var/lib/mongodb")))
+
+(define %mongodb-accounts
+  (list (user-group (name "mongodb") (system? #t))
+        (user-account
+         (name "mongodb")
+         (group "mongodb")
+         (system? #t)
+         (comment "Mongodb server user")
+         (home-directory "/var/lib/mongodb")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define mongodb-activation
+  (match-lambda
+    (($ <mongodb-configuration> mongodb port config-file data-directory)
+     #~(begin
+         (use-modules (guix build utils))
+         (let ((user (getpwnam "mongodb")))
+           (for-each
+            (lambda (directory)
+              (mkdir-p directory)
+              (chown directory
+                     (passwd:uid user) (passwd:gid user)))
+            '("/var/run/mongodb" #$data-directory)))))))
+
+(define mongodb-shepherd-service
+  (match-lambda
+    (($ <mongodb-configuration> mongodb port config-file data-directory)
+     (shepherd-service
+      (provision '(mongodb))
+      (documentation "Run the Mongodb daemon.")
+      (requirement '(user-processes loopback))
+      (start #~(make-forkexec-constructor
+                `(,(string-append #$mongodb "/bin/mongod")
+                  "--config"
+                  ,#$config-file)
+                #:user "mongodb"
+                #:group "mongodb"
+                #:pid-file "/var/run/mongodb/pid"
+                                        ;#:log-file "/var/log/mongodb.log"
+                ))
+      (stop #~(make-kill-destructor))))))
+
+(define mongodb-service-type
+  (service-type
+   (name 'mongodb)
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             (compose list
+                                      mongodb-shepherd-service))
+          (service-extension activation-service-type
+                             mongodb-activation)
+          (service-extension account-service-type
+                             (const %mongodb-accounts))))
+   (default-value
+     (mongodb-configuration))))
+
+
 ;;;
 ;;; MySQL.
 ;;;
-- 
2.14.1

Message #14 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 2/4] gnu: Add mongodb.
Date: Wed, 23 Aug 2017 10:25:14 +0100

* gnu/packages/databases.scm (mongodb): New variable.
---
 gnu/packages/databases.scm | 86 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 86 insertions(+)

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 77abed691..9d98da3c0 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -66,8 +66,10 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages rdf)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages serialization)
   #:use-module (gnu packages tcl)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages valgrind)
   #:use-module (gnu packages xml)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix packages)
@@ -314,6 +316,90 @@ and generic API, and was originally intended for use with dynamic web
 applications.")
     (license license:bsd-3)))
 
+(define-public mongodb
+  (package
+    (name "mongodb")
+    (version "3.4.4")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/mongodb/mongo/archive/r"
+                                  version ".tar.gz"))
+              (file-name (string-append name "-" version ".tar.gz"))
+              (sha256
+               (base32 "1ccd6azplqpi9pp3l6fsi8240kkgagq5j6c2dksppjn7slk1kdy8"))))
+    (build-system gnu-build-system)
+    (native-inputs
+     `(("scons" ,scons)
+       ("python" ,python-2)
+       ("openssl" ,openssl)
+       ("boost" ,boost)
+       ("snappy" ,snappy)
+       ("zlib" ,zlib)
+       ("pcre" ,pcre)
+       ("valgrind" ,valgrind)
+       ("yaml-cpp" ,yaml-cpp)
+       ("perl" ,perl)))
+    (arguments
+     `(#:tests? #f ;; TODO: Check phase currently fails
+       #:phases
+       (let ((common-options
+              `(;; "--use-system-tcmalloc" TODO: Missing gperftools
+                "--use-system-pcre"
+                ;; TODO
+                ;; build/opt/mongo/db/fts/unicode/string.o failed: Error 1
+                ;; "--use-system-boost"
+                "--use-system-snappy"
+                "--use-system-zlib"
+                "--use-system-valgrind"
+                ;; "--use-system-stemmer" TODO: Missing relevant package
+                "--use-system-yaml"
+                "--disable-warnings-as-errors"
+                ,(format #f "--jobs=~a" (parallel-job-count))
+                "--ssl")))
+         (modify-phases %standard-phases
+           (delete 'configure) ; There is no configure phase
+           (add-after 'unpack 'scons-propagate-environment
+             (lambda _
+               ;; Modify the SConstruct file to arrange for
+               ;; environment variables to be propagated.
+               (substitute* "SConstruct"
+                 (("^env = Environment\\(")
+                  "env = Environment(ENV=os.environ, "))))
+           (add-after 'unpack 'create-version-file
+             (lambda _
+               (call-with-output-file "version.json"
+                 (lambda (port)
+                   (display ,(simple-format #f "{
+    \"version\": \"~A\"
+}" version) port)))))
+           (replace 'build
+             (lambda _
+               (zero? (apply system*
+                             `("scons"
+                               ,@common-options
+                               "mongod" "mongo" "mongos")))))
+           (replace 'check
+             (lambda* (#:key tests? #:allow-other-keys)
+               (or (not tests?)
+                   (zero? (apply system*
+                                 `("scons"
+                                   ,@common-options
+                                   "dbtest" "unittests"))))))
+           (replace 'install
+             (lambda _
+               (let ((bin  (string-append (assoc-ref %outputs "out") "/bin")))
+                 (install-file "mongod" bin)
+                 (install-file "mongos" bin)
+                 (install-file "mongo" bin))))))))
+    (home-page "https://www.mongodb.org/")
+    (synopsis "High performance and high availability document database")
+    (description
+     "Mongo is a high-performance, high availability, schema-free
+document-oriented database.  A key goal of MongoDB is to bridge the gap
+between key/value stores (which are fast and highly scalable) and traditional
+RDBMS systems (which are deep in functionality).")
+    (license (list license:agpl3 license:asl2.0))))
+
 (define-public mysql
   (package
     (name "mysql")
-- 
2.14.1

Message #17 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 4/4] tests: databases: Add MongoDB test.
Date: Wed, 23 Aug 2017 10:25:16 +0100

* gnu/tests/databases.scm (%test-mongodb): New variable.
---
 gnu/tests/databases.scm | 77 ++++++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 76 insertions(+), 1 deletion(-)

diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index 9d9a75374..1dbb8a12a 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -25,9 +25,11 @@
   #:use-module (gnu services)
   #:use-module (gnu services databases)
   #:use-module (gnu services networking)
+  #:use-module (gnu packages databases)
   #:use-module (guix gexp)
   #:use-module (guix store)
-  #:export (%test-memcached))
+  #:export (%test-memcached
+            %test-mongodb))
 
 (define %memcached-os
   (simple-operating-system
@@ -121,3 +123,76 @@
    (name "memcached")
    (description "Connect to a running MEMCACHED server.")
    (value (run-memcached-test))))
+
+(define %mongodb-os
+  (operating-system
+    (inherit
+     (simple-operating-system
+      (dhcp-client-service)
+      (service mongodb-service-type)
+      (extra-special-file "/etc/os-release"
+                          (plain-file "os-release" "guix\n"))))
+    (packages (cons* mongodb
+                     %base-packages))))
+
+(define* (run-mongodb-test #:optional (port 27017))
+  "Run tests in %MONGODB-OS, forwarding PORT."
+  (define os
+    (marionette-operating-system
+     %mongodb-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (memory-size 1024)
+     (disk-image-size (* 1024 (expt 2 20)))
+     (port-forwardings `((27017 . ,port)))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (gnu build marionette)
+                       (ice-9 rdelim))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "mongodb")
+
+          ;; Wait for mongodb to be up and running.
+          (test-assert "service running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (match (start-service 'mongodb)
+                  (#f #f)
+                  (('service response-parts ...)
+                   (match (assq-ref response-parts 'running)
+                     ((pid) (number? pid))))))
+             marionette))
+
+          (test-eq "can connect"
+            0
+            (system* (string-append #$mongodb "/bin/mongo")
+                     "test"
+                     "--eval"
+                     "help"))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "mongodb-test" test))
+
+(define %test-mongodb
+  (system-test
+   (name "mongodb")
+   (description "Connect to a running MONGODB server.")
+   (value (run-mongodb-test))))
+
+%mongodb-os
-- 
2.14.1

Message #20 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-devel <at> gnu.org
Cc: Roel Janssen <roel <at> gnu.org>, 28198 <at> debbugs.gnu.org
Subject: Re: [PATCH] gnu: Add mongodb.
Date: Wed, 23 Aug 2017 10:30:06 +0100

[Message part 1 (text/plain, inline)]

On Thu, 17 Mar 2016 16:08:24 +0100
Roel Janssen <roel <at> gnu.org> wrote:

> Dear Guix,
> 
> This is a patch to add MongoDB (server and client) tools.  I used a
> three-line patch to fix the build process's reliance on Git and the
> .git/ directory.

Hey,

I've now created bug #28198 with a slightly modified version of this
patch.

I've started the process of trying to build MongoDB without using the
third party source code that's included. It's going quite slowly,
mostly because the package takes a while to build.

The patches I've sent to #28198 also include a very basic service and
system test. The system test currently fails, as the mongo client
attempts to determine a "distroName", and fails. I haven't worked out
what to do about this yet...

Thanks,

Chris

[Message part 2 (application/pgp-signature, inline)]

Message #23 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Roel Janssen <roel <at> gnu.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH] Add MongoDB package and service.
Date: Sun, 27 Aug 2017 00:33:07 +0200

Christopher Baines writes:

> This is a work in progress issue for supporting MongoDB in Guix.
>
> Back in March 2016, Roel Janssen sent a working package patch to
> guix-devel, but there was concern from those reviewing about the
> security implications of the amount of third party code that is
> included with the MongoDB source.
>
> I've taken that original patch, and started to try and get MongoDB to
> build using libraries and tools from Guix. In addition, I've added a
> very basic service definition and system test.
>
> This isn't ready to be merged yet, but I think it's good to work on in
> public. Patches to follow...

Hello Chris,

Sorry for the late reaction and thanks for picking this up.  I haven't
built mongodb with your patches, but I hope to find the time for this at
some point next week.

I believe I ran into a licensing issue with mongodb's third party stuff.
I cannot find the problematic library so quickly, but we should verify
the licenses of each library in the "third_party" folder:
  https://github.com/mongodb/mongo/tree/master/src/third_party

So, at least the list of licenses for the mongodb package should be
made complete.

Kind regards,
Roel Janssen

Message #26 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Roel Janssen <roel <at> gnu.org>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH] Add MongoDB package and service.
Date: Sun, 27 Aug 2017 23:30:10 +0100

[Message part 1 (text/plain, inline)]

On Sun, 27 Aug 2017 00:33:07 +0200
Roel Janssen <roel <at> gnu.org> wrote:

> Christopher Baines writes:
> 
> > This is a work in progress issue for supporting MongoDB in Guix.
> >
> > Back in March 2016, Roel Janssen sent a working package patch to
> > guix-devel, but there was concern from those reviewing about the
> > security implications of the amount of third party code that is
> > included with the MongoDB source.
> >
> > I've taken that original patch, and started to try and get MongoDB
> > to build using libraries and tools from Guix. In addition, I've
> > added a very basic service definition and system test.
> >
> > This isn't ready to be merged yet, but I think it's good to work on
> > in public. Patches to follow...  
> 
> Hello Chris,
> 
> Sorry for the late reaction and thanks for picking this up.  I haven't
> built mongodb with your patches, but I hope to find the time for this
> at some point next week.
> 
> I believe I ran into a licensing issue with mongodb's third party
> stuff. I cannot find the problematic library so quickly, but we
> should verify the licenses of each library in the "third_party"
> folder: https://github.com/mongodb/mongo/tree/master/src/third_party
> 
> So, at least the list of licenses for the mongodb package should be
> made complete.

Sounds good. Note that I don't think we need to check all the
libraries, as we can hopefully remove the libraries where a
--use-system-... flag is present, which is 5 of them at the moment. 

[Message part 2 (application/pgp-signature, inline)]

Message #29 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 1/4] vm: Add disk-image-size to
 <virtual-machine>.
Date: Thu, 31 Aug 2017 14:29:56 +0200

Hi Chris,

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/system/vm.scm (<virtual-machine>): Add
>   disk-image-size.
>   (port-forwardings->qemu-options): Use disk-image-size from
>   <virtual-machine>.

In which case is it useful?  Perhaps if you want to create lots of data
on the root file system in the MongoDB test?

Currently <virtual-machine> builds a shared-store VM (like ‘guix system
vm’) in which the root file system has a fixed size that’s usually good
enough.

> --- a/gnu/system/vm.scm
> +++ b/gnu/system/vm.scm
> @@ -653,6 +653,8 @@ it is mostly useful when FULL-BOOT?  is true."
>                      (default #f))
>    (memory-size      virtual-machine-memory-size   ;integer (MiB)
>                      (default 256))
> +  (disk-image-size  virtual-machine-disk-image-size   ;integer (bytes)
> +                    (default (* 70 (expt 2 20))))

I think we can use 'guess here as the default value (and we should do
the same in places where #:disk-image-size has an arbitrary default.)

Ludo’.

Message #32 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 2/4] gnu: Add mongodb.
Date: Thu, 31 Aug 2017 14:32:58 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/packages/databases.scm (mongodb): New variable.

[...]

> +    (build-system gnu-build-system)
> +    (native-inputs
> +     `(("scons" ,scons)
> +       ("python" ,python-2)
> +       ("openssl" ,openssl)
> +       ("boost" ,boost)
> +       ("snappy" ,snappy)
> +       ("zlib" ,zlib)
> +       ("pcre" ,pcre)
> +       ("valgrind" ,valgrind)
> +       ("yaml-cpp" ,yaml-cpp)
> +       ("perl" ,perl)))

Seems like some of these should rather go to ‘inputs’, no?

> +    (arguments
> +     `(#:tests? #f ;; TODO: Check phase currently fails

Maybe leave a few details on why this is failing.

> +           (replace 'install
> +             (lambda _
> +               (let ((bin  (string-append (assoc-ref %outputs "out") "/bin")))
> +                 (install-file "mongod" bin)
> +                 (install-file "mongos" bin)
> +                 (install-file "mongo" bin))))))))

Return #t.

> +    (home-page "https://www.mongodb.org/")
> +    (synopsis "High performance and high availability document database")
> +    (description
> +     "Mongo is a high-performance, high availability, schema-free
> +document-oriented database.  A key goal of MongoDB is to bridge the gap
> +between key/value stores (which are fast and highly scalable) and traditional
> +RDBMS systems (which are deep in functionality).")
> +    (license (list license:agpl3 license:asl2.0))))

AGPL version 3 only?  Also please add a comment stating whether it’s
dual-licensed or something else.

OK with these changes, thank you!

Ludo’.

Message #35 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 3/4] services: Add MongoDB.
Date: Thu, 31 Aug 2017 14:34:21 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * ...

Fill in the dots.  :-)

With a few lines in the manual it would be perfect!

Ludo’.

Message #38 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 4/4] tests: databases: Add MongoDB test.
Date: Thu, 31 Aug 2017 14:37:45 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/tests/databases.scm (%test-mongodb): New variable.

FWIW I’d suggest squashing it with the patch that adds the MongoDB
service, since they really go together.

> +(define %mongodb-os
> +  (operating-system
> +    (inherit
> +     (simple-operating-system
> +      (dhcp-client-service)
> +      (service mongodb-service-type)
> +      (extra-special-file "/etc/os-release"
> +                          (plain-file "os-release" "guix\n"))))

Does it require that ‘os-release’ file?  If so, should the service add
it?  That would probably intrusive though, so it’s even better if
MongoDB does not require it.

> +          (test-eq "can connect"
> +            0
> +            (system* (string-append #$mongodb "/bin/mongo")
> +                     "test"
> +                     "--eval"
> +                     "help"))

If it’s easy to do through the CLI, it might be worth trying insert an
element and query it.

> +%mongodb-os

Leftover.

Otherwise LGTM, thank you!

Ludo’.

Message #41 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH] gnu: Add mongodb.
Date: Fri, 15 Sep 2017 18:24:40 +0100

[Message part 1 (text/plain, inline)]

On Wed, 23 Aug 2017 10:30:06 +0100
Christopher Baines <mail <at> cbaines.net> wrote:

> The patches I've sent to #28198 also include a very basic service and
> system test. The system test currently fails, as the mongo client
> attempts to determine a "distroName", and fails. I haven't worked out
> what to do about this yet...

I started a thread about the issue with the mongo client failing on the
mongodb-user mailing list [1], and I've now also got around to creating
a ticket on the MongoDB issue tracker about it [2].

1: https://groups.google.com/d/msg/mongodb-user/jmv1dzc0Kp4/ixvXwyVIBgAJ
2: https://jira.mongodb.org/browse/SERVER-31105

[Message part 2 (application/pgp-signature, inline)]

Message #44 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH] gnu: Add mongodb.
Date: Sat, 16 Sep 2017 22:37:33 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> On Wed, 23 Aug 2017 10:30:06 +0100
> Christopher Baines <mail <at> cbaines.net> wrote:
>
>> The patches I've sent to #28198 also include a very basic service and
>> system test. The system test currently fails, as the mongo client
>> attempts to determine a "distroName", and fails. I haven't worked out
>> what to do about this yet...
>
> I started a thread about the issue with the mongo client failing on the
> mongodb-user mailing list [1], and I've now also got around to creating
> a ticket on the MongoDB issue tracker about it [2].
>
> 1: https://groups.google.com/d/msg/mongodb-user/jmv1dzc0Kp4/ixvXwyVIBgAJ
> 2: https://jira.mongodb.org/browse/SERVER-31105

Given that it’s purely for statistics, IIUC the thread above, I think
the client should have an option to *not* send that information (and it
should be opt-in IMO.)

In the meantime we could patch the code to have a hard-coded
“distroName”.

Ludo’.

Message #47 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 1/4] vm: Add disk-image-size to
 <virtual-machine>.
Date: Mon, 25 Sep 2017 21:36:48 +0100

[Message part 1 (text/plain, inline)]

On Thu, 31 Aug 2017 14:29:56 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hi Chris,

Thanks for reviewing these patches Ludo, unfortunately its taken me
nearly a month to get around to replying.

Thankfully, I've made some progress in that time.

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > * gnu/system/vm.scm (<virtual-machine>): Add
> >   disk-image-size.
> >   (port-forwardings->qemu-options): Use disk-image-size from
> >   <virtual-machine>.  
> 
> In which case is it useful?  Perhaps if you want to create lots of
> data on the root file system in the MongoDB test?
> 
> Currently <virtual-machine> builds a shared-store VM (like ‘guix
> system vm’) in which the root file system has a fixed size that’s
> usually good enough.

The best answer I have at the moment is that I think MongoDB creates a
rather large file, even if it has no data to store in it. I'll do some
more investigation to confirm this though.

> > --- a/gnu/system/vm.scm
> > +++ b/gnu/system/vm.scm
> > @@ -653,6 +653,8 @@ it is mostly useful when FULL-BOOT?  is true."
> >                      (default #f))
> >    (memory-size      virtual-machine-memory-size   ;integer (MiB)
> >                      (default 256))
> > +  (disk-image-size  virtual-machine-disk-image-size   ;integer
> > (bytes)
> > +                    (default (* 70 (expt 2 20))))  
> 
> I think we can use 'guess here as the default value (and we should do
> the same in places where #:disk-image-size has an arbitrary default.)

I've been looking at this in the last few days. Making 'guess the
default value here does mean something, and I think it's a good idea.

As far as I can tell, for this specific test, these are the 3 functions
that are called on the way to using the actual size, and the defaults
they have:

system-qemu-image/shared-store-script default:
  (* (if full-boot? 500 70)
     (expt 2 20)))

system-qemu-image/shared-store default: 
  (* (if full-boot? 500 30) 
     (expt 2 20)))

qemu-image default:
  'guess

From my tests, if the default in the <virtual-machine> is set to
'guess, then for the mongodb test the guess seems to be 0 MiB at the
moment, which doesn't work. I've started looking at setting a sensible
default in qemu-image, so that the root filesystem size isn't 0 MiB.

While testing with the system tests, I noticed that the ones I was
testing with didn't pass without my changes, so I've started looking in
to that now. So far, I've got one patch that helps [1], but I think
there are other issues.

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28600

[Message part 2 (application/pgp-signature, inline)]

Message #50 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 1/4] vm: Add disk-image-size to
 <virtual-machine>.
Date: Tue, 26 Sep 2017 09:14:28 +0200

Hi Chris!

Christopher Baines <mail <at> cbaines.net> skribis:

> On Thu, 31 Aug 2017 14:29:56 +0200
> ludo <at> gnu.org (Ludovic Courtès) wrote:
>
>> Hi Chris,
>
> Thanks for reviewing these patches Ludo, unfortunately its taken me
> nearly a month to get around to replying.
>
> Thankfully, I've made some progress in that time.
>
>> Christopher Baines <mail <at> cbaines.net> skribis:
>> 
>> > * gnu/system/vm.scm (<virtual-machine>): Add
>> >   disk-image-size.
>> >   (port-forwardings->qemu-options): Use disk-image-size from
>> >   <virtual-machine>.  
>> 
>> In which case is it useful?  Perhaps if you want to create lots of
>> data on the root file system in the MongoDB test?
>> 
>> Currently <virtual-machine> builds a shared-store VM (like ‘guix
>> system vm’) in which the root file system has a fixed size that’s
>> usually good enough.
>
> The best answer I have at the moment is that I think MongoDB creates a
> rather large file, even if it has no data to store in it. I'll do some
> more investigation to confirm this though.

OK, that’s a good reason anyway.  :-)

>> > --- a/gnu/system/vm.scm
>> > +++ b/gnu/system/vm.scm
>> > @@ -653,6 +653,8 @@ it is mostly useful when FULL-BOOT?  is true."
>> >                      (default #f))
>> >    (memory-size      virtual-machine-memory-size   ;integer (MiB)
>> >                      (default 256))
>> > +  (disk-image-size  virtual-machine-disk-image-size   ;integer
>> > (bytes)
>> > +                    (default (* 70 (expt 2 20))))  
>> 
>> I think we can use 'guess here as the default value (and we should do
>> the same in places where #:disk-image-size has an arbitrary default.)
>
> I've been looking at this in the last few days. Making 'guess the
> default value here does mean something, and I think it's a good idea.
>
> As far as I can tell, for this specific test, these are the 3 functions
> that are called on the way to using the actual size, and the defaults
> they have:
>
> system-qemu-image/shared-store-script default:
>   (* (if full-boot? 500 70)
>      (expt 2 20)))
>
> system-qemu-image/shared-store default: 
>   (* (if full-boot? 500 30) 
>      (expt 2 20)))
>
> qemu-image default:
>   'guess
>
> From my tests, if the default in the <virtual-machine> is set to
> 'guess, then for the mongodb test the guess seems to be 0 MiB at the
> moment, which doesn't work. I've started looking at setting a sensible
> default in qemu-image, so that the root filesystem size isn't 0 MiB.

Indeed.  Maybe we can go with your patch as-is and investigate the
problem with 'guess separately.  Thoughts?

(I’ll comment on the other issue separately.)

Thanks,
Ludo’.

Message #53 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28635 <at> debbugs.gnu.org, 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 1/4] vm: Add disk-image-size to
 <virtual-machine>.
Date: Fri, 29 Sep 2017 07:48:21 +0100

[Message part 1 (text/plain, inline)]

On Tue, 26 Sep 2017 09:14:28 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Hi Chris!
> 
> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > On Thu, 31 Aug 2017 14:29:56 +0200
> > ludo <at> gnu.org (Ludovic Courtès) wrote:
> >  
> >> Hi Chris,  
> >
> > Thanks for reviewing these patches Ludo, unfortunately its taken me
> > nearly a month to get around to replying.
> >
> > Thankfully, I've made some progress in that time.
> >  
> >> Christopher Baines <mail <at> cbaines.net> skribis:
> >>   
> >> > * gnu/system/vm.scm (<virtual-machine>): Add
> >> >   disk-image-size.
> >> >   (port-forwardings->qemu-options): Use disk-image-size from
> >> >   <virtual-machine>.    
> >> 
> >> In which case is it useful?  Perhaps if you want to create lots of
> >> data on the root file system in the MongoDB test?
> >> 
> >> Currently <virtual-machine> builds a shared-store VM (like ‘guix
> >> system vm’) in which the root file system has a fixed size that’s
> >> usually good enough.  
> >
> > The best answer I have at the moment is that I think MongoDB
> > creates a rather large file, even if it has no data to store in it.
> > I'll do some more investigation to confirm this though.  
> 
> OK, that’s a good reason anyway.  :-)
> 
> >> > --- a/gnu/system/vm.scm
> >> > +++ b/gnu/system/vm.scm
> >> > @@ -653,6 +653,8 @@ it is mostly useful when FULL-BOOT?  is
> >> > true." (default #f))
> >> >    (memory-size      virtual-machine-memory-size   ;integer (MiB)
> >> >                      (default 256))
> >> > +  (disk-image-size  virtual-machine-disk-image-size   ;integer
> >> > (bytes)
> >> > +                    (default (* 70 (expt 2 20))))    
> >> 
> >> I think we can use 'guess here as the default value (and we should
> >> do the same in places where #:disk-image-size has an arbitrary
> >> default.)  
> >
> > I've been looking at this in the last few days. Making 'guess the
> > default value here does mean something, and I think it's a good
> > idea.
> >
> > As far as I can tell, for this specific test, these are the 3
> > functions that are called on the way to using the actual size, and
> > the defaults they have:
> >
> > system-qemu-image/shared-store-script default:
> >   (* (if full-boot? 500 70)
> >      (expt 2 20)))
> >
> > system-qemu-image/shared-store default: 
> >   (* (if full-boot? 500 30) 
> >      (expt 2 20)))
> >
> > qemu-image default:
> >   'guess
> >
> > From my tests, if the default in the <virtual-machine> is set to
> > 'guess, then for the mongodb test the guess seems to be 0 MiB at the
> > moment, which doesn't work. I've started looking at setting a
> > sensible default in qemu-image, so that the root filesystem size
> > isn't 0 MiB.  
> 
> Indeed.  Maybe we can go with your patch as-is and investigate the
> problem with 'guess separately.  Thoughts?

That's fine with me. I've also now created a separate bug with this
patch, and another to add a default value [1].

I tested it with the memcached system test, and the disk image size was
the same plus the test still passed. I haven't checked the other tests
yet.

1: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=28635

[Message part 2 (application/pgp-signature, inline)]

Message #56 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 1/3] vm: Add disk-image-size to <virtual-machine>.
Date: Sat, 30 Sep 2017 08:59:40 +0100

* gnu/system/vm.scm (<virtual-machine>): Add
  disk-image-size.
  (port-forwardings->qemu-options): Use disk-image-size from
  <virtual-machine>.
---
 gnu/system/vm.scm | 13 ++++++++++---
 1 file changed, 10 insertions(+), 3 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 78143e4f7..61fc5c347 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -706,6 +706,8 @@ it is mostly useful when FULL-BOOT?  is true."
                     (default #f))
   (memory-size      virtual-machine-memory-size   ;integer (MiB)
                     (default 256))
+  (disk-image-size  virtual-machine-disk-image-size   ;integer (bytes)
+                    (default (* 70 (expt 2 20))))
   (port-forwardings virtual-machine-port-forwardings ;list of integer pairs
                     (default '())))
 
@@ -734,12 +736,15 @@ FORWARDINGS is a list of host-port/guest-port pairs."
                                                 system target)
   ;; XXX: SYSTEM and TARGET are ignored.
   (match vm
-    (($ <virtual-machine> os qemu graphic? memory-size ())
+    (($ <virtual-machine> os qemu graphic? disk-image-size memory-size ())
      (system-qemu-image/shared-store-script os
                                             #:qemu qemu
                                             #:graphic? graphic?
-                                            #:memory-size memory-size))
-    (($ <virtual-machine> os qemu graphic? memory-size forwardings)
+                                            #:memory-size memory-size
+                                            #:disk-image-size
+                                            disk-image-size))
+    (($ <virtual-machine> os qemu graphic? memory-size disk-image-size
+                          forwardings)
      (let ((options
             `("-net" ,(string-append
                        "user,"
@@ -748,6 +753,8 @@ FORWARDINGS is a list of host-port/guest-port pairs."
                                               #:qemu qemu
                                               #:graphic? graphic?
                                               #:memory-size memory-size
+                                              #:disk-image-size
+                                              disk-image-size
                                               #:options options)))))
 
 ;;; vm.scm ends here
-- 
2.14.1

Message #59 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 3/3] services: Add MongoDB.
Date: Sat, 30 Sep 2017 08:59:42 +0100

* gnu/services/databases.scm (%default-mongodb-configuration-file,
  %mongodb-accounts, mongodb-service-type): New variables.
  (<mongodb-configuration>): New record type.
  (mongodb-activation, mongodb-shepherd-service): New procedures.
* gnu/tests/databases.scm (%test-mongodb): New variable.
* doc/guix.text (Database Services): Add MongoDB documentation.
---
 doc/guix.texi              | 26 ++++++++++++++
 gnu/services/databases.scm | 88 ++++++++++++++++++++++++++++++++++++++++++++++
 gnu/tests/databases.scm    | 86 +++++++++++++++++++++++++++++++++++++++++++-
 3 files changed, 199 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index dd0a46a63..5be427ebd 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -12112,6 +12112,32 @@ Additional command line options to pass to @code{memcached}.
 @end table
 @end deftp
 
+@defvr {Scheme Variable} mongodb-service-type
+This is the service type for @uref{https://www.mongodb.com/, MongoDB}.
+The value for the service type is a @code{mongodb-configuration} object.
+@end defvr
+
+@example
+(service mongodb-service-type)
+@end example
+
+@deftp {Data Type} mongodb-configuration
+Data type representing the configuration of mongodb.
+
+@table @asis
+@item @code{mongodb} (default: @code{mongodb})
+The MongoDB package to use.
+
+@item @code{config-file} (default: @code{%default-mongodb-configuration-file})
+The configuration file for MongoDB.
+
+@item @code{data-directory} (default: @code{"/var/lib/mongodb"})
+This value is used to create the directory, so that it exists and is
+owned by the mongodb user.  It should match the data-directory which
+MongoDB is configured to use through the configuration file.
+@end table
+@end deftp
+
 @defvr {Scheme Variable} redis-service-type
 This is the service type for the @uref{https://redis.io/, Redis}
 key/value store, whose value is a @code{redis-configuration} object.
diff --git a/gnu/services/databases.scm b/gnu/services/databases.scm
index de1f6b841..251dbde3d 100644
--- a/gnu/services/databases.scm
+++ b/gnu/services/databases.scm
@@ -44,6 +44,14 @@
             memcached-configuration-udp-port
             memcached-configuration-additional-options
 
+            <mongodb-configuration>
+            mongodb-configuration
+            mongodb-configuration?
+            mongodb-configuration-mongodb
+            mongodb-configuration-config-file
+            mongodb-configuration-data-directory
+            mongodb-service-type
+
             mysql-service
             mysql-service-type
             mysql-configuration
@@ -263,6 +271,86 @@ and stores the database cluster in @var{data-directory}."
                 (default-value (memcached-configuration))))
 
 
+;;;
+;;; MongoDB
+;;;
+
+(define %default-mongodb-configuration-file
+  (plain-file
+   "mongodb.yaml"
+   "# GNU Guix: MongoDB default configuration file
+processManagement:
+  pidFilePath: /var/run/mongodb/pid
+storage:
+  dbPath: /var/lib/mongodb
+"))
+
+
+(define-record-type* <mongodb-configuration>
+  mongodb-configuration make-mongodb-configuration
+  mongodb-configuration?
+  (mongodb             mongodb-configuration-mongodb
+                       (default mongodb))
+  (config-file         mongodb-configuration-config-file
+                       (default %default-mongodb-configuration-file))
+  (data-directory      mongodb-configuration-data-directory
+                       (default "/var/lib/mongodb")))
+
+(define %mongodb-accounts
+  (list (user-group (name "mongodb") (system? #t))
+        (user-account
+         (name "mongodb")
+         (group "mongodb")
+         (system? #t)
+         (comment "Mongodb server user")
+         (home-directory "/var/lib/mongodb")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define mongodb-activation
+  (match-lambda
+    (($ <mongodb-configuration> mongodb config-file data-directory)
+     #~(begin
+         (use-modules (guix build utils))
+         (let ((user (getpwnam "mongodb")))
+           (for-each
+            (lambda (directory)
+              (mkdir-p directory)
+              (chown directory
+                     (passwd:uid user) (passwd:gid user)))
+            '("/var/run/mongodb" #$data-directory)))))))
+
+(define mongodb-shepherd-service
+  (match-lambda
+    (($ <mongodb-configuration> mongodb config-file data-directory)
+     (shepherd-service
+      (provision '(mongodb))
+      (documentation "Run the Mongodb daemon.")
+      (requirement '(user-processes loopback))
+      (start #~(make-forkexec-constructor
+                `(,(string-append #$mongodb "/bin/mongod")
+                  "--config"
+                  ,#$config-file)
+                #:user "mongodb"
+                #:group "mongodb"
+                #:pid-file "/var/run/mongodb/pid"
+                #:log-file "/var/log/mongodb.log"))
+      (stop #~(make-kill-destructor))))))
+
+(define mongodb-service-type
+  (service-type
+   (name 'mongodb)
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             (compose list
+                                      mongodb-shepherd-service))
+          (service-extension activation-service-type
+                             mongodb-activation)
+          (service-extension account-service-type
+                             (const %mongodb-accounts))))
+   (default-value
+     (mongodb-configuration))))
+
+
 ;;;
 ;;; MySQL.
 ;;;
diff --git a/gnu/tests/databases.scm b/gnu/tests/databases.scm
index 9d9a75374..9e335b27c 100644
--- a/gnu/tests/databases.scm
+++ b/gnu/tests/databases.scm
@@ -25,9 +25,11 @@
   #:use-module (gnu services)
   #:use-module (gnu services databases)
   #:use-module (gnu services networking)
+  #:use-module (gnu packages databases)
   #:use-module (guix gexp)
   #:use-module (guix store)
-  #:export (%test-memcached))
+  #:export (%test-memcached
+            %test-mongodb))
 
 (define %memcached-os
   (simple-operating-system
@@ -121,3 +123,85 @@
    (name "memcached")
    (description "Connect to a running MEMCACHED server.")
    (value (run-memcached-test))))
+
+(define %mongodb-os
+  (operating-system
+    (inherit
+     (simple-operating-system
+      (dhcp-client-service)
+      (service mongodb-service-type)))
+    (packages (cons* mongodb
+                     %base-packages))))
+
+(define* (run-mongodb-test #:optional (port 27017))
+  "Run tests in %MONGODB-OS, forwarding PORT."
+  (define os
+    (marionette-operating-system
+     %mongodb-os
+     #:imported-modules '((gnu services herd)
+                          (guix combinators))))
+
+  (define vm
+    (virtual-machine
+     (operating-system os)
+     (memory-size 1024)
+     (disk-image-size (* 1024 (expt 2 20)))
+     (port-forwardings `((27017 . ,port)))))
+
+  (define test
+    (with-imported-modules '((gnu build marionette))
+      #~(begin
+          (use-modules (srfi srfi-11) (srfi srfi-64)
+                       (gnu build marionette)
+                       (ice-9 popen)
+                       (ice-9 rdelim))
+
+          (define marionette
+            (make-marionette (list #$vm)))
+
+          (mkdir #$output)
+          (chdir #$output)
+
+          (test-begin "mongodb")
+
+          (test-assert "service running"
+            (marionette-eval
+             '(begin
+                (use-modules (gnu services herd))
+                (match (start-service 'mongodb)
+                  (#f #f)
+                  (('service response-parts ...)
+                   (match (assq-ref response-parts 'running)
+                     ((pid) (number? pid))))))
+             marionette))
+
+          (test-eq "test insert"
+            0
+            (system* (string-append #$mongodb "/bin/mongo")
+                     "test"
+                     "--eval"
+                     "db.testCollection.insert({data: 'test-data'})"))
+
+          (test-equal "test find"
+            "test-data"
+            (let* ((port (open-pipe*
+                          OPEN_READ
+                          (string-append #$mongodb "/bin/mongo")
+                          "test"
+                          "--quiet"
+                          "--eval"
+                          "db.testCollection.findOne().data"))
+                   (output (read-line port))
+                   (status (close-pipe port)))
+              output))
+
+          (test-end)
+          (exit (= (test-runner-fail-count (test-runner-current)) 0)))))
+
+  (gexp->derivation "mongodb-test" test))
+
+(define %test-mongodb
+  (system-test
+   (name "mongodb")
+   (description "Connect to a running MONGODB server.")
+   (value (run-mongodb-test))))
-- 
2.14.1

Message #62 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 28198 <at> debbugs.gnu.org
Subject: [PATCH 2/3] gnu: Add mongodb.
Date: Sat, 30 Sep 2017 08:59:41 +0100

* gnu/packages/databases.scm (mongodb): New variable.
---
 gnu/packages/databases.scm                         | 92 ++++++++++++++++++++++
 ...ngodb-support-unknown-linux-distributions.patch | 55 +++++++++++++
 2 files changed, 147 insertions(+)
 create mode 100644 gnu/packages/patches/mongodb-support-unknown-linux-distributions.patch

diff --git a/gnu/packages/databases.scm b/gnu/packages/databases.scm
index 6ce58985e..4ccbe6deb 100644
--- a/gnu/packages/databases.scm
+++ b/gnu/packages/databases.scm
@@ -68,8 +68,10 @@
   #:use-module (gnu packages python)
   #:use-module (gnu packages rdf)
   #:use-module (gnu packages readline)
+  #:use-module (gnu packages serialization)
   #:use-module (gnu packages tcl)
   #:use-module (gnu packages tls)
+  #:use-module (gnu packages valgrind)
   #:use-module (gnu packages xml)
   #:use-module ((guix licenses) #:prefix license:)
   #:use-module (guix packages)
@@ -316,6 +318,96 @@ and generic API, and was originally intended for use with dynamic web
 applications.")
     (license license:bsd-3)))
 
+(define-public mongodb
+  (package
+    (name "mongodb")
+    (version "3.4.9")
+    (source (origin
+              (method url-fetch)
+              (uri (string-append "https://github.com/mongodb/mongo/archive/r"
+                                  version ".tar.gz"))
+              (file-name (string-append name "-" version ".tar.gz"))
+              (sha256
+               (base32 "0gidwyvh3bdwmk2pccgkqkaln4ysgn8iwa7ihjzllsq0rdg95045"))
+              (patches
+               (list
+                (search-patch "mongodb-support-unknown-linux-distributions.patch")))))
+    (build-system gnu-build-system)
+    (inputs
+     `(("openssl" ,openssl)
+       ("pcre" ,pcre)
+       ("yaml-cpp" ,yaml-cpp)
+       ("zlib" ,zlib)
+       ("snappy" ,snappy)
+       ("boost" ,boost)))
+    (native-inputs
+     `(("scons" ,scons)
+       ("python" ,python-2)
+       ("valgrind" ,valgrind)
+       ("perl" ,perl)))
+    (arguments
+     `(#:phases
+       (let ((common-options
+              `(;; "--use-system-tcmalloc" TODO: Missing gperftools
+                "--use-system-pcre"
+                ;; TODO
+                ;; build/opt/mongo/db/fts/unicode/string.o failed: Error 1
+                ;; --use-system-boost
+                "--use-system-snappy"
+                "--use-system-zlib"
+                "--use-system-valgrind"
+                ;; "--use-system-stemmer" TODO: Missing relevant package
+                "--use-system-yaml"
+                "--disable-warnings-as-errors"
+                ,(format #f "--jobs=~a" (parallel-job-count))
+                "--ssl")))
+         (modify-phases %standard-phases
+           (delete 'configure) ; There is no configure phase
+           (add-after 'unpack 'scons-propagate-environment
+             (lambda _
+               ;; Modify the SConstruct file to arrange for
+               ;; environment variables to be propagated.
+               (substitute* "SConstruct"
+                 (("^env = Environment\\(")
+                  "env = Environment(ENV=os.environ, "))))
+           (add-after 'unpack 'create-version-file
+             (lambda _
+               (call-with-output-file "version.json"
+                 (lambda (port)
+                   (display ,(simple-format #f "{
+    \"version\": \"~A\"
+}" version) port)))))
+           (replace 'build
+             (lambda _
+               (zero? (apply system*
+                             `("scons"
+                               ,@common-options
+                               "mongod" "mongo" "mongos")))))
+           (replace 'check
+             (lambda* (#:key tests? #:allow-other-keys)
+               (or (not tests?)
+                   (zero? (apply system*
+                                 `("scons"
+                                   ,@common-options
+                                   "dbtest" "unittests"))))))
+           (replace 'install
+             (lambda _
+               (let ((bin  (string-append (assoc-ref %outputs "out") "/bin")))
+                 (install-file "mongod" bin)
+                 (install-file "mongos" bin)
+                 (install-file "mongo" bin))
+               #t))))))
+    (home-page "https://www.mongodb.org/")
+    (synopsis "High performance and high availability document database")
+    (description
+     "Mongo is a high-performance, high availability, schema-free
+document-oriented database.  A key goal of MongoDB is to bridge the gap
+between key/value stores (which are fast and highly scalable) and traditional
+RDBMS systems (which are deep in functionality).")
+    (license (list license:agpl3
+                   ;; Some parts are licensed under the Apache License
+                   license:asl2.0))))
+
 (define-public mysql
   (package
     (name "mysql")
diff --git a/gnu/packages/patches/mongodb-support-unknown-linux-distributions.patch b/gnu/packages/patches/mongodb-support-unknown-linux-distributions.patch
new file mode 100644
index 000000000..6057ebeb0
--- /dev/null
+++ b/gnu/packages/patches/mongodb-support-unknown-linux-distributions.patch
@@ -0,0 +1,55 @@
+From e724bb7018a482640c4f194f88b554af2c59d76e Mon Sep 17 00:00:00 2001
+From: Mark Benvenuto <mark.benvenuto <at> mongodb.com>
+Date: Wed, 20 Sep 2017 11:50:02 -0400
+Subject: [PATCH] SERVER-30857 Support unknown Linux distributions
+
+---
+ src/mongo/rpc/metadata/client_metadata.cpp | 6 ------
+ src/mongo/util/processinfo_linux.cpp       | 9 ++++++---
+ 2 files changed, 6 insertions(+), 9 deletions(-)
+
+diff --git a/src/mongo/rpc/metadata/client_metadata.cpp b/src/mongo/rpc/metadata/client_metadata.cpp
+index 845a315dd74..a959a4e31e9 100644
+--- a/src/mongo/rpc/metadata/client_metadata.cpp
++++ b/src/mongo/rpc/metadata/client_metadata.cpp
+@@ -302,9 +302,6 @@ void ClientMetadata::serializePrivate(StringData driverName,
+                                       StringData osArchitecture,
+                                       StringData osVersion,
+                                       BSONObjBuilder* builder) {
+-    invariant(!driverName.empty() && !driverVersion.empty() && !osType.empty() && !osName.empty() &&
+-              !osArchitecture.empty() && !osVersion.empty());
+-
+     BSONObjBuilder metaObjBuilder(builder->subobjStart(kMetadataDocumentName));
+ 
+     {
+@@ -347,9 +344,6 @@ Status ClientMetadata::serializePrivate(StringData driverName,
+                                         StringData osVersion,
+                                         StringData appName,
+                                         BSONObjBuilder* builder) {
+-    invariant(!driverName.empty() && !driverVersion.empty() && !osType.empty() && !osName.empty() &&
+-              !osArchitecture.empty() && !osVersion.empty());
+-
+     if (appName.size() > kMaxApplicationNameByteLength) {
+         return Status(ErrorCodes::ClientMetadataAppNameTooLarge,
+                       str::stream() << "The '" << kApplication << "." << kName
+diff --git a/src/mongo/util/processinfo_linux.cpp b/src/mongo/util/processinfo_linux.cpp
+index c3debf377bd..c2813b026b0 100644
+--- a/src/mongo/util/processinfo_linux.cpp
++++ b/src/mongo/util/processinfo_linux.cpp
+@@ -376,10 +376,13 @@ class LinuxSysHelper {
+             if ((nl = name.find('\n', nl)) != string::npos)
+                 // stop at first newline
+                 name.erase(nl);
+-            // no standard format for name and version.  use kernel version
+-            version = "Kernel ";
+-            version += LinuxSysHelper::readLineFromFile("/proc/sys/kernel/osrelease");
++        } else {
++            name = "unknown";
+         }
++
++        // There is no standard format for name and version so use the kernel version.
++        version = "Kernel ";
++        version += LinuxSysHelper::readLineFromFile("/proc/sys/kernel/osrelease");
+     }
+ 
+     /**
-- 
2.14.1

Message #65 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 2/4] gnu: Add mongodb.
Date: Sat, 30 Sep 2017 09:04:20 +0100

[Message part 1 (text/plain, inline)]

On Thu, 31 Aug 2017 14:32:58 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > * gnu/packages/databases.scm (mongodb): New variable.  
> 
> [...]
> 
> > +    (build-system gnu-build-system)
> > +    (native-inputs
> > +     `(("scons" ,scons)
> > +       ("python" ,python-2)
> > +       ("openssl" ,openssl)
> > +       ("boost" ,boost)
> > +       ("snappy" ,snappy)
> > +       ("zlib" ,zlib)
> > +       ("pcre" ,pcre)
> > +       ("valgrind" ,valgrind)
> > +       ("yaml-cpp" ,yaml-cpp)
> > +       ("perl" ,perl)))  
> 
> Seems like some of these should rather go to ‘inputs’, no?

Indeed. I've now sent an updated set of patches, and I've moved some of
these to inputs.

> > +    (arguments
> > +     `(#:tests? #f ;; TODO: Check phase currently fails  
> 
> Maybe leave a few details on why this is failing.

I tried to check, and the tests passed, which seems good :)

> > +           (replace 'install
> > +             (lambda _
> > +               (let ((bin  (string-append (assoc-ref %outputs
> > "out") "/bin")))
> > +                 (install-file "mongod" bin)
> > +                 (install-file "mongos" bin)
> > +                 (install-file "mongo" bin))))))))  
> 
> Return #t.

Good spot, this should be fixed in the latest patch.

> > +    (home-page "https://www.mongodb.org/")
> > +    (synopsis "High performance and high availability document
> > database")
> > +    (description
> > +     "Mongo is a high-performance, high availability, schema-free
> > +document-oriented database.  A key goal of MongoDB is to bridge
> > the gap +between key/value stores (which are fast and highly
> > scalable) and traditional +RDBMS systems (which are deep in
> > functionality).")
> > +    (license (list license:agpl3 license:asl2.0))))  
> 
> AGPL version 3 only?  Also please add a comment stating whether it’s
> dual-licensed or something else.

I've added a comment stating that the Apache license applies to parts
of the source code. I don't believe it's dual licensed.

[Message part 2 (application/pgp-signature, inline)]

Message #68 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 3/4] services: Add MongoDB.
Date: Sat, 30 Sep 2017 09:05:04 +0100

[Message part 1 (text/plain, inline)]

On Thu, 31 Aug 2017 14:34:21 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > * ...  
> 
> Fill in the dots.  :-)
> 
> With a few lines in the manual it would be perfect!

This should be done. I think I've updated the commit message, and added
some documentation to the manual.

[Message part 2 (application/pgp-signature, inline)]

Message #71 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 4/4] tests: databases: Add MongoDB test.
Date: Sat, 30 Sep 2017 09:09:41 +0100

[Message part 1 (text/plain, inline)]

On Thu, 31 Aug 2017 14:37:45 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > * gnu/tests/databases.scm (%test-mongodb): New variable.  
> 
> FWIW I’d suggest squashing it with the patch that adds the MongoDB
> service, since they really go together.

Ok, I've merged these patches together now.
 
> > +(define %mongodb-os
> > +  (operating-system
> > +    (inherit
> > +     (simple-operating-system
> > +      (dhcp-client-service)
> > +      (service mongodb-service-type)
> > +      (extra-special-file "/etc/os-release"
> > +                          (plain-file "os-release" "guix\n"))))  
> 
> Does it require that ‘os-release’ file?  If so, should the service add
> it?  That would probably intrusive though, so it’s even better if
> MongoDB does not require it.

I think this was me trying to get the tests to pass, but it didn't
help, as the mongo client was running outside of the test system. I've
now removed it.

> > +          (test-eq "can connect"
> > +            0
> > +            (system* (string-append #$mongodb "/bin/mongo")
> > +                     "test"
> > +                     "--eval"
> > +                     "help"))  
> 
> If it’s easy to do through the CLI, it might be worth trying insert an
> element and query it.

I've done this now. Also, I've included a patch from upstream that
fixes the crash in the client that was causing the system test to fail,
so the test should now pass.

> > +%mongodb-os  
> 
> Leftover.

I've removed this now.

> Otherwise LGTM, thank you!

Great, thanks for reviewing Ludo :)

[Message part 2 (application/pgp-signature, inline)]

Message #74 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 2/3] gnu: Add mongodb.
Date: Wed, 04 Oct 2017 16:40:50 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/packages/databases.scm (mongodb): New variable.

[...]

> +           (add-after 'unpack 'scons-propagate-environment
> +             (lambda _
> +               ;; Modify the SConstruct file to arrange for
> +               ;; environment variables to be propagated.
> +               (substitute* "SConstruct"
> +                 (("^env = Environment\\(")
> +                  "env = Environment(ENV=os.environ, "))))
> +           (add-after 'unpack 'create-version-file
> +             (lambda _
> +               (call-with-output-file "version.json"
> +                 (lambda (port)
> +                   (display ,(simple-format #f "{
> +    \"version\": \"~A\"
> +}" version) port)))))

These two should explicitly return #t.

OK with this change, thank you!

Ludo’.

Message #77 received at 28198 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Christopher Baines <mail <at> cbaines.net>
Cc: 28198 <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 3/3] services: Add MongoDB.
Date: Wed, 04 Oct 2017 16:44:21 +0200

Christopher Baines <mail <at> cbaines.net> skribis:

> * gnu/services/databases.scm (%default-mongodb-configuration-file,
>   %mongodb-accounts, mongodb-service-type): New variables.
>   (<mongodb-configuration>): New record type.
>   (mongodb-activation, mongodb-shepherd-service): New procedures.
> * gnu/tests/databases.scm (%test-mongodb): New variable.
> * doc/guix.text (Database Services): Add MongoDB documentation.
                ^

texi :-)

[...]

> +(define mongodb-service-type
> +  (service-type
> +   (name 'mongodb)
> +   (extensions
> +    (list (service-extension shepherd-root-service-type
> +                             (compose list
> +                                      mongodb-shepherd-service))
> +          (service-extension activation-service-type
> +                             mongodb-activation)
> +          (service-extension account-service-type
> +                             (const %mongodb-accounts))))
> +   (default-value
> +     (mongodb-configuration))))

Could you add a ‘description’ field?  :-)

Like “Run the MongoDB document database server.”

Apart from that it looks great.

Thank you!

Ludo’.

Message #82 received at 28198-done <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 28198-done <at> debbugs.gnu.org
Subject: Re: [bug#28198] [PATCH 3/3] services: Add MongoDB.
Date: Fri, 6 Oct 2017 21:29:52 +0100

[Message part 1 (text/plain, inline)]

On Wed, 04 Oct 2017 16:44:21 +0200
ludo <at> gnu.org (Ludovic Courtès) wrote:

> Christopher Baines <mail <at> cbaines.net> skribis:
> 
> > * gnu/services/databases.scm (%default-mongodb-configuration-file,
> >   %mongodb-accounts, mongodb-service-type): New variables.
> >   (<mongodb-configuration>): New record type.
> >   (mongodb-activation, mongodb-shepherd-service): New procedures.
> > * gnu/tests/databases.scm (%test-mongodb): New variable.
> > * doc/guix.text (Database Services): Add MongoDB documentation.  
>                 ^
> 
> texi :-)
> 
> [...]
> 
> > +(define mongodb-service-type
> > +  (service-type
> > +   (name 'mongodb)
> > +   (extensions
> > +    (list (service-extension shepherd-root-service-type
> > +                             (compose list
> > +                                      mongodb-shepherd-service))
> > +          (service-extension activation-service-type
> > +                             mongodb-activation)
> > +          (service-extension account-service-type
> > +                             (const %mongodb-accounts))))
> > +   (default-value
> > +     (mongodb-configuration))))  
> 
> Could you add a ‘description’ field?  :-)
> 
> Like “Run the MongoDB document database server.”
> 
> Apart from that it looks great.

I've added the explicit #t values to the two phases, fixed the texi
typo, and added the description, and now pushed.

Thanks again for reviewing Ludo :)

[Message part 2 (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.