GNU bug report logs - #47741
[PATCH 0/3] Add a service for OpenDHT

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 47741 in the body.
You can then email your comments to 47741 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: guix-patches <at> gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: [PATCH 0/3] Add a service for OpenDHT
Date: Tue, 13 Apr 2021 00:14:59 -0400

This series overhaul our opendht package a bit and adds a service for it.  You
can now for example set up your own Jami proxy server easily to use with Jami
on your phone.  Or simply to strengthen the distributed network by
participating in it and adding new stable bootstrap nodes.

Thanks,

Maxim Cournoyer (3):
  gnu: opendht: Add Python bindings, tools.
  services: configuration: Add syntactic sugar to easily generate
    documentation.
  services: Add opendht.

 doc/guix.texi                  |  84 ++++++++++++++++++++
 gnu/packages/networking.scm    |  75 +++++++++++++++---
 gnu/services/configuration.scm |  13 ++++
 gnu/services/networking.scm    | 135 ++++++++++++++++++++++++++++++++-
 4 files changed, 297 insertions(+), 10 deletions(-)

-- 
2.31.1

Message #8 received at 47741 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 47741 <at> debbugs.gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: [PATCH 1/3] gnu: opendht: Add Python bindings, tools.
Date: Tue, 13 Apr 2021 00:17:08 -0400

* gnu/packages/networking.scm (opendht)
[outputs]: Add tools and debug outputs.
[native-inputs]: Add python and python-cython.
[imported-modules, modules, phases]: New arguments.
[configure-flags]: Disable static compilation.  Enable Python support and the
tools.
[synopsis]: Update synopsis.
[description]: Expound description.
---
 gnu/packages/networking.scm | 75 ++++++++++++++++++++++++++++++++-----
 1 file changed, 66 insertions(+), 9 deletions(-)

diff --git a/gnu/packages/networking.scm b/gnu/packages/networking.scm
index baaa9c5707..b3a51c4b82 100644
--- a/gnu/packages/networking.scm
+++ b/gnu/packages/networking.scm
@@ -3436,6 +3436,7 @@ and targeted primarily for asynchronous processing of HTTP-requests.")
                (base32
                 "1wc0f6cnvnlmhxnx64nxqgsx93k4g7ljdaqjl40ml74jg3nqrzcl"))))
     ;; Since 2.0, the gnu-build-system does not seem to work anymore, upstream bug?
+    (outputs '("out" "tools" "debug"))
     (build-system cmake-build-system)
     (inputs
      `(("argon2" ,argon2)
@@ -3452,23 +3453,79 @@ and targeted primarily for asynchronous processing of HTTP-requests.")
      `(("autoconf" ,autoconf)
        ("automake" ,automake)
        ("pkg-config" ,pkg-config)
+       ("python" ,python)
+       ("python-cython" ,python-cython)
        ("libtool" ,libtool)
        ("cppunit" ,cppunit)))
     (arguments
-     `(#:tests? #f                      ; Tests require network connection.
+     `(#:imported-modules ((guix build python-build-system) ;for site-packages
+                           ,@%cmake-build-system-modules)
+       #:modules (((guix build python-build-system) #:prefix python:)
+                  (guix build cmake-build-system)
+                  (guix build utils))
+       #:tests? #f                      ; Tests require network connection.
        #:configure-flags
-       '(;;"-DOPENDHT_TESTS=on"
-         "-DOPENDHT_TOOLS=off"
-         "-DOPENDHT_PYTHON=off"
+       '( ;;"-DOPENDHT_TESTS=on"
+         "-DOPENDHT_STATIC=off"
+         "-DOPENDHT_TOOLS=on"
+         "-DOPENDHT_PYTHON=on"
          "-DOPENDHT_PROXY_SERVER=on"
          "-DOPENDHT_PUSH_NOTIFICATIONS=on"
          "-DOPENDHT_PROXY_SERVER_IDENTITY=on"
-         "-DOPENDHT_PROXY_CLIENT=on")))
+         "-DOPENDHT_PROXY_CLIENT=on")
+       #:phases
+       (modify-phases %standard-phases
+         (add-after 'unpack 'fix-python-installation-prefix
+           ;; Specify the installation prefix for the compiled Python module
+           ;; that would otherwise attempt to installs itself to Python's own
+           ;; site-packages directory.
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (substitute* "python/CMakeLists.txt"
+               (("--root=\\\\\\$ENV\\{DESTDIR\\}")
+                (string-append "--root=/ --single-version-externally-managed "
+                               "--prefix=${CMAKE_INSTALL_PREFIX}")))))
+         (add-after 'unpack 'specify-runpath-for-python-module
+           (lambda* (#:key outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out")))
+               (substitute* "python/setup.py.in"
+                 (("extra_link_args=\\[(.*)\\]" _ args)
+                  (string-append "extra_link_args=[" args
+                                 ", '-Wl,-rpath=" out "/lib']"))))))
+         (add-after 'install 'move-and-wrap-tools
+           (lambda* (#:key inputs outputs #:allow-other-keys)
+             (let ((out (assoc-ref outputs "out"))
+                   (tools (assoc-ref outputs "tools"))
+                   (site-packages (python:site-packages inputs outputs)))
+               (mkdir tools)
+               (rename-file (string-append out "/bin")
+                            (string-append tools "/bin"))
+               (wrap-program (string-append tools "/bin/dhtcluster")
+                 `("PYTHONPATH" prefix (,site-packages)))))))))
     (home-page "https://github.com/savoirfairelinux/opendht/")
-    (synopsis "Distributed Hash Table (DHT) library")
-    (description "OpenDHT is a Distributed Hash Table (DHT) library.  It may
-be used to manage peer-to-peer network connections as needed for real time
-communication.")
+    (synopsis "Lightweight Distributed Hash Table (DHT) library")
+    (description "OpenDHT provides an easy to use distributed in-memory data
+store.  Every node in the network can read and write values to the store.
+Values are distributed over the network, with redundancy.  It includes the
+following features:
+@itemize
+@item Lightweight and scalable, designed for large networks and small devices;
+@item High resilience to network disruption;
+@item Public key cryptography layer providing optional data signature and
+encryption (using GnuTLS);
+@item IPv4 and IPv6 support;
+@item Clean and powerful C++14 map API;
+@item Bindings for C, Rust & Python 3;
+@item REST API with an optional HTTP client and server with push notification
+support.
+@end itemize
+The following tools are also included:
+@table @command
+@item dhtnode
+A command line tool to run a DHT node and perform operations supported by the
+library (get, put, etc.) with text values.
+@item dhtchat
+A very simple IM client working over the DHT.
+@end table")
     (license license:gpl3+)))
 
 (define-public frrouting
-- 
2.31.1

Message #11 received at 47741 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 47741 <at> debbugs.gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: [PATCH 2/3] services: configuration: Add syntactic sugar to easily
 generate documentation.
Date: Tue, 13 Apr 2021 00:17:09 -0400

I found the original (undocumented) interface difficult to understand, which
was the rationale for adding a simpler one on top of it.

* gnu/services/configuration.scm (configuration->documentation): New procedure.
---
 gnu/services/configuration.scm | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/gnu/services/configuration.scm b/gnu/services/configuration.scm
index 90f12a8d39..750535342d 100644
--- a/gnu/services/configuration.scm
+++ b/gnu/services/configuration.scm
@@ -43,6 +43,7 @@
             define-configuration
             validate-configuration
             generate-documentation
+            configuration->documentation
             serialize-package))
 
 ;;; Commentary:
@@ -188,3 +189,15 @@
                       (or (assq-ref sub-documentation field-name) '())))))
             fields)))))
   (stexi->texi `(*fragment* . ,(generate documentation-name))))
+
+(define (configuration->documentation configuration-symbol)
+  "Takes CONFIGURATION-SYMBOL, the symbol used when defining a configuration
+record with DEFINE-CONFIGURATION, and outputs the Texinfo documentation of its
+fields."
+  ;; This is syntax sugar for a simple, straight-forward application of
+  ;; GENERATE-DOCUMENTATION.
+  (let ((fields-getter (module-ref (current-module)
+                                   (symbol-append configuration-symbol
+                                                  '-fields))))
+    (format #t (generate-documentation `((,configuration-symbol ,fields-getter))
+                                       configuration-symbol))))
-- 
2.31.1

Message #14 received at 47741 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 47741 <at> debbugs.gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: [PATCH 3/3] services: Add opendht.
Date: Tue, 13 Apr 2021 00:17:10 -0400

* gnu/services/networking.scm (serialize-boolean)
(serialize-number, serialize-string): New dummy procedures.
(maybe-number, opendht-configuration): New syntaxes.
(%opendht-accounts): New variable.
(opendht-configuration->command-line-arguments): Likewise.
(opendht-shepherd-service, opendht-service-type): New variables.
* doc/guix.texi: Document it.
---
 doc/guix.texi               |  84 ++++++++++++++++++++++
 gnu/services/networking.scm | 135 +++++++++++++++++++++++++++++++++++-
 2 files changed, 218 insertions(+), 1 deletion(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index 84d8bf50be..d2593b2180 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -16653,6 +16653,90 @@ must be @code{'()} or @code{'("internal")}.
 detailed discussion of each configuration field.
 @end deftp
 
+@cindex opendht, distributed hash table network service
+@cindex dhtproxy, for use with jami
+@defvr {Scheme Variable} opendht-service-type
+This is the type of the service running a @uref{https://opendht.net,
+OpenDHT} node, @command{dhtnode}.  The daemon can be used to host your
+own proxy service to the DHT, for example to connect to with Jami, among
+other applications.
+
+The value of this service is a @code{opendht-configuration} object, as
+described below.
+@end defvr
+
+@deftp {Data Type} opendht-configuration
+This is the data type for the OpenDHT service configuration.
+
+@c The fields documentation has been auto-generated using the
+@c configuration->documentation procedure from (gnu services
+@c configuration).
+@c TODO: Tweak the tool so that it generates doc that matches our
+@c current standard.
+Available @code{opendht-configuration} fields are:
+
+@deftypevr {@code{opendht-configuration} parameter} package opendht
+The @code{opendht} package to use.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} boolean peer-discovery?
+Whether to enable the multicast local peer discovery mechanism.
+
+Defaults to @samp{#f}.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} boolean enable-logging?
+Whether to enable logging messages to syslog.  It is disabled by default
+as it is rather verbose.
+
+Defaults to @samp{#f}.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} boolean debug?
+Whether to enable debug-level logging messages.  This has no effect if
+logging is disabled.
+
+Defaults to @samp{#f}.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} maybe-string bootstrap-host
+The node host name that is used to make the first connection to the
+network.  A specific port value can be provided by appending the
+@code{:PORT} suffix.  By default, it uses the Jami bootstrap nodes, but
+any host can be specified here.  It's also possible to disable
+bootsrapping by setting this to the @code{'disabled} symbol.
+
+Defaults to @samp{"bootstrap.jami.net:4222"}.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} maybe-number port
+The UDP port to bind to.  When set to @code{'disabled}, an available
+port is automatically selected.
+
+Defaults to @samp{4222}.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} maybe-number proxy-server-port
+Spawn a proxy server listening on the specified port.
+
+Defaults to @samp{disabled}.
+
+@end deftypevr
+
+@deftypevr {@code{opendht-configuration} parameter} maybe-number proxy-server-port-tls
+Spawn a proxy server listening to TLS connections on the specified port.
+
+Defaults to @samp{disabled}.
+
+@end deftypevr
+@end deftp
+
 @cindex Tor
 @defvr {Scheme Variable} tor-service-type
 This is the type for a service that runs the @uref{https://torproject.org,
diff --git a/gnu/services/networking.scm b/gnu/services/networking.scm
index 231a9f66c7..8c22f0eec3 100644
--- a/gnu/services/networking.scm
+++ b/gnu/services/networking.scm
@@ -10,7 +10,7 @@
 ;;; Copyright © 2018 Chris Marusich <cmmarusich <at> gmail.com>
 ;;; Copyright © 2018 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;; Copyright © 2019 Florian Pelz <pelzflorian <at> pelzflorian.de>
-;;; Copyright © 2019 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
+;;; Copyright © 2019, 2021 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
 ;;; Copyright © 2019 Sou Bunnbu <iyzsong <at> member.fsf.org>
 ;;; Copyright © 2019 Alex Griffin <a <at> ajgrf.com>
 ;;; Copyright © 2020 Brice Waegeneire <brice <at> waegenei.re>
@@ -108,6 +108,18 @@
             inetd-entry
             inetd-service-type
 
+            opendht-configuration
+            opendht-configuration-peer-discovery?
+            opendht-configuration-verbose?
+            opendht-configuration-bootstrap-host
+            opendht-configuration-port
+            opendht-configuration-proxy-server-port
+            opendht-configuration-proxy-server-port-tls
+            opendht-configuration->command-line-arguments
+
+            opendht-shepherd-service
+            opendht-service-type
+
             tor-configuration
             tor-configuration?
             tor-hidden-service
@@ -730,6 +742,127 @@ daemon will keep the system clock synchronized with that of the given servers.")
 for listening on Internet sockets and spawning the corresponding services on
 demand.")))
 
+
+;;;
+;;; OpenDHT, the distributed hash table network used by Jami
+;;;
+
+;;; XXX: These dummy definitions is because there's no way to disable the
+;;; serialization code from define-configuration.
+(define (serialize-boolean option value) "")
+(define (serialize-number option value) "")
+(define (serialize-string option value) "")
+
+(define-maybe number)
+(define-maybe string)
+
+;;; To generate the documentation of the following configuration record, you
+;;; can evaluate: (configuration->documentation 'opendht-configuration)
+(define-configuration opendht-configuration
+  (opendht
+   (package opendht)
+   "The @code{opendht} package to use.")
+  (peer-discovery?
+   (boolean #false)
+   "Whether to enable the multicast local peer discovery mechanism.")
+  (enable-logging?
+   (boolean #false)
+   "Whether to enable logging messages to syslog.  It is disabled by default
+as it is rather verbose.")
+  (debug?
+   (boolean #false)
+   "Whether to enable debug-level logging messages.  This has no effect if
+logging is disabled.")
+  (bootstrap-host
+   (maybe-string "bootstrap.jami.net:4222")
+   "The node host name that is used to make the first connection to the
+network.  A specific port value can be provided by appending the @code{:PORT}
+suffix.  By default, it uses the Jami bootstrap nodes, but any host can be
+specified here.  It's also possible to disable bootstrapping by setting this
+to the @code{'disabled} symbol.")
+  (port
+   (maybe-number 4222)
+   "The UDP port to bind to.  When set to @code{'disabled}, an available port
+is automatically selected.")
+  (proxy-server-port
+   (maybe-number 'disabled)
+   "Spawn a proxy server listening on the specified port.")
+  (proxy-server-port-tls
+   (maybe-number 'disabled)
+   "Spawn a proxy server listening to TLS connections on the specified
+port."))
+
+(define %opendht-accounts
+  ;; User account and groups for Tor.
+  (list (user-group (name "opendht") (system? #t))
+        (user-account
+         (name "opendht")
+         (group "opendht")
+         (system? #t)
+         (comment "OpenDHT daemon user")
+         (home-directory "/var/empty")
+         (shell (file-append shadow "/sbin/nologin")))))
+
+(define (opendht-configuration->command-line-arguments config)
+  "Derive the command line arguments to used from CONFIG, an
+<opendht-configuration> object."
+  (match-record config <opendht-configuration>
+    (opendht bootstrap-host enable-logging? port debug? peer-discovery?
+             proxy-server-port proxy-server-port-tls)
+    (let ((dhtnode #~(string-append #$opendht:tools "/bin/dhtnode")))
+      `(,dhtnode
+        "--service"                     ;non-forking mode
+        ,@(if (string? bootstrap-host)
+              (list "--bootstrap" bootstrap-host))
+        ,@(if enable-logging?
+              (list "--syslog")
+              '())
+        ,@(if (number? port)
+              (list "--port" (number->string port))
+              '())
+        ,@(if debug?
+              (list "--verbose")
+              '())
+        ,@(if peer-discovery?
+              (list "--peer-discovery")
+              '())
+        ,@(if (number? proxy-server-port)
+              (list "--proxyserver" (number->string proxy-server-port))
+              '())
+        ,@(if (number? proxy-server-port-tls)
+              (list "--proxyserverssl" (number->string proxy-server-port-tls))
+              '())))))
+
+(define (opendht-shepherd-service config)
+  "Return a <shepherd-service> running OpenDHT."
+  (shepherd-service
+   (documentation "Run an OpenDHT node.")
+   (provision '(opendht dhtnode dhtproxy))
+   (requirement '(user-processes syslogd))
+   (start #~(make-forkexec-constructor/container
+             (list #$@(opendht-configuration->command-line-arguments config))
+             #:mappings (list (file-system-mapping
+                               (source "/dev/log") ;for syslog
+                               (target source)))
+             #:user "opendht"))
+   (stop #~(make-kill-destructor))))
+
+(define opendht-service-type
+  (service-type
+   (name 'opendht)
+   (default-value (opendht-configuration))
+   (extensions
+    (list (service-extension shepherd-root-service-type
+                             (compose list opendht-shepherd-service))
+          (service-extension account-service-type
+                             (const %opendht-accounts))))
+   (description "Run the OpenDHT @command{dhtnode} command that allows
+participating in the distributed hash table based OpenDHT network.  The
+service can be configured to act as a proxy to the distributed network, which
+can be useful for portable devices where minimizing energy consumption is
+paramount.  OpenDHT was originally based on Kademlia and adapted for
+applications in communication.  It is used by Jami, for example.")))
+
 
 ;;;
 ;;; Tor.
-- 
2.31.1

Message #19 received at 47741-done <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: 47741-done <at> debbugs.gnu.org
Subject: Re: bug#47741: [PATCH 0/3] Add a service for OpenDHT
Date: Mon, 17 May 2021 23:59:44 -0400

Pushed as commit fd449608eb9016fb705eebbea784901b0242cbf2 along some
improvements for (gnu services configuration).

Closing.

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.