X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 0/3] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Thu, 16 Dec 2021 16:18:02 +0000 Resent-Message-ID: <handler.52555.B.163967147614866 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: report 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN X-Debbugs-Original-To: guix-patches@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.163967147614866 (code B ref -1); Thu, 16 Dec 2021 16:18:02 +0000 Received: (at submit) by debbugs.gnu.org; 16 Dec 2021 16:17:56 +0000 Received: from localhost ([127.0.0.1]:36493 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1mxtS1-0003rY-Se for submit <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:17:56 -0500 Received: from lists.gnu.org ([209.51.188.17]:37690) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1mxtRx-0003rN-Ab for submit <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:17:48 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55568) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <pukkamustard@HIDDEN>) id 1mxtRu-0007zZ-Gl for guix-patches@HIDDEN; Thu, 16 Dec 2021 11:17:43 -0500 Received: from mout01.posteo.de ([185.67.36.65]:54721) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <pukkamustard@HIDDEN>) id 1mxtRr-0002xU-RE for guix-patches@HIDDEN; Thu, 16 Dec 2021 11:17:42 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id A7664240027 for <guix-patches@HIDDEN>; Thu, 16 Dec 2021 17:17:36 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1639671456; bh=LCOS0Ruzr1yXnOxpCv/ZshQ+w7IBoj9ooeByvyiV5d8=; h=From:To:Cc:Subject:Date:From; b=owd1T1Cl32kJHXMZf3WHFiPOsjnFVOytH4ghqM0GkhJppwP2dm/+FhcWST8y0jQaY xaz4l+BbbwPuyq2VuvI6a5T3AR5gSTVIcDp78szjoVq5/1egww/hG5Xn+xt538NMSO bkLL6RW2bEKcnQ40FniNb4CeLpO5dJnjkHF7clurTmifj4HuQ1YIRf6FRtud1oivee 3WSf/fz6ccTB+TzqjCLdVnYfCtK4iHo/Nscb5VOx/cB4qL2/zZUT+th9JHJ7O1SDyo oMWAft+8/jnyeQvogoCndtkL+Te/69ZSuMIB90XFQdk/MKs2NoXv7VXTSYp8MDL4so UMHM/LeItkubw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JFHML4ndkz9rxQ; Thu, 16 Dec 2021 17:17:34 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Thu, 16 Dec 2021 16:17:24 +0000 Message-Id: <20211216161724.547-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit Received-SPF: pass client-ip=185.67.36.65; envelope-from=pukkamustard@HIDDEN; helo=mout01.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.0 (/) Hello Guix, This is an initial patch and proposal towards decentralizing substitute distribution with ERIS. ERIS (Encoding for Robust Immutable Storage) [1] is an encoding of content into uniformly sized, encryped and content-addressed blocks. The original content can be reconstructed only with access to a read capability, which can be encoded as an URN. One key advantage of ERIS is that the encoding is protocol agnostic. Any protocol that can transfer small (32KiB) sized blocks referenced by the hash of their content will do. This can be done with things such as GNUNet, IPFS, OpenDHT, HTTP or a USB stick on a bicycle. The following patch allows substitutes to be published over IPFS using ERIS. This is inspired and very similar to previous work on distributing substitutes over IPFS [2]. The narinfos served by `guix publish` look like this: --8<---------------cut here---------------start------------->8--- StorePath: /gnu/store/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 URL: nar/gzip/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 Compression: gzip FileSize: 67363 ERIS: urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDVDAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE URL: nar/zstd/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 Compression: zstd FileSize: 64917 ERIS: urn:erisx2:BIBO7KS7SAWHDNC43DVILOSQ3F3SRRHEV6YPLDCSZ7MMD6LZVCHQMEQ6FUBTJAPSNFF7XR5XPTP4OQ72OPABNEO7UYBUN42O46ARKHBTGM NarHash: sha256:1sagsz1mnlqkr8r8s6gwkzvvhq619rlzhpbxl3h0b111n5hn2w9w NarSize: 220704 References: 2fk1gz2s7ppdicynscra9b19byrrr866-glibc-2.33 81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 90lbavffg0csrf208nw0ayj1bz5knl47-gcc-10.3.0-lib Deriver: 260bk0ch4np4h2yz5yqhf8hjbsyhwpmr-hello-2.10.drv Signature: 1;strawberry;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjNDk4ODkwODZDNTY4MzQyRENFQzk3QzA3NDE4NEQ1RkRCOTNCNDA2MUNCRDM4MUExRjVBQzVDODI0MTgwMTU3OSMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMEU2NDlFODE4QzRFNjNGNEY2OUQ5QTAwRjUwNjRDMzQ3QjY3RDM0RTM0NTg2MkI4NTc3RTg5MUY5Q0Q3NDhBQiMpCiAgIChzICMwMTZGRjA1MDdCQjZGMzA2NUEzMjYzRDA2MTAyRDc5MTBEOEZGODc5RTdENjREODRFODBENDBGMTJFMTBBOTQ1IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjMDRDMkY4ODk1QTU0NDNGNTlCODk2NDEwMEI1MDY0NzU4RjQ1N0YzMENEREE1MTQyQzE0MDc0NjExNTA1NTc5MCMpCiAgICkKICApCiApCg== --8<---------------cut here---------------end--------------->8--- For every compressed nar the ERIS URN is computed and added. If the `--ipfs` is used for `guix publish` then the encoded blocks are also uploaded to the IPFS daemon. The nar could then be retrieved from anywhere like this: --8<---------------cut here---------------start------------->8--- (use-modules (eris) (eris blocks ipfs)) (eris-decode->bytevector "urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDVDAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE" eris-blocks-ipfs-ref) --8<---------------cut here---------------end--------------->8--- These patches do not yet retrieve content from IPFS (TODO). But in principle, anybody connected to IPFS can get the nar with the ERIS URN. This could be used to reduce load on substitute server as they would only need to publish the ERIS URN directly - substitutes could be delivered much more peer-to-peer. Other transports that I have been looking in to and am pretty sure will work include: HTTP (with RFC 2169 [3]), GNUNet, OpenDHT. This is, imho, the advantage of ERIS over IPFS directly or GNUNet directly. The encoding and identifiers (URN) are abstracted away from specific transports (and also applications). ERIS is almost exactly the same encoding as used in GNUNet (ECRS). Blocks can be stored in any kind of databases (see for example the GDBM bindings [4]). A tricky things is figuring out how to multiplex all these different transports and storages... The ERIS specification is still considered "experimental". However we feel confident to stabilize it and intend to do so around February/March 2022 with a release 1.0.0 of the specification. This will ensure that the identifiers remain stable for the forseeable future (until the crypto breaks). Before that there is also a small external security audit of the specification planned (thanks to NGI0/NLnet!). This is just a little demo of the idea and some food for thought and discussion. Give it a try and let me know what you think! I've also pushed the patches to my personal Guix mirror if you want to check it out from there: https://inqlab.net/git/guix.git/log/?h=wip-eris Also CCing ~pukkamustard/eris@HIDDEN where there is some general ERIS related discussion. Thanks, -pukkamustard [1] http://purl.org/eris [2] https://issues.guix.gnu.org/33899 [3] https://www.ietf.org/rfc/rfc2169.txt [4] https://inqlab.net/git/guile-eris.git/tree/eris/blocks/gdbm.scm pukkamustard (3): publish: Add ERIS URN to narinfo WIP: gnu: guile-eris: Update to unreleased git version. publish: Add IPFS support. configure.ac | 5 ++ gnu/packages/guile-xyz.scm | 10 ++-- gnu/packages/package-management.scm | 1 + guix/narinfo.scm | 10 ++-- guix/scripts/publish.scm | 79 ++++++++++++++++++++++------- 5 files changed, 79 insertions(+), 26 deletions(-) -- 2.34.0
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: pukkamustard <pukkamustard@HIDDEN> Subject: bug#52555: Acknowledgement ([RFC PATCH 0/3] Decentralized substitute distribution with ERIS) Message-ID: <handler.52555.B.163967147614866.ack <at> debbugs.gnu.org> References: <20211216161724.547-1-pukkamustard@HIDDEN> X-Gnu-PR-Message: ack 52555 X-Gnu-PR-Package: guix-patches X-Gnu-PR-Keywords: patch Reply-To: 52555 <at> debbugs.gnu.org Date: Thu, 16 Dec 2021 16:18:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): guix-patches@HIDDEN If you wish to submit further information on this problem, please send it to 52555 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 52555: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D52555 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 1/3] publish: Add ERIS URN to narinfo Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Thu, 16 Dec 2021 16:21:02 +0000 Resent-Message-ID: <handler.52555.B52555.163967165715196 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.163967165715196 (code B ref 52555); Thu, 16 Dec 2021 16:21:02 +0000 Received: (at 52555) by debbugs.gnu.org; 16 Dec 2021 16:20:57 +0000 Received: from localhost ([127.0.0.1]:36502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1mxtV2-0003x1-HG for submit <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:20:56 -0500 Received: from mout01.posteo.de ([185.67.36.65]:37321) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1mxtUx-0003wb-RN for 52555 <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:20:54 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 43FC9240026 for <52555 <at> debbugs.gnu.org>; Thu, 16 Dec 2021 17:20:45 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1639671645; bh=c506nOniPbxfWxa5Datyw8oJED0OKeKWWVS7Qozcw10=; h=From:To:Cc:Subject:Date:From; b=GgY5+Cdof2C9QGoRmOaV/LbGZ9OSD/kJwQwxbMx1W50K/MnV3OyMltuBEOI60iAWg nR0Yn7SK+x0Z2b4Epyp3+0gYiVGKkR06lmq2UnI+KYz/gSSr3wwtUD7/f5MFdGxBt8 0BuI1Led8qyeypkADSsiR29ra02FPhd3oFu/TXE5/DiHS6iDjYUzmGpjINGoifaYmV bRtVXXWvp/4GUSzYGqVko3EgfwQnTPt4+tNpjlrhNCmgeoofH1OYeufxysrdBwsOOE zvNVgEOpGA2V9kjDQ+Lm9wSlZV5MdEgaBI0Mv92FojJy83vOrTqJsWZ9iWR+NUVIoB JpsinZnAntYtA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JFHR01ll4z9rxT; Thu, 16 Dec 2021 17:20:43 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Thu, 16 Dec 2021 16:20:34 +0000 Message-Id: <20211216162036.694-1-pukkamustard@HIDDEN> In-Reply-To: <20211216161724.547-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * guix/scripts/publish.scm: (bake-narinfo+nar): Compute ERIS URN of compressed nars. (narinfo-string): Add #:eris-urns parameter and honor it. (store-item->recutils): Add #:eris-urn parameter and honor it. * guix/scripts/narinfo.scm: (<narinfo>)[eris-urns]: New field. (narinfo-maker): Handle ERIS URN. * configure.ac: (HAVE_GUILE_ERIS): New conditional. * gnu/packages/package-management.scm: (guix)[native-inputs]: Add guile-eris. --- configure.ac | 5 +++++ gnu/packages/package-management.scm | 1 + guix/narinfo.scm | 10 ++++++---- guix/scripts/publish.scm | 31 +++++++++++++++++++++-------- 4 files changed, 35 insertions(+), 12 deletions(-) diff --git a/configure.ac b/configure.ac index 341cff8fbd..72396be8aa 100644 --- a/configure.ac +++ b/configure.ac @@ -170,6 +170,11 @@ GUILE_MODULE_AVAILABLE([have_guile_avahi], [(avahi)]) AM_CONDITIONAL([HAVE_GUILE_AVAHI], [test "x$have_guile_avahi" = "xyes"]) +dnl Check for Guile-eris. +GUILE_MODULE_AVAILABLE([have_guile_eris], [(eris)]) +AM_CONDITIONAL([HAVE_GUILE_ERIS], + [test "x$have_guile_eris" = "xyes"]) + dnl Guile-newt is used by the graphical installer. GUILE_MODULE_AVAILABLE([have_guile_newt], [(newt)]) diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm index 9496499850..5c49167782 100644 --- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -394,6 +394,7 @@ (define code ("guile-zstd" ,guile-zstd) ("guile-ssh" ,guile-ssh) ("guile-git" ,guile-git) + ("guile-eris" ,guile-eris) ;; XXX: Keep the development inputs here even though ;; they're unnecessary, just so that 'guix environment diff --git a/guix/narinfo.scm b/guix/narinfo.scm index 4fc550aa6c..0972ede3c1 100644 --- a/guix/narinfo.scm +++ b/guix/narinfo.scm @@ -67,13 +67,14 @@ (define-module (guix narinfo) equivalent-narinfo?)) (define-record-type <narinfo> - (%make-narinfo path uri-base uris compressions file-sizes file-hashes - nar-hash nar-size references deriver system + (%make-narinfo path uri-base uris eris-urns compressions file-sizes + file-hashes nar-hash nar-size references deriver system signature contents) narinfo? (path narinfo-path) (uri-base narinfo-uri-base) ;URI of the cache it originates from (uris narinfo-uris) ;list of strings + (eris-urns narinfo-eris-urns) ;list of (strings | #f) (compressions narinfo-compressions) ;list of strings (file-sizes narinfo-file-sizes) ;list of (integers | #f) (file-hashes narinfo-file-hashes) @@ -134,7 +135,7 @@ (define (narinfo-signature->canonical-sexp str) (define (narinfo-maker str cache-url) "Return a narinfo constructor for narinfos originating from CACHE-URL. STR must contain the original contents of a narinfo file." - (lambda (path urls compressions file-hashes file-sizes + (lambda (path urls eris-urns compressions file-hashes file-sizes nar-hash nar-size references deriver system signature) "Return a new <narinfo> object." @@ -148,6 +149,7 @@ (define len (length urls)) (string-append cache-url url) (string-append cache-url "/" url))))) urls) + eris-urns compressions (match file-sizes (() (make-list len #f)) @@ -186,7 +188,7 @@ (define* (read-narinfo port #:optional url "FileHash" "FileSize" "NarHash" "NarSize" "References" "Deriver" "System" "Signature") - '("URL" "Compression" "FileSize" "FileHash")))) + '("URL" "ERIS" "Compression" "FileSize" "FileHash")))) (define (narinfo-sha256 narinfo) "Return the sha256 hash of NARINFO as a bytevector, or #f if NARINFO lacks a diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 6e2b4368da..8e4b90789b 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -58,6 +58,7 @@ (define-module (guix scripts publish) #:use-module (guix workers) #:use-module (guix store) #:use-module ((guix serialization) #:select (write-file)) + #:use-module (eris) #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port make-lzip-output-port) @@ -308,7 +309,7 @@ (define* (store-item->recutils store-item #:key (nar-path "nar") (compression %no-compression) - file-size) + file-size eris-urn) "Return the 'Compression' and 'URL' fields of the narinfo for STORE-ITEM, with COMPRESSION, starting at NAR-PATH." (let ((url (encode-and-join-uri-path @@ -319,19 +320,22 @@ (define* (store-item->recutils store-item (($ <compression> type) (list (symbol->string type)))) ,(basename store-item))))) - (format #f "URL: ~a~%Compression: ~a~%~@[FileSize: ~a~%~]" - url (compression-type compression) file-size))) + (format #f "URL: ~a~%Compression: ~a~%~@[FileSize: ~a~%~]~@[ERIS: ~a~%~]" + url (compression-type compression) file-size eris-urn))) (define* (narinfo-string store store-path #:key (compressions (list %no-compression)) - (nar-path "nar") (file-sizes '())) + (nar-path "nar") (file-sizes '()) (eris-urns '())) "Generate a narinfo key/value string for STORE-PATH; an exception is raised if STORE-PATH is invalid. Produce a URL that corresponds to COMPRESSION. The narinfo is signed with KEY. NAR-PATH specifies the prefix for nar URLs. Optionally, FILE-SIZES is a list of compression/integer pairs, where the integer is size in bytes of the compressed NAR; it informs the client of how -much needs to be downloaded." +much needs to be downloaded. + +Optionally, ERIS-URNS is a list of compression/string pairs, where the +string is the ERIS URN of the compressed NAR." (let* ((path-info (query-path-info store store-path)) (compressions (actual-compressions store-path compressions)) (hash (bytevector->nix-base32-string @@ -352,9 +356,12 @@ (define* (narinfo-string store store-path store-path (map (lambda (compression) (let ((size (assoc-ref file-sizes - compression))) + compression)) + (eris-urn (assoc-ref eris-urns + compression))) (store-item->recutils store-path #:file-size size + #:eris-urn eris-urn #:nar-path nar-path #:compression compression))) @@ -632,6 +639,12 @@ (define (compressed-nar-size compression) (and stat (cons compression (stat:size stat))))) + (define (compressed-eris-urn compression) + (let* ((nar (nar-cache-file cache item #:compression compression)) + (stat (stat nar #f))) + (and stat + (cons compression (call-with-input-file nar eris-encode->urn))))) + (let ((compression (actual-compressions item compressions))) (for-each (cut compress-nar cache item <>) compressions) @@ -646,11 +659,13 @@ (define (compressed-nar-size compression) ;; thread's connection to the store since we would end up sending ;; stuff concurrently on the same channel. (with-store store - (let ((sizes (filter-map compressed-nar-size compression))) + (let ((sizes (filter-map compressed-nar-size compression)) + (eris-urns (filter-map compressed-eris-urn compression))) (display (narinfo-string store item #:nar-path nar-path #:compressions compressions - #:file-sizes sizes) + #:file-sizes sizes + #:eris-urns eris-urns) port))) ;; Make the cached narinfo world-readable, contrary to what -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 2/3] WIP: gnu: guile-eris: Update to unreleased git version. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Thu, 16 Dec 2021 16:21:02 +0000 Resent-Message-ID: <handler.52555.B52555.163967165715202 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.163967165715202 (code B ref 52555); Thu, 16 Dec 2021 16:21:02 +0000 Received: (at 52555) by debbugs.gnu.org; 16 Dec 2021 16:20:57 +0000 Received: from localhost ([127.0.0.1]:36504 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1mxtV3-0003x3-0g for submit <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:20:57 -0500 Received: from mout01.posteo.de ([185.67.36.65]:44809) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1mxtV0-0003wg-C9 for 52555 <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:20:55 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id C8A4A240027 for <52555 <at> debbugs.gnu.org>; Thu, 16 Dec 2021 17:20:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1639671648; bh=mzF6Dcz6gBsxGgNgXV2yX2To4wOq88FwLdvP0HAzkYo=; h=From:To:Cc:Subject:Date:From; b=n7IfKTN8HDGAo6xEuyFQh2PJ1Kld/Sj/lviOXqQSe/U0XMXzbw0JqTAzPeHhyH2W+ frlB6+zNnf2ug2LBhzBLm+wPFER2hxpmv2TO2DN+nSS6SoCHOTNFVWNu9VhFs5DY2J AMR650+puRcRgAGAQk59mNxHhwGW5FMrTpQoq549I2fZFmf7kIS+/fgapxwxKLIGu9 er1LA+mbm1F1JmBR/oz6HPjkZBFi8lMew85MZRatBy5ZMzqIa9N+W1Q0dnpbRiO8Qw KXAmAprHuQxwUlkG+3dJcIO9IZNW7mDpGjwhesz9C9uWq7x8K41KJwjdMN3vxddy/k 6YE9/vRp4Ep9Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JFHR400WSz9rxX; Thu, 16 Dec 2021 17:20:47 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Thu, 16 Dec 2021 16:20:35 +0000 Message-Id: <20211216162036.694-2-pukkamustard@HIDDEN> In-Reply-To: <20211216162036.694-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20211216162036.694-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * gnu/packages/guile-xyz.schm (guile-eris): Update to unreleased git version. [source]: Update source URI. [propagated-inputs]: Add guile-json-4 and guile-gdbm-ffi. --- gnu/packages/guile-xyz.scm | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm index 8346d99996..5e6f31a6e2 100644 --- a/gnu/packages/guile-xyz.scm +++ b/gnu/packages/guile-xyz.scm @@ -4325,15 +4325,15 @@ (define-public guile-sodium (define-public guile-eris (package (name "guile-eris") - (version "0.2.0") + (version "f1e4dd87988f9a80b05a8051d7f5ba3daf79dcc1") (source (origin (method git-fetch) (uri (git-reference - (url "https://inqlab.net/git/eris.git") - (commit (string-append "v" version)))) + (url "https://inqlab.net/git/guile-eris.git") + (commit version))) (file-name (git-file-name name version)) - (sha256 (base32 "1ijglmwkdy1l87gj429qfjis0v8b1zlxhbyfhx5za8664h68nqka")))) + (sha256 (base32 "0kgm4b4qn2s74wjvxy273gdi1l1m81i2k4kkk1zc6vlcg3np7p06")))) (build-system gnu-build-system) (arguments '()) (native-inputs @@ -4345,7 +4345,7 @@ (define-public guile-eris guile-srfi-180)) (inputs (list guile-3.0)) (propagated-inputs - (list guile-sodium)) + (list guile-sodium guile-json-4 guile-gdbm-ffi)) (synopsis "Guile implementation of the Encoding for Robust Immutable Storage (ERIS)") (description "Guile-ERIS is the reference implementation of the Encoding for Robust -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 3/3] publish: Add IPFS support. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Thu, 16 Dec 2021 16:21:02 +0000 Resent-Message-ID: <handler.52555.B52555.163967166115218 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.163967166115218 (code B ref 52555); Thu, 16 Dec 2021 16:21:02 +0000 Received: (at 52555) by debbugs.gnu.org; 16 Dec 2021 16:21:01 +0000 Received: from localhost ([127.0.0.1]:36508 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1mxtV6-0003xO-Dp for submit <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:21:00 -0500 Received: from mout01.posteo.de ([185.67.36.65]:34165) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1mxtV2-0003wj-VK for 52555 <at> debbugs.gnu.org; Thu, 16 Dec 2021 11:20:57 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 66D91240026 for <52555 <at> debbugs.gnu.org>; Thu, 16 Dec 2021 17:20:51 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1639671651; bh=8FUfzH+H19v7PInjtcmZ9loRZ+gv+GSpvf6oOoA0eR4=; h=From:To:Cc:Subject:Date:From; b=q3677HhT4ire8v1On3rXXk2hHSHVeTfXOJJMYNfGarL65V+/ZMjqP0fAYBPzROczE eas83QDMVCfP6Cru45FGGM4PktcKU+vVuCpRh3MmCqINvg7z46nVn8nJVxhE3r3HrW 2KZZ2vrzt8gZYeVgqymH+gIqvJnG31N8yTe2L8JhgEe4TBxqd2gM4hjE9Re42GZpxx TTVrU15muau8PE+/Z3KNb3VoOrRor3jJvJ0j8aJRHpycDpvEaoOkmAy9S9E9Ho1LDL BWQY3CgIy8F7qnsaDG7ODIY1al8FAVlboSijotsprngtlipcaQeOfTo4blayBSJMgu 1nMOmikrWOjTA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JFHR64dFHz9rxS; Thu, 16 Dec 2021 17:20:50 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Thu, 16 Dec 2021 16:20:36 +0000 Message-Id: <20211216162036.694-3-pukkamustard@HIDDEN> In-Reply-To: <20211216162036.694-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20211216162036.694-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * guix/scripts/publish.scm: (show-help, %options): Add '--ipfs'. (render-narinfo/cached, bake-narinfo+nar, make-request-handler, run-publish-server): Add #:ipfs? and honor it. (guix-publish): Honor '--ipfs' and parameterize %ipfs-base-url. --- guix/scripts/publish.scm | 52 +++++++++++++++++++++++++++++++--------- 1 file changed, 41 insertions(+), 11 deletions(-) diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 8e4b90789b..8e7fb47b9e 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -41,6 +41,8 @@ (define-module (guix scripts publish) #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module (srfi srfi-37) + #:use-module (srfi srfi-71) + #:use-module (srfi srfi-171) #:use-module (web http) #:use-module (web request) #:use-module (web response) @@ -52,6 +54,7 @@ (define-module (guix scripts publish) #:use-module (guix base64) #:use-module (guix config) #:use-module (guix derivations) + #:use-module ((guix ipfs) #:prefix ipfs:) #:use-module (gcrypt hash) #:use-module (guix pki) #:use-module (gcrypt pk-crypto) @@ -59,6 +62,8 @@ (define-module (guix scripts publish) #:use-module (guix store) #:use-module ((guix serialization) #:select (write-file)) #:use-module (eris) + #:use-module (eris read-capability) + #:use-module (eris blocks ipfs) #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port make-lzip-output-port) @@ -83,6 +88,7 @@ (define-module (guix scripts publish) run-publish-server guix-publish)) + (define (show-help) (format #t (G_ "Usage: guix publish [OPTION]... Publish ~a over HTTP.\n") %store-directory) @@ -102,6 +108,8 @@ (define (show-help) (display (G_ " --cache-bypass-threshold=SIZE serve store items below SIZE even when not cached")) + (display (G_ " + --ipfs[=GATEWAY] publish items over IPFS via GATEWAY")) (display (G_ " --workers=N use N workers to bake items")) (display (G_ " @@ -220,6 +228,10 @@ (define %options (lambda (opt name arg result) (alist-cons 'cache-bypass-threshold (size->number arg) result))) + (option '("ipfs") #f #t + (lambda (opt name arg result) + (alist-cons 'ipfs (or arg (ipfs:%ipfs-base-url)) + result))) (option '("workers") #t #f (lambda (opt name arg result) (alist-cons 'workers (string->number* arg) @@ -526,7 +538,7 @@ (define (bypass-cache? store item) (define* (render-narinfo/cached store request hash #:key ttl (compressions (list %no-compression)) (nar-path "nar") negative-ttl - cache pool) + cache pool ipfs?) "Respond to the narinfo request for REQUEST. If the narinfo is available in CACHE, then send it; otherwise, return 404 and \"bake\" that nar and narinfo requested using POOL." @@ -571,7 +583,8 @@ (define (delete-entry narinfo) (bake-narinfo+nar cache item #:ttl ttl #:compressions compressions - #:nar-path nar-path))) + #:nar-path nar-path + #:ipfs? ipfs?))) (when ttl (single-baker 'cache-cleanup @@ -631,7 +644,7 @@ (define (write-compressed-file call-with-compressed-output-port) (define* (bake-narinfo+nar cache item #:key ttl (compressions (list %no-compression)) - (nar-path "/nar")) + (nar-path "/nar") ipfs?) "Write the narinfo and nar for ITEM to CACHE." (define (compressed-nar-size compression) (let* ((nar (nar-cache-file cache item #:compression compression)) @@ -641,9 +654,19 @@ (define (compressed-nar-size compression) (define (compressed-eris-urn compression) (let* ((nar (nar-cache-file cache item #:compression compression)) - (stat (stat nar #f))) + (stat (stat nar #f)) + (block-reducer (if ipfs? + (eris-blocks-ipfs-reducer + #:ipfs-base-url (ipfs:%ipfs-base-url)) + rcount))) (and stat - (cons compression (call-with-input-file nar eris-encode->urn))))) + (cons compression + (call-with-input-file nar + (lambda (port) + (let ((read-cap _ + (eris-encode port #:block-reducer + block-reducer))) + (read-capability->string read-cap)))))))) (let ((compression (actual-compressions item compressions))) @@ -1115,7 +1138,8 @@ (define* (make-request-handler store cache pool narinfo-ttl narinfo-negative-ttl (nar-path "nar") - (compressions (list %no-compression))) + (compressions (list %no-compression)) + ipfs?) (define compression-type? string->compression-type) @@ -1147,7 +1171,8 @@ (define (handle request body) #:ttl narinfo-ttl #:negative-ttl narinfo-negative-ttl #:nar-path nar-path - #:compressions compressions) + #:compressions compressions + #:ipfs? ipfs?) (render-narinfo store request hash #:ttl narinfo-ttl #:negative-ttl narinfo-negative-ttl @@ -1218,7 +1243,7 @@ (define* (run-publish-server socket store advertise? port (compressions (list %no-compression)) (nar-path "nar") narinfo-ttl narinfo-negative-ttl - cache pool) + cache pool ipfs?) (when advertise? (let ((name (service-name))) ;; XXX: Use a callback from Guile-Avahi here, as Avahi can pick a @@ -1234,7 +1259,8 @@ (define* (run-publish-server socket store #:nar-path nar-path #:narinfo-ttl narinfo-ttl #:narinfo-negative-ttl narinfo-negative-ttl - #:compressions compressions) + #:compressions compressions + #:ipfs? ipfs?) concurrent-http-server `(#:socket ,socket))) @@ -1296,6 +1322,8 @@ (define-command (guix-publish . args) (repl-port (assoc-ref opts 'repl)) (cache (assoc-ref opts 'cache)) (workers (assoc-ref opts 'workers)) + (ipfs (assoc-ref opts 'ipfs)) + (ipfs? (if ipfs #t #f)) ;; Read the key right away so that (1) we fail early on if we can't ;; access them, and (2) we can then drop privileges. @@ -1315,7 +1343,8 @@ (define-command (guix-publish . args) (%private-key private-key) (cache-bypass-threshold (or (assoc-ref opts 'cache-bypass-threshold) - (cache-bypass-threshold)))) + (cache-bypass-threshold))) + (ipfs:%ipfs-base-url ipfs)) (info (G_ "publishing ~a on ~a, port ~d~%") %store-directory (inet-ntop (sockaddr:fam address) (sockaddr:addr address)) @@ -1344,7 +1373,8 @@ (define-command (guix-publish . args) #:nar-path nar-path #:compressions compressions #:narinfo-negative-ttl negative-ttl - #:narinfo-ttl ttl)))))) + #:narinfo-ttl ttl + #:ipfs? ipfs?)))))) ;;; Local Variables: ;;; eval: (put 'single-baker 'scheme-indent-function 1) -- 2.34.0
Received: (at control) by debbugs.gnu.org; 18 Dec 2021 20:01:19 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Dec 18 15:01:19 2021 Received: from localhost ([127.0.0.1]:43913 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1myftP-00017M-9z for submit <at> debbugs.gnu.org; Sat, 18 Dec 2021 15:01:19 -0500 Received: from eggs.gnu.org ([209.51.188.92]:40858) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1myftN-00017A-46 for control <at> debbugs.gnu.org; Sat, 18 Dec 2021 15:01:17 -0500 Received: from [2001:470:142:3::e] (port=37648 helo=fencepost.gnu.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1myftH-0008Pl-QH for control <at> debbugs.gnu.org; Sat, 18 Dec 2021 15:01:11 -0500 DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org; s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to: references; bh=tCjw+f0dws25Ri+ZYMz2lTygricKxBDLbCgI8B3C8V0=; b=eZV2MZ+DzwTdWb TsY8uqnePQ3k5kAtam/rFTf2UM+nKk096hnF/f9FJoTAapmzGOKEK/1A2lHYsfElZIT/Eu6jIfE5Z DzK9+xkUVklRSp7CS2n2kVT1esxaQNvZKrcVlBlYgNukZvFNBZlQztcGrruIXdqtKhKzNzyOOqyEH E+N0QKV/WuBnMaOuAuJQOht4A25qErZaVNTFzu94U478W5grfsPwmtA7RsR/yQdT8Ve2L8LeN5N+p zzRcNS25qYEr8fRUWFHkgz3ju6bloM1vCyYuvVpZWr42GPIQfX4l05PBwZFXdMPDw9r0kE/KqjG9x 0iyv6CHYVecWGQjVq1Mg==; Received: from 91-160-117-201.subs.proxad.net ([91.160.117.201]:59382 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1myftH-0007jR-Q1 for control <at> debbugs.gnu.org; Sat, 18 Dec 2021 15:01:12 -0500 Date: Sat, 18 Dec 2021 21:01:10 +0100 Message-Id: <87czltof7d.fsf@HIDDEN> To: control <at> debbugs.gnu.org From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> Subject: control message for bug #52555 MIME-version: 1.0 Content-type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) severity 52555 important quit
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 0/3] Decentralized substitute distribution with ERIS Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Mon, 20 Dec 2021 16:26:02 +0000 Resent-Message-ID: <handler.52555.B52555.164001751919703 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164001751919703 (code B ref 52555); Mon, 20 Dec 2021 16:26:02 +0000 Received: (at 52555) by debbugs.gnu.org; 20 Dec 2021 16:25:19 +0000 Received: from localhost ([127.0.0.1]:51230 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1mzLTT-00057j-1o for submit <at> debbugs.gnu.org; Mon, 20 Dec 2021 11:25:19 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:43422) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1mzLTR-00057Q-De for 52555 <at> debbugs.gnu.org; Mon, 20 Dec 2021 11:25:18 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 927AC361; Mon, 20 Dec 2021 17:25:10 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 1Wfl5kQPKPVV; Mon, 20 Dec 2021 17:25:09 +0100 (CET) Received: from ribbon (unknown [193.50.110.211]) by hera.aquilenet.fr (Postfix) with ESMTPSA id D191B63; Mon, 20 Dec 2021 17:25:08 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> Date: Mon, 20 Dec 2021 17:25:07 +0100 In-Reply-To: <20211216161724.547-1-pukkamustard@HIDDEN> (pukkamustard@HIDDEN's message of "Thu, 16 Dec 2021 16:17:24 +0000") Message-ID: <87h7b3gs64.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / Authentication-Results: hera.aquilenet.fr; none X-Rspamd-Server: hera X-Rspamd-Queue-Id: 927AC361 X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.0 (/) Hi pukkamustard, pukkamustard <pukkamustard@HIDDEN> skribis: > This is an initial patch and proposal towards decentralizing substitute > distribution with ERIS. Woohoo, sounds exciting! > ERIS (Encoding for Robust Immutable Storage) [1] is an encoding of conten= t into > uniformly sized, encryped and content-addressed blocks. The original cont= ent > can be reconstructed only with access to a read capability, which can be > encoded as an URN. > > One key advantage of ERIS is that the encoding is protocol agnostic. Any > protocol that can transfer small (32KiB) sized blocks referenced by the h= ash of > their content will do. This can be done with things such as GNUNet, IPFS, > OpenDHT, HTTP or a USB stick on a bicycle. Yes, that=E2=80=99s nice. > The following patch allows substitutes to be published over IPFS using ER= IS. > This is inspired and very similar to previous work on distributing substi= tutes > over IPFS [2]. > > The narinfos served by `guix publish` look like this: > > StorePath: /gnu/store/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 > URL: nar/gzip/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 > Compression: gzip > FileSize: 67363 > ERIS: urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDV= DAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE > URL: nar/zstd/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 > Compression: zstd > FileSize: 64917 > ERIS: urn:erisx2:BIBO7KS7SAWHDNC43DVILOSQ3F3SRRHEV6YPLDCSZ7MMD6LZVCHQMEQ6= FUBTJAPSNFF7XR5XPTP4OQ72OPABNEO7UYBUN42O46ARKHBTGM Do we really need one URN per compression method? Couldn=E2=80=99t we leave compression (of individual chunks, possibly) as a =E2=80=9Cdetail=E2=80=9D = handled by the encoding or the transport layer? > If the `--ipfs` is used for `guix publish` then the encoded blocks are al= so > uploaded to the IPFS daemon. The nar could then be retrieved from anywher= e like > this: > > (use-modules (eris) > (eris blocks ipfs)) > > (eris-decode->bytevector > "urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDVDAGJ= UEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE" > eris-blocks-ipfs-ref) > > These patches do not yet retrieve content from IPFS (TODO). But in princi= ple, > anybody connected to IPFS can get the nar with the ERIS URN. This could b= e used > to reduce load on substitute server as they would only need to publish th= e ERIS > URN directly - substitutes could be delivered much more peer-to-peer. Nice. So adjusting =E2=80=98guix substitute=E2=80=99 should be relatively = easy? > Other transports that I have been looking in to and am pretty sure will w= ork > include: HTTP (with RFC 2169 [3]), GNUNet, OpenDHT. This is, imho, the > advantage of ERIS over IPFS directly or GNUNet directly. The encoding and > identifiers (URN) are abstracted away from specific transports (and also > applications). ERIS is almost exactly the same encoding as used in GNUNet > (ECRS). As a first step, =E2=80=98guix publish=E2=80=99 could implement RFC 2169, t= oo. I gather implementing the HTTP and IPFS backends in =E2=80=98guix substitut= e=E2=80=99 should be relatively easy, right? > Blocks can be stored in any kind of databases (see for example the GDBM > bindings [4]). > > A tricky things is figuring out how to multiplex all these different > transports and storages... Yes. We don=E2=80=99t know yet what performance and data availability will= be like on IPFS, for instance, so it=E2=80=99s important for users to be able = to set priorities. It=E2=80=99s also important to gracefully fall back to dir= ect HTTP downloads when fancier p2p methods fail, regardless of how they fail. > The ERIS specification is still considered "experimental". However we feel > confident to stabilize it and intend to do so around February/March 2022 = with a > release 1.0.0 of the specification. This will ensure that the identifiers > remain stable for the forseeable future (until the crypto breaks). Before= that > there is also a small external security audit of the specification planned > (thanks to NGI0/NLnet!). Neat. This is all very exciting. I look forward to playing around with it! Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 0/3] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Thu, 23 Dec 2021 12:19:02 +0000 Resent-Message-ID: <handler.52555.B52555.164026190912428 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164026190912428 (code B ref 52555); Thu, 23 Dec 2021 12:19:02 +0000 Received: (at 52555) by debbugs.gnu.org; 23 Dec 2021 12:18:29 +0000 Received: from localhost ([127.0.0.1]:60570 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1n0N3A-0003EI-Ic for submit <at> debbugs.gnu.org; Thu, 23 Dec 2021 07:18:29 -0500 Received: from mout02.posteo.de ([185.67.36.66]:52835) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1n0N38-0003E4-FE for 52555 <at> debbugs.gnu.org; Thu, 23 Dec 2021 07:18:23 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 2F987240105 for <52555 <at> debbugs.gnu.org>; Thu, 23 Dec 2021 13:18:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1640261896; bh=JgqGKyEHLIdnmE6TeYENeP0wVNmtmDc4GKpcUJhHrYU=; h=From:To:Cc:Subject:Date:From; b=iW3bMzBy/BjgyPuHsO5+9LyOGNWnUQFwMfA/pKQ2N7sT2vDJlUho5P5oGdQg9U7cR 2Ezr5kuKqzBVCIoJlu30LWxEMyyQ1OfjizzCEz3NNk5WlTjq7HkqxajPzcaeQJwH+w 9XY7aXwKtfPqkEhYPq776u+6mnysrSdWphxAhC/ghdRNQgz9kT6a0a0bkFaRI1zLDi GiFDbSjV94XgkH54gVWFvomQOn+vccx+fur2ZEnOxTChnW6YYI19Y1k4svuCYlbOLW LPD1Is0csnx0JJkiZ8WexZlR2IFiMxvGLMwVVbs6T73h4WHmM72I8NHM8jXg4aWpOk +lQbMIAcjbkcA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JKTjy5gg2z6tn0; Thu, 23 Dec 2021 13:18:14 +0100 (CET) References: <20211216161724.547-1-pukkamustard@HIDDEN> <87h7b3gs64.fsf@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Thu, 23 Dec 2021 11:42:46 +0000 In-reply-to: <87h7b3gs64.fsf@HIDDEN> Message-ID: <86bl17ms56.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hi Ludo, Thanks for your comments! Ludovic Court=C3=A8s <ludo@HIDDEN> writes: >> StorePath: /gnu/store/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 >> URL: nar/gzip/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 >> Compression: gzip >> FileSize: 67363 >> ERIS: urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDD= VDAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE >> URL: nar/zstd/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 >> Compression: zstd >> FileSize: 64917 >> ERIS: urn:erisx2:BIBO7KS7SAWHDNC43DVILOSQ3F3SRRHEV6YPLDCSZ7MMD6LZVCHQMEQ= 6FUBTJAPSNFF7XR5XPTP4OQ72OPABNEO7UYBUN42O46ARKHBTGM > > Do we really need one URN per compression method? Couldn=E2=80=99t we le= ave > compression (of individual chunks, possibly) as a =E2=80=9Cdetail=E2=80= =9D handled by > the encoding or the transport layer? > I agree that it would be nice to leave this to the encoding layer as that would allow certain optimizations (e.g. de-duplication). Unfortunately, we haven't figured out yet what the most suitable compression/format would be. Something like EROSFS seems good (as it aligns data to fixed block sizes) [1]. But this seems a bit "clunky" for just an archive format and there do not seem to be any libraries that we could use to neatly integrate. It seems possible to block-align a Tar archive, but that seems a bit hackey [2]. Other things to look into might be Tarlz [3] and ZPAQ [4]. To get started I suggest just using one of the compressions/formats already in Guix. zstd seems to be a reasonable choice (for the same reasons why it makes sense to use zstd with `--discover` [5]). Does that sound like a plan? [1] https://inqlab.net/git/guile-eris.git/tree/examples/dedup-fs/Readme.org [2] https://unix.stackexchange.com/questions/276908/make-tar-or-other-archi= ve-with-data-block-aligned-like-in-original-files-for/279384#279384 [3] http://lzip.nongnu.org/tarlz.html [4] http://mattmahoney.net/dc/zpaq.html [5] https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/ >> If the `--ipfs` is used for `guix publish` then the encoded blocks are a= lso >> uploaded to the IPFS daemon. The nar could then be retrieved from anywhe= re like >> this: >> >> (use-modules (eris) >> (eris blocks ipfs)) >> >> (eris-decode->bytevector >> "urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUDDVDAG= JUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE" >> eris-blocks-ipfs-ref) >> >> These patches do not yet retrieve content from IPFS (TODO). But in princ= iple, >> anybody connected to IPFS can get the nar with the ERIS URN. This could = be used >> to reduce load on substitute server as they would only need to publish t= he ERIS >> URN directly - substitutes could be delivered much more peer-to-peer. > > Nice. So adjusting =E2=80=98guix substitute=E2=80=99 should be relativel= y easy? Yes, relatively! :) I meant to send in a V2 that does this before going on holidays, but I'm afraid I won't make it. V2 will come in early January! >> Other transports that I have been looking in to and am pretty sure will = work >> include: HTTP (with RFC 2169 [3]), GNUNet, OpenDHT. This is, imho, the >> advantage of ERIS over IPFS directly or GNUNet directly. The encoding and >> identifiers (URN) are abstracted away from specific transports (and also >> applications). ERIS is almost exactly the same encoding as used in GNUNet >> (ECRS). > > As a first step, =E2=80=98guix publish=E2=80=99 could implement RFC 2169,= too. > > I gather implementing the HTTP and IPFS backends in =E2=80=98guix substit= ute=E2=80=99 > should be relatively easy, right? Yes, those seem to be the two easiest backends to implement. >> A tricky things is figuring out how to multiplex all these different >> transports and storages... > > Yes. We don=E2=80=99t know yet what performance and data availability wi= ll be > like on IPFS, for instance, so it=E2=80=99s important for users to be abl= e to > set priorities. It=E2=80=99s also important to gracefully fall back to d= irect > HTTP downloads when fancier p2p methods fail, regardless of how they > fail. Agree. Thanks, -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH 0/3] Decentralized substitute distribution with ERIS Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Fri, 24 Dec 2021 14:49:01 +0000 Resent-Message-ID: <handler.52555.B52555.164035732520141 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164035732520141 (code B ref 52555); Fri, 24 Dec 2021 14:49:01 +0000 Received: (at 52555) by debbugs.gnu.org; 24 Dec 2021 14:48:45 +0000 Received: from localhost ([127.0.0.1]:35945 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1n0lsD-0005En-H1 for submit <at> debbugs.gnu.org; Fri, 24 Dec 2021 09:48:45 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:57460) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1n0lsA-0005EQ-Mr for 52555 <at> debbugs.gnu.org; Fri, 24 Dec 2021 09:48:43 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 1392B1B2; Fri, 24 Dec 2021 15:48:36 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id qtDbPZFk31Mm; Fri, 24 Dec 2021 15:48:35 +0100 (CET) Received: from ribbon (unknown [78.199.208.147]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 424BABE; Fri, 24 Dec 2021 15:48:34 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <87h7b3gs64.fsf@HIDDEN> <86bl17ms56.fsf@HIDDEN> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 =?UTF-8?Q?Niv=C3=B4se?= an 230 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Fri, 24 Dec 2021 15:48:32 +0100 In-Reply-To: <86bl17ms56.fsf@HIDDEN> (pukkamustard@HIDDEN's message of "Thu, 23 Dec 2021 11:42:46 +0000") Message-ID: <87ilveukhr.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spamd-Bar: / Authentication-Results: hera.aquilenet.fr; none X-Rspamd-Server: hera X-Rspamd-Queue-Id: 1392B1B2 X-Spamd-Result: default: False [-0.10 / 15.00]; ARC_NA(0.00)[]; RCVD_VIA_SMTP_AUTH(0.00)[]; FROM_HAS_DN(0.00)[]; RCPT_COUNT_THREE(0.00)[3]; TO_DN_SOME(0.00)[]; TO_MATCH_ENVRCPT_ALL(0.00)[]; MIME_GOOD(-0.10)[text/plain]; FROM_EQ_ENVFROM(0.00)[]; MIME_TRACE(0.00)[0:+]; RCVD_COUNT_TWO(0.00)[2]; RCVD_TLS_ALL(0.00)[]; MID_RHS_MATCH_FROM(0.00)[] X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.0 (/) Hi! pukkamustard <pukkamustard@HIDDEN> skribis: > Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > >>> StorePath: /gnu/store/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 >>> URL: nar/gzip/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 >>> Compression: gzip >>> FileSize: 67363 >>> ERIS: urn:erisx2:BIBC2LUTIQH43S2KRIAV7TBXNUUVPZTMV6KFA2M7AL5V6FNE77VNUD= DVDAGJUEEAFATVO2QQT67SMOPTO3LGWCJFU7BZVCF5VXEQQW25BE >>> URL: nar/zstd/81bdcd5x4v50i28h98bfkvvkx9cky63w-hello-2.10 >>> Compression: zstd >>> FileSize: 64917 >>> ERIS: urn:erisx2:BIBO7KS7SAWHDNC43DVILOSQ3F3SRRHEV6YPLDCSZ7MMD6LZVCHQME= Q6FUBTJAPSNFF7XR5XPTP4OQ72OPABNEO7UYBUN42O46ARKHBTGM >> >> Do we really need one URN per compression method? Couldn=E2=80=99t we l= eave >> compression (of individual chunks, possibly) as a =E2=80=9Cdetail=E2=80= =9D handled by >> the encoding or the transport layer? >> > > I agree that it would be nice to leave this to the encoding layer as > that would allow certain optimizations (e.g. de-duplication). > > Unfortunately, we haven't figured out yet what the most suitable > compression/format would be. Something like EROSFS seems good (as it > aligns data to fixed block sizes) [1]. But this seems a bit "clunky" for > just an archive format and there do not seem to be any libraries that we > could use to neatly integrate. It seems possible to block-align a Tar > archive, but that seems a bit hackey [2]. Other things to look into > might be Tarlz [3] and ZPAQ [4]. Yeah. Though it may be that deduplication at the block level doesn=E2=80= =99t buy us much. That was the conclusion I reached a long time ago[a], and also seems to be supported by the recent guix-daemon deduplication improvements[b]. [a] https://hal.inria.fr/hal-00187069/en [b] https://issues.guix.gnu.org/24937#20-lineno0 > To get started I suggest just using one of the compressions/formats > already in Guix. zstd seems to be a reasonable choice (for the same > reasons why it makes sense to use zstd with `--discover` [5]). > > Does that sound like a plan? Sure! > I meant to send in a V2 that does this before going on holidays, but I'm > afraid I won't make it. V2 will come in early January! Alright, we=E2=80=99ll see! :-) Until then, enjoy your holidays! Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS References: <20211216161724.547-1-pukkamustard@HIDDEN> In-Reply-To: <20211216161724.547-1-pukkamustard@HIDDEN> Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Tue, 25 Jan 2022 19:23:02 +0000 Resent-Message-ID: <handler.52555.B52555.16431385464633 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16431385464633 (code B ref 52555); Tue, 25 Jan 2022 19:23:02 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:26 +0000 Received: from localhost ([127.0.0.1]:50138 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nCROa-0001CQ-9r for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:26 -0500 Received: from mout02.posteo.de ([185.67.36.66]:55307) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nCROU-0001C0-RN for 52555 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:23 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id B595C240103 for <52555 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:12 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138532; bh=d5WCY24/0Il2IHG4p/6YJQnYqcD4WOhnAqlw7escmoI=; h=From:To:Cc:Subject:Date:From; b=cFaGdg3TV4AcmYezCcN3Sno/LguMEEP14HRAeQQvdbzmzbAEYHeHPor+jpVvhVing 4vo3ejKuSip3vCpYAZmDdC6cGGxQM1V9fuElLSW73+S1zPNoZboPyBPjtWrWuHL+45 JYTjqKJGHUX4TUFH4eJQDqkJm6RzntlHGpCd6y8DspHtLKw5VeJY3l2izWAmwg1IjD y3QARnXu97N+RhBBwkwXtkZzOHV+P4TtdTh2ZnYzFG5eX167Ish7PS8bVZlZa8OwBm Flh9yMADL+aezW4bmqwB4vvA9BfI/3UpA0oGhc/72+Kglcd1SNdVOg0Vya0i/Oc25h wLyl3v2vXo14Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxYv1gq0z9rxM; Tue, 25 Jan 2022 20:22:10 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Tue, 25 Jan 2022 19:21:56 +0000 Message-Id: <20220125192201.7582-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hello Guix, Here comes the V2 of a proposal towards decentralizing substitute distribution with ERIS. A quick summary (as this has become quite long): - This adds support for publishing and getting substitutes over IPFS. - By using the ERIS encoding we are not limited to using IPFS as transport. We can also use GNUNet, Named Data Networking (possibly) or just plain old HTTP. Support for these can be added in (guix eris). - These patches are still very rough and we need better logic for when to use IPFS et. al. and when to fallback to HTTP. - There might be performance issues when using IPFS via the IPFS daemon HTTP API. I found the setup for testing this a bit tricky. I will try and describe how I have been testing it. Please let me know how this can be improved! ** Authorize local substitutes We will be running a local substitute server so we need to add the local signing key to the list of authorized keys. In the system configurations: #+BEGIN_SRC scheme (modify-services %base-services (guix-service-type config => (guix-configuration (inherit config) (authorized-keys (cons* ;; allow substitutes from ourselves for testing purposes (local-file "/etc/signing-key.pub") %default-authorized-guix-keys))))) #+END_SRC ** Configure the local Guix checkout #+BEGIN_SRC shell ./bootstrap && ./configure --localstatedir=/var --sysconfdir=/etc #+END_SRC The ~--sysconfdir~ is required so that guix will use the ACL in ~/etc/guix/acl~. ** Start the IPFS daemon #+BEGIN_SRC shell guix shell go-ipfs -- ipfs daemon #+END_SRC Start a local substitute server: #+BEGIN_SRC shell sudo -E ./pre-inst-env guix publish --public-key=/etc/guix/signing-key.pub --private-key=/etc/guix/signing-key.sec --cache=/tmp/guix-publish-cache/ --port=8081 --compression=zstd:19 #+END_SRC We use port 8081 as IPFS is running on 8080. We use the temporary cache directory ~/tmp/guix-publish-cache~. ** Build some package locally First we build some package: #+BEGIN_SRC shell ./pre-inst-env guix build hello --no-substitutes --no-offload #+END_SRC #+RESULTS: : /gnu/store/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 ** Trigger the substitute server to "bake" a susbtitute #+BEGIN_SRC shell curl http://localhost:8081/khaaib6s836bk5kbik239hlk6n6ianc4.narinfo #+END_SRC --8<---------------cut here---------------start------------->8--- StorePath: /gnu/store/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 URL: nar/zstd/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 Compression: zstd NarHash: sha256:11pk3jsh4zk0gigyjk881ay1nnvjfgpd3xpb4rmbaljhbiis4jbm NarSize: 190480 References: 094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 Deriver: mc7i1cdi42gy89mxl48nhdhgrfa9lpq6-hello-2.11.drv Signature: 1;strawberry;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjOTE0QTVGNTE4NUZGRUIzMzc4QTEwMzgzQzdFMEU1NDI1MEUyREZDRjk1RDUwOTNCMzU4QTFBNDE4OUFBRDVGNCMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDkxMDA2NDlCMkMyMzhEQzE2ODhFQTgyQTdCOEJFMTc5MTVBMjVDQjc1NzcwQjlGRkNGOTFDRTg2MDgyNzAwQiMpCiAgIChzICMwMUFBQ0VERjY0N0VENTQyRTIwNENDMEM1M0VDMEY0QjQ4QzdEOTAyRkFEQTkxREI4NzRGQjE2MTQ4QTIzNUI2IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjMDRDMkY4ODk1QTU0NDNGNTlCODk2NDEwMEI1MDY0NzU4RjQ1N0YzMENEREE1MTQyQzE0MDc0NjExNTA1NTc5MCMpCiAgICkKICApCiApCg== --8<---------------cut here---------------end--------------->8--- If you do this again after a few seconds you will get a different response that has the ERIS URN and the FileSizes. The reason for this is that Guix publish bakes the nars asyncrhonisly in the background: #+BEGIN_SRC shell curl http://localhost:8081/khaaib6s836bk5kbik239hlk6n6ianc4.narinfo #+END_SRC #+RESULTS: --8<---------------cut here---------------start------------->8--- StorePath: /gnu/store/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 URL: nar/zstd/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 Compression: zstd FileSize: 57691 NarHash: sha256:11pk3jsh4zk0gigyjk881ay1nnvjfgpd3xpb4rmbaljhbiis4jbm NarSize: 190480 ERISFormat: application/x-nix-archive+zstd-19 ERIS: urn:erisx2:B4AYPTXLTACB6WJYJ74RKBCVU3RBLHA4PY6HATUWRZNJ6THVSDUFM34K2ASUF3B6EOYEEBRZ5XEUR4PAAAIED7G7YSEZVZ5V7WWZ2PSC7Q References: 094bbaq6glba86h1d4cj16xhdi6fk2jl-gcc-10.3.0-lib 5h2w4qi9hk1qzzgi1w83220ydslinr4s-glibc-2.33 khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 Deriver: mc7i1cdi42gy89mxl48nhdhgrfa9lpq6-hello-2.11.drv Signature: 1;strawberry;KHNpZ25hdHVyZSAKIChkYXRhIAogIChmbGFncyByZmM2OTc5KQogIChoYXNoIHNoYTI1NiAjOTE0QTVGNTE4NUZGRUIzMzc4QTEwMzgzQzdFMEU1NDI1MEUyREZDRjk1RDUwOTNCMzU4QTFBNDE4OUFBRDVGNCMpCiAgKQogKHNpZy12YWwgCiAgKGVjZHNhIAogICAociAjMDkxMDA2NDlCMkMyMzhEQzE2ODhFQTgyQTdCOEJFMTc5MTVBMjVDQjc1NzcwQjlGRkNGOTFDRTg2MDgyNzAwQiMpCiAgIChzICMwMUFBQ0VERjY0N0VENTQyRTIwNENDMEM1M0VDMEY0QjQ4QzdEOTAyRkFEQTkxREI4NzRGQjE2MTQ4QTIzNUI2IykKICAgKQogICkKIChwdWJsaWMta2V5IAogIChlY2MgCiAgIChjdXJ2ZSBFZDI1NTE5KQogICAocSAjMDRDMkY4ODk1QTU0NDNGNTlCODk2NDEwMEI1MDY0NzU4RjQ1N0YzMENEREE1MTQyQzE0MDc0NjExNTA1NTc5MCMpCiAgICkKICApCiApCg== --8<---------------cut here---------------end--------------->8--- These patches have added the ERIS and ERISFormat fields. Eventually we would have figured out what the best format for use over ERIS is, for now we encode it in the ERISFormat field. ** Removing a package from the store This is necessary in order to make guix look for a substitute. #+BEGIN_SRC shell ./pre-inst-env guix gc -D /gnu/store/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 #+END_SRC ** Start the Guix daemon from the repository #+BEGIN_SRC shell sudo -E ./pre-inst-env guix-daemon --build-users-group=guixbuild --debug --substitute-urls=http://localhost:8081/ #+END_SRC Note this will probably stop your system Guix daemon. Run ~sudo herd restart guix-daemon~ to restart it. #+BEGIN_SRC shell ./pre-inst-env guix build hello #+END_SRC --8<---------------cut here---------------start------------->8--- substituting /gnu/store/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11... downloading from urn:erisx2:B4AYPTXLTACB6WJYJ74RKBCVU3RBLHA4PY6HATUWRZNJ6THVSDUFM34K2ASUF3B6EOYEEBRZ5XEUR4PAAAIED7G7YSEZVZ5V7WWZ2PSC7Q ... urn:erisx2:B4AYPTXLTACB6WJYJ74RKBCVU3RBLHA4PY6HATUWRZNJ6THVSDUFM34K2ASUF3B6EOYEEBRZ5XEUR4PAAAIED7G7YSEZVZ5V7WWZ2PSC7Q 502KiB/s 00:00 | 56KiB transferred /gnu/store/khaaib6s836bk5kbik239hlk6n6ianc4-hello-2.11 --8<---------------cut here---------------end--------------->8--- We have just retreived the substitute for the hello package from IPFS. Hello decentralized substitutes! I have only tested this for fairly small packages (up to a few MB). One issue with IPFS might be that we have to create a new HTTP connection to the IPFS daemon for every single block (32KiB). The IPFS daemon does not seem to support HTTP connection re-use and neither does the Guile (web client). I fear this might become a performance issue. It seems possible to use IPFS more directly by exposing the Go code as a C library and then using that with the Guile FFI [1]. This is however a bit complicated and adds a lot of dependencies. In particular, this should not become a dependency of Guix itself. The performance of IPFS itself also needs to be evaluated, maybe the IPFS HTTP API will not be the bottle-neck. As mentioned in previous mail a simple HTTP transport for blocks would be a good fallback. This would allow users to get missing blocks (things that somehow got dropped from IPFS) directly from a substitute server. This is different then getting the entire NAR from a substitute server. A user might be missing a single 32KiB block and should be able to get only that. However, such a HTTP fallback would also suffer from the one-connection-per-block issue. As part of general ERIS research we are investigating CoAP as a better fallback transport. In any case, it would be necessary for the substitute server to store encoded blocks of the NAR. For this I think it makes sense to use a small database. We have bindings to use ERIS with GDBM [2]. It might also make sense to use SQLite, especially if there are other use-cases for such a database. I will be looking into the HTTP fallback and also using BitTorrent and GNUNet as transports. Thanks for making it so far and happy hacking! -pukkamustard [1] https://github.com/scala-network/libipfs/ [2] https://codeberg.org/eris/guile-eris/src/branch/main/eris/blocks/gdbm.scm pukkamustard (5): WIP: gnu: guile-eris: Update to unreleased git version. publish: Add ERIS URN to narinfo Add (guix eris). publish: Add support for storing ERIS encoded blocks to IPFS. substitute: Fetch substitutes using ERIS. Makefile.am | 1 + configure.ac | 5 +++ gnu/packages/guile-xyz.scm | 10 ++--- gnu/packages/package-management.scm | 1 + guix/eris.scm | 60 +++++++++++++++++++++++++++++ guix/narinfo.scm | 14 +++++-- guix/scripts/publish.scm | 32 +++++++++++++-- guix/scripts/substitute.scm | 21 +++++++--- 8 files changed, 126 insertions(+), 18 deletions(-) create mode 100644 guix/eris.scm -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 2/5] publish: Add ERIS URN to narinfo Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Tue, 25 Jan 2022 19:23:02 +0000 Resent-Message-ID: <handler.52555.B52555.16431385624671 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16431385624671 (code B ref 52555); Tue, 25 Jan 2022 19:23:02 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:42 +0000 Received: from localhost ([127.0.0.1]:50144 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nCROe-0001Cs-Gx for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:42 -0500 Received: from mout01.posteo.de ([185.67.36.65]:38629) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nCROZ-0001CA-Dn for 52555 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:25 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id D5D42240027 for <52555 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:17 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138537; bh=W1gZghgCR8earSsN5aBLBtmdrhJyAYyFNtxvyJcaNcI=; h=From:To:Cc:Subject:Date:From; b=KnTlWy6rbh0pmW2mOQtgouzkXQte7i7DPY25YXtv8koUv2u8uy0it7lO9fJYq8SQ8 LR6lkHAxPOEtOgJsVTHI2ZFrpXR4VZ0pAP5zP6ew7D1620U8aJWlRFMHrjsQQkAkaQ VM0lMMrPbquYDs1aMH0L94GQZ3WDiLc2T9pzeKN5xPqtMPzIRG6mnurbQfYzRcPAqz O7gzMbbb1EpRbWbJD3aKqXr0yUf8Jgu/DaKvU70sqUYkcfVDfDktZHQEZM8k6cG9OG Nj2JkNENB60IbyoYOyWuf+DlS0Tp9txBTEzbKaxNnmRLq0Mbj64MLYntaBVO4ZKbyT 8zLPcoEz+Fohw== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ110YGz9rxM; Tue, 25 Jan 2022 20:22:17 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Tue, 25 Jan 2022 19:21:58 +0000 Message-Id: <20220125192201.7582-3-pukkamustard@HIDDEN> In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) * guix/scripts/publish.scm: (bake-narinfo+nar): Compute ERIS URN of compressed nars. (narinfo-string): Add #:eris-urn parameter and honor it. * guix/scripts/narinfo.scm: (<narinfo>)[eris-format,eris-urn]: New fields. (narinfo-maker): Handle ERIS URN and ERIS format. * configure.ac: (HAVE_GUILE_ERIS): New conditional. * gnu/packages/package-management.scm: (guix)[native-inputs]: Add guile-eris. --- configure.ac | 5 +++++ gnu/packages/package-management.scm | 1 + guix/narinfo.scm | 14 ++++++++++---- guix/scripts/publish.scm | 24 ++++++++++++++++++++---- 4 files changed, 36 insertions(+), 8 deletions(-) diff --git a/configure.ac b/configure.ac index 341cff8fbd..72396be8aa 100644 --- a/configure.ac +++ b/configure.ac @@ -170,6 +170,11 @@ GUILE_MODULE_AVAILABLE([have_guile_avahi], [(avahi)]) AM_CONDITIONAL([HAVE_GUILE_AVAHI], [test "x$have_guile_avahi" = "xyes"]) +dnl Check for Guile-eris. +GUILE_MODULE_AVAILABLE([have_guile_eris], [(eris)]) +AM_CONDITIONAL([HAVE_GUILE_ERIS], + [test "x$have_guile_eris" = "xyes"]) + dnl Guile-newt is used by the graphical installer. GUILE_MODULE_AVAILABLE([have_guile_newt], [(newt)]) diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm index 05795824b5..a9094b8b7f 100644 --- a/gnu/packages/package-management.scm +++ b/gnu/packages/package-management.scm @@ -404,6 +404,7 @@ (define code ("guile-zstd" ,guile-zstd) ("guile-ssh" ,guile-ssh) ("guile-git" ,guile-git) + ("guile-eris" ,guile-eris) ;; XXX: Keep the development inputs here even though ;; they're unnecessary, just so that 'guix environment diff --git a/guix/narinfo.scm b/guix/narinfo.scm index 4fc550aa6c..a6a5d3b84b 100644 --- a/guix/narinfo.scm +++ b/guix/narinfo.scm @@ -45,6 +45,8 @@ (define-module (guix narinfo) narinfo-file-sizes narinfo-hash narinfo-size + narinfo-eris-format + narinfo-eris-urn narinfo-references narinfo-deriver narinfo-system @@ -68,8 +70,8 @@ (define-module (guix narinfo) (define-record-type <narinfo> (%make-narinfo path uri-base uris compressions file-sizes file-hashes - nar-hash nar-size references deriver system - signature contents) + nar-hash nar-size eris-format eris-urn references deriver + system signature contents) narinfo? (path narinfo-path) (uri-base narinfo-uri-base) ;URI of the cache it originates from @@ -79,6 +81,8 @@ (define-record-type <narinfo> (file-hashes narinfo-file-hashes) (nar-hash narinfo-hash) (nar-size narinfo-size) + (eris-format narinfo-eris-format) + (eris-urn narinfo-eris-urn) (references narinfo-references) (deriver narinfo-deriver) (system narinfo-system) @@ -135,7 +139,7 @@ (define (narinfo-maker str cache-url) "Return a narinfo constructor for narinfos originating from CACHE-URL. STR must contain the original contents of a narinfo file." (lambda (path urls compressions file-hashes file-sizes - nar-hash nar-size references deriver system + nar-hash nar-size eris-format eris-urn references deriver system signature) "Return a new <narinfo> object." (define len (length urls)) @@ -157,6 +161,8 @@ (define len (length urls)) ((lst ...) (map string->number lst))) nar-hash (and=> nar-size string->number) + eris-format + (if eris-urn (string->uri eris-urn) #f) (string-tokenize references) (match deriver ((or #f "") #f) @@ -184,7 +190,7 @@ (define* (read-narinfo port #:optional url (narinfo-maker str url) '("StorePath" "URL" "Compression" "FileHash" "FileSize" "NarHash" "NarSize" - "References" "Deriver" "System" + "ERISFormat" "ERIS" "References" "Deriver" "System" "Signature") '("URL" "Compression" "FileSize" "FileHash")))) diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 6e2b4368da..9c83f5183d 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -58,6 +58,7 @@ (define-module (guix scripts publish) #:use-module (guix workers) #:use-module (guix store) #:use-module ((guix serialization) #:select (write-file)) + #:use-module (eris) #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port make-lzip-output-port) @@ -146,6 +147,9 @@ (define %default-gzip-compression ;; Since we compress on the fly, default to fast compression. (compression 'gzip 3)) +(define %eris-zstd-compression + (compression 'zstd 19)) + (define (default-compression type) (compression type 3)) @@ -324,7 +328,8 @@ (define* (store-item->recutils store-item (define* (narinfo-string store store-path #:key (compressions (list %no-compression)) - (nar-path "nar") (file-sizes '())) + (nar-path "nar") (file-sizes '()) + eris-urn) "Generate a narinfo key/value string for STORE-PATH; an exception is raised if STORE-PATH is invalid. Produce a URL that corresponds to COMPRESSION. The narinfo is signed with KEY. NAR-PATH specifies the prefix for nar URLs. @@ -347,7 +352,7 @@ (define* (narinfo-string store store-path StorePath: ~a ~{~a~}\ NarHash: sha256:~a -NarSize: ~d +NarSize: ~d~@[~%ERISFormat: application/x-nix-archive+zstd-19~%ERIS: ~a~] References: ~a~%" store-path (map (lambda (compression) @@ -359,7 +364,7 @@ (define* (narinfo-string store store-path #:compression compression))) compressions) - hash size references)) + hash size eris-urn references)) ;; Do not render a "Deriver" line if we are rendering info for a ;; derivation. Also do not render a "System" line that would be ;; expensive to compute and is currently unused. @@ -632,6 +637,16 @@ (define (compressed-nar-size compression) (and stat (cons compression (stat:size stat))))) + (define (eris-encode-nar compressions) + (and (member %eris-zstd-compression compressions) + (let* ((nar (nar-cache-file cache item + #:compression %eris-zstd-compression)) + (stat (stat nar #f))) + (and stat + (call-with-input-file nar + (cut eris-encode->string <> + #:block-size %eris-block-size-large)))))) + (let ((compression (actual-compressions item compressions))) (for-each (cut compress-nar cache item <>) compressions) @@ -650,7 +665,8 @@ (define (compressed-nar-size compression) (display (narinfo-string store item #:nar-path nar-path #:compressions compressions - #:file-sizes sizes) + #:file-sizes sizes + #:eris-urn (eris-encode-nar compression)) port))) ;; Make the cached narinfo world-readable, contrary to what -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 3/5] Add (guix eris). Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Tue, 25 Jan 2022 19:23:03 +0000 Resent-Message-ID: <handler.52555.B52555.16431385634679 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16431385634679 (code B ref 52555); Tue, 25 Jan 2022 19:23:03 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:43 +0000 Received: from localhost ([127.0.0.1]:50148 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nCROs-0001DI-BI for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:43 -0500 Received: from mout01.posteo.de ([185.67.36.65]:36957) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nCROb-0001CE-9T for 52555 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:26 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id B5880240027 for <52555 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:19 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138539; bh=fVwvh/6zrIV6B6hs8ZwDshihybQDuGkAoOjNHeNPWh0=; h=From:To:Cc:Subject:Date:From; b=fTYsV4CtQohwGGN+e0QsRfHniRoyA+0SWQRTT2GBxL7YZcRXo45y1ErrC2zwJvy2O PGflmXpSURrMEQnNriXAl/8Fe66gU++bDKN2+VC6DAoPKaStR8NStByTW+aOSYcv9M 1RhQ0Dlcw1kCPXpJe5qJQPaaTSkGLhFm+xPuNLbXFDxvatMP82Jpmr3917yYvi91eF ClknkBsjLiwHbFnA8oALpQ9uES/59k4i08LYdruI8+2X31Vvoyfx+Wq0DXtlDy8BHk fDTxGjJKyQ+J0yTlRGW+3TvbKzVlwtKrvFq8oB+7h4iYVwfFXgu16tYk8114pkHRo3 lRS+RrCvPAw3Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ25J44z9rxF; Tue, 25 Jan 2022 20:22:18 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Tue, 25 Jan 2022 19:21:59 +0000 Message-Id: <20220125192201.7582-4-pukkamustard@HIDDEN> In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * guix/ipfs.scm: New file. * Makefile.am (MODULES): Add it. --- Makefile.am | 1 + guix/eris.scm | 55 +++++++++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 56 insertions(+) create mode 100644 guix/eris.scm diff --git a/Makefile.am b/Makefile.am index a10aeb817b..7219386361 100644 --- a/Makefile.am +++ b/Makefile.am @@ -131,6 +131,7 @@ MODULES = \ guix/cve.scm \ guix/workers.scm \ guix/ipfs.scm \ + guix/eris.scm \ guix/build-system.scm \ guix/build-system/android-ndk.scm \ guix/build-system/ant.scm \ diff --git a/guix/eris.scm b/guix/eris.scm new file mode 100644 index 0000000000..163bbe05ac --- /dev/null +++ b/guix/eris.scm @@ -0,0 +1,55 @@ +;;; GNU Guix --- Functional package management for GNU +;;; Copyright © 2022 pukkamustard <pukkamustard@HIDDEN> +;;; +;;; This file is part of GNU Guix. +;;; +;;; GNU Guix is free software; you can redistribute it and/or modify it +;;; under the terms of the GNU General Public License as published by +;;; the Free Software Foundation; either version 3 of the License, or (at +;;; your option) any later version. +;;; +;;; GNU Guix is distributed in the hope that it will be useful, but +;;; WITHOUT ANY WARRANTY; without even the implied warranty of +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +;;; GNU General Public License for more details. +;;; +;;; You should have received a copy of the GNU General Public License +;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>. + +(define-module (guix eris) + #:use-module (eris) + #:use-module (eris blocks ipfs) + #:use-module (web client) + #:use-module (web response) + #:use-module (srfi srfi-71) + + #:export (guix-eris-block-reducer)) + +(define (ipfs-daemon-alive?) + "Attempt to connect to the IPFS daemon. Returns #t if the daemon is alive +and #f else." + (with-exception-handler + (const #f) + (lambda _ + (let ((response _ + (http-post (string-append (%ipfs-base-url) + "/api/v0/version")))) + (equal? 200 (response-code response)))) + #:unwind? #t)) + +(define guix-eris-block-reducer + (case-lambda + + ;; Check if IPFS Daemon is running. + (() (if (ipfs-daemon-alive?) + (eris-blocks-ipfs-reducer) + #f)) + + ;; Completion. Nothing to do. + ((_) #t) + + ((ipfs ref-block) + ;; If IPFS has been initialized store block there + (if ipfs + (eris-blocks-ipfs-reducer ipfs ref-block) + ipfs)))) -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 4/5] publish: Add support for storing ERIS encoded blocks to IPFS. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Tue, 25 Jan 2022 19:23:03 +0000 Resent-Message-ID: <handler.52555.B52555.16431385634686 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16431385634686 (code B ref 52555); Tue, 25 Jan 2022 19:23:03 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:43 +0000 Received: from localhost ([127.0.0.1]:50150 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nCROt-0001DQ-8v for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:43 -0500 Received: from mout02.posteo.de ([185.67.36.66]:59993) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nCROd-0001CH-SK for 52555 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:28 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 4C66F240103 for <52555 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:22 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138542; bh=WjdsN5qtP47bgM3Qv/+NX+Fy8ZzMEmAQi9X/KFRZPLo=; h=From:To:Cc:Subject:Date:From; b=QlZsIs6HZgRLJycSuQJN9og4N1nW8ttJnDB8/C1w9CfdkTH18W0i6o1cj1enMolVg psxeNyntX7MBIcBMHLjeFdKe0d8JYP+pmvcLNn2FKoiS5XP+Xnl+TYjQYfylPHUo65 iX+hB9mXQVP1XmxFngAb4CwoYji/B3RQ/p/FrlkBjlsGgjM8noz0jfq8g5nBg2rfmN WvkKaLA8lTM9oHN6OOPU4JibOgz5IqF+d0OWx8G3+5w4OlVG44iC70tsYYVinRHFl3 tjmeTxbM6FT8TMZ4tZj8YXOaTo/Gz/btZ3qd8srheuo5Wr4No9TYT8reev2mVywF+r XbIa2SwyoD0Cg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ54K7cz9rxS; Tue, 25 Jan 2022 20:22:21 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Tue, 25 Jan 2022 19:22:00 +0000 Message-Id: <20220125192201.7582-5-pukkamustard@HIDDEN> In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * guix/scripts/publish.scm (bake-narinfo+nar): Use guix-eris-block-reducer from (guix eris). --- guix/scripts/publish.scm | 12 ++++++++++-- 1 file changed, 10 insertions(+), 2 deletions(-) diff --git a/guix/scripts/publish.scm b/guix/scripts/publish.scm index 9c83f5183d..556107ab7d 100644 --- a/guix/scripts/publish.scm +++ b/guix/scripts/publish.scm @@ -41,6 +41,7 @@ (define-module (guix scripts publish) #:use-module (srfi srfi-26) #:use-module (srfi srfi-34) #:use-module (srfi srfi-37) + #:use-module (srfi srfi-71) #:use-module (web http) #:use-module (web request) #:use-module (web response) @@ -58,6 +59,7 @@ (define-module (guix scripts publish) #:use-module (guix workers) #:use-module (guix store) #:use-module ((guix serialization) #:select (write-file)) + #:use-module (guix eris) #:use-module (eris) #:use-module (zlib) #:autoload (lzlib) (call-with-lzip-output-port @@ -644,8 +646,14 @@ (define (eris-encode-nar compressions) (stat (stat nar #f))) (and stat (call-with-input-file nar - (cut eris-encode->string <> - #:block-size %eris-block-size-large)))))) + (lambda (port) + (let ((eris-urn _ + (eris-encode port + #:block-size + %eris-block-size-large + #:block-reducer + guix-eris-block-reducer))) + (uri->string eris-urn)))))))) (let ((compression (actual-compressions item compressions))) -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 1/5] WIP: gnu: guile-eris: Update to unreleased git version. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Tue, 25 Jan 2022 19:23:03 +0000 Resent-Message-ID: <handler.52555.B52555.16431385634693 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16431385634693 (code B ref 52555); Tue, 25 Jan 2022 19:23:03 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:43 +0000 Received: from localhost ([127.0.0.1]:50152 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nCROt-0001DX-HI for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:43 -0500 Received: from mout01.posteo.de ([185.67.36.65]:41619) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nCROY-0001C7-7H for 52555 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:29 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 4A0B4240027 for <52555 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:15 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138536; bh=qhTWe7UWmXECMl9CT46acau1cZgMduxF1qX8x4pCwgw=; h=From:To:Cc:Subject:Date:From; b=ldwCPusN2dX6HRhDxeqpR67P0+ZOgUpSB0vczXGwylfKlK6bM4WMcgh8+P9NV/wUz Yxb77kEu3qNCbdnD10I6CB+FCmC2Bq9h1msGrCJKg/HgJGIYh1KTspaL65/aGoz9KA ENP7peFHSqTXJSGVQgYoBiTZ80smHY7/DBCO8Ob1UYpj/NfKy9lfUaHeMvXwXE5pUo RnN5pCYNCGA6h0RwRs3s4wJrIjP2Zkid/y88cKWSfOGfAaCJHFULBWDOtDyo5ge6NB d0WEgNqeReTsz/j19C6ZYWjLLOx7fx5Q8y27nvWgL01L6MuFhu5L60FpKD1PMw0Tkc /7FG12IHHEH0Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxYz10tFz9rxM; Tue, 25 Jan 2022 20:22:15 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Tue, 25 Jan 2022 19:21:57 +0000 Message-Id: <20220125192201.7582-2-pukkamustard@HIDDEN> In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * gnu/packages/guile-xyz.schm (guile-eris): Update to unreleased git version. [source]: Update source URI. [propagated-inputs]: Add guile-json-4 and guile-gdbm-ffi. --- gnu/packages/guile-xyz.scm | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/gnu/packages/guile-xyz.scm b/gnu/packages/guile-xyz.scm index e2cf793acc..66ac486a74 100644 --- a/gnu/packages/guile-xyz.scm +++ b/gnu/packages/guile-xyz.scm @@ -4374,15 +4374,15 @@ (define-public guile-sodium (define-public guile-eris (package (name "guile-eris") - (version "0.2.0") + (version "bcbbcbc88f3ec1f2fafcd034ce5b620516bff105") (source (origin (method git-fetch) (uri (git-reference - (url "https://inqlab.net/git/eris.git") - (commit (string-append "v" version)))) + (url "https://codeberg.org/eris/guile-eris") + (commit version))) (file-name (git-file-name name version)) - (sha256 (base32 "1ijglmwkdy1l87gj429qfjis0v8b1zlxhbyfhx5za8664h68nqka")))) + (sha256 (base32 "17v3h2hqx080739rl57nfradp5vlmy24fgqdxry1zal5z9d3i8sr")))) (build-system gnu-build-system) (arguments '()) (native-inputs @@ -4394,7 +4394,7 @@ (define-public guile-eris guile-srfi-180)) (inputs (list guile-3.0)) (propagated-inputs - (list guile-sodium)) + (list guile-sodium guile-json-4 guile-gdbm-ffi)) (synopsis "Guile implementation of the Encoding for Robust Immutable Storage (ERIS)") (description "Guile-ERIS is the reference implementation of the Encoding for Robust -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Tue, 25 Jan 2022 19:23:04 +0000 Resent-Message-ID: <handler.52555.B52555.16431385644699 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: 52555 <at> debbugs.gnu.org Cc: pukkamustard <pukkamustard@HIDDEN>, ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16431385644699 (code B ref 52555); Tue, 25 Jan 2022 19:23:04 +0000 Received: (at 52555) by debbugs.gnu.org; 25 Jan 2022 19:22:44 +0000 Received: from localhost ([127.0.0.1]:50154 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nCROt-0001De-So for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:44 -0500 Received: from mout01.posteo.de ([185.67.36.65]:41807) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nCROf-0001CR-O1 for 52555 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:22:30 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 2BFA2240029 for <52555 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 20:22:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643138544; bh=j7T0v7hpPHZH+YN4wChkVCZNNIqTEpqBrEMRdms1Xc4=; h=From:To:Cc:Subject:Date:From; b=aCsMaZzdU2NgT+wR3r3pQ6awfvPTBY/ROM+4GQsQrm0YW0BD+bgZsQiRm7MdfRV2/ vx/b76cm1NwC+84qbrS2xAyMHdLiC1nNu/YHZ6oRKLoPL6akpybgp3MITSUU99pMXf IJHqO7EuONC5H0GwBXjD9Jz+PfX6nIXkl1a5huc1bKbNF0w7r1NN4pfgVUsARHdVgz 3jw2phmXQCDJ1Ta6NCAHztahZKj9mkP7lsROG9+FjJnDcRjZPThEuE6iSYk8oS7Mzp yAcXjYxeajUzQssVumj3m1pTiaguZYKhdeFVZbwU/OFD/me0V5RKu/Op0zzZeHLP1L 8bmk5mH9vFRxQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JjxZ72hWgz9rxV; Tue, 25 Jan 2022 20:22:23 +0100 (CET) From: pukkamustard <pukkamustard@HIDDEN> Date: Tue, 25 Jan 2022 19:22:01 +0000 Message-Id: <20220125192201.7582-6-pukkamustard@HIDDEN> In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) * guix/scripts/substitute.scm (process-substitution): Fetch substitutes using ERIS. * guix/eris.scm (guix-eris-block-ref): New procedure. --- guix/eris.scm | 7 ++++++- guix/scripts/substitute.scm | 21 ++++++++++++++++----- 2 files changed, 22 insertions(+), 6 deletions(-) diff --git a/guix/eris.scm b/guix/eris.scm index 163bbe05ac..0999564c1f 100644 --- a/guix/eris.scm +++ b/guix/eris.scm @@ -23,7 +23,8 @@ (define-module (guix eris) #:use-module (web response) #:use-module (srfi srfi-71) - #:export (guix-eris-block-reducer)) + #:export (guix-eris-block-reducer + guix-eris-block-ref)) (define (ipfs-daemon-alive?) "Attempt to connect to the IPFS daemon. Returns #t if the daemon is alive @@ -53,3 +54,7 @@ (define guix-eris-block-reducer (if ipfs (eris-blocks-ipfs-reducer ipfs ref-block) ipfs)))) + +(define (guix-eris-block-ref ref) + "Dereference a block for decoding ERIS content" + (eris-blocks-ipfs-ref ref)) diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm index 908a8334a8..852264976e 100755 --- a/guix/scripts/substitute.scm +++ b/guix/scripts/substitute.scm @@ -62,6 +62,8 @@ (define-module (guix scripts substitute) #:use-module (srfi srfi-35) #:use-module (web uri) #:use-module (guix http-client) + #:use-module (guix eris) + #:use-module (eris) #:export (%allow-unauthenticated-substitutes? %reply-file-descriptor @@ -486,18 +488,27 @@ (define (fetch uri) #:port port #:keep-alive? #t #:buffered? #f))))) + (else - (leave (G_ "unsupported substitute URI scheme: ~a~%") - (uri->string uri))))) + (if (and (eris-read-capability? uri)) + (values (eris-decode->port uri + #:block-ref + guix-eris-block-ref) #f) + (leave (G_ "unsupported substitute URI scheme: ~a~%") + (uri->string uri)))))) + + (define* (best-uri narinfo #:key (eris? #f)) + (if (and eris? (narinfo-eris-urn narinfo)) + (values (narinfo-eris-urn narinfo) "zstd" #f) + (narinfo-best-uri narinfo #:fast-decompression? + %prefer-fast-decompression?))) (unless narinfo (leave (G_ "no valid substitute for '~a'~%") store-item)) (let-values (((uri compression file-size) - (narinfo-best-uri narinfo - #:fast-decompression? - %prefer-fast-decompression?))) + (best-uri narinfo #:eris? #t))) (unless print-build-trace? (format (current-error-port) (G_ "Downloading ~a...~%") (uri->string uri))) -- 2.34.0
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:01:02 +0000 Resent-Message-ID: <handler.52555.B52555.16434900417963 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16434900417963 (code B ref 52555); Sat, 29 Jan 2022 21:01:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:00:41 +0000 Received: from localhost ([127.0.0.1]:34922 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDupt-00024M-J8 for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:00:41 -0500 Received: from laurent.telenet-ops.be ([195.130.137.89]:51088) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDups-00024E-1i for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:00:41 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by laurent.telenet-ops.be with bizsmtp id ol0d2600614dU1001l0dcD; Sat, 29 Jan 2022 22:00:38 +0100 Message-ID: <cc7ec17cded7fda6689d23a163df2ba72058a8aa.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:00:31 +0100 In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-NT9v9uBSdFbLolAKqT8C" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643490038; bh=/Nm9xoWmMMvhmMCYSsb8AHBY8F5FMcapsA07KleKORg=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=VWocEWQbRlSx6sbKWXXtHPVpBtJgelH79SS8ytFxdFMimBJv2x6Fmwrjbn5aKD3XL ryEaVm3ZMefk6UyA0NiM+Y7oF4nrBZAlimzLxUc3v/VqvsE0soqUA7vEc6gl7gWXpA Tc/GCYgnAZpuvVVc+7chvhTtyRhvYDnb9sN/z/GM+hI8gCefv9drYmtLPyEINRwdF6 127zIr6tB+LSV0qwCDm+2XOqqQSyopGaD00d6qA1Y8xjRSDhx4RoBfuRcnClymOOym 8HMbYcUjeegRNSVT41sKrD0c+i39l+HQ2hCyCxoidJWCPEhq9DLt4IZ68ssMvIG57G 9gsgpLC8kCCgg== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-NT9v9uBSdFbLolAKqT8C Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > ** Authorize local substitutes >=20 > We will be running a local substitute server so we need to add the > local > signing key to the list of authorized keys. In the system > configurations: >=20 > #+BEGIN_SRC scheme > (modify-services %base-services > (guix-service-type config =3D> [...])) > #+END_SRC > [...] > ** Start the IPFS daemon >=20 > #+BEGIN_SRC shell > guix shell go-ipfs -- ipfs daemon > #+END_SRC There's an ipfs-service-type nowadays, so starting the daemon manually isn't required (if using Guix System). Greetings, Maxime. --=-NT9v9uBSdFbLolAKqT8C Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWq7xccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7kWHAQD8B6ufMY7MntcR+Q44j+roW/MG /b3y8wn1dPnDxb27cQEA+fgcZyuRGc458oFJKqyMP2xsoTGUYQmMC1Vc2aiYRwA= =/y6d -----END PGP SIGNATURE----- --=-NT9v9uBSdFbLolAKqT8C--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:09:01 +0000 Resent-Message-ID: <handler.52555.B52555.16434905208836 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16434905208836 (code B ref 52555); Sat, 29 Jan 2022 21:09:01 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:08:40 +0000 Received: from localhost ([127.0.0.1]:34950 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDuxc-0002IR-B3 for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:08:40 -0500 Received: from laurent.telenet-ops.be ([195.130.137.89]:57380) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDuxZ-0002IJ-Ug for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:08:38 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by laurent.telenet-ops.be with bizsmtp id ol8c2600114dU1001l8c5p; Sat, 29 Jan 2022 22:08:37 +0100 Message-ID: <37651a21b0699b78b0282284b7d7322cb1a8c320.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:08:36 +0100 In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-N1yZtGR0kxSKC69kN7sH" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643490517; bh=RYywVYyfShhfWPYegnqQ7/XCVBNVMenGdK0K4qNegGE=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=Kk1IMzwN/oKN+gDrtKghIAgYvRQXJNoJ8XvtiJ81Frc2C+0U6iqhOjnU7OBZKfwwM 6HdWpbIIGBtnZi1z6Tk09JR+PFNn3i1fwnqZ4e1DfWeiEfZrAXtzEDakSr3jQTB1gf xjRYfsPIVdrFFSoYmG0BsOZxqted0nuenT6MCQjA28EjC05eGRdthFCrIffJyYAFQz jd7OAVVAhM9sHVN0y39Z43Dnqk5xcfdtATgWipAUJA77ONfAujz++lpr5ZjuiM7J+p cwhPoGSGLlTO3bq24Q9xaNe/cProNawiYxTWbXDgGSdL4Sj/mRSCGHR/qPvuE5TQwv sUwr70j+zFLCw== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-N1yZtGR0kxSKC69kN7sH Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > I will be looking into the HTTP fallback and also using BitTorrent and GN= UNet > as transports. I have been writing a (Guile) Scheme port of GNUnet's client libraries (https://git.gnunet.org/gnunet-scheme.git/). Currently only NSE is supported, but I'm working on DHT. DHT search/put already works to a degree (see examples/web.scm), but there are plenty of sharp edges (see TODOs about disconnecting, reconnecting and stopping fibers, and see guix.scm for Guile bugs that are patched out and extra guile- fibers features). Tests are being written a edge cases will be addressed. Greetings, Maxime. --=-N1yZtGR0kxSKC69kN7sH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWs1BccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7iUHAP9GaXu5iS0pWHmlWrCS/xPjChSf 5aDwgq8ex1CbWgGslwEA2yz95+6uNajWeX9xbu6TV6b+AcAPwVUC4p2HgSvxBwA= =tbfK -----END PGP SIGNATURE----- --=-N1yZtGR0kxSKC69kN7sH--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 2/5] publish: Add ERIS URN to narinfo Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:11:02 +0000 Resent-Message-ID: <handler.52555.B52555.16434906119006 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16434906119006 (code B ref 52555); Sat, 29 Jan 2022 21:11:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:10:11 +0000 Received: from localhost ([127.0.0.1]:34955 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDuz4-0002LB-LR for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:10:10 -0500 Received: from albert.telenet-ops.be ([195.130.137.90]:37964) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDuz2-0002KU-VQ for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:10:09 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by albert.telenet-ops.be with bizsmtp id ol9z2600E14dU1006lA0xC; Sat, 29 Jan 2022 22:10:00 +0100 Message-ID: <d0882404dc5153c7cf75846bb5cc70cb90138b78.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:09:59 +0100 In-Reply-To: <20220125192201.7582-3-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-3-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-lz129T6Kuic5c3BJn6uu" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643490600; bh=mP1iYliWUrb2VgKoyCFrRfDk+1XZ0ntQZm9wbzNBEr8=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=eaL6Bl0xJM5SB4kx1qXMEiqcjL1CZH4VCcqkKoeOR7qQZXgWWwDKhoacyOFjiR6XC b409BMTM44HD8io8XMY71fPj8KvbNcsz5+qvIHwnzFAmNcPUtrMUSygJ+6SWE+mLdn YvhCziRQu9si7m8rWvwRnyEuQGvUFilhTLQVL43szF1k9zdZZlF9kFAEdcLFaHqgul ZNHp80obQ4A22C+f4LhsRQqF5KBbJ/V+psOu1CZeA6M9Ctg8GYVOttNsEp9u/0BYHg stuouW/+T+JFH35U7fnOl5kpUyHsVsspw1ATUCe6TTi0WaG9OL25LGKhvN5zSj1Y7p 5VGmtTFgm1Fbw== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-lz129T6Kuic5c3BJn6uu Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > +dnl Check for Guile-eris. > +GUILE_MODULE_AVAILABLE([have_guile_eris], [(eris)]) > +AM_CONDITIONAL([HAVE_GUILE_ERIS], > +=C2=A0 [test "x$have_guile_eris" =3D "xyes"]) This could to be documented in (guix)Requirements. Greetings, Maxime. --=-lz129T6Kuic5c3BJn6uu Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWtJxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7qqUAQCEB1li7ZBR4r0xjFIQwDM+1Z2d DmdnE6ogz16oO4WWjAD9HT5dzf21xMKPiGXXFNsOLwJKtc+2i4ZLOTTOvlCnTgk= =ZFnB -----END PGP SIGNATURE----- --=-lz129T6Kuic5c3BJn6uu--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 2/5] publish: Add ERIS URN to narinfo Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:16:01 +0000 Resent-Message-ID: <handler.52555.B52555.164349094012195 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349094012195 (code B ref 52555); Sat, 29 Jan 2022 21:16:01 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:15:40 +0000 Received: from localhost ([127.0.0.1]:34972 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDv4N-0003AP-LZ for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:15:40 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:58690) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDv4L-000375-NV for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:15:38 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by michel.telenet-ops.be with bizsmtp id olFb2600G14dU1006lFce4; Sat, 29 Jan 2022 22:15:36 +0100 Message-ID: <46558e7148b91f12cdaa92f32adb91bbcb3080bd.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:15:35 +0100 In-Reply-To: <20220125192201.7582-3-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-3-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-dfqLB8R7jnDU5ihy53Cg" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643490936; bh=7Ad8WDveaTcr/d3irjECLOr3MDYQFmLuCGZBOP2Q7ms=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=Mn5jgdX4YzjQn5a6hGhiMLDNEhSQST6qjHX5t3w8CFS/kwGjn1f9YHhpeEfs5YjmF 5as8EARR3xsxeGjMrMaH7e9P/xcyIfGCJvwlXxEx52IyR+YLx/e0OjgNKzaLL/0oDP HRfMdt3R4ZQyqSobfTq3tEElx0pOVC4nkMopRdRkKE5lvaLR3WzG6PeCBsk4ifiWK0 X7aeXfnkSZGm4HOLMD2QqKkyu+4+gPXCJ+FdLdeqLoR+uD1jG0K1E6ogSM3cl+0VQ8 wqkfx8qSmxUIVueR88btcPXl2qF47Bdelp2AO6WCacYxgkZRD6JUgG24KaeHPLdCpO mrIplgP9yZ/Gg== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-dfqLB8R7jnDU5ihy53Cg Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 cHVra2FtdXN0YXJkIHNjaHJlZWYgb3AgZGkgMjUtMDEtMjAyMiBvbSAxOToyMSBbKzAwMDBdOgo+ ICvCoCAoZGVmaW5lIChlcmlzLWVuY29kZS1uYXIgY29tcHJlc3Npb25zKQo+ICvCoMKgwqAgKGFu ZCAobWVtYmVyICVlcmlzLXpzdGQtY29tcHJlc3Npb24gY29tcHJlc3Npb25zKQo+ICvCoMKgwqDC oMKgwqDCoMKgIChsZXQqICgobmFyIChuYXItY2FjaGUtZmlsZSBjYWNoZSBpdGVtCj4gK8KgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoCAjOmNvbXByZXNzaW9uICVlcmlzLXpzdGQtY29tcHJlc3Npb24pKQo+ICvCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgKHN0YXQgKHN0YXQgbmFyICNmKSkpCj4gK8KgwqDC oMKgwqDCoMKgwqDCoMKgIChhbmQgc3RhdAo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqAgKGNhbGwtd2l0aC1pbnB1dC1maWxlIG5hcgo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgIChjdXQgZXJpcy1lbmNvZGUtPnN0cmluZyA8Pgo+ICvCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAjOmJsb2NrLXNpemUgJWVyaXMtYmxvY2stc2l6 ZS1sYXJnZSkpKSkpKQoKV2h5IGFyZSBleGNlcHRpb25zIHR1cm5lZCBpbnRvICNmIChpbiAoc3Rh dCBuYXIgI2YpKT8KU2hvdWxkIHRoaXMgYmUgZG9uZSBmb3IgYWxsIEkvTyBlcnJvcnMsIGluY2x1 ZGluZywgc2F5LCBFT1ZFUkZMT1csCkVOT01FTSBvciBFTkFNRVRPT0xPTkcsIG9yIG9ubHkgZm9y IEVOT0VOVD8KCklzIGEgcmFjZSBjb25kaXRpb24gcG9zc2libGUgaGVyZT8gIElmIHNvLCBtYXli ZSBjb25zaWRlciBkb2luZwpzb21ldGhpbmcgbGlrZQoKICAoY2F0Y2ggJ3N5c3RlbS1lcnJvcgog ICAgKGxhbWJkYSAoKSAoY2FsbC13aXRoLWlucHV0LWZpbGUgLi4uKSkKICAgIChsYW1iZGEgZXhj ZXB0aW9uCiAgICAgIChhbmQgaXQtaXMtYS1FTk9FTlQKICAgICAgICAgICAoYXBwbHkgdGhyb3cg ZXhjZXB0aW9uKSkpKQoKdG8gYXZvaWQgaXQ/CgpHcmVldGluZ3MsCk1heGltZS4K --=-dfqLB8R7jnDU5ihy53Cg Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWudxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7ka/AP0QTYuG8PJyBF3QVw+5ZnP0029l Ax9ryDs/xHGvJ9rfgwEA40JgFHgUhz0M23qld+IUbNztFjshJjFr+Frzf69aow8= =Imy3 -----END PGP SIGNATURE----- --=-dfqLB8R7jnDU5ihy53Cg--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 3/5] Add (guix eris). Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:24:02 +0000 Resent-Message-ID: <handler.52555.B52555.164349140018990 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349140018990 (code B ref 52555); Sat, 29 Jan 2022 21:24:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:23:20 +0000 Received: from localhost ([127.0.0.1]:34985 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvBo-0004wE-GS for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:23:20 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:36426) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvBm-0004w6-JO for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:23:19 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by michel.telenet-ops.be with bizsmtp id olPG2600N14dU1006lPH4M; Sat, 29 Jan 2022 22:23:17 +0100 Message-ID: <0b977ae15a49b051c04922d234149a8e4762404f.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:23:16 +0100 In-Reply-To: <20220125192201.7582-4-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-4-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-cweAFQCPyDn0FuFp9/8n" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643491397; bh=UdL72hfwEQ86TmvVgY+UbhOk7FdlnVWCycecXdq0i9w=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=Yr/TGmmgPI9CmChS1r4u70zhX/uekutqipEsB68GowejRwXQSRt6YEjGF8FoBog2K 4SVIcz1T4DMZTofK2MKMuQCqucoRUX4MLB8c/maCksxlVYa4kI0YUXXNO8RYYiQMGa YidqyV+L45cTo1oIPxybWl0sL0TARE9lS3iuzhUUQsoXQxfLLVgugd3BjHERKMOM7h VO1tSad5R07l80zAWm5JEQc5USY/Er7Ei7e827keYR3R2py+3FUfk0FKemWVNO3oM5 f9YIkG3h3qapSfkAe5myvZY+mP+jkQ+mTxFGyyqNEZuGzcu4zY5zTFIE1MWZbKjOfE 5s2NbR6mA6ZhQ== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-cweAFQCPyDn0FuFp9/8n Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 cHVra2FtdXN0YXJkIHNjaHJlZWYgb3AgZGkgMjUtMDEtMjAyMiBvbSAxOToyMSBbKzAwMDBdOgo+ ICsoZGVmaW5lIChpcGZzLWRhZW1vbi1hbGl2ZT8pCj4gK8KgICJBdHRlbXB0IHRvIGNvbm5lY3Qg dG8gdGhlIElQRlMgZGFlbW9uLiBSZXR1cm5zICN0IGlmIHRoZSBkYWVtb24gaXMgYWxpdmUKPiAr YW5kICNmIGVsc2UuIgo+ICvCoCAod2l0aC1leGNlcHRpb24taGFuZGxlcgo+ICvCoMKgwqDCoMKg IChjb25zdCAjZikKPiArwqDCoMKgIChsYW1iZGEgXwo+ICvCoMKgwqDCoMKgIChsZXQgKChyZXNw b25zZSBfCj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAoaHR0 cC1wb3N0IChzdHJpbmctYXBwZW5kICglaXBmcy1iYXNlLXVybCkKPiArwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAiL2FwaS92MC92ZXJzaW9uIikpKSkKPiArwqDCoMKgwqDC oMKgwqAgKGVxdWFsPyAyMDAgKHJlc3BvbnNlLWNvZGUgcmVzcG9uc2UpKSkpCj4gK8KgwqDCoCAj OnVud2luZD8gI3QpKQoKVGhpcyBzaG91bGQgcHJlZmVyYWJseSBvbmx5IGJlIGNhdGNoaW5nIGV4 Y2VwdGlvbnMgaW5kaWNhdGluZyB0aGF0CnRoZSBkYWVtb24gaXMgZG93biAoZXhjZXB0aW9ucyBp bmRpY2F0aW5nIDQwNHMsIG9yIHN5c3RlbS1lcnJvcnMKaW5kaWNhdGluZyBuZXR3b3JrIGVycm9y cywgLi4uKS4KCj4gKwo+ICsoZGVmaW5lIGd1aXgtZXJpcy1ibG9jay1yZWR1Y2VyCj4gK8KgIChj YXNlLWxhbWJkYQo+ICsKPiArwqDCoMKgIDs7IENoZWNrIGlmIElQRlMgRGFlbW9uIGlzIHJ1bm5p bmcuCj4gK8KgwqDCoCAoKCkgKGlmIChpcGZzLWRhZW1vbi1hbGl2ZT8pCj4gK8KgwqDCoMKgwqDC oMKgwqDCoMKgwqAgKGVyaXMtYmxvY2tzLWlwZnMtcmVkdWNlcikKPiArwqDCoMKgwqDCoMKgwqDC oMKgwqDCoCAjZikpCj4gKwo+ICvCoMKgwqAgOzsgQ29tcGxldGlvbi4gTm90aGluZyB0byBkby4K PiArwqDCoMKgICgoXykgI3QpCj4gKwo+ICvCoMKgwqAgKChpcGZzIHJlZi1ibG9jaykKPiArwqDC oMKgwqAgOzsgSWYgSVBGUyBoYXMgYmVlbiBpbml0aWFsaXplZCBzdG9yZSBibG9jayB0aGVyZQo+ ICvCoMKgwqDCoCAoaWYgaXBmcwo+ICvCoMKgwqDCoMKgwqDCoMKgIChlcmlzLWJsb2Nrcy1pcGZz LXJlZHVjZXIgaXBmcyByZWYtYmxvY2spCj4gK8KgwqDCoMKgwqDCoMKgwqAgaXBmcykpKSkKClRo aXMgKGlwZnMtZGFlbW9uLWFsaXZlPykgc2VlbXMgcmFjeSwgYWx0aG91Z2ggaXQncyBwcm9iYWJs eSBub3QuCkNhbiB3ZSBkbwoKKGRlZmluZSBndWl4LWVyaXMtYmxvY2stcmVkdWNlcgogIChjYXNl LWxhbWJkYQogICAgKCgpIChndWFyZCAoYyAob29wcy1pdC1mYWlscy1iZWNhdXNlLXRoZS1kYWVt b24tY2Fubm90LWJlLQpjb250YWN0ZWQ/IGMpCiAgICAgICAgICAgICAgICAgICNmYWxzZSkKICAg ICAgICAgIChlcmlzLWJsb2NrLWlwZnMtcmVkdWNlcikpCiAgICBbLi4uXSkpCgppbnN0ZWFkPyAo SSBkb24ndCB0aGluayB0aGlzIHdpbGwgd29yayBhcy1pcywgYmVjYXVzZSBmcm9tIHRoZSBuYW1l IGFuZAp0aHVua2luZXNzLCBpdCB3b3VsZCBhcHBlYXIgdGhhdCBlcmlzLWJsb2NrLWlwZnMtcmVk dWNlciByZXR1cm5zIGEKcHJvY2VkdXJlIC4uLgoKR3JlZXRpbmdzLApNYXhpbWUuCg== --=-cweAFQCPyDn0FuFp9/8n Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWwRBccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7nxXAP9CAcbVz3Isk6TX52V5SGhjdExt PLcXaJq7dAgpB44VfwEAqjgtVq1V7j2KDmmrZ9Om8e3E6+ys8okKmYBIMJia/QM= =8qD5 -----END PGP SIGNATURE----- --=-cweAFQCPyDn0FuFp9/8n--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 3/5] Add (guix eris). Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:25:01 +0000 Resent-Message-ID: <handler.52555.B52555.164349146519105 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349146519105 (code B ref 52555); Sat, 29 Jan 2022 21:25:01 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:24:25 +0000 Received: from localhost ([127.0.0.1]:34990 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvCq-0004y5-Rd for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:24:25 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:37170) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvCp-0004xw-7Y for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:24:23 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by michel.telenet-ops.be with bizsmtp id olQM2600X14dU1006lQNAa; Sat, 29 Jan 2022 22:24:22 +0100 Message-ID: <96a3137b8b746fcad9ef1eb75e1abfda5d01bfdb.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:24:21 +0100 In-Reply-To: <20220125192201.7582-4-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-4-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-DwxtgDTYMJtru4xvJYGb" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643491462; bh=kGH8JpYzpJYA7EgmT5S6lEQuTYsZXEPHV8hsIMpxXig=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=WNRix8zRNuus2yH29P7SHBu5wpSr5u9eRdd4Jb+cleuytoV3r6chxfHXsRNwuI2kl z52lOJPM8bM0bSEmsuymHpuAQfj9i6eQevqZ/SeF/nZZK/0eWKVJCUasB5iXZ3FmDC Dr0ikaW/758rwhO2rdImB22P8w7q2ShFYC14eS0zl53Gz/kkQ5Lz9Y+4IRyEtAnIwD S4BmYWVvz33GgK9uUW3VrgKJ8DY8aBJQgpRoi1ph/aNkFAZDtFmXpmceCTvEN/B4Ob pkJLVkQZWBJ2hsOBwrWq8E9wpVhasyuYKxaB10bjp6RvrsobpG+ZujUFZtuBBB5AmG 7r1IDmY88dayA== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-DwxtgDTYMJtru4xvJYGb Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > +=C2=A0 #:use-module (eris) > +=C2=A0 #:use-module (eris blocks ipfs) guile-eris is an optional dependency, so this needs to be autoloaded. Or guix/eris.scm must only be compiled when guile-eris is available. Greetings, Maxime. --=-DwxtgDTYMJtru4xvJYGb Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWwhRccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7tniAP9kASEY2aOmsyLv2tvwFXDDOimF ogDnNWhDkbmvosQthAD/TR5q03yjYDu3Nv/gitR+b5d0ZeZS9eGYT+tQ4PI66A8= =RBt/ -----END PGP SIGNATURE----- --=-DwxtgDTYMJtru4xvJYGb--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 4/5] publish: Add support for storing ERIS encoded blocks to IPFS. Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:29:02 +0000 Resent-Message-ID: <handler.52555.B52555.164349169419502 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349169419502 (code B ref 52555); Sat, 29 Jan 2022 21:29:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:28:14 +0000 Received: from localhost ([127.0.0.1]:35002 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvGY-00054U-3g for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:28:14 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:39892) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvGW-00054L-Jx for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:28:13 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by michel.telenet-ops.be with bizsmtp id olUB2600814dU1006lUBMz; Sat, 29 Jan 2022 22:28:11 +0100 Message-ID: <c6841903c177f934c7fc9d42a270907dbba59554.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:28:11 +0100 In-Reply-To: <20220125192201.7582-5-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-5-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-nOz2PAhFArEiy8P3W2cH" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643491692; bh=YTY4raj8PJNbyQie4baDJiV44aNVM2rNHqRBfjNBDSI=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=ixnjhc9PGkZdyE64CdZ53zLCVkfP1pUzaryAH3zcRvaR6c5pcbpw7HB4W2fIAXPap ro2DsPajCLR+nduX40Uf4Iex7e7J06h1m2NCncpaT3myabzgWHsepTFTS1w3CyZT47 38wIrTOx5wfl8UvXBtigb7OwVxKpFPo2C8uhjFZqbkz8Pk3PStHx82ga+dVXZoWAZ4 DAT9qiPYTW6jhqkhaFkhIwsK/bJkVJZo/P9kRCg8HE8281x6NtHhl09QSwms5hVxAj HXu5qlYWN3naIUYlpDrJK9ByR1egvAu4oy8o2r6e2Kqe/+rbyeFAZ+4TJ6V6X/3u8P Ta7yIK0+SOKQA== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-nOz2PAhFArEiy8P3W2cH Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 cHVra2FtdXN0YXJkIHNjaHJlZWYgb3AgZGkgMjUtMDEtMjAyMiBvbSAxOToyMiBbKzAwMDBdOgo+ ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAobGV0ICgoZXJpcy11cm4g Xwo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgIChlcmlzLWVuY29kZSBwb3J0Cj4gK8KgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoCAjOmJsb2NrLXNpemUKPiArwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgICVlcmlzLWJsb2NrLXNpemUtbGFyZ2UKPiArwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgICM6YmxvY2stcmVkdWNlcgo+ICvCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqAgZ3VpeC1lcmlzLWJsb2NrLXJlZHVjZXIp KSkKPiArICAgICAgICAgICAgICAgICAgICAgICh1cmktPnN0cmluZyBlcmlzLXVybikpKSkpKSkp CgpJSVVDIChhbmQgcXVpdGUgcHJvYmFibHkgSSBkb24ndCwgYmVjYXVzZSBJJ3ZlIG9ubHkgYmVp bmcgcmVhZGluZwp0aGluZ3MgY3Vyc29ybHkpLCBlcmlzLWVuY29kZSByZXR1cm5zICNmYWxzZSB3 aGVuIHRoZSBJUEZTIGRhZW1vbiBpcwpkb3duIChiZWNhdXNlIHRoZW4gZ3VpeC1lcmlzLWJsb2Nr LXJlZHVjZXIgcmV0dXJucyAjZmFsc2UpLgoKSW4gdGhhdCBjYXNlLCAodXJpLT5zdHJpbmcgZXJp cy11cm4pID0gKHVyaS0+c3RyaW5nICNmYWxzZSkgd291bGQKdGhyb3cgYW4gZXhjZXB0aW9uOgoK c2NoZW1lQChndWlsZS11c2VyKT4gKChAICh3ZWIgdXJpKSB1cmktPnN0cmluZykgI2ZhbHNlKQpp Y2UtOS9ib290LTkuc2NtOjE2Njk6MTY6IEluIHByb2NlZHVyZSByYWlzZS1leGNlcHRpb246Cklu IHByb2NlZHVyZSBzdHJ1Y3QtdnRhYmxlOiBXcm9uZyB0eXBlIGFyZ3VtZW50IGluIHBvc2l0aW9u IDEKKGV4cGVjdGluZyBzdHJ1Y3QpOiAjZgoKRW50ZXJpbmcgYSBuZXcgcHJvbXB0LiAgVHlwZSBg LGJ0JyBmb3IgYSBiYWNrdHJhY2Ugb3IgYCxxJyB0byBjb250aW51ZS4Kc2NoZW1lQChndWlsZS11 c2VyKSBbMV0+ICxidApJbiB3ZWIvdXJpLnNjbToKICAgMzM2OjE3ICAxICh1cmktPnN0cmluZyAj ZiAjOmluY2x1ZGUtZnJhZ21lbnQ/IF8pCkluIGljZS05L2Jvb3QtOS5zY206CiAgMTY2OToxNiAg MCAocmFpc2UtZXhjZXB0aW9uIF8gIzpjb250aW51YWJsZT8gXykKCkdyZWV0aW5ncywKTWF4aW1l Lgo= --=-nOz2PAhFArEiy8P3W2cH Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWxaxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hHyAP9u5jfIW9GOQmUDoxjLJrBrB3cA /zhz/MksltEGl16dtwD+M9uQeCUfT4Wqig+4eAxGYD952sqhmdmp8/9iI8W8ngM= =b/X2 -----END PGP SIGNATURE----- --=-nOz2PAhFArEiy8P3W2cH--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:30:02 +0000 Resent-Message-ID: <handler.52555.B52555.164349179819676 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349179819676 (code B ref 52555); Sat, 29 Jan 2022 21:30:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:29:58 +0000 Received: from localhost ([127.0.0.1]:35009 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvIE-00057B-G8 for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:29:58 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:41230) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvIC-000571-5m for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:29:56 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by michel.telenet-ops.be with bizsmtp id olVu2600u14dU1006lVuU8; Sat, 29 Jan 2022 22:29:55 +0100 Message-ID: <00bb85fa4fb62498b354ecb78d248df5209b6f29.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:29:50 +0100 In-Reply-To: <20220125192201.7582-6-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-rxAfJDZfTJSstgj30f4J" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643491795; bh=wsVMVRQvhjmISj+2366mmSBWXbUgnVTQQFw8WX7u1bI=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=If88x+ILAMX+49zdjEj3RmR85SjuRvX+EveJECxvGxVnXlSEBMSSrf7EBTKu8Vtpv r3VAeD1Tn2j1n3FVDQk314WfvC9Rl5XJHllH63/BwH5Rpaq62d/ZnAzAk2uDoARE6p 0A/PlMkq/P4jef5Bp9MYtkXNMi4/GTeC2GZdTuBQpIrQtHYihscOxTJOltuLOOeQ8P 1RPuHxLfBgeQHYsp54w8JH/P+uL8bUwt531Mbt3eVieBq/2t2BH4qFlbhd28x9Ftip qo6EAU9QL40PTchcy7FwmkIH/t2OC2S5hAZbmPZZSmAAnmQ0nWrlYlLUVVK6j22xec Wh3YQFCoRw3QA== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-rxAfJDZfTJSstgj30f4J Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > +(define (guix-eris-block-ref ref) > +=C2=A0 "Dereference a block for decoding ERIS content" > +=C2=A0 (eris-blocks-ipfs-ref ref)) 'guix-eris-block-ref' just calls 'eris-blocks-ipfs-ref', so I'm not seeing the point of this procedure. Greetings, Maxime. --=-rxAfJDZfTJSstgj30f4J Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWxzhccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7uMzAP4u7P60iQCm0ohzXL8c0ZG6Zl0a 66kylFJ4J3x2BE+6UgEAgsWqyNKl0p3k/gg68VcHfsVnJMd0a+zy7V4houhF7As= =iiau -----END PGP SIGNATURE----- --=-rxAfJDZfTJSstgj30f4J--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:35:01 +0000 Resent-Message-ID: <handler.52555.B52555.164349204720212 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349204720212 (code B ref 52555); Sat, 29 Jan 2022 21:35:01 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:34:07 +0000 Received: from localhost ([127.0.0.1]:35023 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvMF-0005Fw-90 for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:34:07 -0500 Received: from laurent.telenet-ops.be ([195.130.137.89]:48134) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvME-0005Fn-Ee for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:34:06 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by laurent.telenet-ops.be with bizsmtp id ola42601814dU1001la4fL; Sat, 29 Jan 2022 22:34:05 +0100 Message-ID: <fbe6d8d556a796b580d0263cac87369bdc62b0d7.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:33:59 +0100 In-Reply-To: <20220125192201.7582-6-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-aR6W/SLDqG5Ad2gUs6YW" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643492045; bh=xpxollq1r5+Oc7W4cL3DwDm8rEntxSO9rINz5KpMr/A=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=fLo9RbtGv1ST+Cs5kXM2Yjm2C31voxokXAEE+vV+GApbgskEiv6jFypcCgwzCfEXP pY6b0z2T8aZsWbS3VvGnAAJ/Z3AODx63un5giYTUMU/DEyHxcGFhujiTyl5x1XqWaF jg8RrT9FBw3wbtBpp7fYwaWGjdIWj5vSAi3WtBWz6LNbutLBU8uVQ1vu7STAp0MFRw o0UN4z5kgNmK7wnKehR+yMqyNnTu5jYlRjRk1ahBtY863P9/ulcrgEaHf6A4QfRnIT BVMOorJBMiO2bQOdD8HIO6Ggl7BH3V/yxocbED7j5D8XKyvTZtghfd/I4jFP/q/0iW q5qeLeO8zIkgw== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-aR6W/SLDqG5Ad2gUs6YW Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > + Superfluous new empty line? > =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (else > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (leave (G_ "unsupported substitute = URI scheme: ~a~%") > -=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 (uri->string uri))))) > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (if (and (eris-read-capability? uri= )) guile-eris (which has the eris-read-capability? procedure) is an optional dependency, so you have to check here is Guix was compiled with guile-eris. Greetings, Maxime. --=-aR6W/SLDqG5Ad2gUs6YW Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWyxxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7v7+AQCV4ylW2AvzZfYFWUufqDp6Fm2+ neoJm3Ip51Rj9gX/EgD9GWPVNtgpSMGfs3nbd9Vg2785Z3K0SsseF3FGI8o/sgA= =1GlX -----END PGP SIGNATURE----- --=-aR6W/SLDqG5Ad2gUs6YW--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:39:02 +0000 Resent-Message-ID: <handler.52555.B52555.164349231120638 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349231120638 (code B ref 52555); Sat, 29 Jan 2022 21:39:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:38:31 +0000 Received: from localhost ([127.0.0.1]:35028 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvQU-0005Mo-Pi for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:38:30 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:47554) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvQT-0005Mc-74 for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:38:29 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by michel.telenet-ops.be with bizsmtp id oleT2600814dU1006leT4U; Sat, 29 Jan 2022 22:38:28 +0100 Message-ID: <0b0401a711d2196400b05a7f98ef5b757c99bf4b.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:38:27 +0100 In-Reply-To: <20220125192201.7582-6-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-jej2Spq+dGk6RML4YgrZ" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643492308; bh=8XWbywET1IiBaaXFd/KhWw3Hvk2XnfQVGcsQ/0cp73E=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=kXZMAxPHC0jpQuxqdpDxP6Yqfw4C4Q6xy+/13BHooFj4S7yay5Xnj1it0z2PZsIRn 1Aw4o5bBbRMPftneIEbuz2qsJAl0vTMtMoqEKs3i0gu+wjZdT7Scn3Sz7uMzfwyFIX 1/m+muS5CNWyUaK2D2OVekLECLcYPirqCDw9mBvQQZMX3EZb30G+faRSTrJKZXBwq+ lsGs/eAlwO1+Iytel5TaFHmV8qD84rvYeYXrmyOTPazuZ2S3//nxk8O0cJSzTig1y8 Xvdj/wzoXJC6Kyw++PcPgkZkP1BHhA7MVn18ENRvjH/LPPs6hH2euZpnfvqDuopiSL 5zR9pTKC2Nufg== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-jej2Spq+dGk6RML4YgrZ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: base64 cHVra2FtdXN0YXJkIHNjaHJlZWYgb3AgZGkgMjUtMDEtMjAyMiBvbSAxOToyMiBbKzAwMDBdOgo+ ICvCoCAoZGVmaW5lKiAoYmVzdC11cmkgbmFyaW5mbyAjOmtleSAoZXJpcz8gI2YpKQo+ICvCoMKg wqAgKGlmIChhbmQgZXJpcz8gKG5hcmluZm8tZXJpcy11cm4gbmFyaW5mbykpCj4gK8KgwqDCoMKg wqDCoMKgICh2YWx1ZXMgKG5hcmluZm8tZXJpcy11cm4gbmFyaW5mbykgInpzdGQiICNmKQo+ICvC oMKgwqDCoMKgwqDCoCAobmFyaW5mby1iZXN0LXVyaSBuYXJpbmZvICM6ZmFzdC1kZWNvbXByZXNz aW9uPwo+ICvCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCAlcHJlZmVyLWZhc3QtZGVjb21wcmVzc2lvbj8pKSkKCldoZW4gR3VpeCBpcyBjb21waWxlZCB3 aXRob3V0IGd1aWxlLWVyaXMgc3VwcG9ydCwgCicoYW5kIGVyaXM/IChuYXJpbmZvLWVyaXMtdXJu IG5hcmluZm8pKScgaXMgdGhlIHdvcnN0LCBub3QgdGhlIGJlc3QuCgoK --=-jej2Spq+dGk6RML4YgrZ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfWz0xccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hfIAP4gQtcdPHk1ATJQ6Y2i1FdWxKRD Hzwh+VxRPOOqAoleBQD+N+YpX65UT4K6zjdj7rmVzn/44mwqm+RRlGa2Kb0NeQg= =9kGm -----END PGP SIGNATURE----- --=-jej2Spq+dGk6RML4YgrZ--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:41:02 +0000 Resent-Message-ID: <handler.52555.B52555.164349243420905 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349243420905 (code B ref 52555); Sat, 29 Jan 2022 21:41:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:40:34 +0000 Received: from localhost ([127.0.0.1]:35038 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvSU-0005R6-9v for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:40:34 -0500 Received: from laurent.telenet-ops.be ([195.130.137.89]:52972) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvSS-0005Qy-KJ for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:40:33 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by laurent.telenet-ops.be with bizsmtp id olgW2600M14dU1001lgX5B; Sat, 29 Jan 2022 22:40:31 +0100 Message-ID: <d6bb147c902c0067d923321daf6d8db4a06fa537.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:40:30 +0100 In-Reply-To: <20220125192201.7582-6-pukkamustard@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-ek3YXnmloixQL2tm2CxQ" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643492431; bh=+O7HSp8wPGV0pNTLTilamPfXqSDSmsRTGpHFfOwZlro=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=ibm1jB219TZygAtI8ytOjFTjkDAct+Uw+x83xEAWhjgW9r7mwm1gMc6WNDP/v5PFH 4C6vSCRugBNKvfv8jlr7LNhMExxPKp/MITtEyWRMxsAiWd4xaPBxWKvYSUwAkmTq48 oxlcRNd7mtk51GNd5q85WhKAiwlexk8eh7k2MVRmN2kMZj90ciK02Vif4WCqiSwIXp /4XqfXPRaDCNo1vxBFHgelv0pEhqzgQJmI7xF33CzYgHecqkGdzAlAmtVsKFYBw+z/ jDnNLwrrXmxQk2jJD1cxlf44q22NjKpDs9xhM1IN67DQgXWtIvMNlVYvt8asRcdSQ9 XKKXJ+zwvInAg== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-ek3YXnmloixQL2tm2CxQ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > +=C2=A0 (define* (best-uri narinfo #:key (eris? #f)) > +=C2=A0=C2=A0=C2=A0 (if (and eris? (narinfo-eris-urn narinfo)) > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (values (narinfo-eris-urn nar= info) "zstd" #f) > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (narinfo-best-uri narinfo #:f= ast-decompression? > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0 %prefer-fast-decompression?))) Why is ERIS the best here? Fast download speeds, decentralisation, less network I/O, less heat production, more pronouncable than HTTPS? I would add a comment here. Greetings, Maxime. --=-ek3YXnmloixQL2tm2CxQ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfW0ThccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rl8AP9JdJvZVUngfsqUuONq1psXpouT FB5+4GT1KNZ8p15qwgEA0BH/56Hi+ezFF4RuMfpqYZkY/yF7KtM6j0a0va/qNgo= =48HT -----END PGP SIGNATURE----- --=-ek3YXnmloixQL2tm2CxQ--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 21:41:02 +0000 Resent-Message-ID: <handler.52555.B52555.164349246120950 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349246120950 (code B ref 52555); Sat, 29 Jan 2022 21:41:02 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 21:41:01 +0000 Received: from localhost ([127.0.0.1]:35042 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDvSv-0005Rq-IX for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:41:01 -0500 Received: from albert.telenet-ops.be ([195.130.137.90]:33302) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDvSu-0005Ri-3n for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 16:41:00 -0500 Received: from [172.20.10.9] ([188.188.235.49]) by albert.telenet-ops.be with bizsmtp id olgx2600P14dU1006lgywd; Sat, 29 Jan 2022 22:40:58 +0100 Message-ID: <e0156ce82b5e9a990d05c0b7bfb6626af5ce171a.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sat, 29 Jan 2022 22:40:57 +0100 In-Reply-To: <0b0401a711d2196400b05a7f98ef5b757c99bf4b.camel@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> <0b0401a711d2196400b05a7f98ef5b757c99bf4b.camel@HIDDEN> Content-Type: text/plain; charset="UTF-8" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 Content-Transfer-Encoding: 8bit DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643492458; bh=swPZZMTGVpOmodbcuvN7XQcaY0RWG76jo7j//usFXhc=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=Id0oa5TA8w5P4hzxbNPbTsMJQTkv4h56kbWE+TlFBRrHjbrGe/dVr9+NG/fftRx81 7augHSvvo5FSNPjSrRgsZm3uHeQk/NuHX1vOVtpttLnJVY2KQ7nOFVIv5WawycjX7o 7sGn30XTvZORTzNwWjiTwD2KU/uo6JIPAIKD4SQ002HME2CtlbIiJ4FbJTpb23dzzT Uc/bm1r/nu39DESzkD5SQweI9YkVMMhRDi2RZqLygzFXqHtFGnrqTT3C1DizyvkI8f T3Tm07NttJPJBD4B12ceexGeGJkrZkGexvVkAuU7C/nIUHm/yERrl4905sODN+4b5K FeSSLp8hHOo4Q== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Maxime Devos schreef op za 29-01-2022 om 22:38 [+0100]: > pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: > > +Â (define* (best-uri narinfo #:key (eris? #f)) > > +Â Â Â (if (and eris? (narinfo-eris-urn narinfo)) > > +Â Â Â Â Â Â Â (values (narinfo-eris-urn narinfo) "zstd" #f) > > +Â Â Â Â Â Â Â (narinfo-best-uri narinfo #:fast-decompression? > > +Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â Â %prefer-fast-decompression?))) > > When Guix is compiled without guile-eris support, > '(and eris? (narinfo-eris-urn narinfo))' is the worst, not the best. Nevermind, that's what (eris? #f) is for, I presume? Greetings, Maxime.
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sat, 29 Jan 2022 22:19:01 +0000 Resent-Message-ID: <handler.52555.B52555.164349469024793 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164349469024793 (code B ref 52555); Sat, 29 Jan 2022 22:19:01 +0000 Received: (at 52555) by debbugs.gnu.org; 29 Jan 2022 22:18:10 +0000 Received: from localhost ([127.0.0.1]:35091 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nDw2r-0006Rp-SD for submit <at> debbugs.gnu.org; Sat, 29 Jan 2022 17:18:10 -0500 Received: from baptiste.telenet-ops.be ([195.130.132.51]:34448) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nDw2o-0006Rd-Ts for 52555 <at> debbugs.gnu.org; Sat, 29 Jan 2022 17:18:07 -0500 Received: from [172.20.10.5] ([5.23.227.239]) by baptiste.telenet-ops.be with bizsmtp id omJ22600E5AYamV01mJ2w6; Sat, 29 Jan 2022 23:18:05 +0100 Message-ID: <73b50ffdca94407ef9fd7ef4875985a3b1c3c568.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-SwekApwKll48sZBJcCBy" Date: Sat, 29 Jan 2022 22:52:49 +0100 MIME-Version: 1.0 User-Agent: Evolution 3.38.3-1 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643494685; bh=IyDH6w3YbsS24u/d8ZJUzQq9ZEAoSyK3ObVp/Dvz80Y=; h=Subject:From:To:Cc:In-Reply-To:References:Date; b=U/WM71znY+1K3HGxcl0s5aFDGvZNJmpIwdctEE+bUs2+hKjQH90LZ3bc1eySUOvjX Fp0nW8MPpOaj1vo6UpwIYc8Dl3FAqUQPsq6KTq0Oerw5DzeBtKPpK3PpEzb076XoP0 JvCpVkfuLBwh7AwLRhtfi+y5tNdfbXfc4Co3t81wnYXRi8rAd87ePkLxkl75kL4zKU e2igzjn3f05BAR8lhiM5u19aODOkaOQTXE8kmmOzQpyd68Ndl28FtroSaH6QMet1nC XgMnExEEh1ahfBIjbqHW5vyXOgDLx5/396HLVfOoKaETc30oKFbMU2FgVGzsi/iwTL ZCqdD13PHuviw== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-SwekApwKll48sZBJcCBy Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable Hi, Is it possible for the following situation to happen? If so, why not? 1. server A is authentic 2. server M is malicious, it tries to trick the client into installing an incorrect substitute 3. (key of) server A is authorised 4. (key of) server M is _not_ authorised 5. server A and M are both in substitute-urls 6. server A only serves =E2=80=98classical=E2=80=99 substitutes, server B= also serves via ERIS+ipfs 7. Both A and M set the same FileHash, References, etc. in the narinfo 8. However, M set an ERIS URN pointing to a backdoored substitute. 9. The client trusts A, and A and B have the same FileHash etc., so the client considers the narinfo of B to be authentic because it has the same FileHash. 10. The client prefers ERIS above HTTP(S), so it downloads via M. 11. The client now installed a backdoored substitute! Greetings, Maxime. --=-SwekApwKll48sZBJcCBy Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfW3KhccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hJxAQDlxH9zz6F8LpkiMv07Hcyt35CG YFb2CeCw2dbFO7qXNwD9HeDCIHVHzYQD/EvcGcYhQIocR7cUvrSnvPyFVHFErQ8= =4RmQ -----END PGP SIGNATURE----- --=-SwekApwKll48sZBJcCBy--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Sun, 30 Jan 2022 11:47:02 +0000 Resent-Message-ID: <handler.52555.B52555.164354321028488 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN>, 52555 <at> debbugs.gnu.org Cc: ~pukkamustard/eris@HIDDEN Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164354321028488 (code B ref 52555); Sun, 30 Jan 2022 11:47:02 +0000 Received: (at 52555) by debbugs.gnu.org; 30 Jan 2022 11:46:50 +0000 Received: from localhost ([127.0.0.1]:35689 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nE8fS-0007PQ-13 for submit <at> debbugs.gnu.org; Sun, 30 Jan 2022 06:46:50 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:44058) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nE8fO-0007PC-6h for 52555 <at> debbugs.gnu.org; Sun, 30 Jan 2022 06:46:48 -0500 Received: from [172.20.10.5] ([5.23.227.239]) by michel.telenet-ops.be with bizsmtp id ozmh260065AYamV06zmiXb; Sun, 30 Jan 2022 12:46:44 +0100 Message-ID: <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Sun, 30 Jan 2022 12:46:23 +0100 In-Reply-To: <20220125192201.7582-1-pukkamustard@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-pmPqROV9QLMWjJsSQXtO" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643543204; bh=/mEX7Ex4Zj3x4I4x3VUqJnt9SI1nWoiWC4yhuNVD5PM=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=lCV8q6vmLstF+lYAkYlg52n65UwLSkBIedOOkxrwZ4LfMoA46q8yBqjIPxL9uMQwX NC4iwQnajmo9Njk4SvA71T3MANvSacSdk5+z6Bd1AYdVZEUoEI8v1reqYXa2H7OS6E l7lxSDq9XAUfA2971kCfncXjjfMJ2u2YnSKptU5wARuq45WEEgurFpovAPhxf86wNH /V4SUPjFqjwHA37scY/ubWwqcK5EEiF75Bqmpcbdgdr/eaA9pRx2XcOvF5M00MoOP1 YDhqng6+MSIwCD1fU1xv+tySl1z1aLAZ3rGIfQ89adwmFK56bPdRBGiqmm/r1KXB6G eYcTFNYzJMSTA== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-pmPqROV9QLMWjJsSQXtO Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > I have only tested this for fairly small packages (up to a few MB). >=20 > One issue with IPFS might be that we have to create a new HTTP connection= to > the IPFS daemon for every single block (32KiB). The IPFS daemon does not= seem > to support HTTP connection re-use According to <https://github.com/ipfs/go-ipfs/issues/3767>, the IPFS daemon supports connection reuse according to some people and doesn't according to other people. > and neither does the Guile (web client). Guix supports connection reuse, see 'call-with-cached-connection' in (guix scripts substitute). > I fear this might become a performance issue. IIUC, the performance problem primarily lies in the round-tripping between the client and the server. If the client and the server are on the same machine, then this round trip time is presumably small compared to, say, localhost contacting ci.guix.gnu.org. Still, connection reuse would be nice. > It seems possible to use IPFS more directly by exposing the Go code as a > C library and then using that with the Guile FFI [1]. This is however a b= it > complicated and adds a lot of dependencies. In particular, this should no= t become > a dependency of Guix itself. The performance of IPFS itself also needs to= be > evaluated, maybe the IPFS HTTP API will not be the bottle-neck. Security-wise, libipfs doesn't seem great: libipfs starts the IPFS daemon inside the process and guix/scripts/substitute.scm is run as root. > As mentioned in previous mail a simple HTTP transport for blocks would be= a > good fallback. This would allow users to get missing blocks (things that > somehow got dropped from IPFS) directly from a substitute server. [...] Seems a good idea to me -- DHTs can be unreliable. I presume this will be implemented with some kind of timeout: if no block is received within N seconds, fallback to HTTP? Also, don't forget to insert this missing block back into IPFS/GNUnet/BitTorrent/..., otherwise less and less blocks will be available until nothing is available anymore. > In any case, it would be necessary for the substitute server to store enc= oded > blocks of the NAR. For this I think it makes sense to use a small databas= e. We > have bindings to use ERIS with GDBM [2]. It might also make sense to use > SQLite, especially if there are other use-cases for such a database. Wouldn't this be a huge database? IIRC, according to logs.guix.gnu.org the size of the nars of the substitute servers are somewhere in the 200G-2T range or something like that. To reduce the size of the database, perhaps you could let the database be a mapping from block ids to the name of the nar + the position in the nar, and encode the block on-demand? The database doesn't seem necessary, the substitute server could have some end-point /publish-this-nar-again-into-IPFS/name-of-the-nar which, when contacted, inserts the nar again into IPFS. Then when a block was unavailable, the client contacts this end-point and retries. Greetings, Maxime. --=-pmPqROV9QLMWjJsSQXtO Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfZ6jxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7rANAP9328I61zxzX3OGMvtg/yulC2fN hA2FYGhhmzeZUgJfWAD+Nz3A9TmanG1T0/QvW6bgstsy7kILYqCvXo7xF8onkAM= =psyK -----END PGP SIGNATURE----- --=-pmPqROV9QLMWjJsSQXtO--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 09:55:01 +0000 Resent-Message-ID: <handler.52555.B52555.164379567931139 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164379567931139 (code B ref 52555); Wed, 02 Feb 2022 09:55:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 09:54:39 +0000 Received: from localhost ([127.0.0.1]:51467 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFCLX-00086A-Ej for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 04:54:39 -0500 Received: from mout02.posteo.de ([185.67.36.66]:50955) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFCLU-00085s-7i for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 04:54:38 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id E72E5240101 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 10:54:29 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643795669; bh=fAUUkVuINH8YzKXULcEe+y9Xtj8OdNtRiMy0UlbOOJ8=; h=From:To:Cc:Subject:Date:From; b=RmXahmNnI2ICfPLlceweypZQaiP8Qw0QOhBd/yNykkLmSSiHxdOFiFc0Zpgrq22h5 Lb73Rm8tb5DW1/qW4ZbKn+HdAt1g8dsDQ9Vnd7PXL4qSfXwnq2+DS9kuj9SRS8U0rC kcnYnnQcLcmj24iLPUQ3vYuU5xfaucCPJtdRHlhGiJsjPbkjSuEBV2LYcGWqBQ1+AN w+7mYLqR4eW9Be9veB9NshUis7pCqpDfdacx7cBn3tXz9kb7GTda0kgAojKrU1k7V2 yncuwPmGq+8//UUCwW/fxK3kn+83pAAvFWeRJdd800PUAli3fuk021LyOS4hwIrqtc lyvFJcbDq4dyQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Jpcb84WNbz6tnc; Wed, 2 Feb 2022 10:54:28 +0100 (CET) References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <cc7ec17cded7fda6689d23a163df2ba72058a8aa.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 09:50:34 +0000 In-reply-to: <cc7ec17cded7fda6689d23a163df2ba72058a8aa.camel@HIDDEN> Message-ID: <86a6f9iofg.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi Maxime, Maxime Devos <maximedevos@HIDDEN> writes: > There's an ipfs-service-type nowadays, so starting the daemon manually > isn't required (if using Guix System). Good point. Starting the daemon manually is only necessary if you don't use the service. I don't use the IPFS service. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 10:10:01 +0000 Resent-Message-ID: <handler.52555.B52555.164379655332684 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164379655332684 (code B ref 52555); Wed, 02 Feb 2022 10:10:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 10:09:13 +0000 Received: from localhost ([127.0.0.1]:51498 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFCZb-0008V4-Kh for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:09:13 -0500 Received: from mout01.posteo.de ([185.67.36.65]:46233) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFCZZ-0008Un-7n for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:09:10 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id C3FD2240026 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 11:09:02 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643796542; bh=32ie/3pdSKouEdEYNKHniVjHfyhqM6t6P+DS7WEuZ9A=; h=From:To:Cc:Subject:Date:From; b=k4Ow1+rBCerL6PiRhqvHix8wAFQ9p7YtEGO1pwuJgo5ZQgmg+ByCpsM+doh0yYUkT sFYDucfyTebypl3/9T+a8YqXqwM3uvFii9QSlQaq3nay5THV4bsqw/ZLZEdeAEV32H eD98BWymfgNh6F2ksbBrcjzNGv9uM85nXHmlNixBW8VlPz7oeVsPiJPFhLVJyqqwqr JdeFQj3g9HROSjZ7xDz1P4XQRQWXkGLS/lK5rI5+ebKUd6oWEKhPFnR7PztQBWqI/z uAOhC9nVCtw1N0h6MuFeFzygexsGlL4+CjTxHy8kZMgSLh0IuWcmXNbx6OLW3QDNMs rP+B2FWIl7JBA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Jpcvx5Jcfz6tm6; Wed, 2 Feb 2022 11:09:01 +0100 (CET) References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <37651a21b0699b78b0282284b7d7322cb1a8c320.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 09:56:50 +0000 In-reply-to: <37651a21b0699b78b0282284b7d7322cb1a8c320.camel@HIDDEN> Message-ID: <865ypxinr8.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > [[PGP Signed Part:Undecided]] > pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: >> I will be looking into the HTTP fallback and also using BitTorrent and GNUNet >> as transports. > > I have been writing a (Guile) Scheme port of GNUnet's client libraries > (https://git.gnunet.org/gnunet-scheme.git/). Currently only NSE is > supported, but I'm working on DHT. DHT search/put already works to a > degree (see examples/web.scm), but there are plenty of sharp edges > (see TODOs about disconnecting, reconnecting and stopping fibers, > and see guix.scm for Guile bugs that are patched out and extra guile- > fibers features). Very interesting! I have been following your work on that a bit. From what I understand gnunet-scheme interacts with the GNUNet services and sends messages to the various GNUNet services. Is that correct? Have you considered implementing the GNUNet protocols themeselves in Guile? I.e. instead of connecting with the GNUNet services and sending messages, implement R5N completely in Guile. IMHO this would be very nice as one could use GNUNet protocols completely in Guile and not rely on the GNUNet C code. I believe this is somewhat the direction being taken with the GNUNet Go implementation (https://github.com/bfix/gnunet-go) and also in line with recent efforts to specify the individual GNUNet components and protocols more independantly of one another (e.g. R5N is specified to work over IP - https://lsd.gnunet.org/lsd0004/). -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 10:16:01 +0000 Resent-Message-ID: <handler.52555.B52555.1643796915898 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.1643796915898 (code B ref 52555); Wed, 02 Feb 2022 10:16:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 10:15:15 +0000 Received: from localhost ([127.0.0.1]:51502 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFCfT-0000EQ-IF for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:15:15 -0500 Received: from mout02.posteo.de ([185.67.36.66]:57605) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFCfS-0000E8-1m for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:15:14 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 1867C240103 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 11:15:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643796908; bh=F24uZxUR4m0QVFxi1PSJnCdTyIx4BfXDU2yMpXzp3Eg=; h=From:To:Cc:Subject:Date:From; b=AITTuw9Lej+lm1oZ3xSAgd20JXrXi7cfFZ2vUT4W3Acqrjfaott1ZOH/TAUMWiSLO h49scgS2+1Lq0Qyva3WbPEWASp3Hbd4A8Tcp2T/ubxVpS7cDWw58YqMiQJkbH8DaxK BeWF/oZ7hHr+aeUuP/YRiI2ozPtkkc1+FnaL9fd7HLeRdXaiup+lyxER79LMf4RnEb mcRIXYyVz7bHoCtRs3EX4E2MdTHCCfAvwBo2nC4hYk1YqKV78HV1EWOd+L1Or3E1bS lXv66xaAmU5PCy8bRUu/Nc+UdJMwTO/Blb7FtTJFyXAPLZ2cZtm70SFxYnnvs1gVHm SfT+wFQ4odTHQ== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Jpd2y5QZzz6tpG; Wed, 2 Feb 2022 11:15:06 +0100 (CET) References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> <00bb85fa4fb62498b354ecb78d248df5209b6f29.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 10:11:18 +0000 In-reply-to: <00bb85fa4fb62498b354ecb78d248df5209b6f29.camel@HIDDEN> Message-ID: <861r0linh1.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > [[PGP Signed Part:Undecided]] > pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: >> +(define (guix-eris-block-ref ref) >> +=C2=A0 "Dereference a block for decoding ERIS content" >> +=C2=A0 (eris-blocks-ipfs-ref ref)) > > 'guix-eris-block-ref' just calls 'eris-blocks-ipfs-ref', > so I'm not seeing the point of this procedure. Yes, currently it is an unnecessary level of abstraction. The idea is that when there are multiple backends/transports they are multiplexed here. E.g. guix-eris-block-ref would attempt to use HTTP/IPFS or whatever to get the block. Whatever calls guix-eris-block-ref does not need to know from where the blocks come. I hope to make this more clear in a V3 that will add HTTP transport. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 2/5] publish: Add ERIS URN to narinfo Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 10:25:02 +0000 Resent-Message-ID: <handler.52555.B52555.16437974491812 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16437974491812 (code B ref 52555); Wed, 02 Feb 2022 10:25:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 10:24:09 +0000 Received: from localhost ([127.0.0.1]:51522 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFCo5-0000TA-6T for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:24:09 -0500 Received: from mout01.posteo.de ([185.67.36.65]:58289) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFCo3-0000Sf-FJ for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:24:08 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 2576F240028 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 11:24:00 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643797441; bh=rp6NqHJ5LqbUHJ89ygY77J91IX+CrsOiM8WOteMtTTE=; h=From:To:Cc:Subject:Date:From; b=QLxFHD+YFOOKlXJqaYCOubZ/fRujoADsSISKGY/AXVque2hyQN4Wtha2R90cbKixL lrZQzHCdRkFa1orooW3TksBUYCevZhFMUjBKYTcc5/NQd2Y/Mbz4ldWnQ4jjjN8RIi bcMpIQREXJXqVGMtG6JZ/JNK5jOmED/XtSI1n3hJGfavCxlBYHeKiG9d2zQixxGHeV db1iBr3Vm5YqxoiFiqydhArqaCeV5Pyal+CflF9+s6EZL2ph+YrYaHAGd5ZS4f60U9 QkOXSMvCWj1CxqxHU1Ky59ubMj/v4VfjAqwkdGp2UCz83noMJJYYLtEb3FArGeD6Nf /lTGJtoNjjniA== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JpdFC1N1Jz6vGj; Wed, 2 Feb 2022 11:23:58 +0100 (CET) References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-3-pukkamustard@HIDDEN> <46558e7148b91f12cdaa92f32adb91bbcb3080bd.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 10:16:12 +0000 In-reply-to: <46558e7148b91f12cdaa92f32adb91bbcb3080bd.camel@HIDDEN> Message-ID: <86wnidh8ht.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: >> +=C2=A0 (define (eris-encode-nar compressions) >> +=C2=A0=C2=A0=C2=A0 (and (member %eris-zstd-compression compressions) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (let* ((nar (nar-cache= -file cache item >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 #:= compression %eris-zstd-compression)) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 (stat (stat nar #f))) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (and stat >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0 (call-with-input-file nar >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (cut eris-encode->string <> >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 #:block-siz= e %eris-block-size-large)))))) > > Why are exceptions turned into #f (in (stat nar #f))? > Should this be done for all I/O errors, including, say, EOVERFLOW, > ENOMEM or ENAMETOOLONG, or only for ENOENT? > > Is a race condition possible here? If so, maybe consider doing > something like > > (catch 'system-error > (lambda () (call-with-input-file ...)) > (lambda exception > (and it-is-a-ENOENT > (apply throw exception)))) > > to avoid it? A valid question. But (stat nar #f) is not something I introduced. It is already in guix/scripts/publish.scm like that. To me turning all exceptions to #f makes sense. Here we only want to know if the file is readable. As the NAR is baked in the background by another thread the case where the compressed NAR does exist yet will happen. In that case we don't worry, we just don't publish the `FileSize` and `ERIS` fields in the narinfo. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 3/5] Add (guix eris). Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 10:37:01 +0000 Resent-Message-ID: <handler.52555.B52555.16437982163121 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16437982163121 (code B ref 52555); Wed, 02 Feb 2022 10:37:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 10:36:56 +0000 Received: from localhost ([127.0.0.1]:51537 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFD0S-0000oH-91 for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:36:56 -0500 Received: from mout02.posteo.de ([185.67.36.66]:42363) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFD0Q-0000o2-L3 for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:36:55 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 56048240104 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 11:36:48 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643798208; bh=QdYp9eTScxciOIUsbmQH0IFCxYyIGa4dEgifTI4NgjI=; h=From:To:Cc:Subject:Date:From; b=nE9f8POhH4rim3g8leJroMQjVo291+DN3Tz3cl5Eg03pWq7nAfb+j2q9hHesm71pb kz4MEughIDred5LHEfUBWBuUKl4ZtZkvGZYq4Va2UNojKtvJ/T/bLaG0QvR0xKX5lx 47aRfMefBC3Kup7xnWVV74R1T5YSON+Mia6J5qGOsKdD7E2GHNoeVtck6BHV5DvREc w4318Xvskg7PXn0MLTzV2jHl59LQaP+GtBzpFj0e+4Zie8AWuz/xk1xTQG4/XNHHcm GLLmIh4nPJLnwXKFo31qChIoUT51nk68YOtm2hOR7CMK3qtdbBT9D7K5JDj72Fzoau 3U1dZhCXcHb9A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JpdWz1gvwz9rxX; Wed, 2 Feb 2022 11:36:46 +0100 (CET) References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-4-pukkamustard@HIDDEN> <0b977ae15a49b051c04922d234149a8e4762404f.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 10:28:18 +0000 In-reply-to: <0b977ae15a49b051c04922d234149a8e4762404f.camel@HIDDEN> Message-ID: <86sft1h7wh.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: >> +(define (ipfs-daemon-alive?) >> +=C2=A0 "Attempt to connect to the IPFS daemon. Returns #t if the daemon= is alive >> +and #f else." >> +=C2=A0 (with-exception-handler >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (const #f) >> +=C2=A0=C2=A0=C2=A0 (lambda _ >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (let ((response _ >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (http-post (strin= g-append (%ipfs-base-url) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 "/api/v0/ve= rsion")))) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (equal? 200 (response-code r= esponse)))) >> +=C2=A0=C2=A0=C2=A0 #:unwind? #t)) > > This should preferably only be catching exceptions indicating that > the daemon is down (exceptions indicating 404s, or system-errors > indicating network errors, ...). Yes, I guess it could be checked a bit finer. But at the end if an exception happens then the IPFS daemon is probably not reachable, right? If we don't care about the reason why it is not reachable then why bother with catching finer grained exceptions? >> + >> +(define guix-eris-block-reducer >> +=C2=A0 (case-lambda >> + >> +=C2=A0=C2=A0=C2=A0 ;; Check if IPFS Daemon is running. >> +=C2=A0=C2=A0=C2=A0 (() (if (ipfs-daemon-alive?) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (eri= s-blocks-ipfs-reducer) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 #f)) >> + >> +=C2=A0=C2=A0=C2=A0 ;; Completion. Nothing to do. >> +=C2=A0=C2=A0=C2=A0 ((_) #t) >> + >> +=C2=A0=C2=A0=C2=A0 ((ipfs ref-block) >> +=C2=A0=C2=A0=C2=A0=C2=A0 ;; If IPFS has been initialized store block th= ere >> +=C2=A0=C2=A0=C2=A0=C2=A0 (if ipfs >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (eris-blocks-ipfs-redu= cer ipfs ref-block) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 ipfs)))) > > This (ipfs-daemon-alive?) seems racy, although it's probably not. > Can we do > > (define guix-eris-block-reducer > (case-lambda > (() (guard (c (oops-it-fails-because-the-daemon-cannot-be- > contacted? c) > #false) > (eris-block-ipfs-reducer)) > [...])) > > instead? (I don't think this will work as-is, because from the name and > thunkiness, it would appear that eris-block-ipfs-reducer returns a > procedure ... Yes, eris-block-ipfs-reducer returns and SRFI-171 reducer. This is a 3-arity procedure that is either initialized, called with a block to reduce and finalized. The #f that the initialization case returns (0-arity call) is the state of the reducer. In the block reducing case (2-ary call) the state (the ipfs variable) is checked if ipfs is alive. If not the blocks are just forgotten. So guix-eris-block-reducer always returns a SRF-171 reducer, regardless of if IPFS is alive or not. This is important as the ERIS URN can still be computed without the IPFS daemon running. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 4/5] publish: Add support for storing ERIS encoded blocks to IPFS. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 10:38:02 +0000 Resent-Message-ID: <handler.52555.B52555.16437982703240 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16437982703240 (code B ref 52555); Wed, 02 Feb 2022 10:38:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 10:37:50 +0000 Received: from localhost ([127.0.0.1]:51546 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFD1J-0000q8-Q9 for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:37:50 -0500 Received: from mout02.posteo.de ([185.67.36.66]:38015) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFD1G-0000pi-Q3 for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:37:47 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 31151240107 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 11:37:41 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643798261; bh=PoygJClA7JDK1t3vahn3hmtLFupTyzon0CNSvF/BarU=; h=From:To:Cc:Subject:Date:From; b=FPM5s/MNtRYc9rGOqj2ZYR5EHqc+nlI01ikSGiLeHJvv4A4dDCZP8LkXAJDrAm8N+ bHIgBQYyyvLVh5d9/1WIHJzjT28oehQo20GpvJfKDb5UPFVqg/T86Xz9OZf0+c7U1c QhDEhNRXZxv9WSQ/TS/ixXaPw9JyoR2L8H5841Fb9yogtz/GuLLLWg/8dsd2OEHFQh pU7nhjFEhK/7XbQqFMkBM6TBsEBH4ia5R10WiyHUzsWV8L+JO5BOZqAnbbqhrHmp18 ZWexnScvORyJQ5+GThc/W78oSJddW0kXyhBJhPU0ykfHEE/NMe8+ynYoIIbTLe0eg7 oE8UpmHjvLR1A== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JpdY02tPjz6tml; Wed, 2 Feb 2022 11:37:40 +0100 (CET) References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-5-pukkamustard@HIDDEN> <c6841903c177f934c7fc9d42a270907dbba59554.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 10:24:48 +0000 In-reply-to: <c6841903c177f934c7fc9d42a270907dbba59554.camel@HIDDEN> Message-ID: <86o83ph7uz.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: base64 X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) DQpNYXhpbWUgRGV2b3MgPG1heGltZWRldm9zQHRlbGVuZXQuYmU+IHdyaXRlczoNCg0KPiBbW1BH UCBTaWduZWQgUGFydDpVbmRlY2lkZWRdXQ0KPiBwdWtrYW11c3RhcmQgc2NocmVlZiBvcCBkaSAy NS0wMS0yMDIyIG9tIDE5OjIyIFsrMDAwMF06DQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqAgKGxldCAoKGVyaXMtdXJuIF8NCj4+ICvCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgIChlcmlz LWVuY29kZSBwb3J0DQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg ICM6YmxvY2stc2l6ZQ0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oCAlZXJpcy1ibG9jay1zaXplLWxhcmdlDQo+PiArwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgICM6YmxvY2stcmVkdWNlcg0KPj4gK8KgwqDCoMKgwqDCoMKgwqDCoMKgwqDC oMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKgwqDCoMKg wqDCoMKgwqDCoMKgwqDCoCBndWl4LWVyaXMtYmxvY2stcmVkdWNlcikpKQ0KPj4gKyAgICAgICAg ICAgICAgICAgICAgICAodXJpLT5zdHJpbmcgZXJpcy11cm4pKSkpKSkpKQ0KPg0KPiBJSVVDIChh bmQgcXVpdGUgcHJvYmFibHkgSSBkb24ndCwgYmVjYXVzZSBJJ3ZlIG9ubHkgYmVpbmcgcmVhZGlu Zw0KPiB0aGluZ3MgY3Vyc29ybHkpLCBlcmlzLWVuY29kZSByZXR1cm5zICNmYWxzZSB3aGVuIHRo ZSBJUEZTIGRhZW1vbiBpcw0KPiBkb3duIChiZWNhdXNlIHRoZW4gZ3VpeC1lcmlzLWJsb2NrLXJl ZHVjZXIgcmV0dXJucyAjZmFsc2UpLg0KDQpObywgZXJpcy1lbmNvZGUgd2lsbCBzdGlsbCByZXR1 cm4gdGhlIEVSSVMgVVJOLiBUaGUgYmxvY2tzIHdpbGwganVzdCBub3QNCmJlIHN0b3JlZCBpbiBJ UEZTIGlmIHRoZSBJUEZTIGRhZW1vbiBpcyBub3QgcnVubmluZy4gU2VlIGFsc28gbXkNCnJlc3Bv bnNlIHRvIHRoZSBxdWVzdGlvbnMgb24gZ3VpeC1lcmlzLWJsb2NrLXJlZHVjZXIgaW4gKGd1aXgg ZXJpcykuDQoNCi1wdWtrYW11c3RhcmQNCg==
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 5/5] substitute: Fetch substitutes using ERIS. Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 10:51:02 +0000 Resent-Message-ID: <handler.52555.B52555.16437990454742 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16437990454742 (code B ref 52555); Wed, 02 Feb 2022 10:51:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 10:50:45 +0000 Received: from localhost ([127.0.0.1]:51576 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFDDp-0001EP-AX for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:50:45 -0500 Received: from mout01.posteo.de ([185.67.36.65]:53719) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFDDl-0001E6-9Y for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 05:50:43 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 331DC240026 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 11:50:34 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643799035; bh=EYSRzA9xqxpfz816dUeBTgp5sO5TyZONbT2I+I5LehU=; h=From:To:Cc:Subject:Date:From; b=UO2e1DF0jBX5nUBoe7glrUPbpM4w7Umdi0/COu/gV/Tsscq04a6ujRLSFWhofE6Re 0iQS0cMgVsHatDmnc3hJdsGC1IzD/6WEiT495Bh6ElqwKKoFuSD2BDCE5c9eErs2L/ 7q8b1r8RS9cyGtlHbedO8ZMl/hl4oCFDBcmnnE5MlnH3SK+tJWUQhPiYgHgRTTokDC C6zp50gzsJICmuDj8XMDDYpj5J77Vin7Fpdasa+PtavgWi40sW10ecEj8INjrzkMtm OxH7HGmZnZ18mddZmmd7sNPpUh0piqnAUdcECet1Y3bsZffy+st6W/NEy6PBWod+eB d8y2iI8LQo1sg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4Jpdqs6YHZz6tq0; Wed, 2 Feb 2022 11:50:33 +0100 (CET) References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-6-pukkamustard@HIDDEN> <d6bb147c902c0067d923321daf6d8db4a06fa537.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 10:38:55 +0000 In-reply-to: <d6bb147c902c0067d923321daf6d8db4a06fa537.camel@HIDDEN> Message-ID: <86k0edh79j.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > [[PGP Signed Part:Undecided]] > pukkamustard schreef op di 25-01-2022 om 19:22 [+0000]: >> +=C2=A0 (define* (best-uri narinfo #:key (eris? #f)) >> +=C2=A0=C2=A0=C2=A0 (if (and eris? (narinfo-eris-urn narinfo)) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (values (narinfo-eris-urn na= rinfo) "zstd" #f) >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (narinfo-best-uri narinfo #:= fast-decompression? >> +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0 %prefer-fast-decompression?))) > > Why is ERIS the best here? Fast download speeds, decentralisation, > less network I/O, less heat production, more pronouncable than HTTPS? > I would add a comment here. Those are all possible reasons. I think we first need to do some experiments to see if any of those claims can be justified. In general, the logic for when to use ERIS transports needs more thought. For one, I think it should be user configurable. I can imagine that certain users do not want to use decentralized substitutes at all. Users should be able to deactivate the entire ERIS thing. For the default, I personally think it would be ok to try and use ERIS. The only thing we absolutely need is a clean fallback logic that transparently falls back to getting the entire NAR by HTTP. Currently such a fallback is not yet implemented. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 11:10:02 +0000 Resent-Message-ID: <handler.52555.B52555.16438001706732 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16438001706732 (code B ref 52555); Wed, 02 Feb 2022 11:10:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 11:09:30 +0000 Received: from localhost ([127.0.0.1]:51615 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFDVy-0001kW-AQ for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:09:30 -0500 Received: from michel.telenet-ops.be ([195.130.137.88]:38916) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nFDVs-0001kK-Vz for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:09:29 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by michel.telenet-ops.be with bizsmtp id qB9N260064UW6Th06B9NZD; Wed, 02 Feb 2022 12:09:22 +0100 Message-ID: <3d9821d7eac6c81a5b1c8a5304a78ec733b2dba4.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Wed, 02 Feb 2022 12:09:17 +0100 In-Reply-To: <865ypxinr8.fsf@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <37651a21b0699b78b0282284b7d7322cb1a8c320.camel@HIDDEN> <865ypxinr8.fsf@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-noX5Y9B6YAsVjwOxOeWJ" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643800163; bh=d45rL6MZKnyXVm+Ug4TV+anvWpdYdx3V7sd4kQXVO7Y=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=nI24bLWVPf9T5fiulJEh819cP8e3goFFfMkxexawOZMvjXFtOVgM7n39D3AhG22+V 1e6gfXvJn1DfrHYJYvHhmX/x8p+sizVGCOVz0woKfhoS6NLS+z22hNd1oeZIAj1/q4 52W5YOroIExw3UnJu2VZQ+Uxlv2bzZ6GxMzHj9yhCO/zVxNF3Tsm92TXOqu/LLF+Rp CWF8JY60K6Qt9v0ERxUU9NC05Wyxf+sXTh10h4oRO5hAQqVFYr92/HsuoCbv8JJTqU DcMBCdQV8wNo+aWgiS5wPMKHKSRHZfWdDyzAvgi17jQ3mtlNSnb6uB5HoyHriaEL71 e/G3XWnYPoiNw== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-noX5Y9B6YAsVjwOxOeWJ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op wo 02-02-2022 om 09:56 [+0000]: > Maxime Devos <maximedevos@HIDDEN> writes: >=20 > > [[PGP Signed Part:Undecided]] > > pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > > > I will be looking into the HTTP fallback and also using BitTorrent an= d GNUNet > > > as transports. > >=20 > > I have been writing a (Guile) Scheme port of GNUnet's client libraries > > (https://git.gnunet.org/gnunet-scheme.git/). Currently only NSE is > > supported, but I'm working on DHT. DHT search/put already works to a > > degree (see examples/web.scm), but there are plenty of sharp edges > > (see TODOs about disconnecting, reconnecting and stopping fibers, > > and see guix.scm for Guile bugs that are patched out and extra guile- > > fibers features). >=20 > Very interesting! I have been following your work on that a bit. >=20 > From what I understand gnunet-scheme interacts with the GNUNet services > and sends messages to the various GNUNet services. Is that correct? Yes, it works like the C GNUnet client libraries, except it's in Guile Scheme and a few different design decisions were made, e.g. w.r.t. concurrency. > Have you considered implementing the GNUNet protocols themeselves in > Guile? I.e. instead of connecting with the GNUNet services and sending > messages, implement R5N completely in Guile. I didn't, at least not _yet_. As-is, things are already complicated enough and the client code seems a lot simpler than the service code. Though perhaps in the future ... E.g., for testing the DHT service, the test code effectively creates a tiny, limited, in-memory DHT service (not communicating to any peers) that's buggy in some respects (not yet committed, but will be in tests/distributed-hash-table.scm). > IMHO this would be very nice as one could use GNUNet protocols completely > in Guile and not rely on the GNUNet C code. While it's not a priority, I'm not opposed to someday implementing the services in Guile and testing whether they can communicate with C peers. However,=C2=A0keep in mind that GNUnet is supposed to be able to eventually replace the TCP/IP stack, and running the DHT, NSE, NAT, FS, CADET, TRANSPORT ... services in every web browser, in every mail client, in all "guix substitute" and "guix perform-download" processes, etc. is rather wasteful (memory-wise and CPU-wise), so I'd prefer this not to be the _default_ option. (I'm not sure if you were referring to that.) Greetings, Maxme. --=-noX5Y9B6YAsVjwOxOeWJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfpmXRccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7mEeAPwOAn/O9DonokYiuo2Yq5+oLnHK fIr8apEPKdTzy/O5sAD/TeShsCEUX0+DdtJF1xDXsYY4uRDd1ubLC/dowbvehA8= =I6pm -----END PGP SIGNATURE----- --=-noX5Y9B6YAsVjwOxOeWJ--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 11:11:02 +0000 Resent-Message-ID: <handler.52555.B52555.16438002366882 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16438002366882 (code B ref 52555); Wed, 02 Feb 2022 11:11:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 11:10:36 +0000 Received: from localhost ([127.0.0.1]:51622 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFDX1-0001mw-VY for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:10:36 -0500 Received: from mout01.posteo.de ([185.67.36.65]:55259) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFDX0-0001mf-DN for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:10:35 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout01.posteo.de (Postfix) with ESMTPS id 70918240026 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 12:10:28 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643800228; bh=fhRCmo5kXYeo7IZGuTKrgKCHCAviFP71C3PQ+lrMfbE=; h=From:To:Cc:Subject:Date:From; b=jMoMRoCb96oBZPOANWNyemN3BQqa4Kf3dAp62rRiqFixY72XJnqcsBxmoDt0QNBLh NPi2fjpO0bEYCVIjJqkjRlWJyKU1WOStnXd3gslJcoYrUuvTSgJWaOzL5cGRS3LbfQ YHyw4GHDnOwvUS+m3WKcVh1T3vPlMRy9Gppj/jQfD8oDALJahA4t84KoBOZLlavWXu ykW4Y4gSB0M/wMDc/hdnzcxAh04Pp5p9MF7ig9CfKKSwdky68OOTWOOoMxR+/f2whq yKdsO8yA8cPDPpm0XuJz5bVKgAJtoLlkC5GnVc8hQhePFcXsb4pedQfXxyJRXU+dzu YL4RRGDXPMw6Q== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JpfGp68ZNz9rxh; Wed, 2 Feb 2022 12:10:26 +0100 (CET) References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 10:51:10 +0000 In-reply-to: <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> Message-ID: <86fsp1h6ce.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > [[PGP Signed Part:Undecided]] > pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: >> I have only tested this for fairly small packages (up to a few MB). >> >> One issue with IPFS might be that we have to create a new HTTP connection to >> the IPFS daemon for every single block (32KiB). The IPFS daemon does not seem >> to support HTTP connection re-use > > According to <https://github.com/ipfs/go-ipfs/issues/3767>, the IPFS > daemon supports connection reuse according to some people and doesn't > according to other people. Hm, from what I understand connection re-use is something introduced in HTTP/2 and go-ipfs does not do HTTP/2 (https://github.com/ipfs/go-ipfs/issues/5974). >> and neither does the Guile (web client). > > Guix supports connection reuse, see 'call-with-cached-connection' > in (guix scripts substitute). Ah ok. Cool! >> I fear this might become a performance issue. > > IIUC, the performance problem primarily lies in the round-tripping > between the client and the server. If the client and the server are on > the same machine, then this round trip time is presumably small > compared to, say, localhost contacting ci.guix.gnu.org. > > Still, connection reuse would be nice. Remains to be seen if this is a problem. It is considerably more pronounced than with regular usage of IPFS as we make a HTTP request to IPFS for every 32KiB block instead of for an entire file (what most people do when using the IPFS daemon). >> It seems possible to use IPFS more directly by exposing the Go code as a >> C library and then using that with the Guile FFI [1]. This is however a bit >> complicated and adds a lot of dependencies. In particular, this should not become >> a dependency of Guix itself. The performance of IPFS itself also needs to be >> evaluated, maybe the IPFS HTTP API will not be the bottle-neck. > > Security-wise, libipfs doesn't seem great: libipfs starts the IPFS > daemon inside the process and guix/scripts/substitute.scm is run > as root. I agree. >> As mentioned in previous mail a simple HTTP transport for blocks would be a >> good fallback. This would allow users to get missing blocks (things that >> somehow got dropped from IPFS) directly from a substitute server. [...] > > Seems a good idea to me -- DHTs can be unreliable. I presume this will > be implemented with some kind of timeout: if no block is received > within N seconds, fallback to HTTP? Yes, exactly. > Also, don't forget to insert this missing block back into > IPFS/GNUnet/BitTorrent/..., otherwise less and less blocks will be > available until nothing is available anymore. This might be a bit of a burden for users. As you mention the size of such a database might become considerable. >> In any case, it would be necessary for the substitute server to store encoded >> blocks of the NAR. For this I think it makes sense to use a small database. We >> have bindings to use ERIS with GDBM [2]. It might also make sense to use >> SQLite, especially if there are other use-cases for such a database. > > Wouldn't this be a huge database? IIRC, according to logs.guix.gnu.org > the size of the nars of the substitute servers are somewhere in the > 200G-2T range or something like that. > > To reduce the size of the database, perhaps you could let the database > be a mapping from block ids to the name of the nar + the position in > the nar, and encode the block on-demand? Yes! I've also been thinking of this - a "in-file" block store. I think this makes a lot of sense for Guix but also other things (e.g. sharing your music collection). Another problem with IPFS/GNUNet is that they have their own storage. So even if are clever about storing blocks in Guix, IPFS and GNUNet will have their own copy of the blocks on disk. I think it would be much nicer if DHTs/transport layers don't do block storage but are provided with a callback from where they can get stored blocks. I believe this is what OpenDHT does (https://github.com/savoirfairelinux/opendht/wiki/API-Overview). I think we should propose such a change to the GNUNet R5N specification (https://lsd.gnunet.org/lsd0004/). > The database doesn't seem necessary, the substitute server could have > some end-point > > /publish-this-nar-again-into-IPFS/name-of-the-nar > > which, when contacted, inserts the nar again into IPFS. Then when a > block was unavailable, the client contacts this end-point and retries. But for a HTTP block endpoint we would still need such a database/block storage. I think it is important that we do not rely on IPFS for block storage. The decentralized block distribution should work even if the IPFS daemon is not available. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 11:22:01 +0000 Resent-Message-ID: <handler.52555.B52555.16438008658047 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16438008658047 (code B ref 52555); Wed, 02 Feb 2022 11:22:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 11:21:05 +0000 Received: from localhost ([127.0.0.1]:51649 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFDhB-00025j-8r for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:21:05 -0500 Received: from mout02.posteo.de ([185.67.36.66]:46045) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFDh8-00024r-5n for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:21:04 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id 285FE240103 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 12:20:55 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643800856; bh=pNHT9Ob/g9tMWWqHoN2a1tCk6AY/iboxTEfny8CifK8=; h=From:To:Cc:Subject:Date:From; b=N9GnxVzwSKNW92Zg8Qgddb5eGbZUR1y30Js0tVxFjUUsaBBh1nNQbBh1r9ALMG4xQ sI/P0SzWhjb7IrYIFzZkrXWKc2UC8LzSQsl4VNdrrEQyKE6gjtQQihLyc9BspJccD2 ZuBQn7X9DWWnT1N6AU+z43svr1AEPSxFYjsmLxq3gekYezDIULwswVx2h293gAy9Dz XtJzVCaPQwLWBh6xhUxsISBAWb8R38Wwt0AXflRs88rlUG9VHyrLIR4DMvguyeF7aD t/Hbv4r8wgMmn07shRuQ+ZKWUCOI3DYZE//T4pxuK/FvNTjzNROmLIe7O6XeWihey8 Op04GXSyjTEog== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JpfVv0TMSz9rxG; Wed, 2 Feb 2022 12:20:54 +0100 (CET) References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <73b50ffdca94407ef9fd7ef4875985a3b1c3c568.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 11:10:58 +0000 In-reply-to: <73b50ffdca94407ef9fd7ef4875985a3b1c3c568.camel@HIDDEN> Message-ID: <86bkzph5ux.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: > [[PGP Signed Part:Undecided]] > Hi, > > Is it possible for the following situation to happen? > If so, why not? > > 1. server A is authentic > 2. server M is malicious, it tries to trick the client into > installing an incorrect substitute > 3. (key of) server A is authorised > 4. (key of) server M is _not_ authorised > 5. server A and M are both in substitute-urls > 6. server A only serves =E2=80=98classical=E2=80=99 substitutes, server= B also serves > via ERIS+ipfs > 7. Both A and M set the same FileHash, References, etc. in the > narinfo > 8. However, M set an ERIS URN pointing to a backdoored substitute. > 9. The client trusts A, and A and B have the same FileHash etc., > so the client considers the narinfo of B to be authentic > because it has the same FileHash. > 10. The client prefers ERIS above HTTP(S), so it downloads via M. > 11. The client now installed a backdoored substitute! > > Greetings, > Maxime. No this should not work. The ERIS URN is only used if the entire narinfo is signed with a authorized signature. The FileHash is not used when getting substitutes via ERIS (being able to decode ERIS content implies integrity). The interesting case that would be allowed with ERIS is following: 1. Server A is authentic and its key is authorized. 2. Servers M1 to MN are potentially malicious and their keys are not authorized. 3. Server A and servers M1 to MN are in the substitute-urls. 4. Client gets Narinfo from server A and uses the ERIS URN from there. 5. Client can get blocks simultaneously from Server A and servers M1 to MN. 6. Client decodes content with the ERIS URN and can be sure that they have the valid substitute. So client only needs to trust A but can use M1-MN (simultaneously) for fetching the content. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 11:28:01 +0000 Resent-Message-ID: <handler.52555.B52555.16438012439305 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.16438012439305 (code B ref 52555); Wed, 02 Feb 2022 11:28:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 11:27:23 +0000 Received: from localhost ([127.0.0.1]:51665 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFDnH-0002Q1-FK for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:27:23 -0500 Received: from albert.telenet-ops.be ([195.130.137.90]:54978) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nFDnE-0002Po-NK for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 06:27:21 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by albert.telenet-ops.be with bizsmtp id qBTG260044UW6Th06BTHxE; Wed, 02 Feb 2022 12:27:17 +0100 Message-ID: <846716544b4424f02e383114ebcb52957b43dd4d.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Wed, 02 Feb 2022 12:27:11 +0100 In-Reply-To: <86fsp1h6ce.fsf@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> <86fsp1h6ce.fsf@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-crNeuanJUAr3xaO0LT5a" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643801237; bh=PleecMB9h/qwk3+XCkghDaV+X30zFidzSB9DufPGVzU=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=Ii2MzhXF3fKvPkGX5+4vmsWk/d+M1jk5ZTFNU0PrLFdpd9bcpHJ5IlLJi/eSMprie rKsYKImG4McwLfwrQABQCA6aADSHtJNJvMZxi0j8UW3ZKg9wbhXMqSdaI+Zce41JxC wx0doKxujWIOw+ZNcLMEKrC55whFlhWlKNLO1NbdOwxBNGls5aIKqXgRUK9ykcycq8 zx+rzXQuBVfH881qKI2j/ZHL76MbHHv2vp2tr8GwDZmN2AX1B9v3s/iERVm9J8stlL 69DlBTWk1I6OMVZNsbZ7YRbU52KYJ0hzjlKJQGhBjd+g3PIv1nDFShEIJL0H5zVo1M uLTqwKeOlsY+w== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-crNeuanJUAr3xaO0LT5a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op wo 02-02-2022 om 10:51 [+0000]: > > The database doesn't seem necessary, the substitute server could > > have > > some end-point > >=20 > > =C2=A0=C2=A0 /publish-this-nar-again-into-IPFS/name-of-the-nar > >=20 > > which, when contacted, inserts the nar again into IPFS.=C2=A0 Then when > > a > > block was unavailable, the client contacts this end-point and > > retries. >=20 > But for a HTTP block endpoint we would still need such a > database/block > storage. >=20 > I think it is important that we do not rely on IPFS for block > storage. The decentralized block distribution should work even if the > IPFS daemon is not available. Do we need a database at all? E.g., if the client cannot download the data in the range [start, end] because the corresponding block has disappeared, can it not simply download that range from https://ci.guix.gnu.org/nar/[...] (not sure about the URI) using a HTTP range request? (Afterwards, the client should insert the block(s) back into IPFS/GNUnet/whatever, maybe using this proposed =E2=80=98in-file block stor= e=E2=80=99 such that other clients (using the same DHT mechanism) can benefit.) Greetings, Maxime. --=-crNeuanJUAr3xaO0LT5a Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfpqjxccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7q/tAQCx5U1HlZF9H7rfUztb00awEgdY KRhb7frTXDXhQfzGwwD/Ro+r9CPRGyK2EQ1AqDDFgZUohQQrKHbAzL8yY+DG/Qk= =93De -----END PGP SIGNATURE----- --=-crNeuanJUAr3xaO0LT5a--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: pukkamustard <pukkamustard@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 12:53:01 +0000 Resent-Message-ID: <handler.52555.B52555.164380633727413 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: Maxime Devos <maximedevos@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164380633727413 (code B ref 52555); Wed, 02 Feb 2022 12:53:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 12:52:17 +0000 Received: from localhost ([127.0.0.1]:51802 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFF7R-000785-ES for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 07:52:17 -0500 Received: from mout02.posteo.de ([185.67.36.66]:60167) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <pukkamustard@HIDDEN>) id 1nFF7O-00077m-7P for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 07:52:15 -0500 Received: from submission (posteo.de [89.146.220.130]) by mout02.posteo.de (Postfix) with ESMTPS id EA813240105 for <52555 <at> debbugs.gnu.org>; Wed, 2 Feb 2022 13:52:07 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1643806327; bh=yNrQsQJfvuyiVKi7mL2Ed53n44JDomBMl49FVbtRM5s=; h=From:To:Cc:Subject:Date:From; b=RpglqmkS0xYfUaxF9GHSJ1C8C5n1G6JRe8zlgclHlC1CvnoXtEMeInL7rv4h3dPhv n7s3JwgaiZnT7DlEbeflp9sM9fSFoycS8pXqn/9P6vyu/DruEebEq+Jzvz1vofgAVf 2gt1sUN0DluCW0M7ZgLI7iBpszZKYv65u/ucjExHXa6ROud60B6HlSPGZhCNNP10b2 jPv5dDCj7SxA28sehry/ie6Q+dEP227NUoINHCA2E4XXRDsGKwPNiEdGFil8LKCvE4 Qr1aLDbSd3t22fx+dLpaZQH6bbzonrgRLT3GtnGoCFv9/P93bIp6Xa8UaimwL8wYrK apAPWoBKYUGCg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4JphX62MhWz6tmm; Wed, 2 Feb 2022 13:52:06 +0100 (CET) References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> <86fsp1h6ce.fsf@HIDDEN> <846716544b4424f02e383114ebcb52957b43dd4d.camel@HIDDEN> From: pukkamustard <pukkamustard@HIDDEN> Date: Wed, 02 Feb 2022 12:42:21 +0000 In-reply-to: <846716544b4424f02e383114ebcb52957b43dd4d.camel@HIDDEN> Message-ID: <8635l1h1mx.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Maxime Devos <maximedevos@HIDDEN> writes: >> I think it is important that we do not rely on IPFS for block >> storage. The decentralized block distribution should work even if the >> IPFS daemon is not available. > > Do we need a database at all? > > E.g., if the client cannot download the data in the range [start, end] > because the corresponding block has disappeared, can it not simply > download that range from https://ci.guix.gnu.org/nar/[...] > (not sure about the URI) using a HTTP range request? This does not work as the mapping from block reference to location in NAR can not be known by the client who only holds the ERIS URN. Furthermore, some blocks will be intermediary nodes - they hold references to content blocks (or other intermediary nodes) but not content itself. > (Afterwards, the client should insert the block(s) back into > IPFS/GNUnet/whatever, maybe using this proposed =E2=80=98in-file block st= ore=E2=80=99 > such that other clients (using the same DHT mechanism) can benefit.) It might make sense for some clients to make content available to other clients and to go trough the extra effort of putting blocks back into IPFS/GNUNet/whatever. But this should be optional. Maybe we can call such clients "caching peers"? IMO A client should by default only deal with things that are strictly necessary for getting substitutes. The substistute servers (and caching peers) should make sure substitutes are available to clients, whether over IPFS/GNUNet/whatever or plain old HTTP. -pukkamustard
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 15:08:02 +0000 Resent-Message-ID: <handler.52555.B52555.164381446419008 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164381446419008 (code B ref 52555); Wed, 02 Feb 2022 15:08:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 15:07:44 +0000 Received: from localhost ([127.0.0.1]:53068 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFHET-0004wP-9L for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 10:07:44 -0500 Received: from xavier.telenet-ops.be ([195.130.132.52]:41928) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nFHEQ-0004wD-43 for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 10:07:40 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by xavier.telenet-ops.be with bizsmtp id qF7b2600C4UW6Th01F7cfL; Wed, 02 Feb 2022 16:07:36 +0100 Message-ID: <45e1dd2f05c6733eb74792af07194347fdd55ebd.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Wed, 02 Feb 2022 16:07:31 +0100 In-Reply-To: <8635l1h1mx.fsf@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> <86fsp1h6ce.fsf@HIDDEN> <846716544b4424f02e383114ebcb52957b43dd4d.camel@HIDDEN> <8635l1h1mx.fsf@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-3lXz/ldhBv2+wwdvsJIY" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643814456; bh=3HVagiaD37/TWambj2cRkJ/5KrREtKSPiKMSvmPXFXc=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=oUMRtSHu2XbpoZpF7vZOQGdJANKwmXgsaWP5GVmEit/o/admL+Kpek9NI8PQBTQZD hPV62qQbjPgk25OvVJTWUgfi7oNmtrMtrNcCJuHOh1XaDgpNjgalWBtVyhTM3Qtg/K D2Xrztdrats76E1iE5ldL/CjHrLVHqryM3+1E0tyybozgWCiJMiVFBZP5/WJH6fTP4 U/qHQeL7Pyjr6KilNwQyVZZiX/pMSXZyLbMq70cDsPDSb9J2HaiDKezxACGdnupD5E yDgiy9pE6N4ZPEsvaQPGlQdhehAPco3CLbk8FSDHDGgpQ8g7dlGC1VCV59P9ifGOp5 2qT95H5azUg8A== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-3lXz/ldhBv2+wwdvsJIY Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op wo 02-02-2022 om 12:42 [+0000]: > > (Afterwards, the client should insert the block(s) back into > > IPFS/GNUnet/whatever, maybe using this proposed =E2=80=98in-file block > > store=E2=80=99 > > such that other clients (using the same DHT mechanism) can > > benefit.) >=20 > It might make sense for some clients to make content available to > other > clients and to go trough the extra effort of putting blocks back into > IPFS/GNUNet/whatever. But this should be optional. Maybe we can call > such clients "caching peers"? >=20 > IMO A client should by default only deal with things that are > strictly > necessary for getting substitutes. The substistute servers (and > caching > peers) should make sure substitutes are available to clients, whether > over IPFS/GNUNet/whatever or plain old HTTP. If re-inserting missing blocks back into the IPFS/GNUnet/whatever is made optional and is off by default, then=C2=A0almost nobody will enable th= e =E2=80=98caching peer=E2=80=99 option and we will have freeloaders, somewha= t defeating the point of GNUnet/whatever. In a classic setting (=E2=80=98plain old HTTP=E2=80=99), serving and downlo= ading is a separate thing. But in a P2P setting, downloading cannot be separated from uploading -- strictly speaking, a peer might be able to download without uploading (depending on the P2P system), but that's anti- social, not something that should be done by default. However, if re-inserting missing blocks is _on_ by default, then there doesn't seem to be any trouble. Greetings, Maxime. --=-3lXz/ldhBv2+wwdvsJIY Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfqeNBccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7hI+AQCyLDdDhhLnScMBo6MNl55F+DKo Yaje3fToELjEOBmOUAD/YOvJCAsYd7BILqDAImAt4T8s+UnANcKzVtbWKXbklwc= =ttyT -----END PGP SIGNATURE----- --=-3lXz/ldhBv2+wwdvsJIY--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 15:29:02 +0000 Resent-Message-ID: <handler.52555.B52555.164381568421149 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164381568421149 (code B ref 52555); Wed, 02 Feb 2022 15:29:02 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 15:28:04 +0000 Received: from localhost ([127.0.0.1]:53095 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFHYC-0005V3-5e for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 10:28:04 -0500 Received: from andre.telenet-ops.be ([195.130.132.53]:48458) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nFHY7-0005Ub-3o for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 10:28:02 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by andre.telenet-ops.be with bizsmtp id qFTw2600T4UW6Th01FTwYL; Wed, 02 Feb 2022 16:27:57 +0100 Message-ID: <84d930adac8c54120b176362e66a63ebf49179c6.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Wed, 02 Feb 2022 16:27:52 +0100 In-Reply-To: <8635l1h1mx.fsf@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <ba490f009be9908063609f71800a2f00b4b1c1ae.camel@HIDDEN> <86fsp1h6ce.fsf@HIDDEN> <846716544b4424f02e383114ebcb52957b43dd4d.camel@HIDDEN> <8635l1h1mx.fsf@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-MLPmcf3VaezyhnWN/EcJ" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643815677; bh=L1cE7iw9wpDs9RBnZWsEw4oIK8YvYcba5uknLmImRGU=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=ElS/TxmLltpabQrQQ28GDGY/JGBbtspCujTfS0pWMjJ2JWZi80ZcumfDMT/28eNVa KhuLco/tCtEg77KqCfTtW5zTD6B39RO4RZLJEIMDO5zY9n03knG8nKXe+ihMZxJnnc IiiKrQbe3uh4Tf2DLipmBYsHNRxtJQWPnZLRp1iZoPFgof0/IV/vLa6X+bmSvJackl Xb7XpmF+bKN1wM+BAoUrpXKRuntXchFhhi1qN4XZ7XhFeNWWU8no+F+G3KgG6QMy6x G/0OTRGeZZcs7jo18t7Zkr2rExGt1LxbmcLtDNoKwhltjCHxeZy+FLEDl5PUHZMEyl sKz98EGtJqKqQ== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-MLPmcf3VaezyhnWN/EcJ Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op wo 02-02-2022 om 12:42 [+0000]: > > E.g., if the client cannot download the data in the range [start, > > end] > > because the corresponding block has disappeared, can it not simply > > download that range from https://ci.guix.gnu.org/nar/[...] > > (not sure about the URI) using a HTTP range request? >=20 > This does not work as the mapping from block reference to location in > NAR can not be known by the client who only holds the ERIS > URN. The client not only knows the ERIS URN, it also knows the location of the nar (over classical HTTP) because it's in the narinfo. > Furthermore, some blocks will be intermediary nodes - they hold > references to content blocks (or other intermediary nodes) but not > content itself. If an intermediary node (responsible for, say, bytes 900--10000) is missing, then the bytes 900--10000 could be downloaded via HTTP. Whether the node is close to the top, or close to the bottom, in ERIS' variant of Merkle trees, doesn't matter much. Granted, if the nar is, say, 1 GiB, and the top-level block is missing, then we'll have to download 1 GiB over HTTP, even if most lower blocks exist on IPFS/GNUnet/whatever, which isn't really great. We could also do some combination of the GDBM database and HTTP Content-Range requests: most nodes are leaf nodes (*). Instead of representing all nodes in the database, we could include only (intermediate) nodes responsible for data of size, say, 4MiB. (*) At least, that's the case for binary trees, presumably something similar holds for ERIS. I don't know the specifics for ERIS, but for (balanced) binary trees, not storing the leaf nodes would save about 50% (**), which is a rather nice space saving. (**) This assumes the =E2=80=98block size=E2=80=99 is the size for storing = two pointers to the children, but in practice the block size would be quite a bit larger, so there would be more space savings? Perhaps we are overthinking things and the GDBM (***) database isn't overly large, or perhaps missing blocks are sufficiently rare such that we could simply download the _entire_ nar from classical HTTP in case of missing blocks ... (***) Guix uses SQlite databases, so I would use SQLite instead of GDBM unless there's a compelling reason to use GDBM instead. Greetings, Maxime. --=-MLPmcf3VaezyhnWN/EcJ Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfqi+BccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7luwAP4x9bFYhc+pM+oN9Jed5nNRDUSl xTW+PIkxKhsPSSkOtAD8DtBwnt0Ju57kJ/OTzouZ0ZwGJRzBUCZ1bkRntYZZPg4= =r/kQ -----END PGP SIGNATURE----- --=-MLPmcf3VaezyhnWN/EcJ--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 3/5] Add (guix eris). Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Wed, 02 Feb 2022 15:37:01 +0000 Resent-Message-ID: <handler.52555.B52555.164381622022144 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164381622022144 (code B ref 52555); Wed, 02 Feb 2022 15:37:01 +0000 Received: (at 52555) by debbugs.gnu.org; 2 Feb 2022 15:37:00 +0000 Received: from localhost ([127.0.0.1]:53109 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFHgq-0005l5-GQ for submit <at> debbugs.gnu.org; Wed, 02 Feb 2022 10:37:00 -0500 Received: from andre.telenet-ops.be ([195.130.132.53]:35326) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nFHgo-0005ks-O9 for 52555 <at> debbugs.gnu.org; Wed, 02 Feb 2022 10:36:59 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by andre.telenet-ops.be with bizsmtp id qFcw2600R4UW6Th01FcweW; Wed, 02 Feb 2022 16:36:57 +0100 Message-ID: <26bc58741a60c972226cd47832b27de955e4a57a.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Wed, 02 Feb 2022 16:36:56 +0100 In-Reply-To: <86sft1h7wh.fsf@HIDDEN> References: <20220125192201.7582-1-pukkamustard@HIDDEN> <20220125192201.7582-4-pukkamustard@HIDDEN> <0b977ae15a49b051c04922d234149a8e4762404f.camel@HIDDEN> <86sft1h7wh.fsf@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-Ye3tASNbF2cSg/wmoG5a" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643816217; bh=0Lel4E0xIIF7zO0TlpU8F7P9LVPeRH9JSCNKMfBDEu0=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=a8Q+gJvXY8G2+ZN9IaAj/RUrZn5iv3AgGEH5i6nZndwnXV+J7iq31wfC5CmxTLTn7 f4yAI5kUtfdoy91JwIUhYC3XQSRMifc4GLf9zfsZo/WQCzoX8S/eSSY3kdWMMVcL4l J4kZZbOz6Pu3mVIKINe/mgoK1SXlTZRUJNwMHCyUp3nT9eOp/NFIe6zoo+mPR4X/8G taeVs7y4USILwrY1QtG0JgSyxweLFnp6lto9rqBwilkofdrcE9gMvLfybJ4meuDcx+ aGc0+Gum7xqqehUV7jvXAoLgAY0QP3mBLFabSGRMq/5SvNDKxKwGSvcmSSEYa7ub2c jxvf5nNtfqWFg== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-Ye3tASNbF2cSg/wmoG5a Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op wo 02-02-2022 om 10:28 [+0000]: > > pukkamustard schreef op di 25-01-2022 om 19:21 [+0000]: > > > +(define (ipfs-daemon-alive?) > > > +=C2=A0 "Attempt to connect to the IPFS daemon. Returns #t if the > > > daemon is alive > > > +and #f else." > > > +=C2=A0 (with-exception-handler > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (const #f) > > > +=C2=A0=C2=A0=C2=A0 (lambda _ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (let ((response _ > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (http-post (st= ring-append (%ipfs-base-url) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2= =A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0= =C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 > > > "/api/v0/version")))) > > > +=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0=C2=A0 (equal? 200 (response-cod= e response)))) > > > +=C2=A0=C2=A0=C2=A0 #:unwind? #t)) > >=20 > > This should preferably only be catching exceptions indicating that > > the daemon is down (exceptions indicating 404s, or system-errors > > indicating network errors, ...). >=20 > Yes, I guess it could be checked a bit finer. But at the end if an > exception happens then the IPFS daemon is probably not reachable, > right? > If we don't care about the reason why it is not reachable then why > bother with catching finer grained exceptions? The exception could be caused by, say: * an unbound variable * wrong arity * type error * stack overflow * prompt tag does not exist in current environment * out of memory Except for the last one, these causes are all bugs and hence shouldn't be surpressed. Granted, this is a bit unlikely since this use of 'http-post' is very simple, but it's far from impossible for (web client) to have a bug. Greetings, Maxime. --=-Ye3tASNbF2cSg/wmoG5a Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfqlGBccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7lMuAQCTlCl2Q5LRrRQiIKgiwuimw1Sm NXVCP9Pf/eOoQZMWAAD+NpY1trIo+TZLI+tmUcKBARROcYliOgUMS0RgLj1l3Qk= =sgcN -----END PGP SIGNATURE----- --=-Ye3tASNbF2cSg/wmoG5a--
X-Loop: help-debbugs@HIDDEN Subject: [bug#52555] [RFC PATCH v2 0/5] Decentralized substitute distribution with ERIS Resent-From: Maxime Devos <maximedevos@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: guix-patches@HIDDEN Resent-Date: Thu, 03 Feb 2022 20:37:02 +0000 Resent-Message-ID: <handler.52555.B52555.164392060531992 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 52555 X-GNU-PR-Package: guix-patches X-GNU-PR-Keywords: patch To: pukkamustard <pukkamustard@HIDDEN> Cc: ~pukkamustard/eris@HIDDEN, 52555 <at> debbugs.gnu.org Received: via spool by 52555-submit <at> debbugs.gnu.org id=B52555.164392060531992 (code B ref 52555); Thu, 03 Feb 2022 20:37:02 +0000 Received: (at 52555) by debbugs.gnu.org; 3 Feb 2022 20:36:45 +0000 Received: from localhost ([127.0.0.1]:58005 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1nFiqT-0008Jw-2q for submit <at> debbugs.gnu.org; Thu, 03 Feb 2022 15:36:45 -0500 Received: from andre.telenet-ops.be ([195.130.132.53]:57620) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maximedevos@HIDDEN>) id 1nFiqQ-0008Jn-GP for 52555 <at> debbugs.gnu.org; Thu, 03 Feb 2022 15:36:43 -0500 Received: from ptr-bvsjgyhxw7psv60dyze.18120a2.ip6.access.telenet.be ([IPv6:2a02:1811:8c09:9d00:3c5f:2eff:feb0:ba5a]) by andre.telenet-ops.be with bizsmtp id qkcg2600A4UW6Th01kcgxL; Thu, 03 Feb 2022 21:36:40 +0100 Message-ID: <52ee517f75c66a8fd9e9823da016b5720b4d5d34.camel@HIDDEN> From: Maxime Devos <maximedevos@HIDDEN> Date: Thu, 03 Feb 2022 21:36:34 +0100 In-Reply-To: <86bkzph5ux.fsf@HIDDEN> References: <20211216161724.547-1-pukkamustard@HIDDEN> <20220125192201.7582-1-pukkamustard@HIDDEN> <73b50ffdca94407ef9fd7ef4875985a3b1c3c568.camel@HIDDEN> <86bkzph5ux.fsf@HIDDEN> Content-Type: multipart/signed; micalg="pgp-sha512"; protocol="application/pgp-signature"; boundary="=-fiDAbmrU5Ni16CfzabJE" User-Agent: Evolution 3.38.3-1 MIME-Version: 1.0 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r22; t=1643920601; bh=1TgKkaZUhhBFFz9KXPG4sUQoFXPGqrPg900RPI+GIoA=; h=Subject:From:To:Cc:Date:In-Reply-To:References; b=UapWoWWerxjPtgeUMjPYvhyHkasv/Lw1/pSmm0BVf4igxlSWbVdyf7wkszFCu7Fwz 4dChT3aSCd+RWdYQfTvHMhpKq757ywg6PKyjIZiHDzNlItxR+iGk49Mc5roF3ozL2i d1ZZNrk3TVfKeHTPjkmENeWFrQOsAJK5QDdwSpONEGyM8rp7l1wgpXEQgbehUjyv6X e/Enl+MmVpn+9nD5n8cLDVc1obYEk3z5ZUgGRhZ18ktE/Kw6B/Nm3q2IaCbCRiYekn 9ExJZHJsExbSrR0S+3yGF6fZTL/iXiDA8vqYXIEKtEyZJQy3Ab461V+HVdVcU8WJrz nrxElbk9bz2lw== X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) --=-fiDAbmrU5Ni16CfzabJE Content-Type: text/plain; charset="UTF-8" Content-Transfer-Encoding: quoted-printable pukkamustard schreef op wo 02-02-2022 om 11:10 [+0000]: > The ERIS URN is only used if the entire narinfo is signed with a > authorized signature. Perhaps I'm missing something here, but in that case, shouldn't "ERIS" be added to %mandatory-fields in (guix narinfo)? Anyway, I don't see what prevents an unauthorised narinfo with a ERIS URN to be used: the narinfo is chosen with (define narinfo (lookup-narinfo cache-urls store-item (if (%allow-unauthenticated-substitutes?) (const #t) (cut valid-narinfo? <> acl)))) where lookup-narinfo is a tiny wrapper around lookup-narinfos/diverse. lookup-narinfos/diverse considers both unauthorised and authorised narinfos, and can choose an unauthorised narinfo if it's =E2=80=98equivalen= t=E2=80=99 to an authorised narinfo (using equivalent-narinfo?) equivalent-narinfo? only looks at the hash, path, references and size, and ignores the ERIS. As such, an unauthorised narinfo with a malicious ERIS URN could be selected. However, it turns out that all this doesn't really matter:=C2=A0whether the port returned by 'fetch' in (guix scripts substitute) came from file://, http://, https:// or ERIS, the file hash is verified later anyway: ;; Compute the actual nar hash as we read it. ((algorithm expected) (narinfo-hash-algorithm+value narinfo)) ((hashed get-hash) (open-hash-input-port algorithm input))) [...] ;; Check whether we got the data announced in NARINFO. (let ((actual (get-hash))) (if (bytevector=3D? actual expected) [...] False alarm I guess! Greetings, Maxime. --=-fiDAbmrU5Ni16CfzabJE Content-Type: application/pgp-signature; name="signature.asc" Content-Description: This is a digitally signed message part Content-Transfer-Encoding: 7bit -----BEGIN PGP SIGNATURE----- iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYfw80hccbWF4aW1lZGV2 b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7s/dAQDJWPpPuJCxdqcJv0tvWer5fKY/ Hr1YGeNT61UHexvazwD/X4/mwomnod+8Urvl9nhTeLUOa8Hs4iZ2tRYZR9nJ7wA= =w/KP -----END PGP SIGNATURE----- --=-fiDAbmrU5Ni16CfzabJE--
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.