GNU bug report logs - #47187
[PATCH] gnu: Add c-lightning.

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: ZmnSCPxj <ZmnSCPxj@HIDDEN>; Keywords: patch; dated Tue, 16 Mar 2021 08:41:02 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.

Message received at 47187 <at> debbugs.gnu.org:


Received: (at 47187) by debbugs.gnu.org; 18 Mar 2021 06:33:19 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Mar 18 02:33:19 2021
Received: from localhost ([127.0.0.1]:44716 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lMmDf-0005f8-7W
	for submit <at> debbugs.gnu.org; Thu, 18 Mar 2021 02:33:19 -0400
Received: from mail.zaclys.net ([178.33.93.72]:56927)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <lle-bout@HIDDEN>) id 1lMmDc-0005es-5o
 for 47187 <at> debbugs.gnu.org; Thu, 18 Mar 2021 02:33:18 -0400
Received: from [192.168.0.27] (82-64-145-38.subs.proxad.net [82.64.145.38])
 (authenticated bits=0)
 by mail.zaclys.net (8.14.7/8.14.7) with ESMTP id 12I6X9T4049359
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Thu, 18 Mar 2021 07:33:09 +0100
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.zaclys.net 12I6X9T4049359
Authentication-Results: mail.zaclys.net;
 dmarc=fail (p=reject dis=none) header.from=zaclys.net
Authentication-Results: mail.zaclys.net;
 spf=fail smtp.mailfrom=lle-bout@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zaclys.net;
 s=default; t=1616049189;
 bh=hkpHsm3tQSls8l/q01QDG8pURoUb/H62Lz8B3P3V5Nc=;
 h=Subject:From:To:Cc:Date:In-Reply-To:References:From;
 b=J6ooI5cZYxzgot/O8Fo/FZgHsPxoMcq6wszc6ce8Ub1xW4VSqZD65hJheYcLqjkHM
 M0+YOAQW2dU0EHPev0x98zygQtODoTvikob+C0FHq/RAz/LamZ4tvhzw64uOWQHxYz
 xAccLfaKs1W+HN4Sl3JjiHuPGfw5ZdKQ1lang8jM=
Message-ID: <d144260cff4e907bd263b4f4dbb1a3022ec94ef5.camel@HIDDEN>
Subject: Re: [bug#47187] [PATCH] gnu: Add c-lightning.
From: =?ISO-8859-1?Q?L=E9o?= Le Bouter <lle-bout@HIDDEN>
To: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Date: Thu, 18 Mar 2021 07:33:04 +0100
In-Reply-To: <X0AJfvGmJvZOXkqcxiL1wDpQGbPYwaMG5V24ltJiXsvMhc8i8OZkWd_uAf18tMpgcSq1izVJTiurVFRaflG2_dOtTi7UzrOZwT9DcV0gFo0=@protonmail.com>
References: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
 <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
 <9436292ea880757cf47a5ce59879e6e7098ff195.camel@HIDDEN>
 <Pnk0YahvVi3QLAGeLyIGEl3tdFaE1AtAtHOvIRjW1gPAUzzjYF9dC0U8Di_kRe84FfmQw_1hr7sSRE4enMKw0Kzd74mMvR-CGQyhWhZJelw=@protonmail.com>
 <8f7d4c04d96fdf8cf1239c476c4c869f92446ada.camel@HIDDEN>
 <X0AJfvGmJvZOXkqcxiL1wDpQGbPYwaMG5V24ltJiXsvMhc8i8OZkWd_uAf18tMpgcSq1izVJTiurVFRaflG2_dOtTi7UzrOZwT9DcV0gFo0=@protonmail.com>
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-XIvGNd8ZJ5PYKJsBIQ+V"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 47187
Cc: "47187 <at> debbugs.gnu.org" <47187 <at> debbugs.gnu.org>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


--=-XIvGNd8ZJ5PYKJsBIQ+V
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Wed, 2021-03-17 at 03:42 +0000, ZmnSCPxj wrote:
> I understand.
>=20
> This will require a largish amount of work I think.
>=20
> Would this technique be acceptable?
>=20
> * `add-before 'configure 'unvendor-externals`
>   * `rm -rf` the vendored externals.
>   * `ln -s` the needed `.h` and `.la`/`.a`/`.so` files from the
> `inputs` to the expected paths within the `external/` directory.
>=20
> ?

Maybe simply add an option like --with-system-libbacktrace etc. in the
build system? And yes remove externals, I think that you should use a
snippet in the origin field instead of a phase for that.

>=20
> We have a `requirements.txt` file which contains this, I duplicate
> below:
>=20
> ```
> # Dependencies required to build and test c-lightning
> https://github.com/ElementsProject/libwally-core/releases/download/releas=
e_0.8.0/wallycore-0.8.0-cp36-cp36m-linux_x86_64.whl; 'linux' in sys_platfor=
m and python_version =3D=3D
> '3.6'
> https://github.com/ElementsProject/libwally-core/releases/download/releas=
e_0.8.0/wallycore-0.8.0-cp37-cp37m-linux_x86_64.whl; 'linux' in sys_platfor=
m and python_version =3D=3D
> '3.7'
> https://github.com/ElementsProject/libwally-core/releases/download/releas=
e_0.8.0/wallycore-0.8.0-cp37-cp37m-macosx_10_14_x86_64.whl; sys_platform =
=3D=3D 'darwin' and python_version =3D=3D
> '3.7'
> mrkd ~=3D 0.1.6
> Mako ~=3D 1.1.3
>=20
> # Dependencies from pyln-client
> Sphinx ~=3D 3.4.0
> flake8=3D=3D3.7.8
> recommonmark>=3D0.7.*
> sphinx-rtd-theme=3D=3D0.4.2
> sphinxcontrib-websupport=3D=3D1.1.0
> tqdm=3D=3D4.32.2
>=20
> # Dependencies from pyln-testing
> Flask=3D=3D1.1.*
> cheroot=3D=3D8.5.*
> ephemeral-port-reserve=3D=3D1.1.1
> filelock=3D=3D3.0.*
> flaky ~=3D 3.7.0
> psutil=3D=3D5.7.*
> psycopg2-binary=3D=3D2.8.*
> pytest-rerunfailures=3D=3D9.1.1
> pytest-timeout ~=3D 1.4.2
> pytest-xdist ~=3D 2.2.0
> pytest=3D=3D6.1.*
> python-bitcoinlib=3D=3D0.11.*
>=20
> # Dependencies from pyln-proto
> base58 ~=3D 2.0.1
> bitstring ~=3D 3.1.6
> coincurve ~=3D 13.0.0
> cryptography ~=3D 3.2
> mypy ~=3D 0.790
> pysocks ~=3D 1.7.1
>=20
> # Dependencies from pyln-spec
> # None
> ```

Thank you, I am going to go through them.

> Incidentally, we also install some Python modules.
> How do I "properly" export the Python modules within Guix?

I will ask someone else here but you might have to mix the python-
build-system in.

L=C3=A9o

--=-XIvGNd8ZJ5PYKJsBIQ+V
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEFIvLi9gL+xax3g6RRaix6GvNEKYFAmBS9CAACgkQRaix6GvN
EKY1Lw/8DksDMQmSXp/4ghYlB5LZDaAwIvIEebtMVVve7ULQFgHoJKhRtFpj4Bes
25rKXL6MCXpFGxwsR0++z0ItPR2WCzq0Y8LIUEBufWIv8H1JAr8K/LrqBS6J4TO9
FuvXz0rdqbuUM3JFUBLsHkI9DfNSMlh62Pq5Z891G/SIClPeXR1WTr43CHTSz3hG
j5vB88n4CF4Jco5TrM6RBxgJg+djVx79+k4W35xrqNNlb2xY1umscXrsc6C6VVBz
6GkELEsEyCosIUCFQu42KkLG7gLctxOL7LN+VmWYinsyYOtYSGzBp+jX5TZ14ap9
eA1VDbVCW9fIn0YZoUnLg3PXXPfeqJxmNC6d6T7U1eOM+ISQ7DtH1S/6SOoeCgm6
NDE7iQNsqm3AMlnRC59fRIfb0XCWlRTnxB3d8Qj7IOSQr5SYP/bMwiiPl1YwCerI
lAF6gRqGa9EeYm8oLbkPZV8qNpu93OytmsUwgkeLBx15xGPr3Fvwk0xtDTBZvDGV
NuA48flNy3dGZapGser4RNwfSbXwccGNnVm4HdZJ8fq36QszpxgJv++CxigXo7Jg
zOz62S4E5pz4z68FO1P6cfvNolLgXKe4lFSb8f6DK60XMF9C3gQtS8O9Mc9ScEe+
Z3D4vKhXEqq1uVXdIDo7PALwrl5MLnsdbp4jjo38MVnhoAl2xFQ=
=SuGc
-----END PGP SIGNATURE-----

--=-XIvGNd8ZJ5PYKJsBIQ+V--





Information forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.

Message received at 47187 <at> debbugs.gnu.org:


Received: (at 47187) by debbugs.gnu.org; 17 Mar 2021 03:42:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 16 23:42:41 2021
Received: from localhost ([127.0.0.1]:41647 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lMN4y-0004eV-OI
	for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 23:42:41 -0400
Received: from mail-40138.protonmail.ch ([185.70.40.138]:56551)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ZmnSCPxj@HIDDEN>) id 1lMN4w-0004eI-2g
 for 47187 <at> debbugs.gnu.org; Tue, 16 Mar 2021 23:42:39 -0400
Date: Wed, 17 Mar 2021 03:42:27 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
 s=protonmail; t=1615952551;
 bh=XuoBC/wHVyU1GLPthFiRR9L9vhKeKQoE8ql1s3jfhfU=;
 h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From;
 b=krbPXf78K0zpyqvbkBWqjMw1whXD2Tk4jaSXhLp8LYJ+MMoMi62slQx2eiDLKp1Yg
 s7tB5g9427KN7jw9ldOzdq32LW2X5f76n+Mrdloah0SxRQ1KuLv8Fx2V2QeHA3iHPj
 VEvoyV5eSm1fbNXbCPa2ElNnUXy9PML9ExZpNa9c=
To: =?utf-8?Q?L=C3=A9o_Le_Bouter?= <lle-bout@HIDDEN>
From: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Subject: Re: [bug#47187] [PATCH] gnu: Add c-lightning.
Message-ID: <X0AJfvGmJvZOXkqcxiL1wDpQGbPYwaMG5V24ltJiXsvMhc8i8OZkWd_uAf18tMpgcSq1izVJTiurVFRaflG2_dOtTi7UzrOZwT9DcV0gFo0=@protonmail.com>
In-Reply-To: <8f7d4c04d96fdf8cf1239c476c4c869f92446ada.camel@HIDDEN>
References: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
 <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
 <9436292ea880757cf47a5ce59879e6e7098ff195.camel@HIDDEN>
 <Pnk0YahvVi3QLAGeLyIGEl3tdFaE1AtAtHOvIRjW1gPAUzzjYF9dC0U8Di_kRe84FfmQw_1hr7sSRE4enMKw0Kzd74mMvR-CGQyhWhZJelw=@protonmail.com>
 <8f7d4c04d96fdf8cf1239c476c4c869f92446ada.camel@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no
 autolearn=disabled version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 mailout.protonmail.ch
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 47187
Cc: "47187 <at> debbugs.gnu.org" <47187 <at> debbugs.gnu.org>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)

Good morning Leo,

> > Yes, it is true that there is something of a requirement of a strict
> > behavior here, I suppose it is possible to use `git-fetch` instead of
> > `url-fetch` for our external libraries.
>
> Yes you can use git-fetch, to make sure we are on the same page, are
> you speaking of strict behavior requirements like for Bitcoin Core's
> consensus code?

Well we need to produce signatures and transactions that pass Bitcoin Core =
signature validation at least, so it is best to use a version of `libsecp25=
6k1` (which produces signatures) that we know works, as well as `libwally-c=
ore` (which produces transactions).
I would personally use the `libsecp256k1` version that `libwally-core` vend=
ors in as well, since there may be specific interactions between `libwally-=
core` and `libsecp256k1` that may be different if we use the Bitcoin Core v=
ersion of `libsecp256k1`.

For `libsodium`, at least the hashing has to work correctly, but I think it=
 is simple enough that strict behavior requirements are not so onerous.
Indeed, we usually get this from the OS (but we need a later feature than t=
hat available on some old Ubuntu versions, which is why it got vendored in)=
, so I should probably "just" add it as an input.

>
> > How do I generate `guix hash` for `git-fetch`ed `source`s?
>
> Actually what I do is put a wrong hash in and then copy the "actual
> hash" from the error. I havent found another way but this definitely
> feels subpar and prevents much verification before putting in the hash,
> better suggestions welcome.

Haha I shall do so as well.

>
> > However it also means that every release of C-Lightning I have to go
> > double-check what git commit to use for each library (though `jsmn`
> > and `libbacktrace` at least seem very stable).
> > But it looks to me that unvendoring will require more extensive
> > patching of the `Makefile` and an even larger maintenance burden on
> > Guix side?
>
> Unvendoring is more or less a policy because we must be able to audit
> each piece of software separately for freedom issues (licenses,
> violations to the GNU FSDG), and it eases work for security-patching
> also.

I understand.

This will require a largish amount of work I think.

Would this technique be acceptable?

* `add-before 'configure 'unvendor-externals`
  * `rm -rf` the vendored externals.
  * `ln -s` the needed `.h` and `.la`/`.a`/`.so` files from the `inputs` to=
 the expected paths within the `external/` directory.

?

> > Please do, I am not very familiar with any Python infrastructure and
> > am primarily a C programmer here, I just barely hack together some
> > kind of test in Python.
>
> If you can list the Python dependencies and their version I can look at
> packaging them.

We have a `requirements.txt` file which contains this, I duplicate below:

```
# Dependencies required to build and test c-lightning
https://github.com/ElementsProject/libwally-core/releases/download/release_=
0.8.0/wallycore-0.8.0-cp36-cp36m-linux_x86_64.whl; 'linux' in sys_platform =
and python_version =3D=3D '3.6'
https://github.com/ElementsProject/libwally-core/releases/download/release_=
0.8.0/wallycore-0.8.0-cp37-cp37m-linux_x86_64.whl; 'linux' in sys_platform =
and python_version =3D=3D '3.7'
https://github.com/ElementsProject/libwally-core/releases/download/release_=
0.8.0/wallycore-0.8.0-cp37-cp37m-macosx_10_14_x86_64.whl; sys_platform =3D=
=3D 'darwin' and python_version =3D=3D '3.7'
mrkd ~=3D 0.1.6
Mako ~=3D 1.1.3

# Dependencies from pyln-client
Sphinx ~=3D 3.4.0
flake8=3D=3D3.7.8
recommonmark>=3D0.7.*
sphinx-rtd-theme=3D=3D0.4.2
sphinxcontrib-websupport=3D=3D1.1.0
tqdm=3D=3D4.32.2

# Dependencies from pyln-testing
Flask=3D=3D1.1.*
cheroot=3D=3D8.5.*
ephemeral-port-reserve=3D=3D1.1.1
filelock=3D=3D3.0.*
flaky ~=3D 3.7.0
psutil=3D=3D5.7.*
psycopg2-binary=3D=3D2.8.*
pytest-rerunfailures=3D=3D9.1.1
pytest-timeout ~=3D 1.4.2
pytest-xdist ~=3D 2.2.0
pytest=3D=3D6.1.*
python-bitcoinlib=3D=3D0.11.*

# Dependencies from pyln-proto
base58 ~=3D 2.0.1
bitstring ~=3D 3.1.6
coincurve ~=3D 13.0.0
cryptography ~=3D 3.2
mypy ~=3D 0.790
pysocks ~=3D 1.7.1

# Dependencies from pyln-spec
# None
```

Incidentally, we also install some Python modules.
How do I "properly" export the Python modules within Guix?

Regards,
ZmnSCPxj




Information forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.

Message received at 47187 <at> debbugs.gnu.org:


Received: (at 47187) by debbugs.gnu.org; 16 Mar 2021 12:46:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 16 08:46:37 2021
Received: from localhost ([127.0.0.1]:38424 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lM95o-0005Xy-SR
	for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 08:46:37 -0400
Received: from mail.zaclys.net ([178.33.93.72]:35141)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <lle-bout@HIDDEN>) id 1lM95l-0005Xj-83
 for 47187 <at> debbugs.gnu.org; Tue, 16 Mar 2021 08:46:35 -0400
Received: from guix-xps.local (82-64-145-38.subs.proxad.net [82.64.145.38])
 (authenticated bits=0)
 by mail.zaclys.net (8.14.7/8.14.7) with ESMTP id 12GCkQnf004376
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Tue, 16 Mar 2021 13:46:26 +0100
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.zaclys.net 12GCkQnf004376
Authentication-Results: mail.zaclys.net;
 dmarc=fail (p=reject dis=none) header.from=zaclys.net
Authentication-Results: mail.zaclys.net;
 spf=fail smtp.mailfrom=lle-bout@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zaclys.net;
 s=default; t=1615898787;
 bh=8S/yRw71eSNVSlh9M0Xb3nE7hl60+nY7+islSaKrzdU=;
 h=Subject:From:To:Cc:Date:In-Reply-To:References:From;
 b=FTeLL6/9ZFb9iopS/69WNes/m7HMq+TD9/hxt5q11/9VcQP471yWJJqqwHScPUl9c
 ydLpTmnz+XiC3zuEDo/nErWNv/mu8txotcDMhVS1Ae/M723L7FowvLiyBB7rxTDDmt
 G2XU+2W/AaQ2alsuM0wkfg5Dn/KFBLOUYNkpkGUM=
Message-ID: <8f7d4c04d96fdf8cf1239c476c4c869f92446ada.camel@HIDDEN>
Subject: Re: [bug#47187] [PATCH] gnu: Add c-lightning.
From: =?ISO-8859-1?Q?L=E9o?= Le Bouter <lle-bout@HIDDEN>
To: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Date: Tue, 16 Mar 2021 13:46:22 +0100
In-Reply-To: <Pnk0YahvVi3QLAGeLyIGEl3tdFaE1AtAtHOvIRjW1gPAUzzjYF9dC0U8Di_kRe84FfmQw_1hr7sSRE4enMKw0Kzd74mMvR-CGQyhWhZJelw=@protonmail.com>
References: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
 <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
 <9436292ea880757cf47a5ce59879e6e7098ff195.camel@HIDDEN>
 <Pnk0YahvVi3QLAGeLyIGEl3tdFaE1AtAtHOvIRjW1gPAUzzjYF9dC0U8Di_kRe84FfmQw_1hr7sSRE4enMKw0Kzd74mMvR-CGQyhWhZJelw=@protonmail.com>
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-N61jddAvNG+oXihVVnBc"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 47187
Cc: "47187 <at> debbugs.gnu.org" <47187 <at> debbugs.gnu.org>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


--=-N61jddAvNG+oXihVVnBc
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

On Tue, 2021-03-16 at 11:27 +0000, ZmnSCPxj wrote:
> Already working on that, but 0.10.0 will release soon and I doubt my
> changes will make it by then, so Guix might get c-lightning to 0.10.1
> or later if we wait for it.

Awesome!

> Yes, it is true that there is something of a requirement of a strict
> behavior here, I suppose it is possible to use `git-fetch` instead of
> `url-fetch` for our external libraries.

Yes you can use git-fetch, to make sure we are on the same page, are
you speaking of strict behavior requirements like for Bitcoin Core's
consensus code?

> How do I generate `guix hash` for `git-fetch`ed `source`s?

Actually what I do is put a wrong hash in and then copy the "actual
hash" from the error. I havent found another way but this definitely
feels subpar and prevents much verification before putting in the hash,
better suggestions welcome.

> However it also means that every release of C-Lightning I have to go
> double-check what git commit to use for each library (though `jsmn`
> and `libbacktrace` at least seem very stable).
>=20
> But it looks to me that unvendoring will require more extensive
> patching of the `Makefile` and an even larger maintenance burden on
> Guix side?

Unvendoring is more or less a policy because we must be able to audit
each piece of software separately for freedom issues (licenses,
violations to the GNU FSDG), and it eases work for security-patching
also.

> `libwally-core` itself depends on another library `libsecp256k1`, so
> I suppose it must transitively be unvendored as well.

Probably, we already have a libsecp256k1 specially for Bitcoin Core.

> The test suite is fairly large and can take a significant amount of
> time to run in full.

That's fine.

> In addition, part of the test includes checks which take advantage of
> `BINTOPKGLIBEXECDIR` relative path we normally use, which I want to
> disable for Guix at least since the relative path only has an
> advantage if the user wants to move the installation after-the-fact
> to a different location (and on Guix the "installation" path cannot
> be moved anyway).
> Using an absolute `BINTOPKGLIBEXECDIR` gives an advantage as
> mentioned in the comments that a profile being upgraded from one
> version of C-Lightning to another does not cause problems for daemons
> currently running off the profile.
> The test can be disabled (but not easily), I suppose.

Problematic or unrealistically expensive tests can be disabled yes.

> Please do, I am not very familiar with any Python infrastructure and
> am primarily a C programmer here, I just barely hack together some
> kind of test in Python.

If you can list the Python dependencies and their version I can look at
packaging them.

> Regards,
> ZmnSCPxj

L=C3=A9o

--=-N61jddAvNG+oXihVVnBc
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEFIvLi9gL+xax3g6RRaix6GvNEKYFAmBQqJ4ACgkQRaix6GvN
EKYMUA//ZJUL8DkBhtuS7WPYCcbnViobj2/R8xmSNa006kXkpG8pzlOrhwwTgeHi
Be0oCbLoDefssky0YAZVYOZjBfFb4QyX7S7f9oEBRTmGlPWTxcyNqs+STt/TYJEw
Uqj/tcdpDfEekADnaggyaOkTaiGpHrHfrgcILmIqTRJe9k9LiCX83O9IELr1nXzp
XrKFdMD/9rShKVobqMKtPzdrO5+H7EquC0az3iI/WXvmc/I0qJsS/WdJC62dneHu
kbrwAQvSd6MxJmO/d01MG+zP1k9nZ0eKrQZH4vJz2aznqe7gaDP5buoK0ZO1t15b
XHpsaHqL+TEsOTuj52eBdO0a7g7c1XHkq8DRFLcUz6EapijkGKIIfqavo8m80Lzx
4VZ0WGX//cmAX4ApvzwIUp5j9MRGf6aCZ5Dur5mJY4+j10sWYL2UfX6DoYjX1LBx
3h5DBAIMh9jWrWFUbtOlTIx1oDCoXuma5nUhfA/yPH98nHxYN1sckWkNhu2Z74nW
/wOk+pGfozU6JwgFhSpWDU3xvipP+j+F7l8Ap4RBQtjZtIVfPziv3ETq86raiK6+
w8UDXhDCN5OQhgXNZpZ4kioRxN2CNZUl5DopAVHoaDUiIN1KL4QfGAYjQ8g8Ytsx
PUKxcY66j7Js57N1P3+HRq75s/vNbHFpLwdLxNDvUnRrCrNyDok=
=Egcs
-----END PGP SIGNATURE-----

--=-N61jddAvNG+oXihVVnBc--





Information forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.

Message received at 47187 <at> debbugs.gnu.org:


Received: (at 47187) by debbugs.gnu.org; 16 Mar 2021 11:27:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 16 07:27:32 2021
Received: from localhost ([127.0.0.1]:38354 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lM7rH-0002qK-Rl
	for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 07:27:32 -0400
Received: from mail-40137.protonmail.ch ([185.70.40.137]:12730)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ZmnSCPxj@HIDDEN>) id 1lM7rF-0002pz-2Y
 for 47187 <at> debbugs.gnu.org; Tue, 16 Mar 2021 07:27:30 -0400
Date: Tue, 16 Mar 2021 11:27:14 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
 s=protonmail; t=1615894042;
 bh=N2l3CP59WcYBg24Bn3fTT1Chet9exmgxUzJIly6XLgI=;
 h=Date:To:From:Cc:Reply-To:Subject:In-Reply-To:References:From;
 b=E7QoKHhzRutHHdypayoDIDTp2n0/AnLyHri1zjsV4uPkFTu0N1scuBwLkfJc4TJut
 VDgJqCn7DoiR04e/B64MoiNRJ6GnaKDkgA0970hhY70VcjHlL/XO94s4EOqw84wafa
 L/Y74ZaH9b4HqxzXDstBSdW0qAwZBZ/PxshfQN3w=
To: =?utf-8?Q?L=C3=A9o_Le_Bouter?= <lle-bout@HIDDEN>
From: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Subject: Re: [bug#47187] [PATCH] gnu: Add c-lightning.
Message-ID: <Pnk0YahvVi3QLAGeLyIGEl3tdFaE1AtAtHOvIRjW1gPAUzzjYF9dC0U8Di_kRe84FfmQw_1hr7sSRE4enMKw0Kzd74mMvR-CGQyhWhZJelw=@protonmail.com>
In-Reply-To: <9436292ea880757cf47a5ce59879e6e7098ff195.camel@HIDDEN>
References: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
 <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
 <9436292ea880757cf47a5ce59879e6e7098ff195.camel@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no
 autolearn=disabled version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 mailout.protonmail.ch
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 47187
Cc: "47187 <at> debbugs.gnu.org" <47187 <at> debbugs.gnu.org>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.5 (/)

Good morning Leo,

> The various hacks in the recipe to get a working build seem troublesome
> maintenance-wise, is it possible you think since you are a developer
> that you make the build scripts a bit more standard in the project?

Already working on that, but 0.10.0 will release soon and I doubt my change=
s will make it by then, so Guix might get c-lightning to 0.10.1 or later if=
 we wait for it.

> I see there's some vendored libraries: gheap, jsmn, libbacktrace,
> libsodium, libwally-core ; Is it possible to unvendor those and create
> separate packages for each of them? Or is there a requirement of strict
> behavior here, either way, you could also create separate packages for
> each of them even if those are specific to c-lightning. It would make
> the main recipe cleaner, since as far as I can see it's also why the
> build system is so stubborn to GNU Guix? That it also attempts to build
> vendored libraries?

I think it is more the custom `configure` script.
The custom `configure` script also compiles a ***C*** program that then gen=
erates the configuration for the `ccan` library of the project, meaning cro=
ss-compilation requires special work for C-Lightning.
And `configure` does not call `configure` of the included libraries as well=
.

Yes, it is true that there is something of a requirement of a strict behavi=
or here, I suppose it is possible to use `git-fetch` instead of `url-fetch`=
 for our external libraries.
How do I generate `guix hash` for `git-fetch`ed `source`s?
However it also means that every release of C-Lightning I have to go double=
-check what git commit to use for each library (though `jsmn` and `libbackt=
race` at least seem very stable).

But it looks to me that unvendoring will require more extensive patching of=
 the `Makefile` and an even larger maintenance burden on Guix side?

`libwally-core` itself depends on another library `libsecp256k1`, so I supp=
ose it must transitively be unvendored as well.

> Additionally, please do enable the test suite it's really valuable in
> GNU Guix.

The test suite is fairly large and can take a significant amount of time to=
 run in full.

In addition, part of the test includes checks which take advantage of `BINT=
OPKGLIBEXECDIR` relative path we normally use, which I want to disable for =
Guix at least since the relative path only has an advantage if the user wan=
ts to move the installation after-the-fact to a different location (and on =
Guix the "installation" path cannot be moved anyway).
Using an absolute `BINTOPKGLIBEXECDIR` gives an advantage as mentioned in t=
he comments that a profile being upgraded from one version of C-Lightning t=
o another does not cause problems for daemons currently running off the pro=
file.
The test can be disabled (but not easily), I suppose.

> I can help packaging the necessary Python dependencies, also we have a
> Python importer, e.g. "$ guix import pypi -r <pkg>" to help us go
> faster at it.

Please do, I am not very familiar with any Python infrastructure and am pri=
marily a C programmer here, I just barely hack together some kind of test i=
n Python.


Regards,
ZmnSCPxj




Information forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.

Message received at 47187 <at> debbugs.gnu.org:


Received: (at 47187) by debbugs.gnu.org; 16 Mar 2021 09:18:07 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 16 05:18:07 2021
Received: from localhost ([127.0.0.1]:38199 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lM5q3-0005gL-5C
	for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 05:18:07 -0400
Received: from mail.zaclys.net ([178.33.93.72]:56205)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <lle-bout@HIDDEN>) id 1lM5q0-0005fq-OX
 for 47187 <at> debbugs.gnu.org; Tue, 16 Mar 2021 05:18:05 -0400
Received: from guix-xps.local (82-64-145-38.subs.proxad.net [82.64.145.38])
 (authenticated bits=0)
 by mail.zaclys.net (8.14.7/8.14.7) with ESMTP id 12G9Hw1H027797
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Tue, 16 Mar 2021 10:17:59 +0100
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.zaclys.net 12G9Hw1H027797
Authentication-Results: mail.zaclys.net;
 dmarc=fail (p=reject dis=none) header.from=zaclys.net
Authentication-Results: mail.zaclys.net;
 spf=fail smtp.mailfrom=lle-bout@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zaclys.net;
 s=default; t=1615886279;
 bh=w7laS2qzZ63zL+9YZ7zBMnrE227taHu5HYVEPPhf2Ag=;
 h=Subject:From:To:Date:In-Reply-To:References:From;
 b=AgdYdLjIV+r+csUTzuV+MPzG8CE7Nme8OYoXv/+2MwZk9WA3wlorT5CiaLWaGOoIG
 3J0E7n3InL1QHVZp35HRzMslgrr/NH+AtmNJma3LJTLGLq3zxUHI6Di+RWmmV86PIs
 QCRS8qs9FpV9kNAcbnezbRFJGNXW3li78vQnIfRI=
Message-ID: <9436292ea880757cf47a5ce59879e6e7098ff195.camel@HIDDEN>
Subject: Re: [bug#47187] [PATCH] gnu: Add c-lightning.
From: =?ISO-8859-1?Q?L=E9o?= Le Bouter <lle-bout@HIDDEN>
To: ZmnSCPxj <ZmnSCPxj@HIDDEN>, 47187 <at> debbugs.gnu.org
Date: Tue, 16 Mar 2021 10:17:58 +0100
In-Reply-To: <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
References: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
 <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-jrALHp1lXj5XqEfEufvE"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 47187
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


--=-jrALHp1lXj5XqEfEufvE
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Additionally, please do enable the test suite it's really valuable in
GNU Guix.

I can help packaging the necessary Python dependencies, also we have a
Python importer, e.g. "$ guix import pypi -r <pkg>" to help us go
faster at it.

--=-jrALHp1lXj5XqEfEufvE
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEFIvLi9gL+xax3g6RRaix6GvNEKYFAmBQd8YACgkQRaix6GvN
EKYqLBAAmakARL2beMnhQCSKYrZ2cSVIYh72Rld8jCG33gK5mI+kY+NzIRHjhnMi
D60Isyxt3JLtTcvnpGO+qZb+7nvDZRfuxJJM20V8lhPd73RZ85cUJqGs052CtatM
dDoGlIeTN1YMdJe65qiaqLA9HRq8PhShRT5PwAIjnzPi0TslXWSw9XOx5jC1lmn7
33wsDiiY0qq+L71zhL5ho21MQDaMqmIDPUqs+cSJonnPo3kC9S6JIT8O20BzejxV
s0F6A5q6cIRl5MMvPAxbLgnA700j/Qi7FYptb6Qi6Wu538XSFKiEzOjm+PHmTqKq
nlnbdXaaeYxh+ljUFNSRgMLsGz58FQUW4QEV2VsJz/gOk4XGspQOOQ7qIsBjQNH8
sqwgToEBTt45EYtzvQRhp02m9VuW3q36CDdz2KnA0WBmSsCAGMDY8Y34QeE2cgsZ
ZlGY4hzUAu/dqamncoiaYsvnuzxi+uP+73qAjHFqD7FSrBOkbOezoeChsCmFuone
bp3VbJ9KFfz8wHIcBt6GSpO8NRE6WAa4hxCLLUyNGYK113o+oTGm+cM+7Y2ns/N+
xo1bgSPMxC4WSqCqyM8TpYslaSDy3KMVv4OuioR2/WVNg8h5Z1kxgkYOxuzBNd5I
kwlsRXjetD+7oLgOMZiKIwCsyatnGW6l4w0eQdhU10/ZMZxMHg4=
=byf2
-----END PGP SIGNATURE-----

--=-jrALHp1lXj5XqEfEufvE--





Information forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.

Message received at 47187 <at> debbugs.gnu.org:


Received: (at 47187) by debbugs.gnu.org; 16 Mar 2021 09:13:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 16 05:13:22 2021
Received: from localhost ([127.0.0.1]:38189 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lM5lS-0005YU-9H
	for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 05:13:22 -0400
Received: from mail.zaclys.net ([178.33.93.72]:39131)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <lle-bout@HIDDEN>) id 1lM5lO-0005YG-Q4
 for 47187 <at> debbugs.gnu.org; Tue, 16 Mar 2021 05:13:20 -0400
Received: from guix-xps.local (82-64-145-38.subs.proxad.net [82.64.145.38])
 (authenticated bits=0)
 by mail.zaclys.net (8.14.7/8.14.7) with ESMTP id 12G9DCqG026755
 (version=TLSv1/SSLv3 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NO);
 Tue, 16 Mar 2021 10:13:12 +0100
DMARC-Filter: OpenDMARC Filter v1.3.2 mail.zaclys.net 12G9DCqG026755
Authentication-Results: mail.zaclys.net;
 dmarc=fail (p=reject dis=none) header.from=zaclys.net
Authentication-Results: mail.zaclys.net;
 spf=fail smtp.mailfrom=lle-bout@HIDDEN
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=zaclys.net;
 s=default; t=1615885992;
 bh=EIugy+6++fIv207AXVbTCIyBMFclI0d7lw24w1s5dpw=;
 h=Subject:From:To:Date:In-Reply-To:References:From;
 b=f3q3kAhvsscCAKsnQnjul7M8YwTVI7t61EpZt2x6xYIEXbME4XIKyks+G4wCh5ZrQ
 mKU0b9fQQyYbZDM/Mib/OguJKvU+GgpPO9khe8OwkiDwwJoTC3t0H+ywyMKndyB7Og
 CC+fdLMFtMrxAC4wPe6bPalzWfnd85GtqgiM1lLY=
Message-ID: <81c654076d3c07c0f7d1657b4e44dce29a01eca2.camel@HIDDEN>
Subject: Re: [bug#47187] [PATCH] gnu: Add c-lightning.
From: =?ISO-8859-1?Q?L=E9o?= Le Bouter <lle-bout@HIDDEN>
To: ZmnSCPxj <ZmnSCPxj@HIDDEN>, 47187 <at> debbugs.gnu.org
Date: Tue, 16 Mar 2021 10:13:07 +0100
In-Reply-To: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
References: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-fTkqj1Uctj9F+r2ceg1h"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 47187
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)


--=-fTkqj1Uctj9F+r2ceg1h
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hello!

Thanks for the patch!

I built it and it was successful, however, few comments:

The various hacks in the recipe to get a working build seem troublesome
maintenance-wise, is it possible you think since you are a developer
that you make the build scripts a bit more standard in the project?

I see there's some vendored libraries: gheap, jsmn, libbacktrace,
libsodium, libwally-core ; Is it possible to unvendor those and create
separate packages for each of them? Or is there a requirement of strict
behavior here, either way, you could also create separate packages for
each of them even if those are specific to c-lightning. It would make
the main recipe cleaner, since as far as I can see it's also why the
build system is so stubborn to GNU Guix? That it also attempts to build
vendored libraries?

L=C3=A9o

--=-fTkqj1Uctj9F+r2ceg1h
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEEFIvLi9gL+xax3g6RRaix6GvNEKYFAmBQdqMACgkQRaix6GvN
EKa21RAAuoYovFsFSLETZcz/z6l0U8QfH/j5arsEgaftiHdxLpPEZr8tLw5mslJG
PM0dCdDC4YPvjhvgRMJRsiqUPwsdK3dya6cqMvWA/P0A10bM6iPPTFyLR4MIzeW0
DpwmZnJricitJcXfh405kRqfzSUfZIW9/vtueZfIaGhxscxwpogPDrfNMIGeZNFW
stSO7BeLusx6TRt2gKzs+/Yg0EkEwT6FKIAGwBloZEfShZ35nWd7G93kkKLMhQKf
5dqCZDSQcmM7uRvThEOV+R1q0kFHPzUBcR8lV8K0S+/urrJ8XQtCf8JRrLGpApgV
GoaMXJVNivmUBENkk6dnw+cHpPFf7dQ0fcpnjmqPve6blB1RQDEeOe0iK441edZU
AmBm6qmmQVgoRR5C7S2vwil+LMPfmBh3RER0+PhqH+AJ9XqQJhxlVfUYdeAluFFw
9Dq8PhZF8fE+sgnGDpNQFNt+RpCgfZiUw9nzeMko1DJGm5U4jZFIyurpuHupoxtn
EXIV9cGhSTPzkgEKI0dEcT3kt75taCYKJiou7Pu+n6g/B+rnMeIbd/NuciPKSbbQ
ArjdzYnNhgwW/joZ6g5bikBup+h5u3ScmJBzPPveGmaZv4tZ/29lhpKD/X1cER3e
VjvFgA42rIN52oow+hvWALyhgpmu/r8JH0Vd0BwAx9aOvAuDs7U=
=jKNa
-----END PGP SIGNATURE-----

--=-fTkqj1Uctj9F+r2ceg1h--





Information forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 16 Mar 2021 08:40:33 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Mar 16 04:40:32 2021
Received: from localhost ([127.0.0.1]:38147 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1lM5Fe-00048j-UQ
	for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 04:40:32 -0400
Received: from lists.gnu.org ([209.51.188.17]:43418)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ZmnSCPxj@HIDDEN>) id 1lM5Fb-00048a-Pz
 for submit <at> debbugs.gnu.org; Tue, 16 Mar 2021 04:40:29 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:44892)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ZmnSCPxj@HIDDEN>)
 id 1lM5Fb-0002vS-AF
 for guix-patches@HIDDEN; Tue, 16 Mar 2021 04:40:27 -0400
Received: from mail-40137.protonmail.ch ([185.70.40.137]:14418)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ZmnSCPxj@HIDDEN>)
 id 1lM5FX-0006yz-Ar
 for guix-patches@HIDDEN; Tue, 16 Mar 2021 04:40:26 -0400
Date: Tue, 16 Mar 2021 08:40:04 +0000
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=protonmail.com;
 s=protonmail; t=1615884014;
 bh=lxEq5WI1mmjXmIcUsvSOCZaFxaO8xLmoSuz+IWIu4pE=;
 h=Date:To:From:Reply-To:Subject:From;
 b=yjnSFcxFqb126UpQO9fhtdvfwVaur9SgOSnRehoW5ryPka0lFkM6HD77K6085BAdk
 U2Z2dpqbTrqvv496XtyBtXOBLXZiyOY3LjDlfadbpPnmpvMkoWEOlHN0D+oeYDYZFL
 GQwoJKRg7Q2VzvSVulfD9u0GxPAd5zkN6fpEISYQ=
To: "guix-patches@HIDDEN" <guix-patches@HIDDEN>
From: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Subject: [PATCH] gnu: Add c-lightning.
Message-ID: <pIzl3qY1MjcwPK5NRfKDs3jl1Gf8zurLPn-2e60bUbwzJgLjgoqtJZ82_CMCCRoIyinyASxSXrIxa4yXMW9VfDCfl8P0Gs2_6_J3Iz0r2Go=@protonmail.com>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Status: No, score=-1.2 required=10.0 tests=ALL_TRUSTED,DKIM_SIGNED,
 DKIM_VALID,DKIM_VALID_AU,DKIM_VALID_EF,FREEMAIL_FROM shortcircuit=no
 autolearn=disabled version=3.4.4
X-Spam-Checker-Version: SpamAssassin 3.4.4 (2020-01-24) on
 mailout.protonmail.ch
Received-SPF: pass client-ip=185.70.40.137;
 envelope-from=ZmnSCPxj@HIDDEN; helo=mail-40137.protonmail.ch
X-Spam_score_int: -15
X-Spam_score: -1.6
X-Spam_bar: -
X-Spam_report: (-1.6 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 FROM_LOCAL_NOVOWEL=0.5, RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H4=0.001,
 RCVD_IN_MSPIKE_WL=0.001, SPF_HELO_PASS=-0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.7 (/)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Reply-To: ZmnSCPxj <ZmnSCPxj@HIDDEN>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.8 (-)

Good morning Guix developers,

C-Lightning is one of the earliest living implementations of the Lightning
Network protocol.
I am one of the developers of this implementation.

I tested this as follows:

* Extracted the package `c-lightning` into a `c-lightning.scm` file.
* Ran `guix build --rounds=3D4 -f c-lightning.scm` on an X86-64 server to
  check for reproducability issues.
* Ran the compiled `lightningd` with `lightningd --version`,
  `lightningd --help`, and the developer-secret undocumented
  `lightningd --test-daemons-only` and checked they work correctly.
* Ran the compiled `lightningd` with `lightningd --proxy=3D127.0.0.1:9050
  --bind-addr=3D127.0.0.1:9735 --addr=3Dstatictor:127.0.0.1:9051
  --always-use-proxy=3Dtrue` on a server with Bitcoin and Tor installed,
  and checked that it indeed created an LN wallet and started operating
  the node.
  * This also tests the SQLITE database backend and that the sub-daemons
    are accessible and runnable.
* From an existing Lightning Network node, did a `lightning-cli connect`
  to the new node running on the above command, and confirmed that the
  above new node was connected to, and that it started downloading the
  gossip map (the first thing every new node does when it initially gets
  connected to the network).
* On a separate X86-64 machine with the same Guix commit version, copied
  the `c-lightning.scm` file and reran `guix build --rounds=3D4 -f
  c-lightning.scm` as well.
* Checked that the same `/gnu/store` path was produced on the separate
  machine as on the first test server.
* Checked that this command produced the exact same text on both computers:
  `find ${GNU_STORE_PATH} -type f | sort | xargs -n1 sha256sum`.
* Extracted the package `c-lightning-postgresql` into a
  `c-lightning-postgresql.scm` file.
* Ran `guix build --rounds=3D4 -f c-lightning-postgresql.scm` on a X86-64
  server to check for reproducibility issues.
* Chekced that `strings ${GNU_STORE_PATH}/bin/lightningd` for the
  `c-lightning-postgresql` had references to `PQ*` functions, and that
  the `c-lightning` version had none, meaning that `c-lightning-postgresql`
  indeed included PostgreSQL support.

Thus, PostgreSQL support is not fully tested yet, but I believe the
produced binaries will work correctly as a C-Lightning instance.
Therefore I would like to submit this patch for inclusion into Guix.

Some notes:

* C-Lightning includes an extensive suite of tests, but these require
  several additional python packages, some of which (e.g.
  `python-bitcoinlib`) are not yet in Guix.
  The tests are extensive and *very long*, running the entire test
  suite can take an hour or more depending on exact settings.
* I took advantage of the functional nature of Guix and disabled the
  relative-path `BINTOPKGLIBEXECDIR`, making it an absolute path, as
  it also incidentally solves the issue of "upgrading C-lightning may
  cause strange problems in a running instance".
  See patch for more details.
* Our `configure` and `Makefile` are weird and I should probably go
  fix those upstream.

Regards,
ZmnSCPxj

From 490aa608765228fbb6bb64b9fa679bb4c8145001 Mon Sep 17 00:00:00 2001
From: ZmnSCPxj jxPCSnmZ <ZmnSCPxj@HIDDEN>
Date: Mon, 15 Mar 2021 11:53:35 +0000
Subject: [PATCH] gnu: Add c-lightning.

* gnu/packages/finance.scm (c-lightning, c-lightning-postgresql): New varia=
bles.
---
 gnu/packages/finance.scm | 153 +++++++++++++++++++++++++++++++++++++++
 1 file changed, 153 insertions(+)

diff --git a/gnu/packages/finance.scm b/gnu/packages/finance.scm
index e6df40c168..4b565ddfeb 100644
--- a/gnu/packages/finance.scm
+++ b/gnu/packages/finance.scm
@@ -22,6 +22,7 @@
 ;;; Copyright =C2=A9 2020 Vinicius Monego <monego@HIDDEN>
 ;;; Copyright =C2=A9 2020 Carlo Holl <carloholl@HIDDEN>
 ;;; Copyright =C2=A9 2020 Giacomo Leidi <goodoldpaul@HIDDEN>
+;;; Copyright =C2=A9 2021 ZmnSCPxj jxPCSnmZ <ZmnSCPxj@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -57,6 +58,7 @@
   #:use-module (gnu packages aidc)
   #:use-module (gnu packages autotools)
   #:use-module (gnu packages base)
+  #:use-module (gnu packages bash)
   #:use-module (gnu packages boost)
   #:use-module (gnu packages check)
   #:use-module (gnu packages compression)
@@ -99,6 +101,7 @@
   #:use-module (gnu packages qt)
   #:use-module (gnu packages readline)
   #:use-module (gnu packages sphinx)
+  #:use-module (gnu packages sqlite)
   #:use-module (gnu packages texinfo)
   #:use-module (gnu packages textutils)
   #:use-module (gnu packages time)
@@ -1676,3 +1679,152 @@ Interface (UI) for the hledger accounting system.  =
It can be used as a
 local, single-user UI, or as a multi-user UI for viewing, adding, and
 editing on the Web.")
     (license license:gpl3)))
+
+(define-public c-lightning
+  (package
+    (name "c-lightning")
+    (version "0.9.3")
+    (source
+      (origin
+        (method url-fetch)
+        (uri (string-append
+               "https://github.com/ElementsProject/lightning/releases/down=
load/"
+               "v" version "/clightning-v" version ".zip"))
+        (sha256
+          (base32 "1chqzxcqpr49vbayrw4213lznmyw4lcghcdh6afxbk4bxlhkjmml"))=
))
+    (build-system gnu-build-system)
+    (native-inputs
+      `(("autoconf" ,autoconf)
+        ("automake" ,automake)
+        ("bash" ,bash)
+        ("libtool" ,libtool)
+        ("pkg-config" ,pkg-config)
+        ("python" ,python)
+        ("python-mako" ,python-mako)
+        ("sed" ,sed)
+        ("unzip" ,unzip)
+        ("which" ,which)))
+    (inputs
+      `(("gmp" ,gmp)
+        ("sqlite" ,sqlite)
+        ("zlib" ,zlib)))
+    (arguments
+      ;; Tests exist, but need a lot of Python packages (some not availabl=
e
+      ;; on Guix) and they are incompatible with our BINTOPKGLIBEXECDIR ha=
ck.
+      `(#:tests? #f
+        #:phases
+        (modify-phases %standard-phases
+          (add-before 'configure 'patch-makefile
+            (lambda _
+              (substitute* "Makefile"
+                ;; The C-lightning Makefile uses the PYTHONPATH
+                ;; variable, which causes Guix builds to fail
+                ;; since the environment variable is masked by
+                ;; the Makefile variable.
+                ;; Insert the contents of the variable into the
+                ;; Makefile.
+                (("^PYTHONPATH=3D")
+                 (string-append
+                   "PYTHONPATH=3D"
+                   (getenv "PYTHONPATH")
+                   ":"))
+                ;; C-lightning will spawn a number of other
+                ;; processes from binaries installed in its
+                ;; libexecdir.
+                ;; It normally uses relative paths so that
+                ;; users can move around the installation
+                ;; location.
+                ;; However, it does have the drawback that if
+                ;; the installation location is overwritten
+                ;; with a newer version while an existing
+                ;; instance is still running, any new
+                ;; sub-processes launched will be the new
+                ;; version, which is likely incompatible with
+                ;; the running instance.
+                ;; Since Guix would not allow the store
+                ;; directory to be moved anyway, we use an
+                ;; absolute path instead in the below
+                ;; substitution.
+                ;; With Guix and an absolute path, even if a
+                ;; running instance was launched from a
+                ;; profile and that profile is upgraded to a
+                ;; newer version, the running instance will
+                ;; refer to the absolute store directory
+                ;; containing binaries of the running version.
+                (("BINTOPKGLIBEXECDIR=3D.*$")
+                 "BINTOPKGLIBEXECDIR=3D'\"'\"$(pkglibexecdir)\"'\"'\n"))))
+          ;; C-lightning configure is unusual, it does not understand
+          ;; the standard parameters Guix passes in, so, provide those
+          ;; by env variables.
+          (replace 'configure
+            (lambda* (#:key outputs inputs (configure-flags '())
+                      #:allow-other-keys)
+              (let* ((bash    (string-append (assoc-ref inputs "bash") "/b=
in/bash"))
+                     (python  (assoc-ref inputs "python"))
+                     (prefix  (assoc-ref outputs "out"))
+                     (flags   (cons*
+                                "CC=3Dgcc"
+                                (string-append "--prefix=3D" prefix)
+                                configure-flags)))
+                (setenv "PYTHON" (string-append python "/bin/python3"))
+                (setenv "CONFIG_SHELL" bash)
+                (setenv "SHELL" bash)
+                (format #t "build directory: ~s~%" (getcwd))
+                (format #t "configure flags: ~s~%" flags)
+                (apply invoke bash
+                       "./configure"
+                       flags))))
+          ;; Rather than call the configure script of its external
+          ;; libraries from its own configure script, the external
+          ;; configure are created and called at build time.
+          ;; Unfortunately, it is a single Makefile stanza which
+          ;; does the autoreconf, configure, and make.
+          ;; This means we cannot "cut" through here by creating
+          ;; the external configure before this step (the Makefile
+          ;; stanza will recreate and overwrite the external
+          ;; configure), we have to modify the autogen.sh scripts
+          ;; of the external libraries instead so that they
+          ;; patch the shebangs after autoreconf.
+          (add-before 'build 'fix-autoreconf
+            (lambda _
+              (substitute* "external/libsodium/autogen.sh"
+                (("exec autoreconf(.*)$" exec-autoreconf flags)
+                 (string-append
+                   "autoreconf" flags
+                   "sed 's:/bin/sh:" (getenv "SHELL") ":g' < configure > c=
onfigure.tmp\n"
+                   "mv configure.tmp configure\n"
+                   "chmod +x configure\n"
+                   "exit 0\n")))
+              (substitute* "external/libwally-core/tools/autogen.sh"
+                (("autoreconf(.*)$" autoreconf)
+                 (string-append
+                   autoreconf
+                   "sed 's:/bin/sh:" (getenv "SHELL") ":g' < configure > c=
onfigure.tmp\n"
+                   "mv configure.tmp configure\n"
+                   "chmod +x configure\n"))))))))
+    (home-page "https://github.com/ElementsProject/lightning")
+    (synopsis "Lightweight Lightning Network protocol implementation in C"=
)
+    (description
+      "c-lightning is a lightweight, highly customizable, and standard
+compliant implementation of the Lightning Network protocol.")
+    (license license:expat)))
+
+(define-public c-lightning-postgresql
+  (package
+    (inherit c-lightning)
+    (name "c-lightning-postgresql")
+    (inputs
+      `(("postgresql" ,postgresql)
+        ;; C-Lightning requires SQLITE3 as of 0.9.3, and will
+        ;; fail to build if it is not found.
+        ;; (The configure script will allow PostgreSQL without
+        ;; SQLITE3 but some build tool of C-Lightning fails if
+        ;; SQLITE3 is not found.)
+        ,@(package-inputs c-lightning)))
+    (description
+      "c-lightning is a lightweight, highly customizable, and standard
+compliant implementation of the Lightning Network protocol.
+
+This package includes support for using a PostgreSQL database to back
+your node; you will need to set up the PostgreSQL separately and pass
+in its details using an appropriate flag setting.")))
--
2.30.1





Acknowledgement sent to ZmnSCPxj <ZmnSCPxj@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#47187; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 18 Mar 2021 06:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.