GNU bug report logs - #38228
Fwd: [PATCH] gnu: boost: Build with python3

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 38228 in the body.
You can then email your comments to 38228 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: goodoldpaul <at> autistici.org
To: guix-patches <at> gnu.org
Subject: Fwd: [PATCH] gnu: boost: Build with python3
Date: Sat, 16 Nov 2019 09:04:44 +0000

[Message part 1 (text/plain, inline)]

Hello I'm opening another thread ( old one was 
http://issues.guix.gnu.org/issue/38128 ) to build boost in core-updates 
with python3. The new patch also parameterizes python versions across 
the package definition.

WDYT?

Bye,

Giacomo

-------- Original Message --------
Subject: Re: bug#38128: [PATCH] gnu: Add boost-with-python3
Date: 2019-11-12 00:06
From: goodoldpaul <at> autistici.org
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: 38128 <at> debbugs.gnu.org, mbakke <at> fastmail.com

On 2019-11-11 09:36, Efraim Flashner wrote:
> I'm going to re-open this one, sorry.
> 
> Can we replace the '--with-python-version=3.7' and 
> 'libboost_python37.so'
> with parameterized python variables so we don't have to bump it when we
> get a new python version?
> 
> Also, I've attached a package that I've never actually built that uses
> boost built with python3 if you want something to test it out with :)

I'm not sure how to send a patch for core-updates, I never did it so I 
attached it here. Please tell me if I should open another patch.

The patch builds boost with python3 and parameterizes the python 
version, as Efraim suggested. I built it successfully on core updates. 
When I tried building Epour on core-updates and saw that Guix was 
starting to build Bash 5.0 I renounced (:D) and I built it on master 
with boost-with-python3 .

Libtorrent-rasterbar seems to build fine on master but some tests fail 
to pass and they all seem to be network related but then again this is 
torrent we are talking about. I didn't investigate further but I attach 
the log.

I also tried boost-with-python3 with Malmo ( 
https://github.com/microsoft/malmo ) and it seemed to compile fine just 
but that package has other problems such as trying to start gradle so I 
nerver managed to actually run it.

Let me know what you think about the patch,

Bye

Giacomo

[0001-gnu-boost-Build-with-python3.patch (text/x-diff, attachment)]

[epour.log (text/plain, attachment)]

Message #8 received at 38228 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: goodoldpaul <at> autistici.org, 38228 <at> debbugs.gnu.org
Subject: Re: [bug#38228] Fwd: [PATCH] gnu: boost: Build with python3
Date: Wed, 20 Nov 2019 01:01:36 +0100

[Message part 1 (text/plain, inline)]

goodoldpaul <at> autistici.org writes:

> Hello I'm opening another thread ( old one was 
> http://issues.guix.gnu.org/issue/38128 ) to build boost in core-updates 
> with python3. The new patch also parameterizes python versions across 
> the package definition.
>
> WDYT?
>
> Bye,
>
> Giacomo
>
> -------- Original Message --------
> Subject: Re: bug#38128: [PATCH] gnu: Add boost-with-python3
> Date: 2019-11-12 00:06
>  From: goodoldpaul <at> autistici.org
> To: Efraim Flashner <efraim <at> flashner.co.il>
> Cc: 38128 <at> debbugs.gnu.org, mbakke <at> fastmail.com
>
> On 2019-11-11 09:36, Efraim Flashner wrote:
>> I'm going to re-open this one, sorry.
>> 
>> Can we replace the '--with-python-version=3.7' and 
>> 'libboost_python37.so'
>> with parameterized python variables so we don't have to bump it when we
>> get a new python version?
>> 
>> Also, I've attached a package that I've never actually built that uses
>> boost built with python3 if you want something to test it out with :)
>
> I'm not sure how to send a patch for core-updates, I never did it so I 
> attached it here. Please tell me if I should open another patch.

Excellent.  Here is fine.  :-)

Note that 'boost-with-python3' has already been merged to core-updates.
What we need now is 1) a patch that makes these adjustments to
'boost-with-python3', and 2) a follow-up patch that removes
boost-with-python3 and adjusts the original 'boost' package instead.

The latter will be applied to core-updates after the first patch gets
merged, but you can make (and test :-)) both patches on a local checkout
of the 'master' branch.

> The patch builds boost with python3 and parameterizes the python 
> version, as Efraim suggested. I built it successfully on core updates. 
> When I tried building Epour on core-updates and saw that Guix was 
> starting to build Bash 5.0 I renounced (:D) and I built it on master 
> with boost-with-python3 .

In general, working on the 'core-updates' branch requires a fairly
powerful computer (and lots of time), sorry about that!

> Libtorrent-rasterbar seems to build fine on master but some tests fail 
> to pass and they all seem to be network related but then again this is 
> torrent we are talking about. I didn't investigate further but I attach 
> the log.

Strange.  I suppose these tests are not run when using Boost + Python 2?
In any case we don't have to worry about that just yet ;-)

> I also tried boost-with-python3 with Malmo ( 
> https://github.com/microsoft/malmo ) and it seemed to compile fine just 
> but that package has other problems such as trying to start gradle so I 
> nerver managed to actually run it.

Cool.  Packaging 'gradle' should be a rewarding experience that makes
many people very happy, if you would like a challenge.  ;-)


> From 91a25fb143ad0e2e20e8ddadea0c0610849adf92 Mon Sep 17 00:00:00 2001
> From: Giacomo Leidi <goodoldpaul <at> autistici.org>
> Date: Tue, 12 Nov 2019 00:24:49 +0100
> Subject: [PATCH] gnu: boost: Build with python3.
>
> * gnu/packages/boost.scm (boost):
> [arguments]: Parameterize python version.
> [native-inputs]: Use python3.

[...]

>      (arguments
> -     `(#:tests? #f
> +     `(#:modules ((guix build gnu-build-system)
> +                  (guix build utils)
> +                  (srfi srfi-1))

If you add (guix build python-build-system) in there ...

>           (replace 'configure
>             (lambda* (#:key inputs outputs #:allow-other-keys)
>               (let ((icu (assoc-ref inputs "icu4c"))
> +                   (python (assoc-ref inputs "python"))
> +                   (python-version
> +                    (take (string-split ,(package-version python) #\.) 2))

... then you can use (python-version (python-version python)) here and
below.

> +               (substitute* "tools/build/src/tools/python.jam"
> +                 (("include/python\\$\\(version\\)")
> +                  "include/python$(version)m"))

I wonder why we have to add the 'm'?  Will that work with Python 2?

Otherwise LGTM.

For 'core-updates', it would be really great to add a 'python2' output
to Boost in case there are packages that do not work with
libboost_python3.so, but I'm not sure how to achieve that.  Future
problems!  :-)

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 38228 <at> debbugs.gnu.org (full text, mbox):

From: goodoldpaul <at> autistici.org
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 38228 <at> debbugs.gnu.org
Subject: Re: [bug#38228] Fwd: [PATCH] gnu: boost: Build with python3
Date: Mon, 09 Dec 2019 11:24:12 +0000

[Message part 1 (text/plain, inline)]

Hello Marius,
I apologize for taking so long: the end of the semester is near and my 
time is a little limited.

On 2019-11-20 00:01, Marius Bakke wrote:
> goodoldpaul <at> autistici.org writes:
> 
>> Hello I'm opening another thread ( old one was
>> http://issues.guix.gnu.org/issue/38128 ) to build boost in 
>> core-updates
>> with python3. The new patch also parameterizes python versions across
>> the package definition.
>> 
>> WDYT?
>> 
>> Bye,
>> 
>> Giacomo
>> 
>> -------- Original Message --------
>> Subject: Re: bug#38128: [PATCH] gnu: Add boost-with-python3
>> Date: 2019-11-12 00:06
>>  From: goodoldpaul <at> autistici.org
>> To: Efraim Flashner <efraim <at> flashner.co.il>
>> Cc: 38128 <at> debbugs.gnu.org, mbakke <at> fastmail.com
>> 
>> On 2019-11-11 09:36, Efraim Flashner wrote:
>>> I'm going to re-open this one, sorry.
>>> 
>>> Can we replace the '--with-python-version=3.7' and
>>> 'libboost_python37.so'
>>> with parameterized python variables so we don't have to bump it when 
>>> we
>>> get a new python version?
>>> 
>>> Also, I've attached a package that I've never actually built that 
>>> uses
>>> boost built with python3 if you want something to test it out with :)
>> 
>> I'm not sure how to send a patch for core-updates, I never did it so I
>> attached it here. Please tell me if I should open another patch.
> 
> Excellent.  Here is fine.  :-)
> 
> Note that 'boost-with-python3' has already been merged to core-updates.
> What we need now is 1) a patch that makes these adjustments to
> 'boost-with-python3', and 2) a follow-up patch that removes
> boost-with-python3 and adjusts the original 'boost' package instead.
> 
> The latter will be applied to core-updates after the first patch gets
> merged, but you can make (and test :-)) both patches on a local 
> checkout
> of the 'master' branch.

I'm attaching the two patches. As you said the one that parameterizes 
the python version of boost-with-python3 must be applied to master, the 
other one to core-updates.

>> The patch builds boost with python3 and parameterizes the python
>> version, as Efraim suggested. I built it successfully on core updates.
>> When I tried building Epour on core-updates and saw that Guix was
>> starting to build Bash 5.0 I renounced (:D) and I built it on master
>> with boost-with-python3 .
> 
> In general, working on the 'core-updates' branch requires a fairly
> powerful computer (and lots of time), sorry about that!
> 
>> Libtorrent-rasterbar seems to build fine on master but some tests fail
>> to pass and they all seem to be network related but then again this is
>> torrent we are talking about. I didn't investigate further but I 
>> attach
>> the log.
> 
> Strange.  I suppose these tests are not run when using Boost + Python 
> 2?
> In any case we don't have to worry about that just yet ;-)

I gave another look at Libtorrent-rasterbar and I noticed that it 
depends directly on python2, so the answer to the failing tests could be 
that upgrading both boost and the python version ( as was done in 
Efraim's code that I used last time to test boost-with-python3 )  broke 
something that a closer look to the failing tests could figure out.

Right now I tested the upgraded version without problems by building the 
following ( randomly selected) packages: innoextract, swig, libarea, 
pbbam, cgal, openimageio. If you have a better idea of what must be 
tested please tell me and I'll do some more tests.

>> I also tried boost-with-python3 with Malmo (
>> https://github.com/microsoft/malmo ) and it seemed to compile fine 
>> just
>> but that package has other problems such as trying to start gradle so 
>> I
>> nerver managed to actually run it.
> 
> Cool.  Packaging 'gradle' should be a rewarding experience that makes
> many people very happy, if you would like a challenge.  ;-)
> 
> 
>> From 91a25fb143ad0e2e20e8ddadea0c0610849adf92 Mon Sep 17 00:00:00 2001
>> From: Giacomo Leidi <goodoldpaul <at> autistici.org>
>> Date: Tue, 12 Nov 2019 00:24:49 +0100
>> Subject: [PATCH] gnu: boost: Build with python3.
>> 
>> * gnu/packages/boost.scm (boost):
>> [arguments]: Parameterize python version.
>> [native-inputs]: Use python3.
> 
> [...]
> 
>>      (arguments
>> -     `(#:tests? #f
>> +     `(#:modules ((guix build gnu-build-system)
>> +                  (guix build utils)
>> +                  (srfi srfi-1))
> 
> If you add (guix build python-build-system) in there ...

[...]

>>           (replace 'configure
>>             (lambda* (#:key inputs outputs #:allow-other-keys)
>>               (let ((icu (assoc-ref inputs "icu4c"))
>> +                   (python (assoc-ref inputs "python"))
>> +                   (python-version
>> +                    (take (string-split ,(package-version python) 
>> #\.) 2))
> 
> ... then you can use (python-version (python-version python)) here and
> below.

I did add python-build-system to #:modules, but Guix kept complaining 
that python-build-system was not importable until I added

#:imported-modules ((guix build python-build-system)
                           ,@%gnu-build-system-modules)

Do you happen to have any idea of the differences from 
#:imported-modules and #:modules? And why was using only 
gnu-build-system possible also without adding it to #:imported-modules?

Or even better: is there any kind of documentation of the arguments and 
keywords and return values of Guix's functions? Something like this 
https://flask.palletsprojects.com/en/1.1.x/api/ .

I don't really like my solution mainly because now there's two lines 
containing 4 times the "python" word, so please if you have any further 
suggestions to how to get this more readable please tell me.

>> +               (substitute* "tools/build/src/tools/python.jam"
>> +                 (("include/python\\$\\(version\\)")
>> +                  "include/python$(version)m"))
> 
> I wonder why we have to add the 'm'?  Will that work with Python 2?

I'm not really sure but grepping for "include/python" inside 
gnu/packages returns some examples where with python3 this "m" seems to 
be standard practice. Maybe the Python team changed their directory 
schema between the two releases? Anyway it won't  work with python2 that 
(also shown by grep -nr "include/python" $GUIX_SOURCE/gnu/packages) 
doesn't use the "m".

Thank you for your review Marius,

Giacomo

[0001-gnu-boost-Build-with-python3.patch (text/x-diff, attachment)]

[0001-gnu-boost-with-python3-Parameterize-python-version.patch (text/x-diff, attachment)]

Message #16 received at 38228-done <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: goodoldpaul <at> autistici.org
Cc: 38228-done <at> debbugs.gnu.org
Subject: Re: [bug#38228] Fwd: [PATCH] gnu: boost: Build with python3
Date: Wed, 08 Jan 2020 22:11:17 +0100

[Message part 1 (text/plain, inline)]

goodoldpaul <at> autistici.org writes:

> Hello Marius,
> I apologize for taking so long: the end of the semester is near and my 
> time is a little limited.

No worries.  :-)

> I'm attaching the two patches. As you said the one that parameterizes 
> the python version of boost-with-python3 must be applied to master, the 
> other one to core-updates.

Excellent!  I went ahead and adjusted it to work with the Boost
cross-compilation changes on 'core-updates', as well as simplified it a
little:

https://git.savannah.gnu.org/cgit/guix.git/commit/?h=core-updates&id=2ac164a8107dbb07ba1ed29986859d3e936f795a

>>> The patch builds boost with python3 and parameterizes the python
>>> version, as Efraim suggested. I built it successfully on core updates.
>>> When I tried building Epour on core-updates and saw that Guix was
>>> starting to build Bash 5.0 I renounced (:D) and I built it on master
>>> with boost-with-python3 .
>> 
>> In general, working on the 'core-updates' branch requires a fairly
>> powerful computer (and lots of time), sorry about that!
>> 
>>> Libtorrent-rasterbar seems to build fine on master but some tests fail
>>> to pass and they all seem to be network related but then again this is
>>> torrent we are talking about. I didn't investigate further but I 
>>> attach
>>> the log.
>> 
>> Strange.  I suppose these tests are not run when using Boost + Python 
>> 2?
>> In any case we don't have to worry about that just yet ;-)
>
> I gave another look at Libtorrent-rasterbar and I noticed that it 
> depends directly on python2, so the answer to the failing tests could be 
> that upgrading both boost and the python version ( as was done in 
> Efraim's code that I used last time to test boost-with-python3 )  broke 
> something that a closer look to the failing tests could figure out.
>
> Right now I tested the upgraded version without problems by building the 
> following ( randomly selected) packages: innoextract, swig, libarea, 
> pbbam, cgal, openimageio. If you have a better idea of what must be 
> tested please tell me and I'll do some more tests.

Thanks for testing!  I had to adjust 'Vigra' to work with Python 3, but
it was straightforward:

https://git.savannah.gnu.org/cgit/guix.git/commit/?h=core-updates&id=a82e6faa8b993d1f3b47a8bd22c4509f7cae7ec1

>>> From 91a25fb143ad0e2e20e8ddadea0c0610849adf92 Mon Sep 17 00:00:00 2001
>>> From: Giacomo Leidi <goodoldpaul <at> autistici.org>
>>> Date: Tue, 12 Nov 2019 00:24:49 +0100
>>> Subject: [PATCH] gnu: boost: Build with python3.
>>> 
>>> * gnu/packages/boost.scm (boost):
>>> [arguments]: Parameterize python version.
>>> [native-inputs]: Use python3.
>> 
>> [...]
>> 
>>>      (arguments
>>> -     `(#:tests? #f
>>> +     `(#:modules ((guix build gnu-build-system)
>>> +                  (guix build utils)
>>> +                  (srfi srfi-1))
>> 
>> If you add (guix build python-build-system) in there ...
>
> [...]
>
>>>           (replace 'configure
>>>             (lambda* (#:key inputs outputs #:allow-other-keys)
>>>               (let ((icu (assoc-ref inputs "icu4c"))
>>> +                   (python (assoc-ref inputs "python"))
>>> +                   (python-version
>>> +                    (take (string-split ,(package-version python) 
>>> #\.) 2))
>> 
>> ... then you can use (python-version (python-version python)) here and
>> below.
>
> I did add python-build-system to #:modules, but Guix kept complaining 
> that python-build-system was not importable until I added
>
> #:imported-modules ((guix build python-build-system)
>                             ,@%gnu-build-system-modules)
>
> Do you happen to have any idea of the differences from 
> #:imported-modules and #:modules? And why was using only 
> gnu-build-system possible also without adding it to #:imported-modules?

Caleb explains it very well here:

https://lists.gnu.org/archive/html/guix-devel/2019-12/msg00105.html

> Or even better: is there any kind of documentation of the arguments and 
> keywords and return values of Guix's functions? Something like this 
> https://flask.palletsprojects.com/en/1.1.x/api/ .

Unfortunately no, but documenting parts of the Guix API is being
discussed.  The procedures are typically documented in the source code
though, so 'git grep' is the best tool for now.  :-/

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.