GNU bug report logs - #75400
[PATCH] Fixing python-spacy

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 75400 in the body.
You can then email your comments to 75400 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alice BRENON <alice.brenon <at> ens-lyon.fr>
To: guix-patches <at> gnu.org
Subject: [PATCH] Fixing python-spacy
Date: Mon, 6 Jan 2025 12:44:14 +0100

[Message part 1 (text/plain, inline)]

Hi all,

The package `python-spacy` seems to have been broken at least partially by the
recent changes in python tooling. The attached patch is a very short first step
into fixing it. This message doesn't hope to get it accepted but is rather meant
to open the discussion on the topic.

- the first issue is naturally the new required dependencies in `native-inputs`
  mentioned in bdde278dc9c565d8e9d11427c3a670ba86808af5 an visible in many
  python packages commit these past weeks. Surprisingly, I managed to compile
  `python-spacy` by adding only `python-wheel`, without `python-setuptools`.
  What makes both required? Are we sure that both were required in each and
  every python package recently modified?

- the second issue which I can't link with the changes in our tooling is caused
  by two packages which tests have broken: `python-aws-xray-sdk` and
  `python-jose`. First one seems to have a problem in sqlalchemy version (it
  seems to be using too recent a syntax; which is weird considering the fact
  that it uses version 2 and that it used to work — `python-aws-xray-sdk` hasn't
  been modified since cec543e7 on april the 27th 2023!). Second one now has
  trouble checking several certificates for mere formatting reasons (expected
  and actual PEMs are almost the same, only newlines are different; fixing the
  code of the test itself, another one breaks somewhere else. It seems that
  these source codes aren't sturdy enough and don't work at all in the package
  environment guix provides.

- these packages are dependencies of `python-moto`. Removing them from the
  dependencies, many tests break as expected for lack of these modules, but
  disabling them isn't enough: I played a game of whack-a-mole yesterday trying
  to get the tests to pass. Strangely, whereas testing with hypothesis runs all
  the tests and then reports all that have failed, fixing the one reported and
  running the tests again doesn't help: new tests break in different files that
  were apparently fine before… I had three iterations of this process, each one
  consuming half and hour on my machine. As far as I am concerned, these tests
  are unreliable and don't bring any additional confidence to the package. I
  have disabled them to test the fix for `python-spacy` and have been able to
  compile it and use the package.

Appart from that, everything looks ok. `python-moto` appears as a dependency of
`python-smart-open` which itself is both a direct dependency of `python-spacy`
and a dependency of `python-pathy` which is also a dependency of `python-spacy`.

python-smart-open -> python-moto
python-pathy -> python-smart-open
python-spacy -> python-smart-open, python-pathy

I understand that disabling the tests on a package entirely isn't a very sound
practice for a distribution so I doubt this is an acceptable fix. As I
understand the situation the next steps will be to sort things between
`python-aws-xray-sdk` and `python-sqlalchemy` and to understand why the tests in
`python-jose` have suddenly broken so bad.

Best,

Alice

[0001-gnu-python-spacy-Fix-build.patch (text/x-patch, attachment)]

Message #10 received at 75400-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Alice BRENON <alice.brenon <at> ens-lyon.fr>
Cc: sharlatanus <at> gmail.com, me <at> bonfacemunyoki.com, lars <at> 6xq.net,
 75400-done <at> debbugs.gnu.org, jgart <at> dismail.de, marius <at> gnu.org,
 tanguy <at> bioneland.org
Subject: Re: [bug#75400] [PATCH] Fixing python-spacy
Date: Mon, 06 Jan 2025 15:58:32 +0100

Hello Alice,

Alice BRENON <alice.brenon <at> ens-lyon.fr> skribis:

>>From 9a349ac01cec33fc1e49ba7f80ae18ef5cfdf922 Mon Sep 17 00:00:00 2001
> Message-ID: <9a349ac01cec33fc1e49ba7f80ae18ef5cfdf922.1736155491.git.alice.brenon <at> ens-lyon.fr>
> From: Alice BRENON <alice.brenon <at> ens-lyon.fr>
> Date: Sat, 4 Jan 2025 23:01:50 +0100
> Subject: [PATCH] gnu: python-spacy: Fix build.
>
> * gnu/packages/machine-learning.scm (python-spacy): Fix build.
> [native-inputs]: Add python-wheel
>
> Change-Id: I6550f6dc0cb1fb88fc733f6471f1831c4ba95977

Applied.  Thanks for the debugging session and thorough analysis.

Cc’ing the Python team for feedback on your other comments:

  https://issues.guix.gnu.org/75400

Ludo’.

Message #13 received at 75400-done <at> debbugs.gnu.org (full text, mbox):

From: Lars-Dominik Braun <lars <at> 6xq.net>
To: Alice BRENON <alice.brenon <at> ens-lyon.fr>,
 Ludovic Courtès <ludo <at> gnu.org>
Cc: tanguy <at> bioneland.org, me <at> bonfacemunyoki.com, 75400-done <at> debbugs.gnu.org,
 jgart <at> dismail.de, marius <at> gnu.org, sharlatanus <at> gmail.com
Subject: Re: [PATCH] Fixing python-spacy
Date: Tue, 7 Jan 2025 15:18:32 +0100

Hi Alice,

via:

> Cc’ing the Python team for feedback on your other comments:
> 
>   https://issues.guix.gnu.org/75400

> - the first issue is naturally the new required dependencies in `native-inputs`
>   mentioned in bdde278dc9c565d8e9d11427c3a670ba86808af5 an visible in many
>   python packages commit these past weeks. Surprisingly, I managed to compile
>   `python-spacy` by adding only `python-wheel`, without `python-setuptools`.
>   What makes both required? Are we sure that both were required in each and
>   every python package recently modified?

in short: yes. I can’t tell for sure for this exact case because
the output of `guix graph` is too big for `xdot` to handle, but it’s
likely python-setuptools is implicitly propagated through some dependency
(perhaps a bug like https://issues.guix.gnu.org/25235) to python-spacy
and thus it looks like it’s “not required”.

Lars

Message #16 received at 75400-done <at> debbugs.gnu.org (full text, mbox):

From: Alice BRENON <alice.brenon <at> ens-lyon.fr>
To: Lars-Dominik Braun <lars <at> 6xq.net>
Cc: tanguy <at> bioneland.org, me <at> bonfacemunyoki.com,
 Ludovic Courtès <ludo <at> gnu.org>, 75400-done <at> debbugs.gnu.org,
 jgart <at> dismail.de, marius <at> gnu.org, sharlatanus <at> gmail.com
Subject: Re: [PATCH] Fixing python-spacy
Date: Tue, 7 Jan 2025 15:27:58 +0100

Hi Lars!

Thanks for the great insight. Yeah, I tried generating that graph when I started
the investigation because at first I couldn't understand where the crashing
version of `python-moto` was still being used… ^^' I got a PNG over 25Mo. I
ended up browsing the dot graph in text directly. Ok, so I think I'll make
another patch to add setuptools (can I send the patch to this closed issue or
will I need to open a new one?).

Any opinion on all the broken tests, anyone?

Alice

Le Tue, 7 Jan 2025 15:18:32 +0100,
Lars-Dominik Braun <lars <at> 6xq.net> a écrit :

> Hi Alice,
> 
> via:
> 
> > Cc’ing the Python team for feedback on your other comments:
> > 
> >   https://issues.guix.gnu.org/75400  
> 
> > - the first issue is naturally the new required dependencies in
> > `native-inputs` mentioned in
> > bdde278dc9c565d8e9d11427c3a670ba86808af5 an visible in many python
> > packages commit these past weeks. Surprisingly, I managed to
> > compile `python-spacy` by adding only `python-wheel`, without
> > `python-setuptools`. What makes both required? Are we sure that
> > both were required in each and every python package recently
> > modified?  
> 
> in short: yes. I can’t tell for sure for this exact case because
> the output of `guix graph` is too big for `xdot` to handle, but it’s
> likely python-setuptools is implicitly propagated through some
> dependency (perhaps a bug
> like https://issues.guix.gnu.org/25235) to python-spacy and thus it looks like it’s “not required”.
> 
> Lars
> 

Message #19 received at 75400-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Alice BRENON <alice.brenon <at> ens-lyon.fr>
Cc: sharlatanus <at> gmail.com, me <at> bonfacemunyoki.com,
 Lars-Dominik Braun <lars <at> 6xq.net>, 75400-done <at> debbugs.gnu.org,
 jgart <at> dismail.de, marius <at> gnu.org, tanguy <at> bioneland.org
Subject: Re: [PATCH] Fixing python-spacy
Date: Wed, 08 Jan 2025 11:08:52 +0100

Alice BRENON <alice.brenon <at> ens-lyon.fr> skribis:

> Thanks for the great insight. Yeah, I tried generating that graph when I started
> the investigation because at first I couldn't understand where the crashing
> version of `python-moto` was still being used… ^^' I got a PNG over 25Mo.

Note that you use, say, ‘guix graph -M3’ to cut the graph at distance 3.
Or you can use things like ‘guix graph --path python-spacy python-moto’.

Ludo’.

Message #22 received at 75400-done <at> debbugs.gnu.org (full text, mbox):

From: "jgart" <jgart <at> dismail.de>
To: "Ludovic Courtès" <ludo <at> gnu.org>, "Alice BRENON"
 <alice.brenon <at> ens-lyon.fr>
Cc: sharlatanus <at> gmail.com, me <at> bonfacemunyoki.com,
 Lars-Dominik Braun <lars <at> 6xq.net>, 75400-done <at> debbugs.gnu.org, marius <at> gnu.org,
 tanguy <at> bioneland.org
Subject: Re: [PATCH] Fixing python-spacy
Date: Wed, 08 Jan 2025 15:30:01 +0000

[Message part 1 (text/plain, inline)]

> Note that you use, say, ‘guix graph -M3’ to cut the graph at distance 3.


Hi, should the CLI documentation for guix graph mention that graphviz is the default backend?

For example, when calling `guix graph --list-backends`?

all best,

jgart

[Message part 2 (text/html, inline)]

Message #25 received at 75400-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: "jgart" <jgart <at> dismail.de>
Cc: sharlatanus <at> gmail.com, me <at> bonfacemunyoki.com,
 Lars-Dominik Braun <lars <at> 6xq.net>, 75400-done <at> debbugs.gnu.org, marius <at> gnu.org,
 tanguy <at> bioneland.org, Alice BRENON <alice.brenon <at> ens-lyon.fr>
Subject: Re: [PATCH] Fixing python-spacy
Date: Thu, 09 Jan 2025 14:30:37 +0100

Hi,

"jgart" <jgart <at> dismail.de> skribis:

>> Note that you use, say, ‘guix graph -M3’ to cut the graph at distance 3.
>
>
> Hi, should the CLI documentation for guix graph mention that graphviz is the default backend?
>
> For example, when calling `guix graph --list-backends`?

Sure, why not!

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.