GNU bug report logs - #50105
[core-updates] Python sitecustomize issue.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 50105 in the body.
You can then email your comments to 50105 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: bug-guix <at> gnu.org
Cc: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: [core-updates] Python sitecustomize issue.
Date: Wed, 18 Aug 2021 11:53:07 +0200

Hello,

I tried to upgrade glade to the 3.38.2 release on core-update and
encountered the following test issue:

--8<---------------cut here---------------start------------->8---
2/5 modules               FAIL            0.29s   killed by signal 5 SIGTRAP
>>> GLADE_MODULE_SEARCH_PATH=/home/mathieu/glade-3.38.2/build/plugins/gtk+:/home/mathieu/glade-3.38.2/build/plugins/python:/home/mathieu/glade-3.38.2/build/plugins/gjs:/home/mathieu/glade-3.38.2/tests/modules MALLOC_PERTURB_=195 GLADE_TESTING=1 GLADE_CATALOG_SEARCH_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+:/home/mathieu/glade-3.38.2/plugins/python:/home/mathieu/glade-3.38.2/plugins/gjs:/home/mathieu/glade-3.38.2/tests/catalogs GLADE_PIXMAP_DIR=/home/mathieu/glade-3.38.2/data/icons GLADE_ICON_THEME_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+/icons/22x22 /home/mathieu/glade-3.38.2/build/tests/modules
―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
stdout:
# random seed: R02Sd222538bc9b76b6e424ec0cb17ee887c
# GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ?gio-vfs?
Bail out! GladeUI-PYTHON-FATAL-WARNING: Error initializing Python interpreter: could not import pygobject
stderr:
Error in sitecustomize; set PYTHONVERBOSE for traceback:
ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list

(/home/mathieu/glade-3.38.2/build/tests/modules:4898): GladeUI-PYTHON-WARNING **: 09:43:58.605: Error initializing Python interpreter: could not import pygobject
--8<---------------cut here---------------end--------------->8---

I think the issue here lies in the sitecustomize.py file introduced with
cb72f9a773e0931ee3758c851d96007ded034e4c.

--8<---------------cut here---------------start------------->8---
python_root = os.path.realpath(sys.executable).split('/bin/')[0]
--8<---------------cut here---------------end--------------->8---

When using the Python C library, sys.executable is not the expected
python binary but the test binary.

So when we try to find the associated libraries in the Python search
path, this way:

--8<---------------cut here---------------start------------->8---
index = sys_path_absolute.index(python_site)
sys.path = sys.path[:index] + matching_sites + sys.path[index:]
--8<---------------cut here---------------end--------------->8---

we get:

--8<---------------cut here---------------start------------->8---
ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list
--8<---------------cut here---------------end--------------->8---

Running the test with the PYTHONPATH set to:

--8<---------------cut here---------------start------------->8---
PYTHONPATH=/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages:$PYTHONPATH 
--8<---------------cut here---------------end--------------->8---

is a way to work around this issue.

The fix here would be to replace sys.executable in the sitecustomize.py
with something more adequate, but I have no idea what could it be.

Thanks,

Mathieu

Message #8 received at 50105 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50105 <at> debbugs.gnu.org
Subject: Re: bug#50105: [core-updates] Python sitecustomize issue.
Date: Tue, 31 Aug 2021 00:03:23 -0400

[Message part 1 (text/plain, inline)]

Hi Mathieu,

[...]

> I think the issue here lies in the sitecustomize.py file introduced with
> cb72f9a773e0931ee3758c851d96007ded034e4c.
>
> python_root = os.path.realpath(sys.executable).split('/bin/')[0]
>
>
> When using the Python C library, sys.executable is not the expected
> python binary but the test binary.
>
> So when we try to find the associated libraries in the Python search
> path, this way:
>
> index = sys_path_absolute.index(python_site)
> sys.path = sys.path[:index] + matching_sites + sys.path[index:]
>
>
> we get:
>
> ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list
>
>
> Running the test with the PYTHONPATH set to:
>
> PYTHONPATH=/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages:$PYTHONPATH 
>
> is a way to work around this issue.
>
> The fix here would be to replace sys.executable in the sitecustomize.py
> with something more adequate, but I have no idea what could it be.

From the doc [0], sys.prefix seems to be more correct.  This had been
suggested by Hartmut some time ago and I had this patch ready recently.
Perhaps it already addresses the issue at hand?

You'll find it attached.  It hasn't gotten much testing yet.

[0]  https://docs.python.org/3/library/sys.html#sys.prefix

[0001-aux-files-sitecustomize-Cleanup-and-add-explanatory-.patch (text/x-patch, attachment)]

Message #15 received at 50105 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50105 <at> debbugs.gnu.org
Subject: Re: bug#50105: [core-updates] Python sitecustomize issue.
Date: Fri, 03 Sep 2021 10:41:28 -0400

Hi Mathieu,

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> Hello,
>
> I tried to upgrade glade to the 3.38.2 release on core-update and
> encountered the following test issue:
>
> 2/5 modules               FAIL            0.29s   killed by signal 5 SIGTRAP
>>>> GLADE_MODULE_SEARCH_PATH=/home/mathieu/glade-3.38.2/build/plugins/gtk+:/home/mathieu/glade-3.38.2/build/plugins/python:/home/mathieu/glade-3.38.2/build/plugins/gjs:/home/mathieu/glade-3.38.2/tests/modules MALLOC_PERTURB_=195 GLADE_TESTING=1 GLADE_CATALOG_SEARCH_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+:/home/mathieu/glade-3.38.2/plugins/python:/home/mathieu/glade-3.38.2/plugins/gjs:/home/mathieu/glade-3.38.2/tests/catalogs GLADE_PIXMAP_DIR=/home/mathieu/glade-3.38.2/data/icons GLADE_ICON_THEME_PATH=/home/mathieu/glade-3.38.2/plugins/gtk+/icons/22x22 /home/mathieu/glade-3.38.2/build/tests/modules
> ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
> stdout:
> # random seed: R02Sd222538bc9b76b6e424ec0cb17ee887c
> # GLib-GIO-DEBUG: _g_io_module_get_default: Found default implementation local (GLocalVfs) for ?gio-vfs?
> Bail out! GladeUI-PYTHON-FATAL-WARNING: Error initializing Python interpreter: could not import pygobject
> stderr:
> Error in sitecustomize; set PYTHONVERBOSE for traceback:
> ValueError: '/home/mathieu/glade-3.38.2/build/lib/python3.9/site-packages' is not in list
>
> (/home/mathieu/glade-3.38.2/build/tests/modules:4898): GladeUI-PYTHON-WARNING **: 09:43:58.605: Error initializing Python interpreter: could not import pygobject
>
>
> I think the issue here lies in the sitecustomize.py file introduced with
> cb72f9a773e0931ee3758c851d96007ded034e4c.

[...]

Is there a simple reproducer to test my fix?  Would just building
current Glade and its test suite trigger it?

Thanks,

Maxim

Message #18 received at 50105 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50105 <at> debbugs.gnu.org
Subject: Re: bug#50105: [core-updates] Python sitecustomize issue.
Date: Fri, 03 Sep 2021 11:21:50 -0400

Hi again,

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

[...]

> Is there a simple reproducer to test my fix?  Would just building
> current Glade and its test suite trigger it?

OK, nevermind, I found that the test case was deleting the 'fix-tests'
build phase from glade3, and building the package.  I confirm the fix
works!

I'll send that patch series I've been talking about.

Thanks,

Maxim

Message #21 received at 50105 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 50105 <at> debbugs.gnu.org
Subject: Re: bug#50105: [core-updates] Python sitecustomize issue.
Date: Fri, 03 Sep 2021 20:46:24 +0200

Hello Maxim,

> OK, nevermind, I found that the test case was deleting the 'fix-tests'
> build phase from glade3, and building the package.  I confirm the fix
> works!

Great, that's exactly what I was trying to test but you beat me to it
:). I'll be afk for a month so I won't be able to confirm that I have
the same result locally. However, this fix looks good to me.

Thanks,

Mathieu

Message #26 received at 50105-done <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 50105-done <at> debbugs.gnu.org
Subject: Re: bug#50105: [core-updates] Python sitecustomize issue.
Date: Fri, 03 Dec 2021 22:08:15 -0500

Hello,

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> Hello Maxim,
>
>> OK, nevermind, I found that the test case was deleting the 'fix-tests'
>> build phase from glade3, and building the package.  I confirm the fix
>> works!
>
> Great, that's exactly what I was trying to test but you beat me to it
> :). I'll be afk for a month so I won't be able to confirm that I have
> the same result locally. However, this fix looks good to me.

I had forgotten to close this.  The fix has been in core-updates-frozen
for a while.

Thank you,

Maxim

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.