GNU bug report logs - #27257
python-apsw bundles a version of SQLite

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27257 in the body.
You can then email your comments to 27257 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jelle Licht <jlicht <at> fsfe.org>
To: bug-guix <at> gnu.org
Subject: python-apsw bundles a version of SQLite
Date: Mon, 5 Jun 2017 18:32:10 +0200

[Message part 1 (text/plain, inline)]

The version of python-apsw (and of course python2-apsw) come with a bundled
copy of SQLite.
The bundled version seems to be a special single-source-file version [1].

I have tried deleting the bundled copy in a new phase, and this allows
python-apsw to build with the system SQLite libraries instead (someone
prematurely added sqlite as an input already, it was just never used).

This leads to a new problem, where that one of the VFS tests uses about 14
gigs of memory, and seems to hang as well. I am not sure if this is
indicative of any real issues when using python-apsw, and as such I am not
confident that just deleting the bundled copy of SQLite will help us, as it
might lead to an unusable package.

Some possible directions on resolving this issue:
- Debug the multiple thousand lines of test code to see exactly where/how
the test fails when using the system SQLite library
- Package the right version of the SQLite amalgation [1] that is now
bundled, although the amalgation should be functionally equivalent to the
normal SQLite version.

Thanks,
Jelle

[1]: https://www.sqlite.org/amalgamation.html

[Message part 2 (text/html, inline)]

Message #8 received at 27257 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Jelle Licht <jlicht <at> fsfe.org>
Cc: 27257 <at> debbugs.gnu.org
Subject: Re: bug#27257: python-apsw bundles a version of SQLite
Date: Mon, 5 Jun 2017 19:27:24 +0200

Hi Jelle,

On Mon, 5 Jun 2017 18:32:10 +0200
Jelle Licht <jlicht <at> fsfe.org> wrote:

> Some possible directions on resolving this issue:
> - Debug the multiple thousand lines of test code to see exactly where/how
> the test fails when using the system SQLite library
> - Package the right version of the SQLite amalgation [1] that is now
> bundled, although the amalgation should be functionally equivalent to the
> normal SQLite version.

I see that apsw bundles SQLite 3.9.2.  Does it work when you replace it with the external version 3.9.2 - amalgation or not ?

I don't necessarily mean permanently - just to find out whether the tests work normally then.

Also, maybe diff the bundled SQLite 3.9.2 and the released SQLite 3.9.2 amalgation of the same version ?

The apsw setup.py seems to do some serious config flags magic.  Are those the same?

It seems that the bundled package was not done by the original apsw author.  See details <https://pypi.python.org/pypi/apsw/3.8.5-r1> and <https://rogerbinns.github.io/apsw/download.html#easy-install-pip-pypi>.  On the first page it says that it will only work with "specific" SQLite versions - which is why the original author provided the setup.py which figures out what to do instead of bundling a random version like that person did.

All in all it would be much better for apsw to become part of sqlite upstream.  It even uses sqlite's configure script and parses sqlite's config flags.  Why not just make it part of the sqlite release then? *shakes head*

Message #11 received at 27257 <at> debbugs.gnu.org (full text, mbox):

From: Jelle Licht <jlicht <at> fsfe.org>
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: 27257 <at> debbugs.gnu.org
Subject: Re: bug#27257: python-apsw bundles a version of SQLite
Date: Mon, 11 Sep 2017 01:50:28 +0200

[Message part 1 (text/plain, inline)]

2017-06-05 19:27 GMT+02:00 Danny Milosavljevic <dannym <at> scratchpost.org>:

> Hi Jelle,
>
> On Mon, 5 Jun 2017 18:32:10 +0200
> Jelle Licht <jlicht <at> fsfe.org> wrote:
>
> > Some possible directions on resolving this issue:
> > - Debug the multiple thousand lines of test code to see exactly where/how
> > the test fails when using the system SQLite library
> > - Package the right version of the SQLite amalgation [1] that is now
> > bundled, although the amalgation should be functionally equivalent to the
> > normal SQLite version.
>
> I see that apsw bundles SQLite 3.9.2.  Does it work when you replace it
> with the external version 3.9.2 - amalgation or not ?
>
Getting version 3.9.2 to build turned out to be a bit of a hassle; some
nonsense about a fossil(?) generated manifest missing or what not.

>
> I don't necessarily mean permanently - just to find out whether the tests
> work normally then.
>
> Also, Perhaps diff the bundled SQLite 3.9.2 and the released SQLite 3.9.2
> amalgation of the same version ?
>
A diff between bundled SQLite 3.9.2 and release 3.9.2. amalgation reveals
no (functional) differences, so at least there will be no extra set of
patches to maintain.

> [...]
>

It seems that the actual upstream for apsw at [1] has continued development
since we packaged it.
Incidentally, our sqlite package is a bit outdated. If I update sqlite to
the latest version (3.20.1), as well as change the python-apsw upstream to
[1], the package seems to build and pass the `check' phase with no issues.

[1] : https://github.com/rogerbinns/apsw

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.