GNU bug report logs - #39079
SBCL CFFI from Guix unable to find dynamic libraries

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 39079 in the body.
You can then email your comments to 39079 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Evan Straw <evan.straw99 <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: SBCL CFFI from Guix unable to find dynamic libraries
Date: Sat, 11 Jan 2020 03:38:19 +0000

[Message part 1 (text/plain, inline)]

Hello all,

I have GNU Guix installed on a foreign distro, Ubuntu 19.04. I have the
sbcl and cl-cffi packages installed from this installation of Guix, and
I am attempting to use the CFFI package to load libcurl, as demonstrated
in the CFFI tutorial here:
https://common-lisp.net/project/cffi/manual/html_node/Tutorial_002dLoading.html

However, this fails with a message saying that there is "no such file or
directory," even when I attempt referencing the library by its exact
name, "libcurl.so.4".

There is only one condition in which I can get this to (sort of) work,
and that is when I manually add the path "/usr/lib/x86_64-linux-gnu/" to
the *foreign-library-directories* list, which is initially NIL. Even
then, however, CFFI fails to automatically load any libraries libcurl is
dependent on, unless I manually load them by directly referencing their
full names as well.

This seems like it isn't intended behavior, especially since the CFFI
documentation discourages the use of the *foreign-library-directories*
variable, and claims that the implementation (SBCL in this case) should
be taking care of this. I'm unsure if this is anything to do with Guix
not using a "traditional" FSH or not.

Does anyone have any thoughts on this? This is my first bug report
submitted here, so if I'm missing any needed information, please let me
know and I'll try my best to provide it.

Thanks,
-- Evan

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: 39079 <at> debbugs.gnu.org
Subject: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Sat, 11 Jan 2020 10:58:07 +0100

[Message part 1 (text/plain, inline)]

Hi Evan,

This is a current limitation on Guix because CFFI hardcodes the lib path
to /usr/lib, etc.
Maybe we could fix this, but it's not very clear how.

In the meantime, you can add this to your ~/.sbclrc:

--8<---------------cut here---------------start------------->8---
(when (ignore-errors (asdf:load-system "cffi"))
  ;; Warning: Trailing "/" matters!
  (dolist (guix-profile
           `(,(when (uiop:directory-exists-p "~/.guix-profile")
                (format nil "~a/.guix-profile/lib/" (uiop:getenv "HOME")))
              ,@(mapcar (lambda (d)
                          (format nil "~a~a/lib/"
                                  (namestring d)
                                  (first (last (pathname-directory d)))))
                        (uiop:subdirectories "~/.guix-extra-profiles/"))))
    (push guix-profile
          (symbol-value (find-symbol (string '*foreign-library-directories*)
                                     (find-package 'cffi))))))
--8<---------------cut here---------------end--------------->8---

In the above, edit "~/.guix-extra-profiles" to point to the directory
where you store all your profiles.

Hope that helps!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Evan Straw <evan.straw99 <at> gmail.com>
To: Pierre Niedhardt <mail <at> ambrevar.xyz>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Mon, 13 Jan 2020 14:11:53 -0800

[Message part 1 (text/plain, inline)]

Hi Pierre,
Thanks for your response.

>In the meantime, you can add this to your ~/.sbclrc:
>
>--8<---------------cut here---------------start------------->8---
>(when (ignore-errors (asdf:load-system "cffi"))
>  ;; Warning: Trailing "/" matters!
>  (dolist (guix-profile
>           `(,(when (uiop:directory-exists-p "~/.guix-profile")
>                (format nil "~a/.guix-profile/lib/" (uiop:getenv "HOME")))
>              ,@(mapcar (lambda (d)
>                          (format nil "~a~a/lib/"
>                                  (namestring d)
>                                  (first (last (pathname-directory d)))))
>                        (uiop:subdirectories "~/.guix-extra-profiles/"))))
>    (push guix-profile
>          (symbol-value (find-symbol (string '*foreign-library-directories*)
>                                     (find-package 'cffi))))))
>--8<---------------cut here---------------end--------------->8---

I attempted to add this to my .sbclrc, and added the "/usr/lib/x86_64-linux-gnu"
directory to that list as well. This seems to have made some progress, because
now, when I load CFFI, the *foreign-library-directories* variable is populated
with a list of directories. Additionally, CFFI is capable of loading libraries
installed through Guix as well with no issues. 

However, the thing that still doesn't appear to be working right is loading
libraries from the foreign distro. I get the same behavior as before when
attempting to load a library under the foreign distro's library directory
(/usr/lib/x86_64-linux-gnu): It loads the library itself but fails upon trying
to load any dependencies, even if they are in the same directory. Ideally, I
would just install the libraries I need from Guix, but I specifically want to
interface with a library from the system, in this case PulseAudio (I've just
been using libcurl as a test to make sure it works). 

I did some investigation and tried both installing just CFFI from another
source, and then SBCL itself. Installing CFFI from Quicklisp but keeping SBCL
from Guix gave the same behavior and was unable to load dependencies of the
library. Installing SBCL from APT and CFFI from Quicklisp was able to load the
library and all dependencies with no issues. From this, I'm wondering if it's a
problem with SBCL itself instead of CFFI? 

Do you (or anyone reading this thread) have any thoughts on this? Is this a bug
in the SBCL package, and if so, is there any way to work around it currently? 

Thanks for the help.
-- Evan

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Mon, 13 Jan 2020 23:22:28 +0100

[Message part 1 (text/plain, inline)]

Hi!

Evan Straw <evan.straw99 <at> gmail.com> writes:

> However, the thing that still doesn't appear to be working right is loading
> libraries from the foreign distro. I get the same behavior as before when
> attempting to load a library under the foreign distro's library directory
> (/usr/lib/x86_64-linux-gnu): It loads the library itself but fails upon trying
> to load any dependencies, even if they are in the same directory.

What are "dependencies" here?
Can you provide a full recipe of what you are trying to do, including
your .sbclrc and the Lisp code snippets you've run to test this?

> Ideally, I would just install the libraries I need from Guix, but I
> specifically want to interface with a library from the system, in this
> case PulseAudio (I've just been using libcurl as a test to make sure
> it works).

Does it work for the libraries installed with Guix?

> I did some investigation and tried both installing just CFFI from another
> source, and then SBCL itself. Installing CFFI from Quicklisp but keeping SBCL
> from Guix gave the same behavior and was unable to load dependencies of the
> library. Installing SBCL from APT and CFFI from Quicklisp was able to load the
> library and all dependencies with no issues. From this, I'm wondering if it's a
> problem with SBCL itself instead of CFFI?

This seems to be a red herring, as Guix does not change how CFFI loads
libraries.  The only thing that changes is where the libraries are found
with Guix packages.

Let's see what we can reproduce with your examples! :)

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>, Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Tue, 14 Jan 2020 06:52:06 +0100

Hi Pierre and Evan,

> This seems to be a red herring, as Guix does not change how CFFI loa > libraries.  The only thing that changes is where the libraries are 
found> with Guix packages.
I don't know about the details of SBCL and its CFFI, but there is a 
difference in how Guix handles shared libraries that I discovered 
recently and which might be related.

Programs compiled under Guix don't use the search path feature for 
shared libraries (via Guix' own copy of ldd.so). All references to 
shared libraries use the full path to the store. So if you compile sbcl 
under Guix, it will never look for libraries that are referenced by name 
only (without the full path). The foreign distro probably relies on that 
lookup, as most Linux distros do.

Note that this is a feature, not a bug, so if that's the cause of the 
problem, there is nothing to do about it. You'd have to compile sbcl 
with the toolchain of the foreign distro.

Cheers,
  Konrad.

Message #20 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Konrad Hinsen <konrad.hinsen <at> fastmail.net>,
 Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Tue, 14 Jan 2020 10:38:56 +0100

[Message part 1 (text/plain, inline)]

Konrad Hinsen <konrad.hinsen <at> fastmail.net> writes:

> Note that this is a feature, not a bug, so if that's the cause of the 
> problem, there is nothing to do about it. You'd have to compile sbcl 
> with the toolchain of the foreign distro.

You don't have to use another SBCL.  If you look at how we package
cffi-based libraries (e.g. sbcl-cl-sqlite), you'll see that we set the
path to the .so manually to the store location.  This should not raise
any issue.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>, Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Tue, 14 Jan 2020 11:11:39 +0100

Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

> Konrad Hinsen <konrad.hinsen <at> fastmail.net> writes:
>
>> Note that this is a feature, not a bug, so if that's the cause of the 
>> problem, there is nothing to do about it. You'd have to compile sbcl 
>> with the toolchain of the foreign distro.
>
> You don't have to use another SBCL.  If you look at how we package
> cffi-based libraries (e.g. sbcl-cl-sqlite), you'll see that we set the
> path to the .so manually to the store location.  This should not raise
> any issue.

Not with shared libraries in Guix, I agree. But the problem was with
shared libraries from the foreign distros. Guix actually tries hard
*not* to make them accessible (in order to ensure reproducibility),
so I wonder if this bug should be closed as "not a bug".

Cheers,
  Konrad.

Message #26 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Konrad Hinsen <konrad.hinsen <at> fastmail.net>,
 Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Tue, 14 Jan 2020 11:25:39 +0100

[Message part 1 (text/plain, inline)]

Konrad Hinsen <konrad.hinsen <at> fastmail.net> writes:

> Not with shared libraries in Guix, I agree. But the problem was with
> shared libraries from the foreign distros. Guix actually tries hard
> *not* to make them accessible (in order to ensure reproducibility),
> so I wonder if this bug should be closed as "not a bug".

The snippet I shared above allows the user to set SBCL to libraries from
the given locations.
I can't see why this wouldn't work on a foreign distribution.  But let's
wait until Evan shares his complete setup.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #29 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Evan Straw <evan.straw99 <at> gmail.com>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>,
 Konrad Hinsen <konrad.hinsen <at> fastmail.net>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Tue, 14 Jan 2020 13:30:55 -0800

[Message part 1 (text/plain, inline)]

Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

> What are "dependencies" here?
> Can you provide a full recipe of what you are trying to do, including
> your .sbclrc and the Lisp code snippets you've run to test this?

Absolutely. Apologies for the wall of text that is about to follow:

My system is Ubuntu 19.04, with Guix 09bad3c, as shown by `guix describe:`

--8<---------------cut here---------------start------------->8---
guix 09bad3c
  repository URL: https://git.savannah.gnu.org/git/guix.git
  branch: master
  commit: 09bad3cd42ca51974f3697f04ce841bf2fe6786c
--8<---------------cut here---------------end--------------->8---

I am using SBCL 2.0.0 and sbcl-cffi 0.19.0, both from Guix. My .sbclrc
file is as follows:

--8<---------------cut here---------------start------------->8---
;;; The following lines added by ql:add-to-init-file:
#-quicklisp
(let ((quicklisp-init (merge-pathnames "quicklisp/setup.lisp"
                                       (user-homedir-pathname))))
  (when (probe-file quicklisp-init)
    (load quicklisp-init)))

(when (ignore-errors (asdf:load-system "cffi"))
  ;; Warning: Trailing "/" matters!
  (dolist (guix-profile
           `(,(when (uiop:directory-exists-p "~/.guix-profile")
                (format nil "~a/.guix-profile/lib/" (uiop:getenv "HOME")))
              ,@(mapcar (lambda (d)
                          (format nil "~a~a/lib/"
                                  (namestring d)
                                  (first (last (pathname-directory d)))))
                        (uiop:subdirectories "~/.guix-extra-profiles/"))
	      "/usr/lib/x86_64-linux-gnu/"))
    (push guix-profile
          (symbol-value (find-symbol (string '*foreign-library-directories*)
                                     (find-package 'cffi))))))
--8<---------------cut here---------------end--------------->8---

I am attempting to load "/usr/lib/x86_64-linux-gnu/libcurl.so.4" since
the tutorial for CFFI starts by loading libcurl. This library depends on
the following other libraries, as reported by `ldd
/usr/lib/x86_64-linux-gnu/libcurl.so.4`:

--8<---------------cut here---------------start------------->8---
linux-vdso.so.1 (0x00007ffd5f393000)
libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f941aae8000)
libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f941aac8000)
librtmp.so.1 => /usr/lib/x86_64-linux-gnu/librtmp.so.1 (0x00007f941a8a8000)
libssh.so.4 => /usr/lib/x86_64-linux-gnu/libssh.so.4 (0x00007f941a828000)
libpsl.so.5 => /usr/lib/x86_64-linux-gnu/libpsl.so.5 (0x00007f941a810000)
libssl.so.1.1 => /usr/lib/x86_64-linux-gnu/libssl.so.1.1 (0x00007f941a780000)
libcrypto.so.1.1 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.1 (0x00007f941a4b0000)
libgssapi_krb5.so.2 => /usr/lib/x86_64-linux-gnu/libgssapi_krb5.so.2 (0x00007f941a460000)
libldap_r-2.4.so.2 => /usr/lib/x86_64-linux-gnu/libldap_r-2.4.so.2 (0x00007f941a408000)
liblber-2.4.so.2 => /usr/lib/x86_64-linux-gnu/liblber-2.4.so.2 (0x00007f941a3f0000)
libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f941a3d0000)
libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f941a3a8000)
libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f941a1b8000)
libunistring.so.2 => /home/evan/.guix-profile/lib/libunistring.so.2 (0x00007f941a030000)
libgnutls.so.30 => /usr/lib/x86_64-linux-gnu/libgnutls.so.30 (0x00007f9419e88000)
libhogweed.so.4 => /usr/lib/x86_64-linux-gnu/libhogweed.so.4 (0x00007f9419e50000)
libnettle.so.6 => /usr/lib/x86_64-linux-gnu/libnettle.so.6 (0x00007f9419e18000)
libgmp.so.10 => /home/evan/.guix-profile/lib/libgmp.so.10 (0x00007f9419d80000)
librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f9419d70000)
/lib64/ld-linux-x86-64.so.2 (0x00007f941abd8000)
libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f9419d68000)
libkrb5.so.3 => /usr/lib/x86_64-linux-gnu/libkrb5.so.3 (0x00007f9419c88000)
libk5crypto.so.3 => /usr/lib/x86_64-linux-gnu/libk5crypto.so.3 (0x00007f9419c50000)
libcom_err.so.2 => /lib/x86_64-linux-gnu/libcom_err.so.2 (0x00007f9419c48000)
libkrb5support.so.0 => /usr/lib/x86_64-linux-gnu/libkrb5support.so.0 (0x00007f9419c38000)
libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f9419c18000)
libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f9419bf8000)
libgssapi.so.3 => /usr/lib/x86_64-linux-gnu/libgssapi.so.3 (0x00007f9419bb0000)
libgcc_s.so.1 => /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/libgcc_s.so.1 (0x00007f9419b90000)
libp11-kit.so.0 => /usr/lib/x86_64-linux-gnu/libp11-kit.so.0 (0x00007f9419a60000)
libtasn1.so.6 => /usr/lib/x86_64-linux-gnu/libtasn1.so.6 (0x00007f9419848000)
libkeyutils.so.1 => /lib/x86_64-linux-gnu/libkeyutils.so.1 (0x00007f9419840000)
libheimntlm.so.0 => /usr/lib/x86_64-linux-gnu/libheimntlm.so.0 (0x00007f9419830000)
libkrb5.so.26 => /usr/lib/x86_64-linux-gnu/libkrb5.so.26 (0x00007f94197a0000)
libasn1.so.8 => /usr/lib/x86_64-linux-gnu/libasn1.so.8 (0x00007f94196f8000)
libhcrypto.so.4 => /usr/lib/x86_64-linux-gnu/libhcrypto.so.4 (0x00007f94196c0000)
libroken.so.18 => /usr/lib/x86_64-linux-gnu/libroken.so.18 (0x00007f94196a0000)
libffi.so.6 => /usr/lib/x86_64-linux-gnu/libffi.so.6 (0x00007f9419690000)
libwind.so.0 => /usr/lib/x86_64-linux-gnu/libwind.so.0 (0x00007f9419660000)
libheimbase.so.1 => /usr/lib/x86_64-linux-gnu/libheimbase.so.1 (0x00007f9419648000)
libhx509.so.5 => /usr/lib/x86_64-linux-gnu/libhx509.so.5 (0x00007f94195f8000)
libsqlite3.so.0 => /usr/lib/x86_64-linux-gnu/libsqlite3.so.0 (0x00007f94194d8000)
libcrypt.so.1 => /lib/x86_64-linux-gnu/libcrypt.so.1 (0x00007f9419498000)
libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f9419348000)
--8<---------------cut here---------------end--------------->8---

In order to load this library, I run the following Lisp code in the SBCL
REPL:

--8<---------------cut here---------------start------------->8---
CL-USER> (asdf:load-system :cffi)
T

;;; *foreign-library-directories* has the right directories in it
CL-USER> cffi:*foreign-library-directories*
("/usr/lib/x86_64-linux-gnu/"
 "/home/evan/.guix-extra-profiles/emacs/emacs/lib/"
 "/home/evan/.guix-profile/lib/")
 
CL-USER> (cffi:load-foreign-library "libcurl.so.4")
--8<---------------cut here---------------end--------------->8---

The last expression fails with the following error, which states that it
failed while trying to load libnghttp2.so.14, listed above as one of the
dependencies for libcurl:

--8<---------------cut here---------------start------------->8---
Unable to load foreign library (LIBCURL.SO.4-464).
  Error opening shared object "/usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0":
  libnghttp2.so.14: cannot open shared object file: No such file or directory.
   [Condition of type CFFI:LOAD-FOREIGN-LIBRARY-ERROR]

Restarts:
 0: [RETRY] Try loading the foreign library again.
 1: [USE-VALUE] Use another library instead.
 2: [RETRY] Retry SLIME REPL evaluation request.
 3: [*ABORT] Return to SLIME's top level.
 4: [ABORT] abort thread (#<THREAD "repl-thread" RUNNING {10046C9BC3}>)

Backtrace:
  0: (CFFI::FL-ERROR "Unable to load foreign library (~A).~%  ~A" #:LIBCURL.SO.4-464 "Error opening shared object \"/usr/lib/x86_64-linux-gnu/libcurl.so.4.5.0\": ..)
  1: (CFFI::REPORT-SIMPLE-ERROR #:LIBCURL.SO.4-464 #<SIMPLE-ERROR "Error opening ~:[runtime~;shared object ~:*~S~]: ..)
  2: (CFFI::LOAD-FOREIGN-LIBRARY-PATH #:LIBCURL.SO.4-464 "libcurl.so.4" NIL)
  3: ((FLET CFFI::%DO-LOAD :IN CFFI::%DO-LOAD-FOREIGN-LIBRARY) #<CFFI:FOREIGN-LIBRARY LIBCURL.SO.4-464> #:LIBCURL.SO.4-464 "libcurl.so.4")
  4: (CFFI::%DO-LOAD-FOREIGN-LIBRARY "libcurl.so.4" NIL)
  5: (CFFI:LOAD-FOREIGN-LIBRARY "libcurl.so.4" :SEARCH-PATH NIL)
  6: (SB-INT:SIMPLE-EVAL-IN-LEXENV (CFFI:LOAD-FOREIGN-LIBRARY "libcurl.so.4") #<NULL-LEXENV>)
  7: (EVAL (CFFI:LOAD-FOREIGN-LIBRARY "libcurl.so.4"))
--8<---------------cut here---------------end--------------->8---



>> Ideally, I would just install the libraries I need from Guix, but I
>> specifically want to interface with a library from the system, in this
>> case PulseAudio (I've just been using libcurl as a test to make sure
>> it works).
>
> Does it work for the libraries installed with Guix?

To test this, I attempted to load `~/.guix-profile/lib/libguile-2.2.so`
since I don't have many libraries installed from Guix. This library
depends on the following libraries, as reported by `ldd`:

--8<---------------cut here---------------start------------->8---
linux-vdso.so.1 (0x00007ffe3a353000)
libgc.so.1 => /gnu/store/3xs3dnc28p9fi8in7hkfcdx20incrdvq-libgc-7.6.12/lib/libgc.so.1 (0x00007faeaee00000)
libpthread.so.0 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libpthread.so.0 (0x00007faeaedd8000)
libffi.so.6 => /gnu/store/ain96mrdwqd4s9shdd3s7m4syp5icdx5-libffi-3.2.1/lib/libffi.so.6 (0x00007faeaedc8000)
libunistring.so.2 => /gnu/store/nsikjxykcaqa0zjpfmkqd569bngbv5nl-libunistring-0.9.10/lib/libunistring.so.2 (0x00007faeaec40000)
libgmp.so.10 => /gnu/store/y7qk8raalgvdnxcglvxa320cfxrjk1x6-gmp-6.1.2/lib/libgmp.so.10 (0x00007faeaeba8000)
libltdl.so.7 => /gnu/store/zasz52va238yyaq68rjm8ljwl4ikij4p-libltdl-2.4.6/lib/libltdl.so.7 (0x00007faeaeb98000)
libdl.so.2 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libdl.so.2 (0x00007faeaeb90000)
libcrypt.so.1 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libcrypt.so.1 (0x00007faeaeb50000)
libm.so.6 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libm.so.6 (0x00007faeaea10000)
libgcc_s.so.1 => /gnu/store/2plcy91lypnbbysb18ymnhaw3zwk8pg1-gcc-7.4.0-lib/lib/libgcc_s.so.1 (0x00007faeae9f0000)
libc.so.6 => /gnu/store/ahqgl4h89xqj695lgqvsaf6zh2nhy4pj-glibc-2.29/lib/libc.so.6 (0x00007faeae830000)
/lib64/ld-linux-x86-64.so.2 (0x00007faeaefb0000)
--8<---------------cut here---------------end--------------->8---

Attempting the same steps in the SBCL REPL again succeeds with no
errors:

--8<---------------cut here---------------start------------->8---
CL-USER> (asdf:load-system :cffi)
T

CL-USER> (cffi:load-foreign-library "libguile-2.2.so")
#<CFFI:FOREIGN-LIBRARY LIBGUILE-2.2.SO-492 "libguile-2.2.so.1.4.1">
--8<---------------cut here---------------end--------------->8---



Hopefully this clarifies the problem somewhat.
-- Evan

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Evan Straw <evan.straw99 <at> gmail.com>,
 Konrad Hinsen <konrad.hinsen <at> fastmail.net>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Wed, 15 Jan 2020 08:45:15 +0100

[Message part 1 (text/plain, inline)]

Evan Straw <evan.straw99 <at> gmail.com> writes:

> This library depends on
> the following other libraries, as reported by `ldd
> /usr/lib/x86_64-linux-gnu/libcurl.so.4`:
>
> --8<---------------cut here---------------start------------->8---
> linux-vdso.so.1 (0x00007ffd5f393000)
> libnghttp2.so.14 => /usr/lib/x86_64-linux-gnu/libnghttp2.so.14 (0x00007f941aae8000)
> libidn2.so.0 => /usr/lib/x86_64-linux-gnu/libidn2.so.0 (0x00007f941aac8000)
...
> --8<---------------cut here---------------end--------------->8---

Which "ldd" did you use?  Ubuntu's or Guix'?
Where did you run it?

My only guess is what Konrad said: the libcurl library references its
dependencies with relative path (which ldd does _not_ display) and the
Guix loaded knows nothing about /usr/lib/x86_64-linux-gnu.

Try exporting LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu in the
environment in which you run SBCL. 

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #35 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>, Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Wed, 15 Jan 2020 09:53:49 +0100

Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

> Which "ldd" did you use?  Ubuntu's or Guix'?
> Where did you run it?

Good suggestion. My expectation is that Evan used Ubuntu's ldd, and that
using Guix' will show different results.

> Try exporting LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu in the
> environment in which you run SBCL. 

Guix' ldd has been patched to ignore LD_LIBRARY_PATH, so my prediction
is that this won't change anything.

Cheers,
  Konrad.

Message #38 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Konrad Hinsen <konrad.hinsen <at> fastmail.net>,
 Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Wed, 15 Jan 2020 10:13:51 +0100

[Message part 1 (text/plain, inline)]

Konrad Hinsen <konrad.hinsen <at> fastmail.net> writes:

> Guix' ldd has been patched to ignore LD_LIBRARY_PATH, so my prediction
> is that this won't change anything.

Maybe ldd (I don't know), but I suggest to run SBCL here.  It should
work.

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #41 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Evan Straw <evan.straw99 <at> gmail.com>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: Konrad Hinsen <konrad.hinsen <at> fastmail.net>, 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Thu, 16 Jan 2020 10:36:51 -0800

[Message part 1 (text/plain, inline)]

Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

> Konrad Hinsen <konrad.hinsen <at> fastmail.net> writes:
>
>> Guix' ldd has been patched to ignore LD_LIBRARY_PATH, so my prediction
>> is that this won't change anything.
>
> Maybe ldd (I don't know), but I suggest to run SBCL here.  It should
> work.

Sorry for the late update, but this actually does work! Setting
LD_LIBRARY_PATH to include all the relevant library directories
(/usr/lib/x86_64-linux-gnu, /lib/x86_64-linux-gnu) makes CFFI load the
library perfectly with no issues, and further steps in the CFFI tutorial
work fine too. It was not set by default on my system so I'll add
something to my .profile or related to set that on login.

Thanks to everybody for their help with this!
-- Evan

[signature.asc (application/pgp-signature, inline)]

Message #44 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Evan Straw <evan.straw99 <at> gmail.com>
Cc: Konrad Hinsen <konrad.hinsen <at> fastmail.net>, 39079 <at> debbugs.gnu.org
Subject: Re: SBCL CFFI from Guix unable to find dynamic libraries
Date: Fri, 17 Jan 2020 16:49:55 +0100

[Message part 1 (text/plain, inline)]

Glad it worked!

Have fun!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #49 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Konrad Hinsen <konrad.hinsen <at> fastmail.net>
Cc: 39079 <at> debbugs.gnu.org, Pierre Neidhardt <mail <at> ambrevar.xyz>,
 Evan Straw <evan.straw99 <at> gmail.com>
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Fri, 17 Jan 2020 22:24:37 +0100

Hi!

Konrad Hinsen <konrad.hinsen <at> fastmail.net> skribis:

> Pierre Neidhardt <mail <at> ambrevar.xyz> writes:

[...]

>> Try exporting LD_LIBRARY_PATH=/usr/lib/x86_64-linux-gnu in the
>> environment in which you run SBCL. 
>
> Guix' ldd has been patched to ignore LD_LIBRARY_PATH, so my prediction
> is that this won't change anything.

‘ldd’ is a script that simply invokes the program with
LD_TRACE_LOADED_OBJECTS=1 (honored by ‘ld.so’), and Guix ‘ld.so’ does
honor ‘LD_LIBRARY_PATH’, it’s no different here.

Ludo’.

Message #52 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Evan Straw <evan.straw99 <at> gmail.com>
Cc: 39079 <at> debbugs.gnu.org, Konrad Hinsen <konrad.hinsen <at> fastmail.net>,
 Pierre Neidhardt <mail <at> ambrevar.xyz>
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Fri, 17 Jan 2020 22:27:41 +0100

Hi Evan,

Evan Straw <evan.straw99 <at> gmail.com> skribis:

> I am attempting to load "/usr/lib/x86_64-linux-gnu/libcurl.so.4" since
> the tutorial for CFFI starts by loading libcurl. This library depends on
> the following other libraries, as reported by `ldd
> /usr/lib/x86_64-linux-gnu/libcurl.so.4`:

That cannot work: this libcurl.so is linked against a different libc
(among other things) than the ‘sbcl’ program from Guix, so trying to
load it in a running Guix ‘sbcl’ is bound to fail.

What would work instead is loading Guix’ libcurl.so.  I suppose you
could do that by removing /usr/lib from ‘*foreign-library-directories*’.

HTH!

Ludo’.

Message #55 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Ludovic Courtès <ludo <at> gnu.org>, Evan Straw
 <evan.straw99 <at> gmail.com>
Cc: Konrad Hinsen <konrad.hinsen <at> fastmail.net>, 39079 <at> debbugs.gnu.org
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Sat, 18 Jan 2020 11:50:08 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> That cannot work: this libcurl.so is linked against a different libc
> (among other things) than the ‘sbcl’ program from Guix, so trying to
> load it in a running Guix ‘sbcl’ is bound to fail.

Are you sure?  I've done this many time before and it worked.

> What would work instead is loading Guix’ libcurl.so.  I suppose you
> could do that by removing /usr/lib from ‘*foreign-library-directories*’.

I think there is a misunderstanding :)
The original question was specifically about loading a .so from the
foreign distribution, so this would do it :p

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/

[signature.asc (application/pgp-signature, inline)]

Message #58 received at 39079 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: Konrad Hinsen <konrad.hinsen <at> fastmail.net>, 39079 <at> debbugs.gnu.org,
 Evan Straw <evan.straw99 <at> gmail.com>
Subject: Re: bug#39079: SBCL CFFI from Guix unable to find dynamic libraries
Date: Sat, 18 Jan 2020 17:17:58 +0100

Hi,

Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> That cannot work: this libcurl.so is linked against a different libc
>> (among other things) than the ‘sbcl’ program from Guix, so trying to
>> load it in a running Guix ‘sbcl’ is bound to fail.
>
> Are you sure?  I've done this many time before and it worked.

It might work to some extent if you’re lucky, but as a rule of thumb,
you should assume that it won’t.

>> What would work instead is loading Guix’ libcurl.so.  I suppose you
>> could do that by removing /usr/lib from ‘*foreign-library-directories*’.
>
> I think there is a misunderstanding :)
> The original question was specifically about loading a .so from the
> foreign distribution, so this would do it :p

My bad!  Well I guess my answer is: don’t even try that.  :-)

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.