GNU bug report logs - #57954
[PATCH] gnu: clapack: Use position-independent code for use as a library.

Previous Next

Package: guix-patches;

Reported by: Nicolas Graves <ngraves <at> ngraves.fr>

Date: Tue, 20 Sep 2022 12:42:02 UTC

Severity: normal

Tags: patch

Done: Nicolas Graves <ngraves <at> ngraves.fr>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 57954 in the body.
You can then email your comments to 57954 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to guix-patches <at> gnu.org:
bug#57954; Package guix-patches. (Tue, 20 Sep 2022 12:42:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Nicolas Graves <ngraves <at> ngraves.fr>:
New bug report received and forwarded. Copy sent to guix-patches <at> gnu.org. (Tue, 20 Sep 2022 12:42:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: guix-patches <at> gnu.org
Cc: ngraves <at> ngraves.fr
Subject: [PATCH] gnu: clapack: Use position-independent code for use as a
 library.
Date: Tue, 20 Sep 2022 14:40:10 +0200
* gnu/packages/maths.scm (clapack): Use position-independent code for use as a library.
---
 gnu/packages/maths.scm | 1 +
 1 file changed, 1 insertion(+)

diff --git a/gnu/packages/maths.scm b/gnu/packages/maths.scm
index 72d5e9a83a..0c4b7ada03 100644
--- a/gnu/packages/maths.scm
+++ b/gnu/packages/maths.scm
@@ -968,6 +968,7 @@ (define-public clapack
     (build-system cmake-build-system)
     (arguments
      `(#:configure-flags '("-DCMAKE_C_FLAGS=-fcommon -O2")
+       #:make-flags '("-fPIC")
        #:phases
        (modify-phases %standard-phases
          ;; These tests use a lot of stack variables and segfault without
-- 
2.37.3





Information forwarded to guix-patches <at> gnu.org:
bug#57954; Package guix-patches. (Tue, 20 Sep 2022 12:59:02 GMT) Full text and rfc822 format available.

Message #8 received at 57954 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: 57954 <at> debbugs.gnu.org
Subject: Some details about why and see if there is no other solution.
Date: Tue, 20 Sep 2022 14:58:52 +0200
I am trying to get a vosk package to work for guix.
The build process is a bit tricky with replaced dependencies etc.

The team from vosk uses a version where they replace fortran by having
both openblas and clapack in libraries (which is incompatible in the
base kaldi configuration, so they have their own fork).

I don't know why exactly the library should be called from another
place, but when compiling kaldi with clapack, I get the following
message (and siblings):

ld: /gnu/store/yvc2w9mg554y6i4frvahjhacj0np9c7s-clapack-3.2.1/lib/liblapack.a(ssptrf.c.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC

I actually does work when recompiling with this flag, but I've also read
that it might make the package a bit slower.

In case it might help to answer, here's where I am for this package,
although not done yet (I still do have to untangle some ffi segmentation
fault issue) :
https://git.sr.ht/~ngraves/dotfiles/tree/main/item/packages/vosk.scm

What is the best option / course of action ?

-- 
Best regards,
Nicolas Graves




Information forwarded to guix-patches <at> gnu.org:
bug#57954; Package guix-patches. (Thu, 22 Sep 2022 15:49:01 GMT) Full text and rfc822 format available.

Message #11 received at 57954 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Nicolas Graves <ngraves <at> ngraves.fr>, 57954 <at> debbugs.gnu.org
Subject: Re: [bug#57954] Some details about why and see if there is no other
 solution.
Date: Thu, 22 Sep 2022 17:48:32 +0200
[Message part 1 (text/plain, inline)]

On 20-09-2022 14:58, Nicolas Graves via Guix-patches via wrote:
> 
> I am trying to get a vosk package to work for guix.
> The build process is a bit tricky with replaced dependencies etc.
> 
> The team from vosk uses a version where they replace fortran by having
> both openblas and clapack in libraries (which is incompatible in the
> base kaldi configuration, so they have their own fork).
> 
> I don't know why exactly the library should be called from another
> place, but when compiling kaldi with clapack, I get the following
> message (and siblings):
> 
> ld: /gnu/store/yvc2w9mg554y6i4frvahjhacj0np9c7s-clapack-3.2.1/lib/liblapack.a(ssptrf.c.o): relocation R_X86_64_32 against `.rodata.str1.1' can not be used when making a shared object; recompile with -fPIC
> 
> I actually does work when recompiling with this flag, but I've also read
> that it might make the package a bit slower.
> 
> In case it might help to answer, here's where I am for this package,
> although not done yet (I still do have to untangle some ffi segmentation
> fault issue) :
> https://git.sr.ht/~ngraves/dotfiles/tree/main/item/packages/vosk.scm
> 
> What is the best option / course of action ?

'kaldi' is compiled as a shared library.  However, going by the error 
message, it is linked to the (static!) clapack.  IIUC, this is not a 
problem per se (the static library would be embedded into the shared 
library IIUC) -- however, shared libraries must be position-independent, 
whereas static libraries aren't by default.

As such, there appear to be three potential solutions:

  * compile the static library as -fPIC (your patch)
  * let kaldi link to a shared clapack (which is -fPIC by default)
  * let 'kaldi' (and its dependent, vosk) be a static library instead
    of a shared library.  Is likely problematic due to 'python-vosk'
    though.

Both 'real' solutions have -fPIC (explicit or implied), so I don't think 
we have to worry about performance.  My default option for deciding 
between static and shared is 'shared' (makes 'clapack' graftable, and 
better interaction with "guix build --repair" and "guix gc --references").

Hence, my proposed (and untested) solution is to make 'kaldi' link to a 
shared clapack.

Greetings,
Maxime.
[OpenPGP_0x49E3EE22191725EE.asc (application/pgp-keys, attachment)]
[OpenPGP_signature (application/pgp-signature, attachment)]

Information forwarded to guix-patches <at> gnu.org:
bug#57954; Package guix-patches. (Sat, 24 Sep 2022 08:39:02 GMT) Full text and rfc822 format available.

Message #14 received at 57954 <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: Maxime Devos <maximedevos <at> telenet.be>, 57954 <at> debbugs.gnu.org
Subject: Re: [bug#57954] Some details about why and see if there is no other
 solution.
Date: Sat, 24 Sep 2022 10:37:54 +0200
>    * compile the static library as -fPIC (your patch)
>    * let kaldi link to a shared clapack (which is -fPIC by default)

> Hence, my proposed (and untested) solution is to make 'kaldi' link to a
> shared clapack.

Thanks Maxime for your quick answer. 

Just to be sure of what that would imply (using the second answer here:
https://stackoverflow.com/questions/2649735/how-to-link-static-library-into-dynamic-library-in-gcc):

1) We keep the version of clapack patched for -fPIC.
2) I need to make a package providing a shared version of clapack based
on this version of clapack compiled with -fPIC.

Also, my original patch is wrong, I'll send a corrected one once I
understand the process.

-- 
Best regards,
Nicolas Graves




Information forwarded to guix-patches <at> gnu.org:
bug#57954; Package guix-patches. (Sat, 24 Sep 2022 09:47:01 GMT) Full text and rfc822 format available.

Message #17 received at 57954 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Nicolas Graves <ngraves <at> ngraves.fr>, 57954 <at> debbugs.gnu.org
Subject: Re: [bug#57954] Some details about why and see if there is no other
 solution.
Date: Sat, 24 Sep 2022 11:46:36 +0200
[Message part 1 (text/plain, inline)]

On 24-09-2022 10:37, Nicolas Graves wrote:
> 
>>     * compile the static library as -fPIC (your patch)
>>     * let kaldi link to a shared clapack (which is -fPIC by default)
> 
>> Hence, my proposed (and untested) solution is to make 'kaldi' link to a
>> shared clapack.
> 
> Thanks Maxime for your quick answer.
> 
> Just to be sure of what that would imply (using the second answer here:
> https://stackoverflow.com/questions/2649735/how-to-link-static-library-into-dynamic-library-in-gcc):
> 
> 1) We keep the version of clapack patched for -fPIC.
> 2) I need to make a package providing a shared version of clapack based
> on this version of clapack compiled with -fPIC.
> 
> Also, my original patch is wrong, I'll send a corrected one once I
> understand the process.

That should do it AFAIK.

However, going by 
https://cmake.org/cmake/help/latest/variable/BUILD_SHARED_LIBS.html#variable:BUILD_SHARED_LIBS 
, you can ask CMake to do these steps instead.

Also, going by the descriptio of 'clapack': ‘"The CLAPACK library was 
built using a Fortran to C conversion utility
called f2c.  The entire Fortran 77 LAPACK library is run through f2c to 
obtain C code, and then modified to improve readability.  CLAPACK's goal 
is to provide LAPACK for someone who does not have access to a Fortran 
compiler.’

... it sounds like clapack could be replaced with lapack without any 
problems (and even removed), and our 'lapack' package already makes 
shared libraries, which seems like a simpler course of action.

Greetings,
Maxime
[OpenPGP_0x49E3EE22191725EE.asc (application/pgp-keys, attachment)]
[OpenPGP_signature (application/pgp-signature, attachment)]

Reply sent to Nicolas Graves <ngraves <at> ngraves.fr>:
You have taken responsibility. (Wed, 28 Sep 2022 12:20:02 GMT) Full text and rfc822 format available.

Notification sent to Nicolas Graves <ngraves <at> ngraves.fr>:
bug acknowledged by developer. (Wed, 28 Sep 2022 12:20:02 GMT) Full text and rfc822 format available.

Message #22 received at 57954-done <at> debbugs.gnu.org (full text, mbox):

From: Nicolas Graves <ngraves <at> ngraves.fr>
To: Maxime Devos <maximedevos <at> telenet.be>, 57954-done <at> debbugs.gnu.org
Subject: Re: [bug#57954] Some details about why and see if there is no other
 solution.
Date: Wed, 28 Sep 2022 14:19:39 +0200
> ... it sounds like clapack could be replaced with lapack without any
> problems (and even removed), and our 'lapack' package already makes
> shared libraries, which seems like a simpler course of action.

You're absolutely right about this. I replaced clapack by lapack and
discarded the flag -lf2c during the build phase.

Everything works fine. I'm closing the issue.

The resulting packages are available in issue 58140.

-- 
Best regards,
Nicolas Graves




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Thu, 27 Oct 2022 11:24:15 GMT) Full text and rfc822 format available.

This bug report was last modified 1 year and 153 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.