GNU bug report logs -
#27866
Handle clang's internal libraries when finding compiler's internal libraries
Previous Next
Reported by: Manoj Gupta <manojgupta <at> chromium.org>
Date: Fri, 28 Jul 2017 21:05:02 UTC
Severity: normal
Done: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27866 in the body.
You can then email your comments to 27866 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 28 Jul 2017 21:05:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
Manoj Gupta <manojgupta <at> chromium.org>
:
New bug report received and forwarded. Copy sent to
bug-libtool <at> gnu.org
.
(Fri, 28 Jul 2017 21:05:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi,
This is Manoj working on ChromeOS. I am facing a problem trying to build it
with clang with its own internal library (compiler-rt) since some packages
like mesa fail to build. The root cause is clang uses an absolute path to
link its internal libraries which libtool does not recognize.
e.g. clang++ -rtlib=compiler-rt main.cpp -v shows use of
/usr/lib64/clang/5.0.0/lib/linux/libclang_rt.builtins-x86_64.a
Libtool currently relies on "-lname" pattern to find the internal
libraries. And this does not work if some code is compiled using +
compiler-rt.
The issue was discovered in building mesa graphics library which uses
-nostdlib flag and relies on libtool to pass the additionally required
compiler internal libraries.
I have a sample fix below for fixing this for clang.
+--- a/m4/libtool.m4
++++ b/m4/libtool.m4
+@@ -7531,7 +7544,7 @@
+ for p in `eval "$output_verbose_link_cmd"`; do
+ case $prev$p in
+
+- -L* | -R* | -l*)
++ -L* | -R* | -l* | *clang_rt*.a)
+ # Some compilers place space between "-{L,R}" and the path.
+ # Remove the space.
+ if test x-L = "$p" ||
+
Please let me know if this is an appropriate fix.
Thanks,
Manoj
Sample linker command line when called by clang with compiler-rt:
"/usr/bin/x86_64-pc-linux-gnu-ld" --build-id --eh-frame-hdr -m elf_x86_64
-dynamic-linker /lib64/ld-linux-x86-64.so.2 -o a.out
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/../../../../lib64/crt1.o
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/../../../../lib64/crti.o
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/crtbegin.o
-L/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x
-L/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/../../../../lib64
-L/usr/bin/../lib64 -L/lib/../lib64 -L/usr/lib/../lib64
-L/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/../../../../x86_64-pc-linux-gnu/lib
-L/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/../../..
-L/usr/bin/../lib -L/lib -L/usr/lib /tmp/main-6b0bb5.o -lc++ -lm
/usr/lib64/clang/5.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lgcc_eh -lc
/usr/lib64/clang/5.0.0/lib/linux/libclang_rt.builtins-x86_64.a -lgcc_eh
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/crtend.o
/usr/bin/../lib64/gcc/x86_64-pc-linux-gnu/4.9.x/../../../../lib64/crtn.o
Thanks,
Manoj
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 28 Jul 2017 21:58:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -7531,7 +7544,7 @@
for p in `eval "$output_verbose_link_cmd"`; do
case $prev$p in
- -L* | -R* | -l*)
+ -L* | -R* | -l* | */libclang_rt.*.a)
# Some compilers place space between "-{L,R}" and the path.
# Remove the space.
if test x-L = "$p" ||
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 19 Jan 2018 00:50:02 GMT)
Full text and
rfc822 format available.
Message #11 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Friendly ping.
This issue keeps on biting us since many packages ship a copy of libtool.m4
file. Can some one look at the patch?
Thanks,
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 19 Jan 2018 22:17:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 28 Jul 2017 13:36, Manoj Gupta wrote:
> This is Manoj working on ChromeOS. I am facing a problem trying to build it
> with clang with its own internal library (compiler-rt) since some packages
> like mesa fail to build. The root cause is clang uses an absolute path to
> link its internal libraries which libtool does not recognize.
>
> e.g. clang++ -rtlib=compiler-rt main.cpp -v shows use of
> /usr/lib64/clang/5.0.0/lib/linux/libclang_rt.builtins-x86_64.a
>
> Libtool currently relies on "-lname" pattern to find the internal
> libraries. And this does not work if some code is compiled using +
> compiler-rt.
> The issue was discovered in building mesa graphics library which uses
> -nostdlib flag and relies on libtool to pass the additionally required
> compiler internal libraries.
>
> I have a sample fix below for fixing this for clang.
>
> +--- a/m4/libtool.m4
> ++++ b/m4/libtool.m4
> +@@ -7531,7 +7544,7 @@
> + for p in `eval "$output_verbose_link_cmd"`; do
> + case $prev$p in
> +
> +- -L* | -R* | -l*)
> ++ -L* | -R* | -l* | *clang_rt*.a)
> + # Some compilers place space between "-{L,R}" and the path.
> + # Remove the space.
> + if test x-L = "$p" ||
> +
i don't think hardcoding any specific library is correct, especially with
an expansive glob like this.
i wonder if leveraging libext would be a bad idea here.
-L* | -R* | -l* | *.${libext})
-mike
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sat, 20 Jan 2018 01:36:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
I think that both .a and .so libraries should be handled here. Will *.${libext}
handle both cases?
On Fri, Jan 19, 2018 at 2:16 PM, Mike Frysinger <vapier <at> gentoo.org> wrote:
> On 28 Jul 2017 13:36, Manoj Gupta wrote:
> > This is Manoj working on ChromeOS. I am facing a problem trying to build
> it
> > with clang with its own internal library (compiler-rt) since some
> packages
> > like mesa fail to build. The root cause is clang uses an absolute path to
> > link its internal libraries which libtool does not recognize.
> >
> > e.g. clang++ -rtlib=compiler-rt main.cpp -v shows use of
> > /usr/lib64/clang/5.0.0/lib/linux/libclang_rt.builtins-x86_64.a
> >
> > Libtool currently relies on "-lname" pattern to find the internal
> > libraries. And this does not work if some code is compiled using +
> > compiler-rt.
> > The issue was discovered in building mesa graphics library which uses
> > -nostdlib flag and relies on libtool to pass the additionally required
> > compiler internal libraries.
> >
> > I have a sample fix below for fixing this for clang.
> >
> > +--- a/m4/libtool.m4
> > ++++ b/m4/libtool.m4
> > +@@ -7531,7 +7544,7 @@
> > + for p in `eval "$output_verbose_link_cmd"`; do
> > + case $prev$p in
> > +
> > +- -L* | -R* | -l*)
> > ++ -L* | -R* | -l* | *clang_rt*.a)
> > + # Some compilers place space between "-{L,R}" and the path.
> > + # Remove the space.
> > + if test x-L = "$p" ||
> > +
>
> i don't think hardcoding any specific library is correct, especially with
> an expansive glob like this.
>
> i wonder if leveraging libext would be a bad idea here.
> -L* | -R* | -l* | *.${libext})
> -mike
>
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sat, 20 Jan 2018 01:51:01 GMT)
Full text and
rfc822 format available.
Message #20 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 19 Jan 2018 17:34, Manoj Gupta wrote:
> I think that both .a and .so libraries should be handled here. Will *.${libext}
> handle both cases?
libext is only "a". for shared libs, it can be calculated from shrext_cmds.
eval std_shrext=\"$shrext_cmds\"
-L* | -R* | -l* | *.${libext} | *${std_shrext})
that would only support libs that end in ".so". but maybe that's OK.
-mike
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Wed, 28 Feb 2018 21:13:01 GMT)
Full text and
rfc822 format available.
Message #23 received at 27866 <at> debbugs.gnu.org (full text, mbox):
On Fri, 19 Jan 2018, Mike Frysinger wrote:
> On 19 Jan 2018 17:34, Manoj Gupta wrote:
>> I think that both .a and .so libraries should be handled here. Will *.${libext}
>> handle both cases?
>
> libext is only "a". for shared libs, it can be calculated from shrext_cmds.
> eval std_shrext=\"$shrext_cmds\"
> -L* | -R* | -l* | *.${libext} | *${std_shrext})
>
> that would only support libs that end in ".so". but maybe that's OK.
Gentle ping - I'm also running into this issue, and would like to have a
canonical fix for it upstream.
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 18 Mar 2018 02:40:02 GMT)
Full text and
rfc822 format available.
Message #26 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Mike,
Any ideas who can commit this to upstream libtool?
On Wed, Feb 28, 2018 at 12:55 PM, Martin Storsjö <martin <at> martin.st> wrote:
> On Fri, 19 Jan 2018, Mike Frysinger wrote:
>
> On 19 Jan 2018 17:34, Manoj Gupta wrote:
>>
>>> I think that both .a and .so libraries should be handled here. Will
>>> *.${libext}
>>> handle both cases?
>>>
>>
>> libext is only "a". for shared libs, it can be calculated from
>> shrext_cmds.
>> eval std_shrext=\"$shrext_cmds\"
>> -L* | -R* | -l* | *.${libext} | *${std_shrext})
>>
>> that would only support libs that end in ".so". but maybe that's OK.
>>
>
> Gentle ping - I'm also running into this issue, and would like to have a
> canonical fix for it upstream.
>
> // Martin
>
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 23 Jul 2018 15:34:01 GMT)
Full text and
rfc822 format available.
Message #29 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Mike and Manoj,
Another gentle ping on this subject...
// Martin
On Sat, 17 Mar 2018, Manoj Gupta wrote:
> Mike,
> Any ideas who can commit this to upstream libtool?
>
> On Wed, Feb 28, 2018 at 12:55 PM, Martin Storsjö
> <martin <at> martin.st> wrote:
> On Fri, 19 Jan 2018, Mike Frysinger wrote:
>
> On 19 Jan 2018 17:34, Manoj
> Gupta wrote:
> I think that both .a
> and .so libraries
> should be handled
> here. Will
> *.${libext}
> handle both cases?
>
>
> libext is only "a". for shared
> libs, it can be calculated from
> shrext_cmds.
> eval
> std_shrext=\"$shrext_cmds\"
> -L* | -R* | -l* | *.${libext} |
> *${std_shrext})
>
> that would only support libs
> that end in ".so". but maybe
> that's OK.
>
>
> Gentle ping - I'm also running into this
> issue, and would like to have a canonical
> fix for it upstream.
>
> // Martin
>
>
>
>
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 23 Jul 2018 16:17:02 GMT)
Full text and
rfc822 format available.
Message #32 received at 27866 <at> debbugs.gnu.org (full text, mbox):
Mike,
Do you know who can commit this?
On Mon, Jul 23, 2018 at 8:33 AM, Martin Storsjö <martin <at> martin.st> wrote:
> Mike and Manoj,
>
> Another gentle ping on this subject...
>
> // Martin
>
>
> On Sat, 17 Mar 2018, Manoj Gupta wrote:
>
>> Mike,
>> Any ideas who can commit this to upstream libtool?
>>
>> On Wed, Feb 28, 2018 at 12:55 PM, Martin Storsjö
>> <martin <at> martin.st> wrote:
>> On Fri, 19 Jan 2018, Mike Frysinger wrote:
>>
>> On 19 Jan 2018 17:34, Manoj
>> Gupta wrote:
>> I think that both .a
>> and .so libraries
>> should be handled
>> here. Will
>> *.${libext}
>> handle both cases?
>>
>>
>> libext is only "a". for shared
>> libs, it can be calculated from
>> shrext_cmds.
>> eval
>> std_shrext=\"$shrext_cmds\"
>> -L* | -R* | -l* | *.${libext} |
>> *${std_shrext})
>>
>> that would only support libs
>> that end in ".so". but maybe
>> that's OK.
>>
>>
>> Gentle ping - I'm also running into this
>> issue, and would like to have a canonical
>> fix for it upstream.
>>
>> // Martin
>>
>>
>>
>
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Wed, 14 Aug 2019 20:47:02 GMT)
Full text and
rfc822 format available.
Message #35 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Manoj, Mike, Pavel and others,
While this isn't yet fixed upstream, I noted that this fix isn't enough
for me in mingw setups.
I've mostly been running into this issue in trying to compile VLC, where
this fix has been enough, but with other projects the issue remains. The
root cause for this seems to be that VLC overrides one libtool decision
here,
http://git.videolan.org/?p=vlc.git;a=blob;f=configure.ac;h=4aef56f06e3d16c8fe378055155126943d7ed69#l526
by manually setting this:
lt_cv_deplibs_check_method=pass_all
In other projects that don't set this, linking with libtool prints this
warning:
*** Warning: Trying to link with static lib archive C:/code/llvm-mingw/lib/clang/9.0.0/lib/windows/libclang_rt.builtins-x86_64.a.
*** I have the capability to make that library automatically link in when
*** you link to this library. But I can only do this if you have a
*** shared version of the library, which you do not appear to have
*** because the file extensions .a of this argument makes me believe
*** that it is just a static archive that I should not use here.
Manoj, did you run into this issue anywhere?
I'm able to work around it by patching libtool to do essentially the same,
to set lt_cv_deplibs_check_method to pass_all, e.g. like this:
diff --git a/m4/libtool.m4 b/m4/libtool.m4
index b55a6e57..c1eebf4c 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -3487,17 +3487,7 @@ cygwin*)
;;
mingw* | pw32*)
- # Base MSYS/MinGW do not provide the 'file' command needed by
- # func_win32_libid shell function, so use a weaker test based on 'objdump',
- # unless we find 'file', for example because we are cross-compiling.
- if ( file / ) >/dev/null 2>&1; then
- lt_cv_deplibs_check_method='file_magic ^x86 archive import|^x86 DLL'
- lt_cv_file_magic_cmd='func_win32_libid'
- else
- # Keep this pattern in sync with the one in func_win32_libid.
- lt_cv_deplibs_check_method='file_magic file format (pei*-i386(.*architecture: i386)?|pe-arm-wince|pe-x86-64)'
- lt_cv_file_magic_cmd='$OBJDUMP -f'
- fi
+ lt_cv_deplibs_check_method=pass_all
;;
cegcc*)
However, this doesn't seem ideal. Is there any other way around it, to
link against a literal path to an .a file while building a shared library?
// Martin
On Mon, 23 Jul 2018, Manoj Gupta wrote:
> Mike,
>
> Do you know who can commit this?
>
> On Mon, Jul 23, 2018 at 8:33 AM, Martin Storsjö <martin <at> martin.st> wrote:
>> Mike and Manoj,
>>
>> Another gentle ping on this subject...
>>
>> // Martin
>>
>>
>> On Sat, 17 Mar 2018, Manoj Gupta wrote:
>>
>>> Mike,
>>> Any ideas who can commit this to upstream libtool?
>>>
>>> On Wed, Feb 28, 2018 at 12:55 PM, Martin Storsjö
>>> <martin <at> martin.st> wrote:
>>> On Fri, 19 Jan 2018, Mike Frysinger wrote:
>>>
>>> On 19 Jan 2018 17:34, Manoj
>>> Gupta wrote:
>>> I think that both .a
>>> and .so libraries
>>> should be handled
>>> here. Will
>>> *.${libext}
>>> handle both cases?
>>>
>>>
>>> libext is only "a". for shared
>>> libs, it can be calculated from
>>> shrext_cmds.
>>> eval
>>> std_shrext=\"$shrext_cmds\"
>>> -L* | -R* | -l* | *.${libext} |
>>> *${std_shrext})
>>>
>>> that would only support libs
>>> that end in ".so". but maybe
>>> that's OK.
>>>
>>>
>>> Gentle ping - I'm also running into this
>>> issue, and would like to have a canonical
>>> fix for it upstream.
>>>
>>> // Martin
>>>
>>>
>>>
>>
>
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 10:01:02 GMT)
Full text and
rfc822 format available.
Message #38 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Wed, 14 Aug 2019, Martin Storsjö wrote:
> by manually setting this:
>
> lt_cv_deplibs_check_method=pass_all
>
> In other projects that don't set this, linking with libtool prints this
> warning:
>
> *** Warning: Trying to link with static lib archive
> C:/code/llvm-mingw/lib/clang/9.0.0/lib/windows/libclang_rt.builtins-x86_64.a.
> *** I have the capability to make that library automatically link in when
> *** you link to this library. But I can only do this if you have a
> *** shared version of the library, which you do not appear to have
> *** because the file extensions .a of this argument makes me believe
> *** that it is just a static archive that I should not use here.
>
> Manoj, did you run into this issue anywhere?
I see that libtool always sets lt_cv_deplibs_check_method=pass_all on
Linux, so that's probably why you haven't run into it.
So, it basically boils down to, what the actual purpose of inspecting
dependency libs is (what real scenario does it protect from), as it breaks
linking to compiler_rt's builtins (which are referred to as an absolute
path to the .a file)?
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 13:02:01 GMT)
Full text and
rfc822 format available.
Message #41 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, 15 Aug 2019, Martin Storsjö wrote:
>
> So, it basically boils down to, what the actual purpose of inspecting
> dependency libs is (what real scenario does it protect from), as it breaks
> linking to compiler_rt's builtins (which are referred to as an absolute path
> to the .a file)?
The purpose of inspecting dependency libs is that often code needs to
be compiled with special options (e.g. for PIC code) in order to
function in shared libraries or DLLs. Code which was compiled
properly can be included in the shared library but code which lacks
the necessary options needs to be saved for later and linked directly
with the dependent program. Libtool's ".la" files contain enough
information that libtool can make the correct decision when a
dependent program is linked.
If code which is not prepared for use in a shared library is included
into the shared library, the linking may fail, or the program may
crash, or run very inefficiently.
Since clang is intended to be gcc compatible, it would be most useful
for clang and its linker to emulate the GNU equivalents closely enough
that existing build infrastructure does not need to change.
Bob
--
Bob Friesenhahn
bfriesen <at> simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 13:03:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 14:57:02 GMT)
Full text and
rfc822 format available.
Message #47 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, 15 Aug 2019, Bob Friesenhahn wrote:
> On Thu, 15 Aug 2019, Martin Storsjö wrote:
>>
>> So, it basically boils down to, what the actual purpose of inspecting
>> dependency libs is (what real scenario does it protect from), as it breaks
>> linking to compiler_rt's builtins (which are referred to as an absolute
>> path to the .a file)?
>
> The purpose of inspecting dependency libs is that often code needs to be
> compiled with special options (e.g. for PIC code) in order to function in
> shared libraries or DLLs. Code which was compiled properly can be included
> in the shared library but code which lacks the necessary options needs to be
> saved for later and linked directly with the dependent program. Libtool's
> ".la" files contain enough information that libtool can make the correct
> decision when a dependent program is linked.
>
> If code which is not prepared for use in a shared library is included into
> the shared library, the linking may fail, or the program may crash, or run
> very inefficiently.
Ah, thanks for the explanation. Ok, if libtool has the ability to defer
the use of such libraries to the the executable instead of the shared
library, that's clearly neat.
But on Windows, the DLLs aren't allowed to have undefined references, so
that mechanism of deferring linking of certain libraries don't work there.
(And shouldn't this mechanism be sidestepped altogether if linking with
-no-undefined in general?)
Additionally, I don't know of any special options that need to be used to
build code for a shared library on Windows (either MSVC or mingw), as e.g.
-fPIC doesn't apply on windows at all. So given that, it seems to me that
lt_cv_deplibs_check_method=pass_all on windows/mingw should be safe?
> Since clang is intended to be gcc compatible, it would be most useful for
> clang and its linker to emulate the GNU equivalents closely enough that
> existing build infrastructure does not need to change.
Yes, that would of course be ideal, but for various reasons it doesn't
always happen to the full extent.
In general, clang does link to libgcc just like gcc does, by passing
-L<gccdir> -lgcc, but when using compiler_rt, it does so by passing the
full absolute path to the static library instead.
I started out making a patch for changing this some time ago, but there
were arguments against it; apparently it's deemed a safety feature to be
more exact in how the compiler_rt libraries are specified:
https://reviews.llvm.org/D51440
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 14:58:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 15:21:02 GMT)
Full text and
rfc822 format available.
Message #53 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, 15 Aug 2019, Martin Storsjö wrote:
>
> But on Windows, the DLLs aren't allowed to have undefined references, so that
> mechanism of deferring linking of certain libraries don't work there. (And
> shouldn't this mechanism be sidestepped altogether if linking with
> -no-undefined in general?)
Libraries provided by the compiler should have a special status since
they are built in a well-defined way and it should be possible to make
assumptions about their suitability for use. The failure to ascribe
this special status appears to be the problem here.
> Additionally, I don't know of any special options that need to be used to
> build code for a shared library on Windows (either MSVC or mingw), as e.g.
> -fPIC doesn't apply on windows at all. So given that, it seems to me that
> lt_cv_deplibs_check_method=pass_all on windows/mingw should be safe?
Actually, Windows DLL code does often require special options so that
symbols are exposed and used in the correct way. It may be that GCC
and Clang help by automating symbol export and import in a way that
compilers like Visual Studio / MSVC do not. There are also often
issues with exception handling since throwing exceptions across a DLL
boundary is a special case, which historically has been handled in a
couple of different ways by GCC.
Bob
--
Bob Friesenhahn
bfriesen <at> simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
Public Key, http://www.simplesystems.org/users/bfriesen/public-key.txt
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 15:21:03 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 21:08:02 GMT)
Full text and
rfc822 format available.
Message #59 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, 15 Aug 2019, Bob Friesenhahn wrote:
> On Thu, 15 Aug 2019, Martin Storsjö wrote:
>>
>> But on Windows, the DLLs aren't allowed to have undefined references, so
>> that mechanism of deferring linking of certain libraries don't work there.
>> (And shouldn't this mechanism be sidestepped altogether if linking with
>> -no-undefined in general?)
>
> Libraries provided by the compiler should have a special status since they
> are built in a well-defined way and it should be possible to make assumptions
> about their suitability for use. The failure to ascribe this special status
> appears to be the problem here.
Hmm, ok... Is there code in libtool that actually tries to make this
distinction, that should be adjusted so that it triggers here as well, or
is that just a general thing that would be sensible to have?
>> Additionally, I don't know of any special options that need to be used to
>> build code for a shared library on Windows (either MSVC or mingw), as e.g.
>> -fPIC doesn't apply on windows at all. So given that, it seems to me that
>> lt_cv_deplibs_check_method=pass_all on windows/mingw should be safe?
>
> Actually, Windows DLL code does often require special options so that symbols
> are exposed and used in the correct way. It may be that GCC and Clang help
> by automating symbol export and import in a way that compilers like Visual
> Studio / MSVC do not.
Right, yes, the use or lack of dllimport is an issue, yes. But that
doesn't mean one can't/shouldn't link to static libraries when building a
shared one either, it only implies that one has to link against one that
matches the attributes used when compiling the calling code.
And yes, GCC/Clang and ld.bfd and lld have special code that make most of
this issue go away (making dllimport essentially unnecessary).
> There are also often issues with exception handling since throwing
> exceptions across a DLL boundary is a special case, which historically
> has been handled in a couple of different ways by GCC.
Yeah, that's also a big potential issue.
But as far as I know, libtool doesn't actually check for either of these
issues at the momemnt anyway, or does it? I.e. does the dependency check
actually help detecting any of these issues specifically, or just refuse
to link in static libraries in general (in certain cases) when building a
shared one?
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 15 Aug 2019 21:08:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 19 Aug 2019 10:46:02 GMT)
Full text and
rfc822 format available.
Message #65 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Thu, 15 Aug 2019, Bob Friesenhahn wrote:
> On Thu, 15 Aug 2019, Martin Storsjö wrote:
>>
>> But on Windows, the DLLs aren't allowed to have undefined references, so
>> that mechanism of deferring linking of certain libraries don't work there.
>> (And shouldn't this mechanism be sidestepped altogether if linking with
>> -no-undefined in general?)
>
> Libraries provided by the compiler should have a special status since they
> are built in a well-defined way and it should be possible to make assumptions
> about their suitability for use. The failure to ascribe this special status
> appears to be the problem here.
I tried implementing this, see patch attached patch 2.
It's not exactly great to explicitly have to list libraries by name like
libgcc* and libclang_rt*, but the existing general mechanisms like
deplibs_check_method aren't really used fully here (if
deplibs_check_method="file_magic ...", then the static library is just
outright rejected without even inspecting it).
Or should this case be extended to also try file_magic if that's what has
been chosen, and then put the special case code for libclang_rt in e.g.
func_win32_libid? (But that wouldn't work for the other mingw case that
uses $OBJDUMP -f instead of func_win32_libid.)
I also tried updating Manoj's original patch with the suggestions from
Mike earlier in this discussion thread in attach patch 1.
// Martin
[0001-Pick-up-clang_rt-static-archives-compiler-internal-l.patch (text/x-diff, attachment)]
[0002-Allow-statically-linking-compiler-support-libraries-.patch (text/x-diff, attachment)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 19 Aug 2019 10:46:03 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 09 Sep 2019 08:40:02 GMT)
Full text and
rfc822 format available.
Message #71 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On Mon, 19 Aug 2019, Martin Storsjö wrote:
> On Thu, 15 Aug 2019, Bob Friesenhahn wrote:
>
>> On Thu, 15 Aug 2019, Martin Storsjö wrote:
>>>
>>> But on Windows, the DLLs aren't allowed to have undefined references, so
>>> that mechanism of deferring linking of certain libraries don't work there.
>>> (And shouldn't this mechanism be sidestepped altogether if linking with
>>> -no-undefined in general?)
>>
>> Libraries provided by the compiler should have a special status since they
>> are built in a well-defined way and it should be possible to make
>> assumptions about their suitability for use. The failure to ascribe this
>> special status appears to be the problem here.
>
> I tried implementing this, see patch attached patch 2.
Any comments or suggestions for the patches?
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 09 Sep 2019 08:40:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 03 Mar 2022 12:30:02 GMT)
Full text and
rfc822 format available.
Message #77 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Alex,
I've understood you're a new maintainer of libtool. Can you have a look at
this bug (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27866)?
In the last few posts, there's a couple patches attached. They have been
used downstream within e.g. MSYS2 since a couple years:
https://github.com/msys2/MSYS2-packages/blob/master/libtool/0011-Pick-up-clang_rt-static-archives-compiler-internal-l.patch
https://github.com/msys2/MSYS2-packages/blob/master/libtool/0013-Allow-statically-linking-compiler-support-libraries-.patch
// Martin
On Mon, 19 Aug 2019, Martin Storsjö wrote:
> On Thu, 15 Aug 2019, Bob Friesenhahn wrote:
>
>> On Thu, 15 Aug 2019, Martin Storsjö wrote:
>>>
>>> But on Windows, the DLLs aren't allowed to have undefined references, so
>>> that mechanism of deferring linking of certain libraries don't work there.
>>> (And shouldn't this mechanism be sidestepped altogether if linking with
>>> -no-undefined in general?)
>>
>> Libraries provided by the compiler should have a special status since they
>> are built in a well-defined way and it should be possible to make
>> assumptions about their suitability for use. The failure to ascribe this
>> special status appears to be the problem here.
>
> I tried implementing this, see patch attached patch 2.
>
> It's not exactly great to explicitly have to list libraries by name like
> libgcc* and libclang_rt*, but the existing general mechanisms like
> deplibs_check_method aren't really used fully here (if
> deplibs_check_method="file_magic ...", then the static library is just
> outright rejected without even inspecting it).
>
> Or should this case be extended to also try file_magic if that's what has
> been chosen, and then put the special case code for libclang_rt in e.g.
> func_win32_libid? (But that wouldn't work for the other mingw case that uses
> $OBJDUMP -f instead of func_win32_libid.)
>
> I also tried updating Manoj's original patch with the suggestions from Mike
> earlier in this discussion thread in attach patch 1.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Thu, 03 Mar 2022 12:30:03 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 06 May 2022 10:07:01 GMT)
Full text and
rfc822 format available.
Message #83 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi all,
Is there any status on this?
Thanks,
Shea
Martin Storsjö <martin <at> martin.st> writes:
> Hi Alex,
> I've understood you're a new maintainer of libtool. Can you have a look at this bug (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27866)?
> In the last few posts, there's a couple patches attached. They have been used downstream within e.g. MSYS2 since a couple years:
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 06 May 2022 10:07:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 08 May 2022 20:38:02 GMT)
Full text and
rfc822 format available.
Message #89 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
No not yet. Was traveling for Mother's Day, but I can look at it this week
or next weekend.
On Fri, May 6, 2022, 4:59 AM Shea Levy <shea <at> shealevy.com> wrote:
> Hi all,
>
> Is there any status on this?
>
> Thanks,
> Shea
>
> Martin Storsjö <martin <at> martin.st> writes:
>
> > Hi Alex,
>
> > I've understood you're a new maintainer of libtool. Can you have a look
> at this bug (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27866)?
>
> > In the last few posts, there's a couple patches attached. They have been
> used downstream within e.g. MSYS2 since a couple years:
>
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 08 May 2022 20:38:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 15 May 2022 18:34:01 GMT)
Full text and
rfc822 format available.
Message #95 received at submit <at> debbugs.gnu.org (full text, mbox):
Earlier this week I read through the thread, and created a patch based
on the ones posted. This was checked if you would like to experiment
with it.
What I did notice was that this change has a wider effect than the
problem statement initially suggests. I'm not crazy about the way it has
a conditional behavior for two specific libraries since it is an ad-hoc
solution directed at two compiler-collections, as opposed to a general
purpose solution; but for the time being I see this as a practical change.
As a side effect this change should also resolve issues with certain
flag-specs such as `-fsanitize' which is nice; but the impact of unknown
side effects is something I expect will rear its head in the near
future. With that in mind, I think this is a necessary change, but I
want to express up front that "I'm confident this will break a lot of
existing builds, and I consider this to be a first draft".
I would greatly appreciate y'all taking this for a spin on any available
projects you have to get a sense of how it will behave "in the field".
This change really effects "unspecified behavior" that the test-suite
isn't designed to audit, but nonetheless has a practical effect on users.
On 5/6/22 04:59, Shea Levy wrote:
> Hi all,
>
> Is there any status on this?
>
> Thanks,
> Shea
>
> Martin Storsjö <martin <at> martin.st> writes:
>
>> Hi Alex,
>> I've understood you're a new maintainer of libtool. Can you have a look at this bug (https://debbugs.gnu.org/cgi/bugreport.cgi?bug=27866)?
>> In the last few posts, there's a couple patches attached. They have been used downstream within e.g. MSYS2 since a couple years:
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 15 May 2022 18:34:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 15 May 2022 20:22:02 GMT)
Full text and
rfc822 format available.
Message #101 received at submit <at> debbugs.gnu.org (full text, mbox):
On Sun, 15 May 2022, Alex Ameen wrote:
> Earlier this week I read through the thread, and created a patch based on the
> ones posted. This was checked if you would like to experiment with it.
>
> What I did notice was that this change has a wider effect than the problem
> statement initially suggests. I'm not crazy about the way it has a
> conditional behavior for two specific libraries since it is an ad-hoc
> solution directed at two compiler-collections, as opposed to a general
> purpose solution; but for the time being I see this as a practical change.
>
> As a side effect this change should also resolve issues with certain
> flag-specs such as `-fsanitize' which is nice; but the impact of unknown side
> effects is something I expect will rear its head in the near future. With
> that in mind, I think this is a necessary change, but I want to express up
> front that "I'm confident this will break a lot of existing builds, and I
> consider this to be a first draft".
>
> I would greatly appreciate y'all taking this for a spin on any available
> projects you have to get a sense of how it will behave "in the field". This
> change really effects "unspecified behavior" that the test-suite isn't
> designed to audit, but nonetheless has a practical effect on users.
Thanks!
I tested this now, and it doesn't work quite as is - it needs this
modification:
diff --git a/m4/libtool.m4 b/m4/libtool.m4
index ab5af335..9c084816 100644
--- a/m4/libtool.m4
+++ b/m4/libtool.m4
@@ -7554,7 +7554,7 @@ if AC_TRY_EVAL(ac_compile); then
for p in `eval "$output_verbose_link_cmd"`; do
case $prev$p in
- -L* | -R* | -l* | */clang_rt*.a)
+ -L* | -R* | -l* | */libclang_rt*.a)
# Some compilers place space between "-{L,R}" and the path.
# Remove the space.
if test x-L = "$p" ||
(This was correct in one out of two instances in
1d2577357ee704da2d6d7c7da119ad82ba8ca172.)
With that changed, it does seem to work as it should for me on a test
project.
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 15 May 2022 20:22:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 15 May 2022 22:47:02 GMT)
Full text and
rfc822 format available.
Message #107 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Nice catch, I fat-fingered that one.
Could you share the workspace you tested in, or a snippet? I need to
make a test case.
There was one aspect I wanted some clarity on though from the patches
MSYS developed, which was "why is it that `m4/libtool.m4' used
`*NAME*.a' when `build-aux/ltmain.in' used `*/libNAME*.$libext', and why
they allowed `libgcc*' in `ltmain.in' but not `m4/libtool.m4'. I'm glad
I looked into it because when I investigated the original patches'
commit logs I realized that these were really just aimed at getting the
builds for a small number of MSYS2 packages to succeed ( in the author's
case VLC was the one they were focused on ); and that really this wasn't
really designed to be a general purpose patch to `libtool'. From what I
can tell MSYS2 is only using `libtool' for like 15 packages, so for
their use case hard coding a few whitelisted libraries is totally fine.
I'm actually really glad I dug into this more because In the field this
would have caused serious UB issues - the likes of which are actually
what led me to learn about linking/loading and eventually towards `libtool'.
I think my initial gut feeling on this one was right, and I'm probably
going to revert that change and iron this out in a project branch that
handles this alongside some related issues with flag-specs. The
underlying problem here, which has been discussed in a few other threads
is that `libtool' tries to outsmart the user and the compiler-collection
by reinterpreting flags library linkage flags like `-l:libfoo.a' or
`./libfoo.so', replacing them with `libfoo.la' and in rare cases even
swapping system libs which entirely different alternatives for a
particular platform.
In the MSYS patches their issue was that `libtool' doesn't respect the
flag-spec/implicit linkage for either GCC or Clang - so libraries which
are implicitly linked by the compiler-collection such as `libgcc_s.so'
or `libclang_rt.a' don't get added, usually what you get instead is a
manual invocation of `ld'. The result is that libtool removes certain
libs which are normally added by the compiler-collection's invocation of
`ld':
```
ld /path-to-libc/{crt1,crti}.o \
/path-to-cc/lib/crt<CC-init>.o \ # CC adds, but not `libtool'
<USER-FLAGS> \
-L/path-to-cc/lib -l<CC-libc-overrides> \ # CC adds, but not `libtool' ( almost always a static lib )
-L/path-to-libc/lib -lc \
-l<CC-runtime> \ # CC adds, but not `libtool' ( usually a shared lib )
/path-to-cc/lib/crt<CC-cleanup>.o \ # CC adds, but not `libtool' ( this one can get you in the most trouble )
/path-to-libc/crtn.o;
```
While the issues they're dealing with for Clang really focus on the RT
and libc override style libraries - the CC specific `.o' files can be a
real silent killer ( I know from experience before becoming maintainer
). The solution I'd like for this is going to take some time to develop,
and is going to require some real changes to the stated behavior of
`libtool' ( definitely not backwards compatible ) - but `libtool' needs
to either parse flag-specs and perfectly reproduce the underlying CC's
implicit libs, or it needs to stop directly invoking `ld'.
The big issue I see with the MSYS patches, and similar ad-hoc patches
that have been submitted to work around issues in flag-specs
`-fsanitize=' for example ) is that they often explicitly add linkage
for the relevant libraries, and fail to add linkage for the `.o' files.
The nasty thing about this is that it'll probably work fine 99% of the
time, but the 1% of the time that it misbehaves will be nearly
impossible to debug. You have initialization/cleanup routine in those
`.o' files that alter the initialization of executables and shared
libraries in discreet ways which effect a subset of of the RT and libc
override implementations provided by CCs - if those routines aren't
linked you often won't get a failure at link-editing time, rather you'll
get bizarre UB at runtime and ( in my experience ) by the time you
figure out what's wrong you'll have read two books, read the manuals for
CC and `ld' backwards and forwards a dozen times, GDB sessions will
haunt your dreams, and four months of your life have passed by lol.
In my case the mistake didn't involve `libtool', it was just a dev
dropping the ball while porting a Makefile from AIX to Linux. It's worth
noting, that as obnoxious as it is that `libtool' doesn't try to
replicate the CC linkage - in cases where people didn't try to work
around this by manually linking their CC specific libs, they would be
protected from the issue I encountered. I startled myself today
realizing that I checked in a patch that would have caused `libtool' to
make the exact issue that led me to learn about linking ( the irony
would have been palpable ). I am interested to learn if the patches MSYS
is using have caused the sort of UB I dealt with, I've got a list of at
least 3 issues I'd wager they run into with C++ "old ABI", `pthread',
and `dlopen'.
On 5/15/22 15:21, Martin Storsjö wrote:
> On Sun, 15 May 2022, Alex Ameen wrote:
>
>> Earlier this week I read through the thread, and created a patch
>> based on the ones posted. This was checked if you would like to
>> experiment with it.
>>
>> What I did notice was that this change has a wider effect than the
>> problem statement initially suggests. I'm not crazy about the way it
>> has a conditional behavior for two specific libraries since it is an
>> ad-hoc solution directed at two compiler-collections, as opposed to a
>> general purpose solution; but for the time being I see this as a
>> practical change.
>>
>> As a side effect this change should also resolve issues with certain
>> flag-specs such as `-fsanitize' which is nice; but the impact of
>> unknown side effects is something I expect will rear its head in the
>> near future. With that in mind, I think this is a necessary change,
>> but I want to express up front that "I'm confident this will break a
>> lot of existing builds, and I consider this to be a first draft".
>>
>> I would greatly appreciate y'all taking this for a spin on any
>> available projects you have to get a sense of how it will behave "in
>> the field". This change really effects "unspecified behavior" that
>> the test-suite isn't designed to audit, but nonetheless has a
>> practical effect on users.
>
> Thanks!
>
> I tested this now, and it doesn't work quite as is - it needs this
> modification:
>
> diff --git a/m4/libtool.m4 b/m4/libtool.m4
> index ab5af335..9c084816 100644
> --- a/m4/libtool.m4
> +++ b/m4/libtool.m4
> @@ -7554,7 +7554,7 @@ if AC_TRY_EVAL(ac_compile); then
> for p in `eval "$output_verbose_link_cmd"`; do
> case $prev$p in
>
> - -L* | -R* | -l* | */clang_rt*.a)
> + -L* | -R* | -l* | */libclang_rt*.a)
> # Some compilers place space between "-{L,R}" and the path.
> # Remove the space.
> if test x-L = "$p" ||
>
> (This was correct in one out of two instances in
> 1d2577357ee704da2d6d7c7da119ad82ba8ca172.)
>
> With that changed, it does seem to work as it should for me on a test
> project.
>
> // Martin
>
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sun, 15 May 2022 22:48:01 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 16 May 2022 07:27:02 GMT)
Full text and
rfc822 format available.
Message #113 received at submit <at> debbugs.gnu.org (full text, mbox):
On Sun, 15 May 2022, Alex Ameen wrote:
> Could you share the workspace you tested in, or a snippet? I need to make a
> test case.
This can reproduced with most libtool based projects that include C++
code. One example that I used for small standalone testing is this:
http://github.com/mstorsjo/fdk-aac
This project normally links the library as if it was C code, not C++, to
avoid unnecessary dependencies on the C++ standard library (which also
implicitly avoids this issue). To expose this issue, uncomment this line
in Makefile.am:
libfdk_aac_la_LINK = $(LINK) $(libfdk_aac_la_LDFLAGS)
Then regenerate the build files with ./autogen.sh.
Then build the project with a toolchain from
https://github.com/mstorsjo/llvm-mingw/releases - there are prebuilt ones
for x86_64 and aarch64 linux and macos (and windows). Assuming you run on
linux, configure the project like this:
../fdk-aac/configure --host=x86_64-w64-mingw32
When building, you'll end up with this build error:
CXXLD libfdk-aac.la
ld.lld: error: undefined symbol: ___chkstk_ms
>>> referenced by ../fdk-aac/libAACdec/src/FDK_delay.cpp:130
>>>
libAACdec/src/.libs/FDK_delay.o:(FDK_Delay_Apply(FDK_SignalDelay*, long*,
unsigned int, unsigned char))
>>> referenced by ../fdk-aac/libAACdec/src/aacdec_hcr.cpp:435
>>>
libAACdec/src/.libs/aacdec_hcr.o:(HcrDecoder(CErHcrInfo*,
CAacDecoderChannelInfo*, SamplingRateInfo const*, FDK_BITSTREAM*))
>>> referenced by ../fdk-aac/libAACdec/src/aacdecoder.cpp:2520
>>>
libAACdec/src/.libs/aacdecoder.o:(CAacDecoder_DecodeFrame(AAC_DECODER_INSTANCE*,
unsigned int, long*, int, int))
>>> referenced 14 more times
Are you able to reproduce this setup, or do you want me to package up a
full build environment with the pre-setup code in this configuration?
> There was one aspect I wanted some clarity on though from the patches MSYS
> developed, which was "why is it that `m4/libtool.m4' used `*NAME*.a' when
> `build-aux/ltmain.in' used `*/libNAME*.$libext',
Which patch is that? In both
https://github.com/msys2/MSYS2-packages/blob/master/libtool/0011-Pick-up-clang_rt-static-archives-compiler-internal-l.patch
and
https://github.com/msys2/MSYS2-packages/blob/master/libtool/0013-Allow-statically-linking-compiler-support-libraries-.patch
it's spelled out as libclang_rt and libgcc, not *clang_rt.
> and why they allowed `libgcc*' in `ltmain.in' but not `m4/libtool.m4'.
The reason for that is that when clang refers to the clang_rt builtins
library, it does so by passing an absolute path to
libclang_rt.builtins-<arch>.a is specified in the link command (which
libtool tries to analyze and pick up, by running $CC -v).
https://github.com/msys2/MSYS2-packages/blob/master/libtool/0011-Pick-up-clang_rt-static-archives-compiler-internal-l.patch
makes sure that this library is picked up here. For the case of libgcc,
it's never specified as an absolute path to the library, but always
specified as -lgcc<suffixes> which libtool does pick up on. So the patch
to m4/libtool.m4 only needs special casing of libclang_rt.builtins; libgcc
is handled just fine by the existing pattern of -l*.
But later if either libclang_rt.builtins, or libgcc, is linked statically
(the only option for libclang_rt.builtins), they need to be marked as
allowed in build-aux/ltmain.in, in
https://github.com/msys2/MSYS2-packages/blob/master/libtool/0013-Allow-statically-linking-compiler-support-libraries-.patch.
> I'm glad I looked into it because when I investigated the original
> patches' commit logs I realized that these were really just aimed at
> getting the builds for a small number of MSYS2 packages to succeed ( in
> the author's case VLC was the one they were focused on );
I think that author you refer to is me.
> and that really this wasn't really designed to be a general purpose
> patch to `libtool'.
It was meant to be a as general purpose patch as possible, to fix the
issues I had observed - without having the entirely full picture as I'm
not the libtool maintainer of course. But they were indeed meant to be
upstreamable.
> From what I can tell MSYS2 is only using `libtool' for like 15 packages,
> so for their use case hard coding a few whitelisted libraries is totally
> fine.
That sounds like an incorrect conclusion.
The MSYS2 packaged and patched libtool is used when building the packages
in https://github.com/msys2/MINGW-packages/. Currently this is a bit over
2000 packages. Out of those, 320 have 'autoreconf' as part of their build
instructions, 63 contain 'autogen.sh', 13 contain 'libtoolize'.
So roughly at least around 400 packages are routinely built with the MSYS2
patched libtool installed in the packages - possibly some more that update
their bundled libtool via some other command.
> but `libtool' needs to either parse flag-specs and perfectly reproduce
> the underlying CC's implicit libs, or it needs to stop directly invoking
> `ld'.
Indeed, I never quite understood why libtool needs to build C++ libraries
with -nostdlib and try to replicate the compiler's default libraries in
the first place.
> The big issue I see with the MSYS patches, and similar ad-hoc patches
> that have been submitted to work around issues in flag-specs
> `-fsanitize=' for example ) is that they often explicitly add linkage
> for the relevant libraries, and fail to add linkage for the `.o' files.
I'm not familiar with the patches for '-fsanitize=' - but in the case of
the patch I'm primarily talking about, it's all about the single static
library libclang_rt.builtins-<arch>.a, which doesn't have any associated
object files. I'm not referring to the sanitizers (which do have a much
more tricky setup), but just the compiler helper builtins library, which
doesn't have any of the tricky-init-object-files issues.
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 16 May 2022 07:27:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 16 May 2022 14:19:02 GMT)
Full text and
rfc822 format available.
Message #119 received at submit <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hey I reread my message, and will make my way through your responses today,
but something I forgot to mention was : really thank you for submitting
these. I hope my response didn't come across as critical. I really do
appreciate the work you've done here, and the effort you've made towards
clarifying.
On Mon, May 16, 2022, 2:26 AM Martin Storsjö <martin <at> martin.st> wrote:
> On Sun, 15 May 2022, Alex Ameen wrote:
>
> > Could you share the workspace you tested in, or a snippet? I need to
> make a
> > test case.
>
> This can reproduced with most libtool based projects that include C++
> code. One example that I used for small standalone testing is this:
> http://github.com/mstorsjo/fdk-aac
>
> This project normally links the library as if it was C code, not C++, to
> avoid unnecessary dependencies on the C++ standard library (which also
> implicitly avoids this issue). To expose this issue, uncomment this line
> in Makefile.am:
>
> libfdk_aac_la_LINK = $(LINK) $(libfdk_aac_la_LDFLAGS)
>
> Then regenerate the build files with ./autogen.sh.
>
> Then build the project with a toolchain from
> https://github.com/mstorsjo/llvm-mingw/releases - there are prebuilt ones
> for x86_64 and aarch64 linux and macos (and windows). Assuming you run on
> linux, configure the project like this:
>
> ../fdk-aac/configure --host=x86_64-w64-mingw32
>
> When building, you'll end up with this build error:
>
> CXXLD libfdk-aac.la
> ld.lld: error: undefined symbol: ___chkstk_ms
> >>> referenced by ../fdk-aac/libAACdec/src/FDK_delay.cpp:130
> >>>
> libAACdec/src/.libs/FDK_delay.o:(FDK_Delay_Apply(FDK_SignalDelay*, long*,
> unsigned int, unsigned char))
> >>> referenced by ../fdk-aac/libAACdec/src/aacdec_hcr.cpp:435
> >>>
> libAACdec/src/.libs/aacdec_hcr.o:(HcrDecoder(CErHcrInfo*,
> CAacDecoderChannelInfo*, SamplingRateInfo const*, FDK_BITSTREAM*))
> >>> referenced by ../fdk-aac/libAACdec/src/aacdecoder.cpp:2520
> >>>
> libAACdec/src/.libs/aacdecoder.o:(CAacDecoder_DecodeFrame(AAC_DECODER_INSTANCE*,
>
> unsigned int, long*, int, int))
> >>> referenced 14 more times
>
> Are you able to reproduce this setup, or do you want me to package up a
> full build environment with the pre-setup code in this configuration?
>
>
> > There was one aspect I wanted some clarity on though from the patches
> MSYS
> > developed, which was "why is it that `m4/libtool.m4' used `*NAME*.a' when
> > `build-aux/ltmain.in' used `*/libNAME*.$libext',
>
> Which patch is that? In both
>
> https://github.com/msys2/MSYS2-packages/blob/master/libtool/0011-Pick-up-clang_rt-static-archives-compiler-internal-l.patch
> and
>
> https://github.com/msys2/MSYS2-packages/blob/master/libtool/0013-Allow-statically-linking-compiler-support-libraries-.patch
> it's spelled out as libclang_rt and libgcc, not *clang_rt.
>
> > and why they allowed `libgcc*' in `ltmain.in' but not `m4/libtool.m4'.
>
> The reason for that is that when clang refers to the clang_rt builtins
> library, it does so by passing an absolute path to
> libclang_rt.builtins-<arch>.a is specified in the link command (which
> libtool tries to analyze and pick up, by running $CC -v).
>
>
> https://github.com/msys2/MSYS2-packages/blob/master/libtool/0011-Pick-up-clang_rt-static-archives-compiler-internal-l.patch
> makes sure that this library is picked up here. For the case of libgcc,
> it's never specified as an absolute path to the library, but always
> specified as -lgcc<suffixes> which libtool does pick up on. So the patch
> to m4/libtool.m4 only needs special casing of libclang_rt.builtins; libgcc
> is handled just fine by the existing pattern of -l*.
>
> But later if either libclang_rt.builtins, or libgcc, is linked statically
> (the only option for libclang_rt.builtins), they need to be marked as
> allowed in build-aux/ltmain.in, in
>
> https://github.com/msys2/MSYS2-packages/blob/master/libtool/0013-Allow-statically-linking-compiler-support-libraries-.patch
> .
>
> > I'm glad I looked into it because when I investigated the original
> > patches' commit logs I realized that these were really just aimed at
> > getting the builds for a small number of MSYS2 packages to succeed ( in
> > the author's case VLC was the one they were focused on );
>
> I think that author you refer to is me.
>
> > and that really this wasn't really designed to be a general purpose
> > patch to `libtool'.
>
> It was meant to be a as general purpose patch as possible, to fix the
> issues I had observed - without having the entirely full picture as I'm
> not the libtool maintainer of course. But they were indeed meant to be
> upstreamable.
>
> > From what I can tell MSYS2 is only using `libtool' for like 15 packages,
> > so for their use case hard coding a few whitelisted libraries is totally
> > fine.
>
> That sounds like an incorrect conclusion.
>
> The MSYS2 packaged and patched libtool is used when building the packages
> in https://github.com/msys2/MINGW-packages/. Currently this is a bit over
> 2000 packages. Out of those, 320 have 'autoreconf' as part of their build
> instructions, 63 contain 'autogen.sh', 13 contain 'libtoolize'.
>
> So roughly at least around 400 packages are routinely built with the MSYS2
> patched libtool installed in the packages - possibly some more that update
> their bundled libtool via some other command.
>
> > but `libtool' needs to either parse flag-specs and perfectly reproduce
> > the underlying CC's implicit libs, or it needs to stop directly invoking
> > `ld'.
>
> Indeed, I never quite understood why libtool needs to build C++ libraries
> with -nostdlib and try to replicate the compiler's default libraries in
> the first place.
>
> > The big issue I see with the MSYS patches, and similar ad-hoc patches
> > that have been submitted to work around issues in flag-specs
> > `-fsanitize=' for example ) is that they often explicitly add linkage
> > for the relevant libraries, and fail to add linkage for the `.o' files.
>
> I'm not familiar with the patches for '-fsanitize=' - but in the case of
> the patch I'm primarily talking about, it's all about the single static
> library libclang_rt.builtins-<arch>.a, which doesn't have any associated
> object files. I'm not referring to the sanitizers (which do have a much
> more tricky setup), but just the compiler helper builtins library, which
> doesn't have any of the tricky-init-object-files issues.
>
> // Martin
>
>
[Message part 2 (text/html, inline)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Mon, 16 May 2022 14:19:02 GMT)
Full text and
rfc822 format available.
Reply sent
to
Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
:
You have taken responsibility.
(Tue, 15 Apr 2025 15:39:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Manoj Gupta <manojgupta <at> chromium.org>
:
bug acknowledged by developer.
(Tue, 15 Apr 2025 15:39:03 GMT)
Full text and
rfc822 format available.
Message #127 received at 27866-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Thank you for your bug submission. This issue may be fixed by
utilizing a new configuration option, --enable-cxx-stdlib [1], since it
has been confirmed to fix a similar bug report [2]. If re-configuring
does not fix the issue, I will reopen this bug.
[1]
https://git.savannah.gnu.org/cgit/libtool.git/commit/?id=9bc7536bd8755f3b63118613496a89671512f2db
[2] https://savannah.gnu.org/support/index.php?111193
--
Ileana Dumitrescu
GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354
[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]
[OpenPGP_signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Fri, 25 Apr 2025 13:45:02 GMT)
Full text and
rfc822 format available.
Message #130 received at 27866-done <at> debbugs.gnu.org (full text, mbox):
Hi Ileana,
On Tue, 15 Apr 2025, Ileana Dumitrescu wrote:
> Thank you for your bug submission. This issue may be fixed by
> utilizing a new configuration option, --enable-cxx-stdlib [1], since it
> has been confirmed to fix a similar bug report [2]. If re-configuring
> does not fix the issue, I will reopen this bug.
Thanks for looking into this bug (and sorry for taking a little while to
follow up on this).
Yes, using the new option --enable-cxx-stdlib does avoid hitting this
issue. However, as this requires explicit opt-in from users (either from
project maintainers, enabling it via LT_INIT, or from users to configure
with --enable-cxx-stdlib), most users affected by this issue will still
keep hitting this issue (although they might have a way to work around it)
- so I would appreciate if this bug would be reopened.
If --enable-cxx-stdlib would be enabled by default (as it perhaps might do
at some point in the future?), this wouldn't be as much of an issue any
longer.
The issue can be reproduced with a very minimal testcase at [1], using a
toolchain from [2], e.g. [3].
To reproduce the issue, unpack the toolchain, add it to $PATH, unpack the
testcase, run ./autogen.sh, and configure and build with "./configure
--host=x86_64-w64-mingw32 && make".
The issue can be fixed by the referenced patches, or by reapplying
1d2577357ee704da2d6d7c7da119ad82ba8ca172 (which was reverted quickly
afterwards, without much explanation). That fix commit in itself is
mildly incorrect though, it requires changing m4/libtool.m4 to reference
*/libclang_rt*.a instead of */clang_rt*.a.
[1] https://martin.st/temp/libtool-compiler-rt.tar.gz
[2] https://github.com/mstorsjo/llvm-mingw/releases
[3] https://github.com/mstorsjo/llvm-mingw/releases/download/20250417/llvm-mingw-20250417-ucrt-ubuntu-22.04-x86_64.tar.xz
Thanks for the consideration!
// Martin
Did not alter fixed versions and reopened.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Sat, 26 Apr 2025 12:56:02 GMT)
Full text and
rfc822 format available.
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Sat, 26 Apr 2025 13:29:03 GMT)
Full text and
rfc822 format available.
Message #135 received at 27866 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
On 25/04/2025 16:43, Martin Storsjö wrote:
> Hi Ileana,
Hi Martin,
> On Tue, 15 Apr 2025, Ileana Dumitrescu wrote:
>
>> Thank you for your bug submission. This issue may be fixed by
>> utilizing a new configuration option, --enable-cxx-stdlib [1], since it
>> has been confirmed to fix a similar bug report [2]. If re-configuring
>> does not fix the issue, I will reopen this bug.
>
> Thanks for looking into this bug (and sorry for taking a little while to
> follow up on this).
Thank you for replying to me and testing my changes! You have waited
much longer for this issue to be fixed upstream, so I am happy to wait
on your responses.
> Yes, using the new option --enable-cxx-stdlib does avoid hitting this
> issue. However, as this requires explicit opt-in from users (either from
> project maintainers, enabling it via LT_INIT, or from users to configure
> with --enable-cxx-stdlib), most users affected by this issue will still
> keep hitting this issue (although they might have a way to work around
> it) - so I would appreciate if this bug would be reopened.
>
> If --enable-cxx-stdlib would be enabled by default (as it perhaps might
> do at some point in the future?), this wouldn't be as much of an issue
> any longer.
I would like to enable this by default, but I do not want to break
functionality in older systems that may be relying on '-nostdlib'. I
wish I understood better why it was added originally...
> The issue can be reproduced with a very minimal testcase at [1], using a
> toolchain from [2], e.g. [3].
>
> To reproduce the issue, unpack the toolchain, add it to $PATH, unpack
> the testcase, run ./autogen.sh, and configure and build with "./
> configure --host=x86_64-w64-mingw32 && make".
Thank you for providing an example to test on. I have done some local
testing to reproduce the issue, which was fixed with '--enable-cxx-stdlib'
and separately with the patches you made. I want to do some more
testing and think about what might be best to do for this.
--
Ileana Dumitrescu
GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354
[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]
[OpenPGP_signature.asc (application/pgp-signature, attachment)]
Reply sent
to
Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
:
You have taken responsibility.
(Mon, 19 May 2025 17:02:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
Manoj Gupta <manojgupta <at> chromium.org>
:
bug acknowledged by developer.
(Mon, 19 May 2025 17:02:02 GMT)
Full text and
rfc822 format available.
Message #140 received at 27866-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi,
I decided to apply the proposed patches to development [1][2] for now.
If there are any issues, I might revert these commits, but there will
still be the option to use '--enable-cxx-stdlib' during configuration.
Thank you again for your bug report and patches!
[1]
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=e606cd2358dd526d79837ce9f98da159c1d02bf7
[2]
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=2f505131b5f29b0de6338bf9d13b7ceadf617d96
--
Ileana Dumitrescu
GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354
[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]
[OpenPGP_signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Wed, 21 May 2025 08:18:02 GMT)
Full text and
rfc822 format available.
Message #143 received at 27866-done <at> debbugs.gnu.org (full text, mbox):
Hi Ileana,
On Mon, 19 May 2025, Ileana Dumitrescu wrote:
> I decided to apply the proposed patches to development [1][2] for now.
> If there are any issues, I might revert these commits, but there will
> still be the option to use '--enable-cxx-stdlib' during configuration.
Thanks for taking care of this! What's the expected process for migrating
patches further from development into master; will you revisit them in
e.g. 6-12 months and consider including them in master if they haven't
caused issues in the development branch since?
// Martin
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Wed, 21 May 2025 15:01:02 GMT)
Full text and
rfc822 format available.
Message #146 received at 27866-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Hi Martin,
On 21/05/2025 11:17, Martin Storsjö wrote:
> Hi Ileana,
>
> On Mon, 19 May 2025, Ileana Dumitrescu wrote:
>
>> I decided to apply the proposed patches to development [1][2] for now.
>> If there are any issues, I might revert these commits, but there will
>> still be the option to use '--enable-cxx-stdlib' during configuration.
>
> Thanks for taking care of this! What's the expected process for
> migrating patches further from development into master; will you revisit
> them in e.g. 6-12 months and consider including them in master if they
> haven't caused issues in the development branch since?
Thank you for the fixes! Commits are cherry-picked from development to
master periodically. There is not a set time frame, but it should not
be longer than a couple of months for general code changes. I want to
have time for testing libtool's behaviour and for users to give
feedback.
--
Ileana Dumitrescu
GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354
[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]
[OpenPGP_signature.asc (application/pgp-signature, attachment)]
Information forwarded
to
bug-libtool <at> gnu.org
:
bug#27866
; Package
libtool
.
(Wed, 21 May 2025 15:48:01 GMT)
Full text and
rfc822 format available.
Message #149 received at 27866-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
Thanks Ileana for applying the patch.
On Wed, May 21, 2025 at 8:00 AM Ileana Dumitrescu <
ileanadumitrescu95 <at> gmail.com> wrote:
> Hi Martin,
>
> On 21/05/2025 11:17, Martin Storsjö wrote:
> > Hi Ileana,
> >
> > On Mon, 19 May 2025, Ileana Dumitrescu wrote:
> >
> >> I decided to apply the proposed patches to development [1][2] for now.
> >> If there are any issues, I might revert these commits, but there will
> >> still be the option to use '--enable-cxx-stdlib' during configuration.
> >
> > Thanks for taking care of this! What's the expected process for
> > migrating patches further from development into master; will you revisit
> > them in e.g. 6-12 months and consider including them in master if they
> > haven't caused issues in the development branch since?
>
> Thank you for the fixes! Commits are cherry-picked from development to
> master periodically. There is not a set time frame, but it should not
> be longer than a couple of months for general code changes. I want to
> have time for testing libtool's behaviour and for users to give
> feedback.
>
> --
> Ileana Dumitrescu
>
> GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354
>
>
[Message part 2 (text/html, inline)]
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Thu, 19 Jun 2025 11:24:11 GMT)
Full text and
rfc822 format available.
This bug report was last modified 21 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.