GNU bug report logs - #70982
test suite failures on OpenBSD

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 70982 in the body.
You can then email your comments to 70982 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-libtool <at> gnu.org
Subject: test suite failures on OpenBSD
Date: Thu, 16 May 2024 15:47:13 +0200

[Message part 1 (text/plain, inline)]

Hi,

An automated test of the current libtool HEAD [1] exhibits
test failures on OpenBSD 7.5/x86_64.

testsuite: 34 81 169 170 failed

The relevant test, AFAIK, are tests 34 and 81. Find attached the log files.

Bruno

[1] https://github.com/gnu-libtool/ci-check/actions

[logs.zip (application/zip, attachment)]

Message #10 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
To: 70982-done <at> debbugs.gnu.org
Cc: Bruno Haible <bruno <at> clisp.org>
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Thu, 7 Nov 2024 20:09:20 +0200

[Message part 1 (text/plain, inline)]

On 16/05/2024 16:47, Bruno Haible wrote:
> Hi,
> 
> An automated test of the current libtool HEAD [1] exhibits
> test failures on OpenBSD 7.5/x86_64.
> 
> testsuite: 34 81 169 170 failed
> 
> The relevant test, AFAIK, are tests 34 and 81. Find attached the log files.
> 
> Bruno
> 
> [1] https://github.com/gnu-libtool/ci-check/actions

Tests 34 and 81 have been fixed [1] with these commits [2][3] in
OpenBSD, and an additional test, 173, for using local versions of shared
libraries is fixed by utilizing functionality added in this commit [4]. 
The entire OpenBSD testsuite should be fixed in development now!

[1] https://github.com/gnu-libtool/ci-check/actions/runs/11726885546
[2] 
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=7c310c9b0439a020add4c23d84ab33eeb486bd75
[3] 
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=c7bfd6884ceae4e792b17c9ea89097e8d00dbe2b
[4] 
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=3996b1fddcaa543c8bf349314cac3ab4b07b4d1d

-- 
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354

[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #13 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: 70982-done <at> debbugs.gnu.org,
 Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Thu, 07 Nov 2024 20:49:50 +0100

Hi Ileana,

Ileana Dumitrescu wrote:
> Tests 34 and 81 have been fixed [1] with these commits [2][3] in
> OpenBSD, and an additional test, 173, for using local versions of shared
> libraries is fixed by utilizing functionality added in this commit [4]. 
> The entire OpenBSD testsuite should be fixed in development now!
> 
> [1] https://github.com/gnu-libtool/ci-check/actions/runs/11726885546
> [2] 
> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=7c310c9b0439a020add4c23d84ab33eeb486bd75
> [3] 
> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=c7bfd6884ceae4e792b17c9ea89097e8d00dbe2b
> [4] 
> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=3996b1fddcaa543c8bf349314cac3ab4b07b4d1d

Thanks for working on this!

However, there are two things that I don't understand:

* Why two options, --test and --check, that do the same thing?
  In my understanding, "check" means doing a "test" with the a certain
  expectation, whereas "test" is unbiased.

* In which situations is the user expected to pass one of these options?
  - Is it only for libtool's own test suite?
  - Is it also for the test suite of a package (like GNU gettext)?
    If yes, does this mean that instead of executing a program through
      ../src/prog <some_arguments>
    the package's test suite needs to run
      ../libtool --mode=execute --test ../src/prog <some_arguments>  ?
    And what about interactive invocations of src/prog then?

Bruno

Message #16 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
To: Bruno Haible <bruno <at> clisp.org>, 70982-done <at> debbugs.gnu.org
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Fri, 8 Nov 2024 17:24:01 +0200

[Message part 1 (text/plain, inline)]

On 07/11/2024 21:49, Bruno Haible wrote:
> Hi Ileana,

Hi Bruno,

> Ileana Dumitrescu wrote:
>> Tests 34 and 81 have been fixed [1] with these commits [2][3] in
>> OpenBSD, and an additional test, 173, for using local versions of shared
>> libraries is fixed by utilizing functionality added in this commit [4].
>> The entire OpenBSD testsuite should be fixed in development now!
>>
>> [1] https://github.com/gnu-libtool/ci-check/actions/runs/11726885546
>> [2]
>> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=7c310c9b0439a020add4c23d84ab33eeb486bd75
>> [3]
>> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=c7bfd6884ceae4e792b17c9ea89097e8d00dbe2b
>> [4]
>> https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=3996b1fddcaa543c8bf349314cac3ab4b07b4d1d
> 
> Thanks for working on this!

Thank you for submitting the issue and logs!

> However, there are two things that I don't understand:
> 
> * Why two options, --test and --check, that do the same thing?
>    In my understanding, "check" means doing a "test" with the a certain
>    expectation, whereas "test" is unbiased.

I have seen check and test used interchangeably for executing a
testsuite. I thought users would appreciate the ability to choose
whichever term they generally use, but if one option seems sufficient,
I can remove the other.

> * In which situations is the user expected to pass one of these options?
>    - Is it only for libtool's own test suite?

No, but I can see where I may have created that confusion. I can update
the documentation to make this more clear.

>    - Is it also for the test suite of a package (like GNU gettext)?

Yes? Depending on what a package's testsuite looks like, it should
ensure that local changes to a shared library (that has previously been
installed) will be utilized when linking to an executable, instead of an
old installed version.

>      If yes, does this mean that instead of executing a program through
>        ../src/prog <some_arguments>
>      the package's test suite needs to run
>        ../libtool --mode=execute --test ../src/prog <some_arguments>  ?

No, this should be specific to "--mode=finish" and "--mode=install" for
shared libraries where the shared library cache may be updated by
finish_cmds. The command(s) should not have any effect on other libtool
modes. For libtool's testsuite, I have been utilizing:

  make check LIBTOOLFLAGS=--test

For packages using libtool, a package testsuite could be built, and
then, the shared libraries within could be changed and rebuilt without
the need to install those changes.

>      And what about interactive invocations of src/prog then?

Interactive invocations of src/prog should operate as expected for the
shared libraries it has been linked to. I think --test and --check
should only be useful for checking and testing of local changes to
shared libraries, without the need to install the updated shared
library. I have only seen this as an issue with ldconfig on OpenBSD 7.5.
Other versions of ldconfig do not seem to update the shared library
cache for these local libraries during testing, but I have not put much
time into understanding the differences.

-- 
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354

[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #19 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: 70982-done <at> debbugs.gnu.org,
 Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Fri, 08 Nov 2024 19:56:51 +0100

Hi Ileana,

> > * In which situations is the user expected to pass one of these options?
> >    - Is it only for libtool's own test suite?
> 
> No, but I can see where I may have created that confusion. I can update
> the documentation to make this more clear.
> ...
> No, this should be specific to "--mode=finish" and "--mode=install" for
> shared libraries where the shared library cache may be updated by
> finish_cmds. The command(s) should not have any effect on other libtool
> modes.

Documenting this would make sense. It was not clear to me.

> Depending on what a package's testsuite looks like, it should
> ensure that local changes to a shared library (that has previously been
> installed) will be utilized when linking to an executable, instead of an
> old installed version.
> ...
> I think --test and --check
> should only be useful for checking and testing of local changes to
> shared libraries, without the need to install the updated shared
> library. I have only seen this as an issue with ldconfig on OpenBSD 7.5.

I am still confused.

On one hand, when you say "only on OpenBSD", it reminds me of the problem
that I have seen repeatedly on OpenBSD over the last years (e.g. with
GNU libunistring, when built without --disable-shared): If I have an
older version of GNU libunistring installed (in the directory specified
through --prefix), when building a newer version of GNU libunistring
(i.e. "make && make check") the "make check" part fails because it
apparently uses the older version. Whereas "make && make install && make check"
succeeds. OpenBSD is the only ELF platform on which this 'make install'
before 'make check' is (or used to be) needed.

On the other hand, during "make && make check", none of the libtool
modes --mode=finish and --mode=install is used. Automake does not
generate invocations of "libtool --mode=finish ..." at all, and it
generates invocations of "libtool --mode=install ..." only as part
of 'make install', which is typically the last thing I do in a build
tree. (In the build tree, doing source code modifications and "make"
will regenerate the libraries using "libtool --mode=link ...", but
not invoke "libtool --mode=finish ..." nor "libtool --mode=install ...".)

So, it's not clear to me how your change can have any effect in a
usual build tree as generated by Automake.

Bruno

Message #22 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
To: Bruno Haible <bruno <at> clisp.org>, 70982-done <at> debbugs.gnu.org
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Sat, 9 Nov 2024 17:17:41 +0200

[Message part 1 (text/plain, inline)]

Hi Bruno,

>> Depending on what a package's testsuite looks like, it should
>> ensure that local changes to a shared library (that has previously been
>> installed) will be utilized when linking to an executable, instead of an
>> old installed version.
>> ...
>> I think --test and --check
>> should only be useful for checking and testing of local changes to
>> shared libraries, without the need to install the updated shared
>> library. I have only seen this as an issue with ldconfig on OpenBSD 7.5.
> 
> I am still confused.
> 
> On one hand, when you say "only on OpenBSD", it reminds me of the problem
> that I have seen repeatedly on OpenBSD over the last years (e.g. with
> GNU libunistring, when built without --disable-shared): If I have an
> older version of GNU libunistring installed (in the directory specified
> through --prefix), when building a newer version of GNU libunistring
> (i.e. "make && make check") the "make check" part fails because it
> apparently uses the older version. Whereas "make && make install && make check"
> succeeds. OpenBSD is the only ELF platform on which this 'make install'
> before 'make check' is (or used to be) needed.

I believe this would also be an issue in NetBSD ELF versions, but I have
not verified with libtool's testsuite. The OpenBSD manpage for ldconfig
states that it is the same as the ldconfig in NetBSD 0.9A [1], and the
finish_cmds in libtool are identical for both operating systems.

Performing a make install before make check is limiting, so I would like
to have libtool work around this. I think I understand the issue better
now.

If the shared library cache does not contain paths for an existing
package, the --test and --check options should be useful for verifying
the package is operating properly or for testing different
implementations of shared libraries before finalizing an install, and I
am sure users could find other use cases for it.

However, the --test and --check options would not work for shared
libraries that have existing paths in the shared library cache. I think
this could be fixed if the option(s) are combined with directories to
unconfigure. Users could supply something like this

  'make "LIBTOOLFLAGS=--test=/usr/local/lib" check'

which would execute this

  'ldconfig -U /usr/local/lib'

or a better option could be supplied to libtool to reorder the shared
library cache? The user would pass a list of directories to libtool in
the preferred order, and libtool could do minimal parsing before passing
this list to 'ldconfig -U <dir list>' and 'ldconfig -m <dir list>'. A
reorder option should help ensure that the shared library cache is not
left empty and programs broken.

> On the other hand, during "make && make check", none of the libtool
> modes --mode=finish and --mode=install is used. Automake does not
> generate invocations of "libtool --mode=finish ..." at all, and it
> generates invocations of "libtool --mode=install ..." only as part
> of 'make install', which is typically the last thing I do in a build
> tree. (In the build tree, doing source code modifications and "make"
> will regenerate the libraries using "libtool --mode=link ...", but
> not invoke "libtool --mode=finish ..." nor "libtool --mode=install ...".)
> 
> So, it's not clear to me how your change can have any effect in a
> usual build tree as generated by Automake.

I would agree that this should have no effect on "usual" build trees.
However, it will allow users who do install and later make changes to
shared libraries the ability to avoid ldconfig changing the shared
library cache. I can imagine this may be useful for users
cross-compiling shared libraries, and I know this does resolve the test
failure in OpenBSD for 'Use local version', where old installed
libraries were linked to the executable instead of the intended local
versions.

[1] https://man.openbsd.org/OpenBSD-7.5/ldconfig

-- 
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354

[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #25 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
To: Bruno Haible <bruno <at> clisp.org>, 70982-done <at> debbugs.gnu.org
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Tue, 12 Nov 2024 20:58:07 +0200

[Message part 1 (text/plain, inline)]

On 09/11/2024 17:17, Ileana Dumitrescu wrote:
> Hi Bruno,
> 
>>> Depending on what a package's testsuite looks like, it should
>>> ensure that local changes to a shared library (that has previously been
>>> installed) will be utilized when linking to an executable, instead of an
>>> old installed version.
>>> ...
>>> I think --test and --check
>>> should only be useful for checking and testing of local changes to
>>> shared libraries, without the need to install the updated shared
>>> library. I have only seen this as an issue with ldconfig on OpenBSD 7.5.
>>
>> I am still confused.
>>
>> On one hand, when you say "only on OpenBSD", it reminds me of the problem
>> that I have seen repeatedly on OpenBSD over the last years (e.g. with
>> GNU libunistring, when built without --disable-shared): If I have an
>> older version of GNU libunistring installed (in the directory specified
>> through --prefix), when building a newer version of GNU libunistring
>> (i.e. "make && make check") the "make check" part fails because it
>> apparently uses the older version. Whereas "make && make install && 
>> make check"
>> succeeds. OpenBSD is the only ELF platform on which this 'make install'
>> before 'make check' is (or used to be) needed.
> 
> I believe this would also be an issue in NetBSD ELF versions, but I have
> not verified with libtool's testsuite. The OpenBSD manpage for ldconfig
> states that it is the same as the ldconfig in NetBSD 0.9A [1], and the
> finish_cmds in libtool are identical for both operating systems.
> 
> Performing a make install before make check is limiting, so I would like
> to have libtool work around this. I think I understand the issue better
> now.
> 
> If the shared library cache does not contain paths for an existing
> package, the --test and --check options should be useful for verifying
> the package is operating properly or for testing different
> implementations of shared libraries before finalizing an install, and I
> am sure users could find other use cases for it.
> 
> However, the --test and --check options would not work for shared
> libraries that have existing paths in the shared library cache. I think
> this could be fixed if the option(s) are combined with directories to
> unconfigure. Users could supply something like this
> 
>    'make "LIBTOOLFLAGS=--test=/usr/local/lib" check'
> 
> which would execute this
> 
>    'ldconfig -U /usr/local/lib'
> 
> or a better option could be supplied to libtool to reorder the shared
> library cache? The user would pass a list of directories to libtool in
> the preferred order, and libtool could do minimal parsing before passing
> this list to 'ldconfig -U <dir list>' and 'ldconfig -m <dir list>'. A
> reorder option should help ensure that the shared library cache is not
> left empty and programs broken.

I have implemented a separate option '--reorder-cache=DIRS' for OpenBSD
users to reorder the shared library cache [2]. If verified that NetBSD
ELF versions have this issue also, I will update the documentation and
case statements.

>> On the other hand, during "make && make check", none of the libtool
>> modes --mode=finish and --mode=install is used. Automake does not
>> generate invocations of "libtool --mode=finish ..." at all, and it
>> generates invocations of "libtool --mode=install ..." only as part
>> of 'make install', which is typically the last thing I do in a build
>> tree. (In the build tree, doing source code modifications and "make"
>> will regenerate the libraries using "libtool --mode=link ...", but
>> not invoke "libtool --mode=finish ..." nor "libtool --mode=install ...".)
>>
>> So, it's not clear to me how your change can have any effect in a
>> usual build tree as generated by Automake.
> 
> I would agree that this should have no effect on "usual" build trees.
> However, it will allow users who do install and later make changes to
> shared libraries the ability to avoid ldconfig changing the shared
> library cache. I can imagine this may be useful for users
> cross-compiling shared libraries, and I know this does resolve the test
> failure in OpenBSD for 'Use local version', where old installed
> libraries were linked to the executable instead of the intended local
> versions.

The 'Use local version' test should work without the use of LIBTOOLFLAGS
now, since it is using '--reorder-cache=DIRS'. I think '--test' and/or
'--check' are still useful options for users of OpenBSD, and it should
be clearer on how to use them with the new option and documentation
updates.

I would like to have these options available in the next stable release,
which I have committed to doing before the end of next week. If there
are any more issues that you observe, I am happy to improve these
features.

> [1] https://man.openbsd.org/OpenBSD-7.5/ldconfig
> 

[2] 
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=0a1e894220521e5955371eea5408fd6f7a5149c1

-- 
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354

[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Message #28 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: 70982-done <at> debbugs.gnu.org,
 Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Sun, 17 Nov 2024 17:54:56 +0100

Hi Ileana,

> > On one hand, when you say "only on OpenBSD", it reminds me of the problem
> > that I have seen repeatedly on OpenBSD over the last years (e.g. with
> > GNU libunistring, when built without --disable-shared): If I have an
> > older version of GNU libunistring installed (in the directory specified
> > through --prefix), when building a newer version of GNU libunistring
> > (i.e. "make && make check") the "make check" part fails because it
> > apparently uses the older version. Whereas "make && make install && make check"
> > succeeds. OpenBSD is the only ELF platform on which this 'make install'
> > before 'make check' is (or used to be) needed.
> 
> I believe this would also be an issue in NetBSD ELF versions, but I have
> not verified with libtool's testsuite.

I haven't been able to reproduce said problem on OpenBSD (tried OpenBSD 6.0
and 7.6), nor on NetBSD 10.0. So, at least for the usual / simple cases,
there's nothing to worry about.

Bruno

Message #31 received at 70982-done <at> debbugs.gnu.org (full text, mbox):

From: Ileana Dumitrescu <ileanadumitrescu95 <at> gmail.com>
To: Bruno Haible <bruno <at> clisp.org>, 70982-done <at> debbugs.gnu.org
Subject: Re: bug#70982: test suite failures on OpenBSD
Date: Mon, 18 Nov 2024 18:59:34 +0200

[Message part 1 (text/plain, inline)]

Hi Bruno,

>>> On one hand, when you say "only on OpenBSD", it reminds me of the problem
>>> that I have seen repeatedly on OpenBSD over the last years (e.g. with
>>> GNU libunistring, when built without --disable-shared): If I have an
>>> older version of GNU libunistring installed (in the directory specified
>>> through --prefix), when building a newer version of GNU libunistring
>>> (i.e. "make && make check") the "make check" part fails because it
>>> apparently uses the older version. Whereas "make && make install && make check"
>>> succeeds. OpenBSD is the only ELF platform on which this 'make install'
>>> before 'make check' is (or used to be) needed.
>>
>> I believe this would also be an issue in NetBSD ELF versions, but I have
>> not verified with libtool's testsuite.
> 
> I haven't been able to reproduce said problem on OpenBSD (tried OpenBSD 6.0
> and 7.6), nor on NetBSD 10.0. So, at least for the usual / simple cases,
> there's nothing to worry about.

Thank you for trying to test this issue. I agree that this should not be
an issue on NetBSD 10.0 [1], even though ldconfig is still available for
use. For OpenBSD, version 7.6 has been added to the CI [2], and it
exhibits the same behaviour as version 7.5. Until the finish_cmds for
OpenBSD are removed or updated sufficiently, the --reorder-cache [3] and
--no-finish [4] commands should be useful for development and testing of
shared libraries for these unusual / complex cases that libtool's 'Use
local version' test demonstrates.

[1] https://www.netbsd.org/docs/elf.html#elf-ldconfig
[2] https://github.com/gnu-libtool/ci-check/actions/runs/11896549018
[3] 
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=0a1e894220521e5955371eea5408fd6f7a5149c1
[4] 
https://git.savannah.gnu.org/cgit/libtool.git/commit/?h=development&id=92725b31ebf2d4eb182ca9dba4692778191e0aaf

-- 
Ileana Dumitrescu

GPG Public Key: FA26 CA78 4BE1 8892 7F22 B99F 6570 EA01 146F 7354

[OpenPGP_0x6570EA01146F7354.asc (application/pgp-keys, attachment)]

[OpenPGP_signature.asc (application/pgp-signature, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.