GNU bug report logs - #65324
"make check" hangs on NetBSD 9.3

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 65324 in the body.
You can then email your comments to 65324 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: bug-gnu-emacs <at> gnu.org
Subject: "make check" hangs on NetBSD 9.3
Date: Wed, 16 Aug 2023 00:02:13 +0200

[Message part 1 (text/plain, inline)]

Building Emacs (git sources from today) on NetBSD 9.3/x86_64:
1. I configured in a subdirectory (VPATH build) with options
     --disable-silent-rules --without-all --without-x
2. "make" succeeds.
3. "make check" leads to a process that eats 100% CPU time
   over several hours. Log file 'make-check.log' is attached.
   This process can be killed through Ctrl-C.

Bruno

[configure.log (text/x-log, attachment)]

[make.log.gz (application/gzip, attachment)]

[make-check.log (text/x-log, attachment)]

[netbsd93-hang.png (image/png, inline)]

Message #8 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefankangas <at> gmail.com>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Michael Albinus <michael.albinus <at> gmx.de>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Fri, 1 Sep 2023 22:23:01 +0200

Bruno Haible <bruno <at> clisp.org> writes:

> Building Emacs (git sources from today) on NetBSD 9.3/x86_64:
> 1. I configured in a subdirectory (VPATH build) with options
>      --disable-silent-rules --without-all --without-x
> 2. "make" succeeds.
> 3. "make check" leads to a process that eats 100% CPU time
>    over several hours. Log file 'make-check.log' is attached.
>    This process can be killed through Ctrl-C.

From the screenshot, this seems to be about Tramp.  Michael, could you
take a look at this?

Message #11 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefankangas <at> gmail.com>
Cc: 65324 <at> debbugs.gnu.org, Bruno Haible <bruno <at> clisp.org>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 13:24:52 +0200

Stefan Kangas <stefankangas <at> gmail.com> writes:

Hi,

> Bruno Haible <bruno <at> clisp.org> writes:
>
>> Building Emacs (git sources from today) on NetBSD 9.3/x86_64:
>> 1. I configured in a subdirectory (VPATH build) with options
>>      --disable-silent-rules --without-all --without-x
>> 2. "make" succeeds.
>> 3. "make check" leads to a process that eats 100% CPU time
>>    over several hours. Log file 'make-check.log' is attached.
>>    This process can be killed through Ctrl-C.
>
> From the screenshot, this seems to be about Tramp.  Michael, could you
> take a look at this?

Yep. There should be a file test/lisp/net/tramp-tests.log which I would
like to see.

Best regards, Michael.

Message #16 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 13:33:11 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> There should be a file test/lisp/net/tramp-tests.log which I would
> like to see.

Here it is (attached).

[tramp-tests.log (text/x-log, attachment)]

Message #19 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 13:54:39 +0200

[Message part 1 (text/plain, inline)]

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

> Here it is (attached).

Thanks.

>    FAILED  32/58  tramp-test21-file-links (0.833248 sec) at ../../test/lisp/net/tramp-tests.el:4112

Hmm, something else to investigate. Let's do it later.

>    passed  47/58  tramp-test40-make-nearby-temp-file (0.445061 sec)

The last passed test, so the problem happens in
tramp-test41-special-characters. Could you pls apply the appended patch,
and rerun? And I'll need the new tramp-tests.log file.

Btw, we skip the test already for macOS. Perhaps there is a similar
problem with NetBSD?

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #22 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 14:16:08 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> >    passed  47/58  tramp-test40-make-nearby-temp-file (0.445061 sec)
> 
> The last passed test, so the problem happens in
> tramp-test41-special-characters. Could you pls apply the appended patch,
> and rerun? And I'll need the new tramp-tests.log file.

Attached. It contains no new information. Should I try to stop some
processes and get stack traces using gdb? Or is this useless?

> Btw, we skip the test already for macOS. Perhaps there is a similar
> problem with NetBSD?

macOS is usually closer to FreeBSD than to NetBSD.
But how would I check that it's the same problem on NetBSD as on macOS?

Bruno

[tramp-tests.log (text/x-log, attachment)]

Message #25 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 16:08:31 +0200

[Message part 1 (text/plain, inline)]

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

>> The last passed test, so the problem happens in
>> tramp-test41-special-characters. Could you pls apply the appended patch,
>> and rerun? And I'll need the new tramp-tests.log file.
>
> Attached. It contains no new information. Should I try to stop some
> processes and get stack traces using gdb? Or is this useless?

gdb won't help. I've extended the patch (appended), so it writes traces
to your tmp directory in parallel. Could you pls try?

There should be a file /tmp/*debug tramp mock netbsd93.bruno.haible.de*
which might tell us the story.

>> Btw, we skip the test already for macOS. Perhaps there is a similar
>> problem with NetBSD?
>
> macOS is usually closer to FreeBSD than to NetBSD.
> But how would I check that it's the same problem on NetBSD as on macOS?

No idea. I cannot test macOS or NetBSD myself.

> Bruno

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #28 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 17:25:05 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> gdb won't help. I've extended the patch (appended), so it writes traces
> to your tmp directory in parallel. Could you pls try?
> 
> There should be a file /tmp/*debug tramp mock netbsd93.bruno.haible.de*
> which might tell us the story.

Find attached the resulting log files. They don't have the file names that
you expected, but anyway.

[netbsd93-tramp-logs.tar.gz (application/x-compressed-tar, attachment)]

Message #31 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 18:30:37 +0200

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

>> There should be a file /tmp/*debug tramp mock netbsd93.bruno.haible.de*
>> which might tell us the story.
>
> Find attached the resulting log files. They don't have the file names that
> you expected, but anyway.

These are the test files in "/tmp/tramp-testMWoYrx/". Not important.

I'm speaking about the debug file in /tmp (or wherever your temprary
directory is). A file name which starts with a star, and ends with a
star, as said above.

Or do you happen to own a directory "~/.cache/emacs"? Then it might be
located there.

Best regards, Michael.

Message #34 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Sat, 02 Sep 2023 19:55:31 +0200

Michael Albinus wrote:
> >> There should be a file /tmp/*debug tramp mock netbsd93.bruno.haible.de*
> >> which might tell us the story.
> ...
> I'm speaking about the debug file in /tmp (or wherever your temprary
> directory is). A file name which starts with a star, and ends with a
> star, as said above.

There is no file with such a name anywhere on this machine's disk.

Bruno

Message #39 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Mon, 04 Sep 2023 11:52:55 +0200

[Message part 1 (text/plain, inline)]

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

>> I'm speaking about the debug file in /tmp (or wherever your temprary
>> directory is). A file name which starts with a star, and ends with a
>> star, as said above.
>
> There is no file with such a name anywhere on this machine's disk.

Hmm, so the problem is deeper than expected. I've modified the patch to
add traces from the very beginning of the test case, see appended. Could
you pls try?

And just to make it sure: you're running Emacs 30?

> Bruno

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #42 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Mon, 04 Sep 2023 12:19:50 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> And just to make it sure: you're running Emacs 30?

I'm running Emacs from git on 2023-08-15; it identifies itself as
"GNU Emacs 30.0.50".

> >> I'm speaking about the debug file in /tmp (or wherever your temprary
> >> directory is). A file name which starts with a star, and ends with a
> >> star, as said above.
> >
> > There is no file with such a name anywhere on this machine's disk.
> 
> Hmm, so the problem is deeper than expected. I've modified the patch to
> add traces from the very beginning of the test case, see appended. Could
> you pls try?

Still no file with such a name on the disk. But some new logs in /tmp
(attached).

[netbsd93-tramp-logs.tar.gz (application/x-compressed-tar, attachment)]

Message #45 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Mon, 04 Sep 2023 12:28:34 +0200

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

>> >> I'm speaking about the debug file in /tmp (or wherever your temprary
>> >> directory is). A file name which starts with a star, and ends with a
>> >> star, as said above.
>> >
>> > There is no file with such a name anywhere on this machine's disk.
>>
>> Hmm, so the problem is deeper than expected. I've modified the patch to
>> add traces from the very beginning of the test case, see appended. Could
>> you pls try?
>
> Still no file with such a name on the disk. But some new logs in /tmp
> (attached).

Thanks. However, more important is whether there's some output from the
make call. In order to reduce test time, what happens if you call

--8<---------------cut here---------------start------------->8---
# make -C test tramp-tests SELECTOR=tramp-test41-special-characters
--8<---------------cut here---------------end--------------->8---

Best regards, Michael.

Message #48 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Mon, 04 Sep 2023 12:39:40 +0200

Michael Albinus wrote:
> In order to reduce test time, what happens if you call
> 
> --8<---------------cut here---------------start------------->8---
> # make -C test tramp-tests SELECTOR=tramp-test41-special-characters
> --8<---------------cut here---------------end--------------->8---

Run as a normal user (not root):

$ gmake -C test tramp-tests SELECTOR=tramp-test41-special-characters
gmake: Entering directory '/home/bruno/emacs/build/test'
gmake[1]: Entering directory '/home/bruno/emacs/build/test'
../../build-aux/install-sh -c -d lisp/net/
HOME=/nonexistent LANG=C EMACS_TEST_DIRECTORY=/home/bruno/emacs/test   "../src/emacs"  --no-init-file --no-site-file --no-site-lisp -L ":../../test"  \
  -l ert  -l lisp/net/tramp-tests.el \
  --batch --eval '(ert-run-tests-batch-and-exit (quote tramp-test41-special-characters))' 
Running 1 tests (2023-09-04 12:36:53+0200, selector `tramp-test41-special-characters')
Test tramp-test41-special-characters backtrace:
  (tramp-test-instrument-test-case 10 (setq tramp-debug-to-file t) (le
  (closure (tramp-use-connection-share tramp-inline-compress-start-siz
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name tramp-test41-special-characters :docu
  ert-run-or-rerun-test(#s(ert--stats :selector tramp-test41-special-c
  ert-run-tests(tramp-test41-special-characters #f(compiled-function (
  ert-run-tests-batch(tramp-test41-special-characters)
  ert-run-tests-batch-and-exit(tramp-test41-special-characters)
  eval((ert-run-tests-batch-and-exit 'tramp-test41-special-characters)
  command-line-1(("-L" ":../../test" "-l" "ert" "-l" "lisp/net/tramp-t
  command-line()
  normal-top-level()
Test tramp-test41-special-characters condition:
    (void-function tramp-test-instrument-test-case)
   FAILED  1/1  tramp-test41-special-characters (0.000283 sec) at ../../test/lisp/net/tramp-tests.el:7408

Ran 1 tests, 0 results as expected, 1 unexpected (2023-09-04 12:36:53+0200, 0.048956 sec)

1 unexpected results:
   FAILED  tramp-test41-special-characters

gmake[1]: *** [Makefile:181: lisp/net/tramp-tests.log] Error 1
gmake[1]: Leaving directory '/home/bruno/emacs/build/test'
gmake: *** [Makefile:247: lisp/net/tramp-tests] Error 2
gmake: Leaving directory '/home/bruno/emacs/build/test'

Message #51 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Tue, 05 Sep 2023 19:56:20 +0200

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

>> --8<---------------cut here---------------start------------->8---
>> # make -C test tramp-tests SELECTOR=tramp-test41-special-characters
>> --8<---------------cut here---------------end--------------->8---
>
> Run as a normal user (not root):
>
> $ gmake -C test tramp-tests SELECTOR=tramp-test41-special-characters
> gmake: Entering directory '/home/bruno/emacs/build/test'
> gmake[1]: Entering directory '/home/bruno/emacs/build/test'
> ../../build-aux/install-sh -c -d lisp/net/
> HOME=/nonexistent LANG=C EMACS_TEST_DIRECTORY=/home/bruno/emacs/test   "../src/emacs"  --no-init-file --no-site-file --no-site-lisp -L ":../../test"  \
>   -l ert  -l lisp/net/tramp-tests.el \
>   --batch --eval '(ert-run-tests-batch-and-exit (quote tramp-test41-special-characters))'
> Running 1 tests (2023-09-04 12:36:53+0200, selector `tramp-test41-special-characters')
> Test tramp-test41-special-characters backtrace:
>   (tramp-test-instrument-test-case 10 (setq tramp-debug-to-file t) (le
>   (closure (tramp-use-connection-share tramp-inline-compress-start-siz
>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>   ert-run-test(#s(ert-test :name tramp-test41-special-characters :docu
>   ert-run-or-rerun-test(#s(ert--stats :selector tramp-test41-special-c
>   ert-run-tests(tramp-test41-special-characters #f(compiled-function (
>   ert-run-tests-batch(tramp-test41-special-characters)
>   ert-run-tests-batch-and-exit(tramp-test41-special-characters)
>   eval((ert-run-tests-batch-and-exit 'tramp-test41-special-characters)
>   command-line-1(("-L" ":../../test" "-l" "ert" "-l" "lisp/net/tramp-t
>   command-line()
>   normal-top-level()
> Test tramp-test41-special-characters condition:
>     (void-function tramp-test-instrument-test-case)
>    FAILED  1/1  tramp-test41-special-characters (0.000283 sec) at ../../test/lisp/net/tramp-tests.el:7408

I don't understand anything. My patch has added a call to
tramp--test-instrument-test-case (two hyphens). What goes wrong?

Best regards, Michael.

Message #54 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Mon, 11 Sep 2023 14:26:28 +0200

Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Bruno,
>
>>> --8<---------------cut here---------------start------------->8---
>>> # make -C test tramp-tests SELECTOR=tramp-test41-special-characters
>>> --8<---------------cut here---------------end--------------->8---
>>
>> Run as a normal user (not root):
>>
>> $ gmake -C test tramp-tests SELECTOR=tramp-test41-special-characters
>> gmake: Entering directory '/home/bruno/emacs/build/test'
>> gmake[1]: Entering directory '/home/bruno/emacs/build/test'
>> ../../build-aux/install-sh -c -d lisp/net/
>> HOME=/nonexistent LANG=C EMACS_TEST_DIRECTORY=/home/bruno/emacs/test   "../src/emacs"  --no-init-file --no-site-file --no-site-lisp -L ":../../test"  \
>>   -l ert  -l lisp/net/tramp-tests.el \
>>   --batch --eval '(ert-run-tests-batch-and-exit (quote tramp-test41-special-characters))'
>> Running 1 tests (2023-09-04 12:36:53+0200, selector `tramp-test41-special-characters')
>> Test tramp-test41-special-characters backtrace:
>>   (tramp-test-instrument-test-case 10 (setq tramp-debug-to-file t) (le
>>   (closure (tramp-use-connection-share tramp-inline-compress-start-siz
>>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
>>   ert-run-test(#s(ert-test :name tramp-test41-special-characters :docu
>>   ert-run-or-rerun-test(#s(ert--stats :selector tramp-test41-special-c
>>   ert-run-tests(tramp-test41-special-characters #f(compiled-function (
>>   ert-run-tests-batch(tramp-test41-special-characters)
>>   ert-run-tests-batch-and-exit(tramp-test41-special-characters)
>>   eval((ert-run-tests-batch-and-exit 'tramp-test41-special-characters)
>>   command-line-1(("-L" ":../../test" "-l" "ert" "-l" "lisp/net/tramp-t
>>   command-line()
>>   normal-top-level()
>> Test tramp-test41-special-characters condition:
>>     (void-function tramp-test-instrument-test-case)
>>    FAILED  1/1  tramp-test41-special-characters (0.000283 sec) at ../../test/lisp/net/tramp-tests.el:7408
>
> I don't understand anything. My patch has added a call to
> tramp--test-instrument-test-case (two hyphens). What goes wrong?

Any News here? Have you tried to rerun the test with the proper function
tramp--test-instrument-test-case?

Best regards, Michael.

Message #57 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Tue, 12 Sep 2023 16:31:12 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> >> Test tramp-test41-special-characters backtrace:
> >>   (tramp-test-instrument-test-case 10 (setq tramp-debug-to-file t) (le
> >>   (closure (tramp-use-connection-share tramp-inline-compress-start-siz
> >>   ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
> >>   ert-run-test(#s(ert-test :name tramp-test41-special-characters :docu
> >>   ert-run-or-rerun-test(#s(ert--stats :selector tramp-test41-special-c
> >>   ert-run-tests(tramp-test41-special-characters #f(compiled-function (
> >>   ert-run-tests-batch(tramp-test41-special-characters)
> >>   ert-run-tests-batch-and-exit(tramp-test41-special-characters)
> >>   eval((ert-run-tests-batch-and-exit 'tramp-test41-special-characters)
> >>   command-line-1(("-L" ":../../test" "-l" "ert" "-l" "lisp/net/tramp-t
> >>   command-line()
> >>   normal-top-level()
> >> Test tramp-test41-special-characters condition:
> >>     (void-function tramp-test-instrument-test-case)
> >>    FAILED  1/1  tramp-test41-special-characters (0.000283 sec) at ../../test/lisp/net/tramp-tests.el:7408
> >
> > I don't understand anything. My patch has added a call to
> > tramp--test-instrument-test-case (two hyphens). What goes wrong?

When you wrote this, I thought you were out of debugging ideas.

> Any News here? Have you tried to rerun the test with the proper function
> tramp--test-instrument-test-case?

Ah, you meant, I made a mistake while applying your patch? Indeed, I had
not noticed the change from tramp-test-instrument-test-case to
tramp--test-instrument-test-case. Now I fixed that. Find attached the two
output files. I truncated the file '*debug tramp mock netbsd93.bruno.haible.de*'
to 10000 lines, since it starts to repeat itself (modulo time stamps) after
line 6499.

Bruno

[netbsd93-tramp-logs.tar.gz (application/x-compressed-tar, attachment)]

Message #60 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Tue, 12 Sep 2023 22:12:18 +0200

[Message part 1 (text/plain, inline)]

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

> Ah, you meant, I made a mistake while applying your patch? Indeed, I had
> not noticed the change from tramp-test-instrument-test-case to
> tramp--test-instrument-test-case. Now I fixed that. Find attached the two
> output files. I truncated the file '*debug tramp mock netbsd93.bruno.haible.de*'
> to 10000 lines, since it starts to repeat itself (modulo time stamps) after
> line 6499.

Thanks, this has told us the story, indeed. The last command Tramp has
sent is

--8<---------------cut here---------------start------------->8---
16:04:12.381001 tramp-send-command (6) # /bin/ls -ild -w /tmp/tramp-testXwo544/foo/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}; /bin/ls -lnd -w /tmp/tramp-testXwo544/foo/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}
--8<---------------cut here---------------end--------------->8---

Yes, one line. This is too long for your shell. I have modified
tramp-sh.el (see appended), could you pls give it a test?

> Bruno

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #63 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Wed, 13 Sep 2023 01:03:24 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> I have modified
> tramp-sh.el (see appended), could you pls give it a test?

I've applied the patch to lisp/net/tramp-sh.el (since there
is no lisp/tramp-sh.el in this version of emacs).

'patch' also gives a diagnostic:

$ patch tramp-sh.el < ~/emacs.patch
patching file tramp-sh.el
patch unexpectedly ends in middle of line
Hunk #1 succeeded at 1292 with fuzz 1 (offset -3 lines).

But anyway. With this change, the "gmake check" run goes further along:
The debug log repeats itself starting around line 7783.
Find it attached (again, truncated to 10000 lines).

Bruno

[netbsd93-tramp-logs.tar.gz (application/x-compressed-tar, attachment)]

Message #66 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Wed, 13 Sep 2023 16:34:49 +0200

[Message part 1 (text/plain, inline)]

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

> I've applied the patch to lisp/net/tramp-sh.el (since there
> is no lisp/tramp-sh.el in this version of emacs).

Yep, I did the work in the Tramp repository, which has a slightly
different directory structure.

> But anyway. With this change, the "gmake check" run goes further along:

Good. After the night, I didn't like the patch anymore, it was too
complicate. So I have rewitten it, and pushed it to Emacs master.

> The debug log repeats itself starting around line 7783.
> Find it attached (again, truncated to 10000 lines).

Hmm, the debug traces tell us:

--8<---------------cut here---------------start------------->8---
00:54:42.448637 tramp-send-command (6) # cd /tmp/tramp-test1ig90x/foo/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}/ && \ln -sf /tmp/tramp-test1ig90x/foo/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}/\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\} ./\	foo\ bar\ baz\	\@foo\@bar\@baz\@\$foo\$bar\$\$baz\$-foo-bar-baz-\%foo\%bar\%baz\%\&foo\&bar\&baz\&\?foo\?bar\?baz\?\*foo\+bar\*baz\+\'foo\"bar\'baz\"\#foo\~bar\#baz\~\!foo\|bar\!baz\|\:foo\;bar\:baz\;\<foo\>bar\<baz\>\(foo\)bar\(baz\)\[foo\]bar\[baz\]\{foo\}bar\{baz\}foo 2>/dev/null; echo tramp_exit_status $?
--8<---------------cut here---------------end--------------->8---

Again, one line. Here I cannot fix too much, the files are such long in
the testsuite. And NetBSD has restricted ressources, for example it
reports PIPE_BUF being 512, where other systems report 4096 ...

So I exclude this special "ln -sf" call for NetBSD, such longuish file
names shouldn't happen in the wild (I hope). See the reworked patch for
tramp-tests.el.

> Bruno

Best regards, Michael.

[Message part 2 (text/x-patch, attachment)]

Message #69 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Bruno Haible <bruno <at> clisp.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Wed, 13 Sep 2023 21:17:25 +0200

[Message part 1 (text/plain, inline)]

Michael Albinus wrote:
> So I have rewitten it, and pushed it to Emacs master.
> ...
> So I exclude this special "ln -sf" call for NetBSD, such longuish file
> names shouldn't happen in the wild (I hope). See the reworked patch for
> tramp-tests.el.

With these two new files, the tramp part of "gmake check" terminates.
Find attached the log files.

> And NetBSD has restricted ressources, for example it
> reports PIPE_BUF being 512, where other systems report 4096 ...

If your code and tests depend on the value of PIPE_BUF, that explains it.
Whereas the maximum command line length (according to the libtool configure
test) is:
  checking the maximum length of command line arguments... 196608

The PIPE_BUF in POSIX [1] can be as low as 512 [2]. Here are the values
on various platforms:
  - 512 on macOS, FreeBSD, NetBSD, OpenBSD, MirBSD, native Windows.
  - 4 KiB on Linux, OSF/1, Cygwin, Haiku.
  - 5 KiB on Solaris.
  - 8 KiB on HP-UX, Plan9.
  - 10 KiB on IRIX.
  - 32 KiB on AIX, Minix.

[1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html
[2] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/limits.h.html

> such longuish file names shouldn't happen in the wild (I hope).

But the problem is:
  - It's not only NetBSD. It's all *BSDs that have a PIPE_BUF value of 512.
  - tramp seems not only to fail in this case, but to go into an endless loop,
    which is much much worse.

Btw, I know that passing long commands through pipes is hard to implement.
A frequent mistake - that leads to endlessly hanging processes - is to leave
a file descriptor open in both the parent and the child process, when it
should only be open in one of the two.

Bruno

[netbsd93-tramp-logs.tar.gz (application/x-compressed-tar, attachment)]

Message #72 received at 65324 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324 <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Thu, 14 Sep 2023 14:51:41 +0200

Bruno Haible <bruno <at> clisp.org> writes:

Hi Bruno,

> With these two new files, the tramp part of "gmake check" terminates.
> Find attached the log files.

Thanks. I've pushed the fix to Emacs master.

>> And NetBSD has restricted ressources, for example it
>> reports PIPE_BUF being 512, where other systems report 4096 ...
>
> If your code and tests depend on the value of PIPE_BUF, that explains it.

No, this special case doesn't depend on PIPE_BUF I believe. I've used
PIPE_BUF as an example to compare NetBSD with Linux, because I have seen
it in the traces, and because it it shows the more limited ressources on
NetBSD.

> Whereas the maximum command line length (according to the libtool configure
> test) is:
>   checking the maximum length of command line arguments... 196608
>
> The PIPE_BUF in POSIX [1] can be as low as 512 [2]. Here are the values
> on various platforms:
>   - 512 on macOS, FreeBSD, NetBSD, OpenBSD, MirBSD, native Windows.
>   - 4 KiB on Linux, OSF/1, Cygwin, Haiku.
>   - 5 KiB on Solaris.
>   - 8 KiB on HP-UX, Plan9.
>   - 10 KiB on IRIX.
>   - 32 KiB on AIX, Minix.
>
> [1] https://pubs.opengroup.org/onlinepubs/9699919799/functions/write.html
> [2] https://pubs.opengroup.org/onlinepubs/9699919799/basedefs/limits.h.html

Thanks to the explanation, much appreciated!

>> such longuish file names shouldn't happen in the wild (I hope).
>
> But the problem is:
>   - It's not only NetBSD. It's all *BSDs that have a PIPE_BUF value of 512.

I have an OpenBSD VM as test system, and it didn't show this error. And
I didn't get a similar bug report yet from other users, for example
users with macOS.

But as said, I don't believe it is a PIPE_BUF problem.

>   - tramp seems not only to fail in this case, but to go into an endless loop,
>     which is much much worse.

Yes. Tramp waits for a response from remote, which didn't arrive. One
could thing about a timeout, but since Tramp sends arbitrary commands
over arbitrary slow lines to arbitrary slow machines, I don't know of a
proper value of such a timeout.

Well, I believe we have nailed it for *this* bug report. OK for you if I
close it?

> Bruno

Best regards, Michael.

Message #77 received at 65324-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Bruno Haible <bruno <at> clisp.org>
Cc: 65324-done <at> debbugs.gnu.org, Stefan Kangas <stefankangas <at> gmail.com>
Subject: Re: bug#65324: "make check" hangs on NetBSD 9.3
Date: Fri, 22 Sep 2023 20:06:07 +0200

Version: 29.1

Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Bruno,

> Well, I believe we have nailed it for *this* bug report. OK for you if I
> close it?

Well, no further comment, so I'm closing the bug.

>> Bruno

Best regards, Michael.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.