GNU bug report logs - #36940
tests slowness and failure after recent Tramp changes

Previous Next

Package: emacs;

Reported by: Paul Eggert <eggert <at> cs.ucla.edu>

Date: Tue, 6 Aug 2019 00:29:02 UTC

Severity: normal

Done: Paul Eggert <eggert <at> cs.ucla.edu>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 36940 in the body.
You can then email your comments to 36940 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 06 Aug 2019 00:29:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Paul Eggert <eggert <at> cs.ucla.edu>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Tue, 06 Aug 2019 00:29:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Emacs bug reports and feature requests <bug-gnu-emacs <at> gnu.org>
Cc: Michael Albinus <michael.albinus <at> gmx.de>
Subject: tests slowness and failure after recent Tramp changes
Date: Mon, 5 Aug 2019 17:28:12 -0700
[Message part 1 (text/plain, inline)]
I'm having trouble with 'make check' after the recent Tramp changes on master. 
First, the Tramp tests are now too slow; they take over 6 minutes on my platform 
(Fedora 30 x86-4, AMD Phenom II X4 910e, default optimization), which is so slow 
that I don't want to run 'make check' while developing, which would be a bad 
habit to get into.

Second, the tramp-test19-directory-files-and-attributes test now fails.

A nit: the output has some weird spacing and log lines.

Attached is a log illustrating the problem.
[tramp-log.txt (text/plain, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 06 Aug 2019 13:05:01 GMT) Full text and rfc822 format available.

Message #8 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 06 Aug 2019 14:59:17 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> I'm having trouble with 'make check' after the recent Tramp changes on
> master. First, the Tramp tests are now too slow; they take over 6
> minutes on my platform (Fedora 30 x86-4, AMD Phenom II X4 910e,
> default optimization), which is so slow that I don't want to run 'make
> check' while developing, which would be a bad habit to get into.

You have run the expensive tests with this call. If you want to run the
default tests, which are active while "make check", you must use the
default selector. See test/README. On my local machine, this makes the
following difference:

$ make tramp-tests
Ran 67 tests, 65 results as expected, 1 unexpected, 1 skipped (2019-08-06 14:45:25+0200, 420.067490 sec)

$ make tramp-tests SELECTOR='$(SELECTOR_DEFAULT)'
Ran 47 tests, 45 results as expected, 1 unexpected, 1 skipped (2019-08-06 14:49:12+0200, 29.771561 sec)

Note, that you don't need the whole path of the test file, its basename
is sufficient.

> Second, the tramp-test19-directory-files-and-attributes test now fails.

Hmm, prior pushing the patch yesterday, all tests have passed. Today, I
can reproduce the problem. Maybe some of the time conversion changes,
arrived over night, have caused this failure? Will check.

> A nit: the output has some weird spacing and log lines.

I know. Its a continued hunt for years to remove these quirks. Some of
them (like the empty lines) are mysterious, and hard to detect where they
come from.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 06 Aug 2019 16:14:01 GMT) Full text and rfc822 format available.

Message #11 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 6 Aug 2019 09:13:05 -0700
Michael Albinus wrote:
>> Second, the tramp-test19-directory-files-and-attributes test now fails.
> Hmm, prior pushing the patch yesterday, all tests have passed. Today, I
> can reproduce the problem. Maybe some of the time conversion changes,
> arrived over night, have caused this failure? Will check.

Thanks for looking into it. The failure started with commit 
2019-08-05T11:09:26Z!michael.albinus <at> gmx.de 
(6c1d0d53b34d9350d55ebbd83ea56aa751a55f1b), which predates those time-conversion 
changes.

> If you want to run the
> default tests, which are active while "make check", you must use the
> default selector. See test/README.

Thanks, I didn't know that. But that's confusing, as it's natural to assume that 
if FOO is an individual check, then 'make FOO' will run it the same way that 
'make check' would. Would it make sense to change this test to be more natural?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 06 Aug 2019 19:17:02 GMT) Full text and rfc822 format available.

Message #14 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 06 Aug 2019 21:16:09 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

>> If you want to run the
>> default tests, which are active while "make check", you must use the
>> default selector. See test/README.
>
> Thanks, I didn't know that. But that's confusing, as it's natural to
> assume that if FOO is an individual check, then 'make FOO' will run it
> the same way that 'make check' would. Would it make sense to change
> this test to be more natural?

It is intended this way. If I call 'make FOO', I'm interested in package
FOO. For example, because I have changed something in FOO, and I want
to know that I didn't break anything. Expensive tests are part of this.

This is the most applied use case of 'make FOO'. At least I call it
several times a week for exactly this reason. That's why we have the
behavior as it is.

All of this has been discussed already, somewhere in the emacs-devel ML
archives.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 06 Aug 2019 19:55:01 GMT) Full text and rfc822 format available.

Message #17 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 06 Aug 2019 21:54:35 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

>> Hmm, prior pushing the patch yesterday, all tests have passed. Today, I
>> can reproduce the problem. Maybe some of the time conversion changes,
>> arrived over night, have caused this failure? Will check.
>
> Thanks for looking into it. The failure started with commit
> 2019-08-05T11:09:26Z!michael.albinus <at> gmx.de
> (6c1d0d53b34d9350d55ebbd83ea56aa751a55f1b), which predates those
> time-conversion changes.

I've pushed a patch to master, which ought to fix it. Could you, pls,
check?

It was a problem inside tramp-tests, indeed. My final check yesterday
must have been too sloppy, don't know ... :-(

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 06 Aug 2019 23:28:01 GMT) Full text and rfc822 format available.

Message #20 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 6 Aug 2019 16:27:34 -0700
[Message part 1 (text/plain, inline)]
Michael Albinus wrote:
> I've pushed a patch to master, which ought to fix it. Could you, pls,
> check?

Unfortunately I'm still having a problem with that commit 
(998f3612f7e3732d43d8cc6827a16a29008f5db5) on the same platform. Log attached. 
For what it's worth, the test does work for me on another desktop that runs 
Ubuntu 18.04.2, which uses Linux 4.15.0 instead of the Linux 5.1.20 in the 
failing Fedora 30 platform.
[tramp-tests.log (text/x-log, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Wed, 07 Aug 2019 15:22:01 GMT) Full text and rfc822 format available.

Message #23 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Wed, 07 Aug 2019 17:20:36 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> Unfortunately I'm still having a problem with that commit
> (998f3612f7e3732d43d8cc6827a16a29008f5db5) on the same platform. Log
> attached. For what it's worth, the test does work for me on another
> desktop that runs Ubuntu 18.04.2, which uses Linux 4.15.0 instead of
> the Linux 5.1.20 in the failing Fedora 30 platform.

My most recent commit has passed the tests on emba. Maybe it works for
you as well?

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Wed, 07 Aug 2019 19:37:02 GMT) Full text and rfc822 format available.

Message #26 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Wed, 7 Aug 2019 12:36:02 -0700
[Message part 1 (text/plain, inline)]
Michael Albinus wrote:

> My most recent commit has passed the tests on emba. Maybe it works for
> you as well?

Thanks, it does fix the failure. However, the test is still quite a bit slower 
for me than you even when I invoke it the way you suggest, as it takes 108 
seconds for me compared to 30 seconds for you. The four slowest tests are:

   passed  37/47  tramp-test35-remote-path (25.712913 sec)
   passed  42/47  tramp-test41-utf8 (14.036743 sec)
   passed  17/47  tramp-test12-rename-file (8.772317 sec)
   passed  16/47  tramp-test11-copy-file (7.754914 sec)

Log attached.
[tramp-tests.log (text/x-log, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Wed, 07 Aug 2019 21:43:01 GMT) Full text and rfc822 format available.

Message #29 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Wed, 07 Aug 2019 17:42:32 -0400
I suspect it is now unstable. On RHEL 7.6 at d5622eb:

for ((n=1;n<=100;n++)); do
  make lisp/net/tramp-tests SELECTOR='tramp-test19-directory-files-and-attributes' 2>&1 | grep -q '[^0] unexpected' && let fail++
done

echo $fail

returned 7 failures for me.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Thu, 08 Aug 2019 13:53:03 GMT) Full text and rfc822 format available.

Message #32 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Thu, 08 Aug 2019 15:52:05 +0200
Glenn Morris <rgm <at> gnu.org> writes:

Hi Glenn,

> I suspect it is now unstable. On RHEL 7.6 at d5622eb:
>
> for ((n=1;n<=100;n++)); do
>   make lisp/net/tramp-tests
> SELECTOR='tramp-test19-directory-files-and-attributes' 2>&1 | grep -q
> '[^0] unexpected' && let fail++
> done
>
> echo $fail
>
> returned 7 failures for me.

Thanks, this is a very useful test. I've committed another patch to
master. With that patch, your test passes.

(Damn time arithmetic!)

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Thu, 08 Aug 2019 14:15:02 GMT) Full text and rfc822 format available.

Message #35 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Thu, 08 Aug 2019 16:14:18 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> Thanks, it does fix the failure. However, the test is still quite a
> bit slower for me than you even when I invoke it the way you suggest,
> as it takes 108 seconds for me compared to 30 seconds for you. The
> four slowest tests are:
>
>    passed  37/47  tramp-test35-remote-path (25.712913 sec)
>    passed  42/47  tramp-test41-utf8 (14.036743 sec)
>    passed  17/47  tramp-test12-rename-file (8.772317 sec)
>    passed  16/47  tramp-test11-copy-file (7.754914 sec)

My machine is about 10 years old (Ubuntu 19.04, Intel Core2 Duo T95,
4GB). It runs today

   passed  37/47  tramp-test35-remote-path (4.826182 sec)
   passed  42/47  tramp-test41-utf8 (2.885723 sec)
   passed  17/47  tramp-test12-rename-file (1.725029 sec)
   passed  16/47  tramp-test11-copy-file (1.437843 sec)

But experience has shown, that it easily doubles or triples the time if
there are parallel tasks. It would need more debugging on your machine
in order to understand, where the time is spent.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sat, 10 Aug 2019 01:40:03 GMT) Full text and rfc822 format available.

Message #38 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>, Glenn Morris <rgm <at> gnu.org>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Fri, 9 Aug 2019 18:39:13 -0700
Michael Albinus wrote:
> With that patch, your test passes.
> 
> (Damn time arithmetic!)

No, time arithmetic is fun! :-)

Your patch has several instances of (floor (float-time X)) that would be 
slightly better if written as (time-convert X 'integer), as the former has 
(unlikely) rounding errors that the latter lacks.

I assume that the test now uses integer timestamps because file systems 
routinely drop low-order parts of timestamps. But won't the test continue to 
have problems with filesystems whose timestamp resolution is 2 seconds? I have 
heard of such things.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sat, 10 Aug 2019 09:44:02 GMT) Full text and rfc822 format available.

Message #41 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Glenn Morris <rgm <at> gnu.org>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sat, 10 Aug 2019 11:43:33 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> Your patch has several instances of (floor (float-time X)) that would
> be slightly better if written as (time-convert X 'integer), as the
> former has (unlikely) rounding errors that the latter lacks.
>
> I assume that the test now uses integer timestamps because file
> systems routinely drop low-order parts of timestamps. But won't the
> test continue to have problems with filesystems whose timestamp
> resolution is 2 seconds? I have heard of such things.

Thanks for the explanation.

Honestly, we have also a further problem. The tests compare time values
taken on different machines. It cannot be assumed that they are always
synchronized. Until now it works, but who knows.

The tests in tramp--test-file-attributes-equal-p do not require exact
time stamps. They just want to ensure, that time values in file
attributes are newer than the time the ert test case has started. I've
changed the test such a way, that it is compared with the start time
minus 60 seconds. This solves the two problems you have mentioned, and
it also allows a time offset of up to 60 seconds for the two machines in
question. This shall suffice for most of the caes in practice.

A more precise test would be to retrieve the ert test start time, as taken
from the remote machine. But I don't know of a general function doing this.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sat, 10 Aug 2019 20:25:01 GMT) Full text and rfc822 format available.

Message #44 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Glenn Morris <rgm <at> gnu.org>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sat, 10 Aug 2019 13:24:07 -0700
Michael Albinus wrote:

> A more precise test would be to retrieve the ert test start time, as taken
> from the remote machine. But I don't know of a general function doing this.

Can we do the equivalent of 'touch nonce-file' on the remote machine, and then 
retrieve the nonce-file's timestamp? That should be a reasonably-reliable way of 
getting a remote timestamp at the start of the test.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 11 Aug 2019 10:13:02 GMT) Full text and rfc822 format available.

Message #47 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Glenn Morris <rgm <at> gnu.org>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 11 Aug 2019 12:12:21 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> Can we do the equivalent of 'touch nonce-file' on the remote machine,
> and then retrieve the nonce-file's timestamp? That should be a
> reasonably-reliable way of getting a remote timestamp at the start of
> the test.

"touch ..." would work only for ssh-like Tramp methods, which is the
minority. I've applioed a similar approach, using the modification time
of the very first remote file created in the test, as start test time.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Wed, 14 Aug 2019 10:33:01 GMT) Full text and rfc822 format available.

Message #50 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Glenn Morris <rgm <at> gnu.org>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Wed, 14 Aug 2019 12:31:49 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Paul,

>> Can we do the equivalent of 'touch nonce-file' on the remote machine,
>> and then retrieve the nonce-file's timestamp? That should be a
>> reasonably-reliable way of getting a remote timestamp at the start of
>> the test.
>
> "touch ..." would work only for ssh-like Tramp methods, which is the
> minority. I've applied a similar approach, using the modification time
> of the very first remote file created in the test, as start test time.

I guess there's nothing left to do with this bug. Is it OK to close?

Best regards, Michael.




Reply sent to Paul Eggert <eggert <at> cs.ucla.edu>:
You have taken responsibility. (Thu, 15 Aug 2019 04:27:02 GMT) Full text and rfc822 format available.

Notification sent to Paul Eggert <eggert <at> cs.ucla.edu>:
bug acknowledged by developer. (Thu, 15 Aug 2019 04:27:02 GMT) Full text and rfc822 format available.

Message #55 received at 36940-done <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Glenn Morris <rgm <at> gnu.org>, 36940-done <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Wed, 14 Aug 2019 21:26:09 -0700
Michael Albinus wrote:
> I guess there's nothing left to do with this bug. Is it OK to close?

Sure, closing it now.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sat, 24 Aug 2019 01:52:02 GMT) Full text and rfc822 format available.

Message #58 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sat, 24 Aug 2019 03:51:26 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> I guess there's nothing left to do with this bug. Is it OK to close?

Hi Michael,

I'm still seeing failures on my MacOS machine on current master.  I
have attached a log with two failing tests from when I run "make
check" in the top level emacs folder as the file tramp-tests.log.

Weirdly enough, when I say "cd test ; make tramp-tests", I get six
errors instead -- but no log file is saved.  Not sure if I'm doing
something wrong here, but I've included the console output from this
test as tramp-tests2.console.log.  I tried running the tests in both
ways a couple of times, and this odd result is reproducible every
time.

Please let me know if there's anything I can do to help you investigate.

(Not sure if you want to reopen the bug report for this.)

Thanks,
Stefan Kangas
[tramp-tests.log (application/octet-stream, attachment)]
[tramp-tests2.console.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sat, 24 Aug 2019 08:09:02 GMT) Full text and rfc822 format available.

Message #61 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sat, 24 Aug 2019 10:08:22 +0200
[Message part 1 (text/plain, inline)]
Stefan Kangas <stefan <at> marxist.se> writes:

> Hi Michael,

Hi Stefan,

> I'm still seeing failures on my MacOS machine on current master.  I
> have attached a log with two failing tests from when I run "make
> check" in the top level emacs folder as the file tramp-tests.log.
>
> Weirdly enough, when I say "cd test ; make tramp-tests", I get six
> errors instead -- but no log file is saved.  Not sure if I'm doing
> something wrong here, but I've included the console output from this
> test as tramp-tests2.console.log.  I tried running the tests in both
> ways a couple of times, and this odd result is reproducible every
> time.

If you say "make check", the default ert selector is applied, which
excludes expensive tests. if you say "make FOO", all test from package
foo are applied, also the expensive tests. You see it by the different
numbers of tests, 47 vs 67 tests.

If you want to get a log file when running tests for package FOO, say
"make FOO.log". This seems to be broken now; will check.

All of this is described in test/README.

> (Not sure if you want to reopen the bug report for this.)

For the time being, I'll continue to reply here. If needed (more complex
failure, not reported initially here) we might need to create another
bug report.

> (t 3 501 20 (0 0 0 1000) (0 0 0 1000) (0 0 0 1000) 0 "drwxr-xr-x" nil 41369843 (-1 . 18))
> (t 5 501 20 (0 0 0 1000) (0 0 0 1000) (0 0 0 1000) 0 "drwxr-xr-x" nil (631 . 16627) (-1 . 18))
> Test tramp-test19-directory-files-and-attributes condition:
>     (ert-test-failed
>      ((should
>        (tramp--test-file-attributes-equal-p
> 	(file-attributes ...)
> 	(cdr elt)))
>       :form
>       (tramp--test-file-attributes-equal-p
>        (t 3 501 20
> 	  (23904 37780)
> 	  (23904 37780)
> 	  (23904 37780)
> 	  102 "drwxr-xr-x" nil ...)
>        (t 5 501 20
> 	  (23904 37780)
> 	  (23904 37780)
> 	  (23904 37780)
> 	  170 "drwxr-xr-x" nil ...))
>       :value nil :explanation
>       (list-elt 1
> 		(different-atoms
> 		 (3 "#x3" "?")
> 		 (5 "#x5" "?")))))
>    FAILED  30/67  tramp-test19-directory-files-and-attributes (0.996916 sec)

Well, this time it is because of a different inode number, 41369843 vs
(631 . 16627). Will check what's up.

> Test tramp-test22-file-times condition:
>     (ert-test-failed
>      ((should
>        (equal
> 	(tramp-compat-file-attribute-modification-time ...)
> 	(seconds-to-time 1)))
>       :form
>       (equal
>        (0 1)
>        (0 1 0 0))
>       :value nil :explanation
>       (proper-lists-of-different-length 2 4
> 					(0 1)
> 					(0 1 0 0)
> 					first-mismatch-at 2)))
>    FAILED  33/67  tramp-test22-file-times (0.567633 sec)

Grrr, the time stamps (0 1) and (0 1 0 0) are the same, but handled
differently internally. Fixed in master.

> Test tramp-test30-make-process condition:
>     (ert-test-failed
>      ((should
>        (string-match "killed
> \\'"
> 		     (buffer-string)))
>       :form
>       (string-match "killed
> \\'" "killed: 9
> ")
>       :value nil))
>    FAILED  41/67  tramp-test30-make-process (1.024851 sec)

Ahh, the message on macOS is slightly different. Fixed in master.

> Test tramp-test41-utf8 condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  58/67  tramp-test41-utf8 (35.641851 sec)

> Test tramp-test41-utf8-with-ls condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  59/67  tramp-test41-utf8-with-ls (29.942527 sec)

> Test tramp-test41-utf8-with-perl condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  60/67  tramp-test41-utf8-with-perl (36.502001 sec)

These three errors are basically the same. Could you please apply the
following patch to tramp-tests.el, and show me the output? You might run

make -C test tramp-tests >tramp-tests.log 2>&1

[Message part 2 (text/x-diff, inline)]
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 180f746c64..42d44f780b 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -5459,7 +5459,8 @@ tramp--test-utf8
   (skip-unless (not (tramp--test-windows-nt-and-batch)))
   (skip-unless (not (tramp--test-windows-nt-and-pscp-psftp-p)))

-  (tramp--test-utf8))
+  (tramp--test-instrument-test-case 10
+  (tramp--test-utf8)))

 (ert-deftest tramp-test41-utf8-with-stat ()
   "Check UTF8 encoding in file names and file contents.
[Message part 3 (text/plain, inline)]
> Thanks,
> Stefan Kangas

Best regards, Michael.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sat, 24 Aug 2019 12:52:01 GMT) Full text and rfc822 format available.

Message #64 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sat, 24 Aug 2019 14:51:40 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

> Grrr, the time stamps (0 1) and (0 1 0 0) are the same, but handled
> differently internally. Fixed in master.

> Ahh, the message on macOS is slightly different. Fixed in master.

Thanks, down to 4 unexpected results now.

> These three errors are basically the same. Could you please apply the
> following patch to tramp-tests.el, and show me the output? You might run
>
> make -C test tramp-tests >tramp-tests.log 2>&1

I've done that, but I'll send you the file off list since it's
approximately 10 MB.

Best regards,
Stefan Kangas




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 09:28:02 GMT) Full text and rfc822 format available.

Message #67 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 11:27:26 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> Hi again,

Hi Stefan,

> Thanks a lot for working on this.  Please find attached the log file
> you requested, and do tell me if there is anything more I can do to
> help.

What I see is that the string to be searched for in the tests is
different from what the test should use. The test declares the string

"Γυρίστε το Γαλαξία με Ώτο Στοπ"

The fourth character is "ί", which is 

character: ί (displayed as ί) (codepoint 943, #o1657, #x3af)
name: GREEK SMALL LETTER IOTA WITH TONOS

In your search, the following string is used instead

"Γυρίστε το Γαλαξία με Ώτο Στοπ"

The fourth character is replaced by the two characters "ί", which are

character: ι (displayed as ι) (codepoint 953, #o1671, #x3b9)
name: GREEK SMALL LETTER IOTA

character: ́ (displayed as ́) (codepoint 769, #o1401, #x301)
name: COMBINING ACUTE ACCENT

I have no idea how and why the replacement happened. Could somebody with
UTF-8 background chime in, please? Or is it a font issue?

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 09:52:02 GMT) Full text and rfc822 format available.

Message #70 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 12:51:05 +0300
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Date: Sun, 25 Aug 2019 11:27:26 +0200
> Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
> 
> What I see is that the string to be searched for in the tests is
> different from what the test should use. The test declares the string
> 
> "Γυρίστε το Γαλαξία με Ώτο Στοπ"
> 
> The fourth character is "ί", which is 
> 
> character: ί (displayed as ί) (codepoint 943, #o1657, #x3af)
> name: GREEK SMALL LETTER IOTA WITH TONOS
> 
> In your search, the following string is used instead
> 
> "Γυρίστε το Γαλαξία με Ώτο Στοπ"
> 
> The fourth character is replaced by the two characters "ί", which are
> 
> character: ι (displayed as ι) (codepoint 953, #o1671, #x3b9)
> name: GREEK SMALL LETTER IOTA
> 
> character: ́ (displayed as ́) (codepoint 769, #o1401, #x301)
> name: COMBINING ACUTE ACCENT
> 
> I have no idea how and why the replacement happened. Could somebody with
> UTF-8 background chime in, please? Or is it a font issue?

Darwin filesystem under utf-8-hfs uses decomposed strings for file
names.  What you see is the result of that decomposition: ί was
decomposed into ι followed by the acute accent, a combination that
displays the same as the original precomposed letter, but is different
when binary-compared.

I think you should run the strings through
ucs-normalize-HFS-NFD-string before comparing them, when the
en/decoding is done with utf-8-hfs.  Alternatively, use the facilities
in char-fold.el to generate a more lax regexp for your search in
tramp--test-check-files.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 10:08:02 GMT) Full text and rfc822 format available.

Message #73 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: michael.albinus <at> gmx.de
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 13:07:51 +0300
> Date: Sun, 25 Aug 2019 12:51:05 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
> 
> I think you should run the strings through
> ucs-normalize-HFS-NFD-string before comparing them, when the
> en/decoding is done with utf-8-hfs.  Alternatively, use the facilities
> in char-fold.el to generate a more lax regexp for your search in
> tramp--test-check-files.

As yet another possibility, avoid decomposable characters in the
strings you use altogether.  You can find out whether a given
character CH can be decomposed like this:

  (get-char-code-property CH 'decomposition)

For characters that have no decomposition, this should return a list
with a single element equal to CH.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 11:28:01 GMT) Full text and rfc822 format available.

Message #76 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 13:26:30 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

>> I think you should run the strings through
>> ucs-normalize-HFS-NFD-string before comparing them, when the
>> en/decoding is done with utf-8-hfs.  Alternatively, use the facilities
>> in char-fold.el to generate a more lax regexp for your search in
>> tramp--test-check-files.
>
> As yet another possibility, avoid decomposable characters in the
> strings you use altogether.  You can find out whether a given
> character CH can be decomposed like this:
>
>   (get-char-code-property CH 'decomposition)
>
> For characters that have no decomposition, this should return a list
> with a single element equal to CH.

Well, tramp-test41-utf8 is intended to handle all whistles and bells
related to utf8 and Tramp. So this case shall be handled as well.

The interesting point is, that the test does *not* fail with such file
names. It is rather a regexp search in a shell output, which fails.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 11:30:02 GMT) Full text and rfc822 format available.

Message #79 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 13:28:46 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

> Darwin filesystem under utf-8-hfs uses decomposed strings for file
> names.  What you see is the result of that decomposition: ί was
> decomposed into ι followed by the acute accent, a combination that
> displays the same as the original precomposed letter, but is different
> when binary-compared.
>
> I think you should run the strings through
> ucs-normalize-HFS-NFD-string before comparing them, when the
> en/decoding is done with utf-8-hfs.  Alternatively, use the facilities
> in char-fold.el to generate a more lax regexp for your search in
> tramp--test-check-files.

Thanks, I will try to adapt the test accordingly. In core Tramp, nothing
needs to be changed I believe.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 11:40:02 GMT) Full text and rfc822 format available.

Message #82 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 14:39:46 +0300
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: eggert <at> cs.ucla.edu,  stefan <at> marxist.se,  36940 <at> debbugs.gnu.org
> Date: Sun, 25 Aug 2019 13:26:30 +0200
> 
> The interesting point is, that the test does *not* fail with such file
> names.

What does "not failing with such file names" entail?  Does the test
compare file names as strings, for example?




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 11:47:02 GMT) Full text and rfc822 format available.

Message #85 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 13:46:30 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

>> The interesting point is, that the test does *not* fail with such file
>> names.
>
> What does "not failing with such file names" entail?  Does the test
> compare file names as strings, for example?

Yes. But it compares file names as returned from the remote host, so
both strings are transformed the same way. I believe.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 11:49:02 GMT) Full text and rfc822 format available.

Message #88 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 13:48:08 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Stefan,

>>    FAILED  30/67  tramp-test19-directory-files-and-attributes (0.996916 sec)
>
> Well, this time it is because of a different inode number, 41369843 vs
> (631 . 16627). Will check what's up.

Fixed in master.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 12:00:02 GMT) Full text and rfc822 format available.

Message #91 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 14:58:54 +0300
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: eggert <at> cs.ucla.edu,  stefan <at> marxist.se,  36940 <at> debbugs.gnu.org
> Date: Sun, 25 Aug 2019 13:46:30 +0200
> 
> > What does "not failing with such file names" entail?  Does the test
> > compare file names as strings, for example?
> 
> Yes. But it compares file names as returned from the remote host, so
> both strings are transformed the same way. I believe.

Are the strings returned by the remote compared as unibyte strings?
If not, maybe there's a subtle bug in the comparison that does fail.
For example, could it be that you decode those strings as utf-8, not
as utf-8-hfs?  The latter should produce back the composed characters,
I think.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 15:40:02 GMT) Full text and rfc822 format available.

Message #94 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>, Stefan Kangas <stefan <at> marxist.se>
Cc: 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 08:39:01 -0700
Michael Albinus wrote:
> Fixed in master.

Surely the fix won't work for most inode numbers greater than 2**53, because 
converting them to double and back will lose information due to rounding. Such 
inode numbers do exist in the wild, with FUSE and the like; see, e.g.:

https://forum.rclone.org/t/fuse-inode-number-aufs/215




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 16:35:01 GMT) Full text and rfc822 format available.

Message #97 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Stefan Kangas <stefan <at> marxist.se>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 18:34:21 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> Surely the fix won't work for most inode numbers greater than 2**53,
> because converting them to double and back will lose information due
> to rounding.

Tramp uses this approach already at other places, I have just unified
the way inode numbers are retrieved. No problem reported over the years.

At which places in the code do you believe such errors will happen? The
floating number will be converted in tramp-convert-file-attributes, I
cannot see how a rounding error could happen there.

Practically, the inode slot of file-attributes doesn't seem to be
relevant. I haven't seen any code in Emacs which needs this
slot. Likely, Tramp spends too much effort computing it, I believe.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Sun, 25 Aug 2019 20:37:01 GMT) Full text and rfc822 format available.

Message #100 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Stefan Kangas <stefan <at> marxist.se>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Sun, 25 Aug 2019 13:36:46 -0700
[Message part 1 (text/plain, inline)]
Michael Albinus wrote:

> At which places in the code do you believe such errors will happen? The
> floating number will be converted in tramp-convert-file-attributes, I
> cannot see how a rounding error could happen there.

I think the rounding error doesn't happen there, as that code accurately 
converts the Lisp float to an integer or to a cons of integers. The rounding 
error happens in functions like tramp-send-command-and-read when the Emacs Lisp 
reader scans the string generated by the Perl code, and converts the string to a 
Lisp float.

For example, suppose the inode number is 2**63 - 512 and the Perl code generates 
"9223372036854775296.0", which is exact. When the Emacs Lisp reader converts 
this to double it must round since there is no exact double representation, and 
rounding yields 2**63, i.e., 9223372036854775808.0, which is off by 512.

A simple fix is to change the Perl code to omit the trailing ".0", e.g., 
"9223372036854775296" rather than "9223372036854775296.0". This will cause the 
Emacs Lisp reader in master to return an exact integer instead of a float, thus 
avoiding the rounding error. This fix will still suffer from the same rounding 
errors in older Emacs releases where the reader returns a float for integers out 
of fixnum range, but it shouldn't hurt those older releases and it should fix 
the problem in master.

Please see attached patch. I haven't tested or installed the patch as I don't 
use Tramp, but you should be able to test it with something like this:

$ echo '' | dd obs=1 seek=9007199254740992 of=file
0+1 records in
1+0 records out
1 byte (1 B) copied, 0.015429 seconds, 0.1 kB/s
$ ls -l file
-rw-rw-r-- 1 eggert faculty 9007199254740993 2019-08-25 13:28 file

assuming your filesystem supports files containing more than 2**53 bytes - since 
there's a similar bug for file sizes.

> I haven't seen any code in Emacs which needs this slot.

It's used by ede--inode-for-dir, eshell-shuffle-files, ls-lisp-format, 
find-lisp-format, nnmaildir--group-maxnum, nnmaildir--new-number. Typical uses 
are to determine whether two directory entries identify the same file, e.g., 
(equal (file-attribute-inode-number file1) (file-attribute-inode-number file2)) 
or to use it in a hash table.

As I mentioned above, a similar bug occurs for the file size slot: files 
containing more than 2**53 bytes have rounding errors in their sizes. The 
attached patch should fix that too.
[0001-Fix-Tramp-rounding-of-file-sizes-and-inode-numbers.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 08:45:01 GMT) Full text and rfc822 format available.

Message #103 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Stefan Kangas <stefan <at> marxist.se>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 10:44:06 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> For example, suppose the inode number is 2**63 - 512 and the Perl code
> generates "9223372036854775296.0", which is exact. When the Emacs Lisp
> reader converts this to double it must round since there is no exact
> double representation, and rounding yields 2**63, i.e.,
> 9223372036854775808.0, which is off by 512.
>
> A simple fix is to change the Perl code to omit the trailing ".0",
> e.g., "9223372036854775296" rather than "9223372036854775296.0". This
> will cause the Emacs Lisp reader in master to return an exact integer
> instead of a float, thus avoiding the rounding error. This fix will
> still suffer from the same rounding errors in older Emacs releases
> where the reader returns a float for integers out of fixnum range, but
> it shouldn't hurt those older releases and it should fix the problem
> in master.

Indeed, up to Emacs 26 this rounding error still applies. With master
(improved big number support) it works fine.

> Please see attached patch. I haven't tested or installed the patch as
> I don't use Tramp, but you should be able to test it with something
> like this:

OK for me. You might install the patch, and also adapt the templates in
`tramp-do-file-attributes-with-stat and'
`tramp-do-directory-files-and-attributes-with-stat'.

>> I haven't seen any code in Emacs which needs this slot.
>
> It's used by ede--inode-for-dir, eshell-shuffle-files, ls-lisp-format,
> find-lisp-format, nnmaildir--group-maxnum,
> nnmaildir--new-number.

`ls-lisp-format' and `find-lisp-format' don't count, they *provide* the
inode number; they don't *use* it.

> Typical uses are to determine whether two
> directory entries identify the same file, e.g., (equal
> (file-attribute-inode-number file1) (file-attribute-inode-number
> file2)) or to use it in a hash table.

`ede--inode-for-dir' and `nnmaildir--*' are a little bit lazy, they
don't compare the device number. Might be acceptable; one could assume
that a project dir or a mail dir handle only files on the same file system.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 09:23:02 GMT) Full text and rfc822 format available.

Message #106 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 11:22:16 +0200
Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

>> Yes. But it compares file names as returned from the remote host, so
>> both strings are transformed the same way. I believe.
>
> Are the strings returned by the remote compared as unibyte strings?
> If not, maybe there's a subtle bug in the comparison that does fail.
> For example, could it be that you decode those strings as utf-8, not
> as utf-8-hfs?  The latter should produce back the composed characters,
> I think.

The error happens while searching in a process buffer, which is set properly:

14:40:39.573073 tramp-open-connection-setup-interactive-shell (5) # Setting coding system to `utf-8-hfs' and `utf-8-hfs-mac'

But the search string needs to be normalized as proposed by you. Will do.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 10:00:03 GMT) Full text and rfc822 format available.

Message #109 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 12:59:33 +0300
> From: Michael Albinus <michael.albinus <at> gmx.de>
> Cc: eggert <at> cs.ucla.edu,  stefan <at> marxist.se,  36940 <at> debbugs.gnu.org
> Date: Mon, 26 Aug 2019 11:22:16 +0200
> 
> > Are the strings returned by the remote compared as unibyte strings?
> > If not, maybe there's a subtle bug in the comparison that does fail.
> > For example, could it be that you decode those strings as utf-8, not
> > as utf-8-hfs?  The latter should produce back the composed characters,
> > I think.
> 
> The error happens while searching in a process buffer, which is set properly:
> 
> 14:40:39.573073 tramp-open-connection-setup-interactive-shell (5) # Setting coding system to `utf-8-hfs' and `utf-8-hfs-mac'
> 
> But the search string needs to be normalized as proposed by you. Will do.

Sorry for bothering you, but now I am confused.  If the process
buffer's contents is decoded with utf-8-hfs, then decoding should have
composed the characters back, I believe.  So the text to be searched
in the buffer should use the precomposed character U+03AF GREEK SMALL
LETTER IOTA WITH TONOS.  Is that so?  And if so, is the problem with
the regexp you pass to re-search-forward?  Then where did that regexp
come from -- maybe the problem happens while you produce that regexp?

Feel free to ignore my questions if they are just a distraction, due
to my unfamiliarity with the text internals.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 11:48:01 GMT) Full text and rfc822 format available.

Message #112 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: eggert <at> cs.ucla.edu, stefan <at> marxist.se, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 13:47:02 +0200
[Message part 1 (text/plain, inline)]
Eli Zaretskii <eliz <at> gnu.org> writes:

Hi Eli,

>> The error happens while searching in a process buffer, which is set properly:
>>
>> 14:40:39.573073 tramp-open-connection-setup-interactive-shell (5) #
>> Setting coding system to `utf-8-hfs' and `utf-8-hfs-mac'
>>
>> But the search string needs to be normalized as proposed by you. Will do.
>
> Sorry for bothering you, but now I am confused.  If the process
> buffer's contents is decoded with utf-8-hfs, then decoding should have
> composed the characters back, I believe.  So the text to be searched
> in the buffer should use the precomposed character U+03AF GREEK SMALL
> LETTER IOTA WITH TONOS.  Is that so?  And if so, is the problem with
> the regexp you pass to re-search-forward?  Then where did that regexp
> come from -- maybe the problem happens while you produce that regexp?

Now I start to understand what happens. In tramp--test-check-files,
there is a loop over the test strings, which are used for different test
purposes. At the very end, an environment variable is set as

		  (setenv envvar elt)

When the failing regexp search is applied, the original string is not
taken, but the set environment variable

		    (should
		     (re-search-forward
		      (format
		       "^%s=%s$"
		       (regexp-quote envvar)
		       (regexp-quote (getenv envvar))))))))))

Likely, (getenv envvar) returns the encoded string, which fails. Using
the original string shall work.

> Feel free to ignore my questions if they are just a distraction, due
> to my unfamiliarity with the text internals.

No, you're very helpful as usual! Thanks!

Stefan, does the following patch solves the problem for you?

[Message part 2 (text/x-patch, inline)]
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 557536a0eb..6ce1afa4a7 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -5288,7 +5288,7 @@ tramp--test-check-files
 		      (format
 		       "^%s=%s$"
 		       (regexp-quote envvar)
-		       (regexp-quote (getenv envvar))))))))))
+		       (regexp-quote elt)))))))))

 	;; Cleanup.
 	(ignore-errors (delete-directory tmp-name1 'recursive))
[Message part 3 (text/plain, inline)]
Best regards, Michael.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 12:55:01 GMT) Full text and rfc822 format available.

Message #115 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 14:54:39 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> Stefan, does the following patch solves the problem for you?

Current master gives the same results with or without that patch, unfortunately.

Without the patch:

4 unexpected results:
   FAILED  tramp-test19-directory-files-and-attributes
   FAILED  tramp-test41-utf8
   FAILED  tramp-test41-utf8-with-ls
   FAILED  tramp-test41-utf8-with-perl

With the patch:

4 unexpected results:
   FAILED  tramp-test19-directory-files-and-attributes
   FAILED  tramp-test41-utf8
   FAILED  tramp-test41-utf8-with-ls
   FAILED  tramp-test41-utf8-with-perl

Please find attached the console log of the test with that patch.

Thanks,
Stefan Kangas
[tramp-tests3.console.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 14:20:01 GMT) Full text and rfc822 format available.

Message #118 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 16:19:22 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

>> Stefan, does the following patch solves the problem for you?
>
> Current master gives the same results with or without that patch, unfortunately.

It's a pity I have no Mac for own testing ...

> Without the patch:
>
> 4 unexpected results:
>    FAILED  tramp-test19-directory-files-and-attributes

Well, there was also a difference in the link number. Fixed in master.

(I have no idea why it happens only for you.)

>    FAILED  tramp-test41-utf8
>    FAILED  tramp-test41-utf8-with-ls
>    FAILED  tramp-test41-utf8-with-perl

Hmm, under macOS the search string must be encoded as Eli has
suggested. I've pushed a respective patch to master.

Could you, please, check again?

> Thanks,
> Stefan Kangas

Thanks for your patience, and best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 14:37:01 GMT) Full text and rfc822 format available.

Message #121 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 16:36:35 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> It's a pity I have no Mac for own testing ...

I can imagine it must be frustrating.  I'm trying to respond quickly
in order to minimize the turnaround time.  Thanks for your continued
efforts to fix this.

> Could you, please, check again?

Absolutely.  On current master (commit f87ace2aed), I'm now seeing:

3 unexpected results:
   FAILED  tramp-test41-utf8
   FAILED  tramp-test41-utf8-with-ls
   FAILED  tramp-test41-utf8-with-perl

Please find attached the console log for that run, using:
make -C test tramp-tests >tramp-tests.log 2>&1

Best regards,
Stefan Kangas
[tramp-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 15:10:01 GMT) Full text and rfc822 format available.

Message #124 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 17:09:28 +0200
[Message part 1 (text/plain, inline)]
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

> Absolutely.  On current master (commit f87ace2aed), I'm now seeing:
>
> 3 unexpected results:
>    FAILED  tramp-test41-utf8
>    FAILED  tramp-test41-utf8-with-ls
>    FAILED  tramp-test41-utf8-with-perl

Well, so we've fixed tramp-test19-directory-files-and-attributes at least.
Progress!

> Test tramp-test41-utf8 condition:
>     (search-failed "^VAR_971C287AFA5BBEDD54ACB58B1CE718B4=Γυρίστε το Γαλαξία με Ώτο Στοπ$")
>    FAILED  58/67  tramp-test41-utf8 (35.618526 sec)

Hmm, the string is still decomposed. Let's give it a last try, writing
some traces. Could you pls apply the patch, and send the result. You
could call

make -C test lisp/net/tramp-tests.log SELECTOR='tramp-test41-utf8'

[Message part 2 (text/x-patch, inline)]
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 9930a2c9e1..69dadcd107 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -5288,6 +5288,14 @@ tramp--test-check-files
 		  (setenv "PS1")
 		  (with-temp-buffer
 		    (should (zerop (process-file "env" nil t nil)))
+                    (tramp--test-message
+                     "coding-sytem: `%s' `%s'"
+                     coding-system-for-read coding-system-for-write )
+                    (tramp--test-message "search: `%s'" (getenv envvar))
+                    (let ((coding-system-for-read 'no-conversion)
+                          (coding-system-for-write 'no-conversion))
+                      (tramp--test-message "search: `%s'" (getenv envvar))
+                      (tramp--test-message "buffer: `%s'" (buffer-string)))
 		    (goto-char (point-min))
 		    (should
 		     (re-search-forward
[Message part 3 (text/plain, inline)]
Best regards, Michael.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 15:47:02 GMT) Full text and rfc822 format available.

Message #127 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 17:46:16 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> Well, so we've fixed tramp-test19-directory-files-and-attributes at least.
> Progress!

Indeed, congratulations.

> Hmm, the string is still decomposed. Let's give it a last try, writing
> some traces. Could you pls apply the patch, and send the result. You
> could call
>
> make -C test lisp/net/tramp-tests.log SELECTOR='tramp-test41-utf8'

Please find attached the log of running that with your patch.

Best regards,
Stefan Kangas
[tramp-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 16:44:01 GMT) Full text and rfc822 format available.

Message #130 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 18:43:35 +0200
[Message part 1 (text/plain, inline)]
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

> Please find attached the log of running that with your patch.

So let's try the following patch.

[Message part 2 (text/x-patch, inline)]
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 9930a2c9e1..8afd9f4ba4 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -5288,18 +5288,28 @@ tramp--test-check-files
 		  (setenv "PS1")
 		  (with-temp-buffer
 		    (should (zerop (process-file "env" nil t nil)))
-		    (goto-char (point-min))
-		    (should
-		     (re-search-forward
-                      ;; We must use proper encoding on macOS.  See
-                      ;; Bug#36940.
-                      (funcall
-	               (if (eq coding-system-for-read 'utf-8-hfs)
-                           'ucs-normalize-HFS-NFD-string 'identity)
-		       (format
-		        "^%s=%s$"
-		        (regexp-quote envvar)
-		        (regexp-quote (getenv envvar)))))))))))
+                    (tramp--test-message
+                     "coding-sytem: `%s' `%s'"
+                     coding-system-for-read coding-system-for-write )
+                    (tramp--test-message "search: `%s'" (getenv envvar))
+                    (let ((coding-system-for-read 'no-conversion)
+                          (coding-system-for-write 'no-conversion))
+                      (tramp--test-message
+                       "search: `%s'"
+                       (ucs-normalize-HFS-NFD-string (getenv envvar)))
+                      (tramp--test-message "buffer: `%s'" (buffer-string))
+		      (goto-char (point-min))
+		      (should
+		       (re-search-forward
+                        ;; We must use proper encoding on macOS.  See
+                        ;; Bug#36940.
+                        (funcall
+	                 (if (eq system-type 'darwin)
+                             'ucs-normalize-HFS-NFD-string 'identity)
+		         (format
+		          "^%s=%s$"
+		          (regexp-quote envvar)
+		          (regexp-quote (getenv envvar))))))))))))

 	;; Cleanup.
 	(ignore-errors (delete-directory tmp-name1 'recursive))
[Message part 3 (text/plain, inline)]
> Best regards,
> Stefan Kangas

Best regards, Michael.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 17:47:01 GMT) Full text and rfc822 format available.

Message #133 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 19:46:33 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> So let's try the following patch.

Sorry, it's still failing.  I've attached the log.

Best regards,
Stefan Kangas
[tramp-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Mon, 26 Aug 2019 19:49:01 GMT) Full text and rfc822 format available.

Message #136 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 21:47:37 +0200
[Message part 1 (text/plain, inline)]
Stefan Kangas <stefan <at> marxist.se> writes:

> Sorry, it's still failing.  I've attached the log.

Let's try this one. If it also fails, I will disable this part of the
test for macOS. Tramp works correctly, and the check is a minor one only.

[Message part 2 (text/x-patch, inline)]
diff --git a/test/lisp/net/tramp-tests.el b/test/lisp/net/tramp-tests.el
index 9930a2c9e1..3d5949b033 100644
--- a/test/lisp/net/tramp-tests.el
+++ b/test/lisp/net/tramp-tests.el
@@ -5288,18 +5288,26 @@ tramp--test-check-files
 		  (setenv "PS1")
 		  (with-temp-buffer
 		    (should (zerop (process-file "env" nil t nil)))
-		    (goto-char (point-min))
-		    (should
-		     (re-search-forward
-                      ;; We must use proper encoding on macOS.  See
-                      ;; Bug#36940.
-                      (funcall
-	               (if (eq coding-system-for-read 'utf-8-hfs)
-                           'ucs-normalize-HFS-NFD-string 'identity)
-		       (format
-		        "^%s=%s$"
-		        (regexp-quote envvar)
-		        (regexp-quote (getenv envvar)))))))))))
+                    (tramp--test-message
+                     "coding-sytem: `%s' `%s'"
+                     coding-system-for-read coding-system-for-write )
+                    (tramp--test-message "search: `%s'" (getenv envvar))
+                    ;; We must use proper encoding on macOS.  See
+                    ;; Bug#36940.
+                    (let ((system-type 'gnu/linux)
+                          (coding-system-for-read 'no-conversion)
+                          (coding-system-for-write 'no-conversion))
+                      (tramp--test-message
+                       "search: `%s'"
+                       (ucs-normalize-HFS-NFD-string (getenv envvar)))
+                      (tramp--test-message "buffer: `%s'" (buffer-string))
+		      (goto-char (point-min))
+		      (should
+		       (re-search-forward
+		        (format
+		         "^%s=%s$"
+		         (regexp-quote envvar)
+		         (regexp-quote (getenv envvar)))))))))))

 	;; Cleanup.
 	(ignore-errors (delete-directory tmp-name1 'recursive))
[Message part 3 (text/plain, inline)]
> Best regards,
> Stefan Kangas

Best regards, Michael.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 27 Aug 2019 02:18:01 GMT) Full text and rfc822 format available.

Message #139 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Stefan Kangas <stefan <at> marxist.se>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Mon, 26 Aug 2019 19:17:01 -0700
[Message part 1 (text/plain, inline)]
Michael Albinus wrote:
> You might install the patch, and also adapt the templates in
> `tramp-do-file-attributes-with-stat and'
> `tramp-do-directory-files-and-attributes-with-stat'.

Thanks, I missed those templates. I adapted them too and installed the attached 
combined patch into master.
[0001-Fix-Tramp-rounding-of-file-sizes-and-inode-numbers.patch (text/x-patch, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 27 Aug 2019 11:04:01 GMT) Full text and rfc822 format available.

Message #142 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 27 Aug 2019 13:03:03 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Stefan,

> If you want to get a log file when running tests for package FOO, say
> "make FOO.log". This seems to be broken; will check.

Fixed in master. You can run "make tramp-tests.log" now, for example.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 27 Aug 2019 16:35:02 GMT) Full text and rfc822 format available.

Message #145 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 27 Aug 2019 18:34:11 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> Let's try this one. If it also fails, I will disable this part of the
> test for macOS. Tramp works correctly, and the check is a minor one only.

Sorry for the delay.  I had a chance to run this on my macOS machine
again, and unfortunately it fails too.  I've attached the log.

Best regards,
Stefan Kangas
[tramp-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Tue, 27 Aug 2019 16:57:01 GMT) Full text and rfc822 format available.

Message #148 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Tue, 27 Aug 2019 18:56:00 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

>> Let's try this one. If it also fails, I will disable this part of the
>> test for macOS. Tramp works correctly, and the check is a minor one only.
>
> Sorry for the delay.  I had a chance to run this on my macOS machine
> again, and unfortunately it fails too.  I've attached the log.

So I have deactivated this part of the test for macOS. Tramp tests shall
pass now for you.

> Best regards,
> Stefan Kangas

Thanks for your continuous patience, and best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#36940; Package emacs. (Wed, 28 Aug 2019 00:25:02 GMT) Full text and rfc822 format available.

Message #151 received at 36940 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Paul Eggert <eggert <at> cs.ucla.edu>,
 36940 <at> debbugs.gnu.org
Subject: Re: bug#36940: tests slowness and failure after recent Tramp changes
Date: Wed, 28 Aug 2019 02:23:55 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

> > Sorry for the delay.  I had a chance to run this on my macOS machine
> > again, and unfortunately it fails too.  I've attached the log.
>
> So I have deactivated this part of the test for macOS. Tramp tests shall
> pass now for you.

Sounds good.  And indeed, the tests are now passing.

> Thanks for your continuous patience, and best regards, Michael.

Thank you again for looking into this.

Best regards,
Stefan Kangas




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Wed, 25 Sep 2019 11:24:05 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 187 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.