GNU bug report logs - #37202
shadowfile-tests: two failures on macOS

Previous Next

Package: emacs;

Reported by: Stefan Kangas <stefan <at> marxist.se>

Date: Wed, 28 Aug 2019 00:42:02 UTC

Severity: normal

Fixed in version 27.1

Done: Michael Albinus <michael.albinus <at> gmx.de>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 37202 in the body.
You can then email your comments to 37202 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Wed, 28 Aug 2019 00:42:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Stefan Kangas <stefan <at> marxist.se>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Wed, 28 Aug 2019 00:42:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: bug-gnu-emacs <at> gnu.org
Subject: shadowfile-tests: two failures on macOS
Date: Wed, 28 Aug 2019 02:41:04 +0200
I'm seeing two test failures in shadowfile-tests on macOS on current master.

Running 10 tests (2019-08-28 02:27:08+0200, selector `(not (or (tag
:expensive-test) (tag :unstable)))')
   passed   1/10  shadow-test00-clusters (4.016950 sec)
   passed   2/10  shadow-test01-sites (0.001616 sec)
   passed   3/10  shadow-test02-files (0.002011 sec)
   passed   4/10  shadow-test03-expand-cluster-in-file-name (0.016575 sec)
   passed   5/10  shadow-test04-contract-file-name (0.105286 sec)
   passed   6/10  shadow-test05-file-match (0.001446 sec)
   passed   7/10  shadow-test06-literal-groups (0.005732 sec)
   passed   8/10  shadow-test07-regexp-groups (0.001333 sec)
Test shadow-test08-shadow-todo backtrace:
  signal(ert-test-failed (((should (member (cons file (shadow-contract
  ert-fail(((should (member (cons file (shadow-contract-file-name (con
  #f(compiled-function () #<bytecode 0x1ffc602329d1>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name shadow-test08-shadow-todo :documentat
  ert-run-or-rerun-test(#s(ert--stats :selector (not (or ... ...)) :te
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/shadowfile-tests" "-
  command-line()
  normal-top-level()
Test shadow-test08-shadow-todo condition:
    (ert-test-failed
     ((should
       (member
    (cons file ...)
    shadow-files-to-copy))
      :form
      (member
       ("/var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-tests6rxEJv"
. "/cluster2:/var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-tests6rxEJv")
       nil)
      :value nil))
   FAILED   9/10  shadow-test08-shadow-todo (0.231305 sec)


Test shadow-test09-shadow-copy-files backtrace:
  signal(ert-test-failed (((should (looking-at (regexp-quote "(setq sh
  ert-fail(((should (looking-at (regexp-quote "(setq shadow-files-to-c
  #f(compiled-function () #<bytecode 0x1ffc60232b79>)()
  ert--run-test-internal(#s(ert--test-execution-info :test #s(ert-test
  ert-run-test(#s(ert-test :name shadow-test09-shadow-copy-files :docu
  ert-run-or-rerun-test(#s(ert--stats :selector (not (or ... ...)) :te
  ert-run-tests((not (or (tag :expensive-test) (tag :unstable))) #f(co
  ert-run-tests-batch((not (or (tag :expensive-test) (tag :unstable)))
  ert-run-tests-batch-and-exit((not (or (tag :expensive-test) (tag :un
  eval((ert-run-tests-batch-and-exit '(not (or (tag :expensive-test) (
  command-line-1(("-L" ":." "-l" "ert" "-l" "lisp/shadowfile-tests" "-
  command-line()
  normal-top-level()
Test shadow-test09-shadow-copy-files condition:
    (ert-test-failed
     ((should
       (looking-at
    (regexp-quote "(setq shadow-files-to-copy nil)")))
      :form
      (looking-at "(setq shadow-files-to-copy nil)")
      :value nil))
   FAILED  10/10  shadow-test09-shadow-copy-files (0.756282 sec)

Ran 10 tests, 8 results as expected, 2 unexpected (2019-08-28
02:27:13+0200, 5.665292 sec)

2 unexpected results:
   FAILED  shadow-test08-shadow-todo
   FAILED  shadow-test09-shadow-copy-files

Best regards,
Stefan Kangas

In GNU Emacs 27.0.50 (build 10, x86_64-apple-darwin16.7.0, NS
appkit-1504.83 Version 10.12.6 (Build 16G2128))
 of 2019-08-28 built on Stefans-MBP
Repository revision: 4118297ae2fab4886b20d193ba511a229637aea3
Repository branch: master
Windowing system distributor 'Apple', version 10.3.1504
System Description:  Mac OS X 10.12.6

Configured features:
NOTIFY KQUEUE ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS NS THREADS
PDUMPER LCMS2 GMP

Important settings:
  value of $LC_CTYPE: UTF-8
  value of $LANG: en_SE <at> calendar=iso8601.UTF-8
  locale-coding-system: utf-8-unix




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Wed, 28 Aug 2019 08:19:02 GMT) Full text and rfc822 format available.

Message #8 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Wed, 28 Aug 2019 10:18:39 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

> I'm seeing two test failures in shadowfile-tests on macOS on current master.

> 2 unexpected results:
>    FAILED  shadow-test08-shadow-todo
>    FAILED  shadow-test09-shadow-copy-files

So you've hit me, again :-(

Will check later today what's up.

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Thu, 29 Aug 2019 13:53:02 GMT) Full text and rfc822 format available.

Message #11 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Thu, 29 Aug 2019 15:51:58 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Stefan,

>> I'm seeing two test failures in shadowfile-tests on macOS on current master.
>
>> 2 unexpected results:
>>    FAILED  shadow-test08-shadow-todo
>>    FAILED  shadow-test09-shadow-copy-files
>
> Will check later today what's up.

After adding some trace messages (intended for you, originally), the
error happens also to me. Looks like some race conditions. I still don't
understand what happens, but I'm able now to debug myself.

>> Best regards,
>> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Tue, 03 Sep 2019 12:13:01 GMT) Full text and rfc822 format available.

Message #14 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Tue, 03 Sep 2019 14:12:04 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

>> After adding some trace messages (intended for you, originally), the
>> error happens also to me. Looks like some race conditions. I still don't
>> understand what happens, but I'm able now to debug myself.
>
> Thanks.  As usual, let me know if you would like me to run any tests.

I've pushed some changes to master, which fix the problem at least for
me. Major change is to improve the cleanup of internal shadowfile
variables between the tests.

Could you, pls, check? In case it still fails, pls set shadow-debug to t
(in shadowfile-tests.el, line 67), and rerun the test.

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Tue, 03 Sep 2019 13:14:02 GMT) Full text and rfc822 format available.

Message #17 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Tue, 3 Sep 2019 15:13:07 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Michael,

> Could you, pls, check? In case it still fails, pls set shadow-debug to t
> (in shadowfile-tests.el, line 67), and rerun the test.

Thanks for looking into this.  Unfortunately, I'm still seeing
failures on latest master (commit ea5d591f29):

2 unexpected results:
   FAILED  shadow-test08-shadow-todo
   FAILED  shadow-test09-shadow-copy-files

I've attached the output of "make -C test shadowfile-tests" with
shadow-debug set to t.

Best regards,
Stefan Kangas
[shadowfile-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Tue, 03 Sep 2019 15:05:02 GMT) Full text and rfc822 format available.

Message #20 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Tue, 03 Sep 2019 17:04:19 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> Hi Michael,

Hi Stefan,

>> Could you, pls, check? In case it still fails, pls set shadow-debug to t
>> (in shadowfile-tests.el, line 67), and rerun the test.
>
> Thanks for looking into this.  Unfortunately, I'm still seeing
> failures on latest master (commit ea5d591f29):
>
> 2 unexpected results:
>    FAILED  shadow-test08-shadow-todo
>    FAILED  shadow-test09-shadow-copy-files
>
> I've attached the output of "make -C test shadowfile-tests" with
> shadow-debug set to t.

Thanks. It's a pity that the error doesn't happen anymore to me after
the recent changes. Will investigate further.

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Wed, 04 Sep 2019 12:18:02 GMT) Full text and rfc822 format available.

Message #23 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Wed, 04 Sep 2019 14:16:55 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> Hi Michael,

Hi Stefan,

> Thanks for looking into this.  Unfortunately, I'm still seeing
> failures on latest master (commit ea5d591f29):
>
> 2 unexpected results:
>    FAILED  shadow-test08-shadow-todo
>    FAILED  shadow-test09-shadow-copy-files
>
> I've attached the output of "make -C test shadowfile-tests" with
> shadow-debug set to t.

I've made some changes in shadowfile-tests.el in order to improve
robustness. Since I cannot reproduce locally, I don't know whether it
solves the problem. But it might be worth to check,

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Thu, 05 Sep 2019 15:09:02 GMT) Full text and rfc822 format available.

Message #26 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Thu, 5 Sep 2019 17:07:52 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> > 2 unexpected results:
> >    FAILED  shadow-test08-shadow-todo
> >    FAILED  shadow-test09-shadow-copy-files
> >
> > I've attached the output of "make -C test shadowfile-tests" with
> > shadow-debug set to t.
>
> I've made some changes in shadowfile-tests.el in order to improve
> robustness. Since I cannot reproduce locally, I don't know whether it
> solves the problem. But it might be worth to check,

Thanks.  I tried your changes, but I still see the same failures.
I've attached the log.

Best regards,
Stefan Kangas
[shadowfile-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sat, 07 Sep 2019 10:34:01 GMT) Full text and rfc822 format available.

Message #29 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sat, 07 Sep 2019 12:33:43 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> Thanks.  I tried your changes, but I still see the same failures.
> I've attached the log.

So I've added more traces, since it doesn't happen to me. Pls run.

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sat, 07 Sep 2019 12:46:02 GMT) Full text and rfc822 format available.

Message #32 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sat, 7 Sep 2019 14:45:19 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> So I've added more traces, since it doesn't happen to me. Pls run.

OK.  Log file attached.

Thanks,
Stefan Kangas
[shadowfile-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sun, 08 Sep 2019 08:22:02 GMT) Full text and rfc822 format available.

Message #35 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sun, 08 Sep 2019 10:21:01 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

>> So I've added more traces, since it doesn't happen to me. Pls run.
>
> OK.  Log file attached.

That helped, indeed.

>    passed   8/10  shadow-test07-regexp-groups (0.032929 sec)
> shadow-test08-shadow-todo: cluster1 /Stefans-MBP: \`/Stefans-MBP:\' ((shadow-cluster cluster1 /Stefans-MBP: \`/Stefans-MBP:\'))
> shadow-test08-shadow-todo: cluster2 /mock:Stefans-MBP: \`/mock:Stefans-MBP:\' ((shadow-cluster cluster2 /mock:Stefans-MBP: \`/mock:Stefans-MBP:\') (shadow-cluster cluster1 /Stefans-MBP: \`/Stefans-MBP:\'))
> shadow-test08-shadow-todo: /var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsmTKsYe ((/cluster1:/var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsmTKsYe /cluster2:/var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsmTKsYe))
> shadow-add-to-todo: nil nil

That's the first difference to my traces. Locally, I have here

shadow-add-to-todo: ((/tmp/shadowfile-tests0mPIFq . /cluster2:/tmp/shadowfile-tests0mPIFq)) nil

So you have a nil value of `shadows' in that trace. Looking at the code,
I see that `file-truename' is used to compute `shadows'. That rings a
bell - `temporary-file-directory' on macOS is located on a symlink. I
ran into this trap earlier already.

I hope, that my fix, pushed to master, fixes this now.

> Thanks,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sun, 08 Sep 2019 20:24:02 GMT) Full text and rfc822 format available.

Message #38 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sun, 8 Sep 2019 22:23:34 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> That's the first difference to my traces. Locally, I have here
>
> shadow-add-to-todo: ((/tmp/shadowfile-tests0mPIFq . /cluster2:/tmp/shadowfile-tests0mPIFq)) nil
>
> So you have a nil value of `shadows' in that trace. Looking at the code,
> I see that `file-truename' is used to compute `shadows'. That rings a
> bell - `temporary-file-directory' on macOS is located on a symlink. I
> ran into this trap earlier already.
>
> I hope, that my fix, pushed to master, fixes this now.

I'm still seing failures on current master on these tests, including
the line you noted above:

shadow-add-to-todo: nil nil

I've attached the complete log of the latest run (commit 034f9363e).

Thanks again,
Stefan Kangas
[shadowfile-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Fri, 13 Sep 2019 12:30:03 GMT) Full text and rfc822 format available.

Message #41 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Fri, 13 Sep 2019 14:29:10 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> I'm still seing failures on current master on these tests, including
> the line you noted above:

I still have no idea what's up. Added more traces, for test.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sat, 14 Sep 2019 16:41:01 GMT) Full text and rfc822 format available.

Message #44 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sat, 14 Sep 2019 18:39:56 +0200
[Message part 1 (text/plain, inline)]
Michael Albinus <michael.albinus <at> gmx.de> writes:

> I still have no idea what's up. Added more traces, for test.

Thanks, please find attached the log file.

Best regards,
Stefan Kangas
[shadowfile-tests.log (application/octet-stream, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sun, 15 Sep 2019 14:57:02 GMT) Full text and rfc822 format available.

Message #47 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sun, 15 Sep 2019 16:56:18 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

Hi Stefan,

> Thanks, please find attached the log file.

That helps.

> shadow-add-to-todo: /var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsUMHFDw /private/var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsUMHFDw

This tells us, that we must not only use the truename of
shadow-test-remote-temporary-file-directory, but also of
temporary-file-directory. I've pushed a fix to master, let's see how it runs.

> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Mon, 16 Sep 2019 02:09:01 GMT) Full text and rfc822 format available.

Message #50 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Mon, 16 Sep 2019 04:07:45 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

Hi Michael,

> > shadow-add-to-todo: /var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsUMHFDw /private/var/folders/pj/rhx0gxy15tv3vx6l3mdy0qvm0000gn/T/shadowfile-testsUMHFDw
>
> This tells us, that we must not only use the truename of
> shadow-test-remote-temporary-file-directory, but also of
> temporary-file-directory. I've pushed a fix to master, let's see how it runs.

That seems to have done the trick; I see no more failures.  I'll send
you the test log privately, but I think the relevant line is this one:

Ran 10 tests, 10 results as expected, 0 unexpected (2019-09-15
22:02:48+0200, 8.197759 sec)

Congratulations!  And thanks for investigating and fixing it.

Best regards,
Stefan Kangas




Reply sent to Michael Albinus <michael.albinus <at> gmx.de>:
You have taken responsibility. (Mon, 16 Sep 2019 13:15:01 GMT) Full text and rfc822 format available.

Notification sent to Stefan Kangas <stefan <at> marxist.se>:
bug acknowledged by developer. (Mon, 16 Sep 2019 13:15:01 GMT) Full text and rfc822 format available.

Message #55 received at 37202-done <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 37202-done <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Mon, 16 Sep 2019 15:13:51 +0200
Version: 27.1

Stefan Kangas <stefan <at> marxist.se> writes:

> Hi Michael,

Hi Stefan,

> That seems to have done the trick; I see no more failures.  I'll send
> you the test log privately, but I think the relevant line is this one:
>
> Ran 10 tests, 10 results as expected, 0 unexpected (2019-09-15
> 22:02:48+0200, 8.197759 sec)

Thanks for your patient tests! I've disabled the traces in
shadowfile-tests.el. Closing the bug.

> Congratulations!  And thanks for investigating and fixing it.
>
> Best regards,
> Stefan Kangas

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Mon, 16 Sep 2019 19:53:02 GMT) Full text and rfc822 format available.

Message #58 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Stefan Kangas <stefan <at> marxist.se>, 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Mon, 16 Sep 2019 15:52:23 -0400
Michael Albinus wrote:

> This tells us, that we must not only use the truename of
> shadow-test-remote-temporary-file-directory, but also of
> temporary-file-directory. I've pushed a fix to master, let's see how it runs.

This change makes it fail every time on hydra.nixos.org :)
https://hydra.nixos.org/build/101137999




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Thu, 19 Sep 2019 11:11:01 GMT) Full text and rfc822 format available.

Message #61 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, Stefan Kangas <stefan <at> marxist.se>,
 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Thu, 19 Sep 2019 13:09:58 +0200
Glenn Morris <rgm <at> gnu.org> writes:

Hi Glenn,

>> This tells us, that we must not only use the truename of
>> shadow-test-remote-temporary-file-directory, but also of
>> temporary-file-directory. I've pushed a fix to master, let's see how it runs.
>
> This change makes it fail every time on hydra.nixos.org :)
> https://hydra.nixos.org/build/101137999

Analysing the backtrace, it doesn't look like "this change" is
responsible. It seems rather that this is the first sign that Paul's
recent changes have damaged Tramp. I mean, that file-attributes returns
an error now instead of nil, if a file is not accessible.

My analysis could be wrong, but the question still stands: why do we
need this change in file-attributes and friends?

(And I propose to start with a new bug. It is not related to
shadowfile-tests at all).

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Fri, 20 Sep 2019 00:20:02 GMT) Full text and rfc822 format available.

Message #64 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>, Glenn Morris <rgm <at> gnu.org>
Cc: Stefan Kangas <stefan <at> marxist.se>, 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Thu, 19 Sep 2019 17:19:35 -0700
On 9/19/19 4:09 AM, Michael Albinus wrote:

>> https://hydra.nixos.org/build/101137999
> 
> ... It seems rather that this is the first sign that Paul's
> recent changes have damaged Tramp. I mean, that file-attributes returns
> an error now instead of nil, if a file is not accessible.

Could you explain how the backtrace implicates file-attributes? I see 
the following in the backtrace, but I don't see file-attributes listed 
there, and when I look at the tramp-handle-file-modes source it appears 
that file-attributes returned nil which is what I think you expected.

Test shadow-test08-shadow-todo backtrace:
  signal(args-out-of-range ("([] 1)"))
  tramp-signal-hook-function(args-out-of-range ([] 1))
  tramp-mode-string-to-int(nil)

tramp-handle-file-modes(#("/mock:localhost:/build/shadowfile-tests75GTBI" 
6 15 (tramp-default t)))
  apply(tramp-handle-file-modes 
#("/mock:localhost:/build/shadowfile-tests75GTBI" 6 15 (tramp-default t)))
  tramp-sh-file-name-handler(file-modes 
#("/mock:localhost:/build/shadowfile-tests75GTBI" 6 15 (tramp-default t)))
  apply(tramp-sh-file-name-handler file-modes 
#("/mock:localhost:/build/shadowfile-tests75GTBI" 6 15 (tramp-default t)))
  tramp-file-name-handler(file-modes 
#("/mock:localhost:/build/shadowfile-tests75GTBI" 6 15 (tramp-default t)))
  file-modes(#("/mock:localhost:/build/shadowfile-tests75GTBI" 6 15 
(tramp-default t)))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Fri, 20 Sep 2019 12:37:02 GMT) Full text and rfc822 format available.

Message #67 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Glenn Morris <rgm <at> gnu.org>, Stefan Kangas <stefan <at> marxist.se>,
 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Fri, 20 Sep 2019 14:35:52 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> Could you explain how the backtrace implicates file-attributes? I see
> the following in the backtrace, but I don't see file-attributes listed
> there, and when I look at the tramp-handle-file-modes source it
> appears that file-attributes returned nil which is what I think you
> expected.
>
> Test shadow-test08-shadow-todo backtrace:
>   signal(args-out-of-range ("([] 1)"))
>   tramp-signal-hook-function(args-out-of-range ([] 1))
>   tramp-mode-string-to-int(nil)
>
> tramp-handle-file-modes(#("/mock:localhost:/build/shadowfile-tests75GTBI"
> 6 15 (tramp-default t)))

Right, `file-attributes' does not seem to have raised an error. But it
still seems to behave differently. The code is

--8<---------------cut here---------------start------------->8---
(defun tramp-handle-file-modes (filename)
  "Like `file-modes' for Tramp files."
  (let ((truename (or (file-truename filename) filename)))
    (when (file-exists-p truename)
      (tramp-mode-string-to-int
       (tramp-compat-file-attribute-modes (file-attributes truename))))))
--8<---------------cut here---------------end--------------->8---

That means, (file-attributes truename) returns nil although
(file-exists-p truename) returns non-nil. That seems to be a changed
behaviour to me, I don't remember having seen this before.

Arguably, the test shall better use `access-file' but
`file-exists-p'. But anyway, such subtle differences could break Tramp.

I still use conditional wording, because I cannot debug on hydra. And I
cannot say for sure that your changes have caused the problem.

For the time being, I'll adapt `tramp-handle-file-modes' in order to
unbreak the errors on hydra.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Fri, 20 Sep 2019 19:05:02 GMT) Full text and rfc822 format available.

Message #70 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Albinus <michael.albinus <at> gmx.de>
Cc: Glenn Morris <rgm <at> gnu.org>, Stefan Kangas <stefan <at> marxist.se>,
 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Fri, 20 Sep 2019 12:04:41 -0700
On 9/20/19 5:35 AM, Michael Albinus wrote:
> That means, (file-attributes truename) returns nil although
> (file-exists-p truename) returns non-nil.

I don't see why the recent changes would cause file-attributes to return nil on 
a local file. If file-exists-p succeeds, the file has been confirmed to exist. 
In that case, when file-attributes uses fstatat (or openat+stat) the syscalls 
should either succeed, or should fail with an unusual errno value like 
EOVERFLOW, so file-attributes should either return non-nil or signal an error.

Perhaps the file in question has file handlers that are interposing themselves? 
That might explain the situation.

I should mention that under Emacs 26 file-attributes can return nil here, even 
for a local file: e.g., (file-exists-p F) uses faccessat which succeeds, then 
(file-attributes F) uses fstatat (or openat+fstat) which fails with 
errno==EOVERFLOW because F's timestamp or inode number is out of range for the 
kernel.

Also, of course using two calls suffers from a race if the filesystem changes 
between the calls.

So, regardless of the the recent access changes, it appears that this code in 
tramp-handle-file-modes:

    (when (file-exists-p truename)
      (tramp-mode-string-to-int
       (tramp-compat-file-attribute-modes (file-attributes truename))))

should be rewritten to be something like this:

    (when-let ((attrs (file-attributes truename)))
      (tramp-mode-string-to-int
       (tramp-compat-file-attribute-modes attrs)))




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Fri, 20 Sep 2019 19:14:02 GMT) Full text and rfc822 format available.

Message #73 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Glenn Morris <rgm <at> gnu.org>, Stefan Kangas <stefan <at> marxist.se>,
 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Fri, 20 Sep 2019 21:13:25 +0200
Paul Eggert <eggert <at> cs.ucla.edu> writes:

Hi Paul,

> So, regardless of the the recent access changes, it appears that this
> code in tramp-handle-file-modes:
>
>     (when (file-exists-p truename)
>       (tramp-mode-string-to-int
>        (tramp-compat-file-attribute-modes (file-attributes truename))))
>
> should be rewritten to be something like this:
>
>     (when-let ((attrs (file-attributes truename)))
>       (tramp-mode-string-to-int
>        (tramp-compat-file-attribute-modes attrs)))

Something like this, yes. Except, that when-let does not exist in Emacs 24.4.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Sat, 21 Sep 2019 07:56:02 GMT) Full text and rfc822 format available.

Message #76 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Stefan Kangas <stefan <at> marxist.se>, 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Sat, 21 Sep 2019 09:55:18 +0200
Glenn Morris <rgm <at> gnu.org> writes:

> This change makes it fail every time on hydra.nixos.org :)
> https://hydra.nixos.org/build/101137999

Should be fixed now, hopefully.

Best regards, Michael.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#37202; Package emacs. (Thu, 03 Oct 2019 13:51:01 GMT) Full text and rfc822 format available.

Message #79 received at 37202 <at> debbugs.gnu.org (full text, mbox):

From: Michael Albinus <michael.albinus <at> gmx.de>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Stefan Kangas <stefan <at> marxist.se>, 37202 <at> debbugs.gnu.org
Subject: Re: bug#37202: shadowfile-tests: two failures on macOS
Date: Thu, 03 Oct 2019 15:50:23 +0200
Michael Albinus <michael.albinus <at> gmx.de> writes:

> Glenn Morris <rgm <at> gnu.org> writes:
>
>> This change makes it fail every time on hydra.nixos.org :)
>> https://hydra.nixos.org/build/101137999
>
> Should be fixed now, hopefully.

After some further iterations, hydra doesn't report an error anymore.

Best regards, Michael.




bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Fri, 01 Nov 2019 11:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 4 years and 149 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.