GNU bug report logs - #17330
files.el cd-absolute overcome false negative from file-executable-p

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 17330 in the body.
You can then email your comments to 17330 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: bug-gnu-emacs <at> gnu.org
Subject: files.el cd-absolute overcome false negative from file-executable-p
Date: Wed, 23 Apr 2014 22:54:18 +0200

[using emacs-w32.exe 24.3 in cygwin but also applies to other platforms.]

Symptom:
When I try to cd to my samba-mounted directory,
or try to run ediff-files, it refuses with the error
"Cannot cd to my_directory_name: Permission denied"
The same directory opens fine in dired-mode, and I can open files within it.

A workaround is to make my directory and its parents executable for "others",
but of course it would be much better to keep the correct mode in force.
Maybe a more recent or better configured samba on the server would help.

This is just one way in which file-executable-p can produce a false negative,
where executing the file or searching the directory may succeed after all.

[False positives, where the operation refuses, have no impact so long as
the operation fails quickly with an equivalent but authoritative error message.]

This is typical of many situations where it is not good enough just
to check quickly whether it just looks like it can or cannot be done;
you have to also actually try it, and report the outcome of
that particular attempt at that particular moment.
A preliminary check may still be useful too if it produces a better error
message quickly up front with no dashed expectations or cleaning up to do.

The comments in fileio.c check_executable (and check_writeable) point out
that on some filesystems there might be an access control list (ACL) in force,
and that the effective user might have more permission than the real user.

Suggested fix:
In files.el the function cd-absolute can ask for confirmation in case
check-executable-p might be producing a false negative:

;; in emacs-24.3/lisp/files.el [cd-absolute] replace the expression
    (unless (file-executable-p dir)
      (error "Cannot cd to %s:  Permission denied" dir))
;; with
    (unless (file-executable-p dir)
      (unless (yes-or-no-p
			   (format
				"Directory does not look searchable; try to cd to %s anyway? "
				dir))
		(error "Cannot cd to %s:  Permission denied" dir)))

I encountered the cd failure originally on invoking ediff-files to merge
source code from a samba mount, then noticed cd itself is affected too.
I haven't run into any other false negative cases.
The 24.3 distribution contains about 50 calls to
file-executable-p where a fix like this might potentially be useful,
not to mention custom code and community packages, and other similar functions.

In directory search contexts file-searchable-p might be a better name.

Message #8 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Fri, 02 May 2014 20:24:04 -0400

Philip Hodges wrote:

> [using emacs-w32.exe 24.3 in cygwin but also applies to other platforms.]
>
> Symptom:
> When I try to cd to my samba-mounted directory,
> or try to run ediff-files, it refuses with the error
> "Cannot cd to my_directory_name: Permission denied"
> The same directory opens fine in dired-mode, and I can open files within it.

> This is just one way in which file-executable-p can produce a false negative,
> where executing the file or searching the directory may succeed after all.
>
> [False positives, where the operation refuses, have no impact so long as
> the operation fails quickly with an equivalent but authoritative error message.]
>
> This is typical of many situations where it is not good enough just
> to check quickly whether it just looks like it can or cannot be done;
> you have to also actually try it, and report the outcome of
> that particular attempt at that particular moment.
> A preliminary check may still be useful too if it produces a better error
> message quickly up front with no dashed expectations or cleaning up to do.
>
> The comments in fileio.c check_executable (and check_writeable) point out
> that on some filesystems there might be an access control list (ACL) in force,
> and that the effective user might have more permission than the real user.


Perhaps there is something Cygwin-specific at work, because I don't
think I can reproduce such a problem on eg RHEL 6.5 GNU/Linux.

As user A:
mkdir foo1 foo2
chmod 700 foo1 foo2
setfacl -m u:b:r-x foo1

As user B:
In the shell: [ -x foo1 ]      # -> true

In Emacs 24.3: file-executable-p foo1   ; -> true


(There's also file-accessible-directory-p; does that work any better for
you?).

Message #11 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sat, 03 May 2014 11:17:43 +0200

On 2014-05-03 02:24, Glenn Morris wrote:
> Philip Hodges wrote:
>
>> [using emacs-w32.exe 24.3 in cygwin but also applies to other platforms.]
>>
>> Symptom:
>> When I try to cd to my samba-mounted directory,
>> or try to run ediff-files, it refuses with the error
>> "Cannot cd to my_directory_name: Permission denied"
>> The same directory opens fine in dired-mode, and I can open files within it.
>
>> This is just one way in which file-executable-p can produce a false negative,
>> where executing the file or searching the directory may succeed after all.
>>
>> [False positives, where the operation refuses, have no impact so long as
>> the operation fails quickly with an equivalent but authoritative error message.]
>>
>> This is typical of many situations where it is not good enough just
>> to check quickly whether it just looks like it can or cannot be done;
>> you have to also actually try it, and report the outcome of
>> that particular attempt at that particular moment.
>> A preliminary check may still be useful too if it produces a better error
>> message quickly up front with no dashed expectations or cleaning up to do.
>>
>> The comments in fileio.c check_executable (and check_writeable) point out
>> that on some filesystems there might be an access control list (ACL) in force,
>> and that the effective user might have more permission than the real user.
>
>
> Perhaps there is something Cygwin-specific at work, because I don't
> think I can reproduce such a problem on eg RHEL 6.5 GNU/Linux.
>
> As user A:
> mkdir foo1 foo2
> chmod 700 foo1 foo2
> setfacl -m u:b:r-x foo1
>
> As user B:
> In the shell: [ -x foo1 ]      # -> true
>
> In Emacs 24.3: file-executable-p foo1   ; -> true
>
>
> (There's also file-accessible-directory-p; does that work any better for
> you?).
>

Your example platform likely has euidaccess and so takes the effective 
user id and ACL into account. Thanks for checking. I'm glad it works.

Otherwise fileio.c:check_executable calls access which ignores any 
effective id, or looks at bits returned by stat.

Calling something like GetFileAttributes might give a truer picture, but 
apparently there are several different ways the id(s) a user has can be 
permitted to access files in a directory, so interpreting the results 
might not be easy. Would simply opening the directory, or starting a 
directory entries listing, be more reliable, or too slow?

In the end what counts is whether listing the directory at a particular 
moment actually works. To concoct a hopefully absurd example, the 
directory might be on a filesystem that has been cracked to grant 
directory listing access to guest users during NSA overnight batch job 
hours. There is no way a security audit system calling check_executable 
on an arbitrary client system can possibly discover that.

Samba can be configured to allow or deny various kinds of access to 
various users, and to map the permission bits in artificial ways. The 
host access and client access permissions actually applied can differ 
wildly from what stat returns. I dare say the configuration of my samba 
share can be improved, but I think there will still be valid use cases 
as well as misconfigurations where false negatives cannot be completely 
ruled out.

I'm happy that ediff-directories works for my samba folder when I tell 
it to let me override the outcome of check_executable and cd to it 
anyway. If I do cd to a directory that is genuinely not searchable, 
which I think would be something I could always avoid, the question 
actually helps me think about how I got there and whether it really is 
correct that I don't seem to have access.

It's not the whole story. There are still circumstances where I get a 
slightly different error message about no permission to set current 
directory, after callproc.c:call-process or proc.c:start-process calls 
file-accessible-directory-p. It is C code calling C code, I don't 
suppose it can be intercepted in Lisp?

A process does not necessarily even always have or need a current 
directory (for example MacOS before MacOSX). It might not even use 
relative filenames that need prefixing at all. Why does a directory even 
have to exist to use it to expand a relative pathname? Is Macavity's 
parent ever there? (That cat is free as in spirit, not as in beer). Do 
all GNU/Linux processes inevitably open the current directory for read? 
Does Windows open it and keep a handle, does it fail if it cannot? I 
will have to experiment and find out.

The function file-accessible-directory-p is more appropriately named for 
this context but it is not better. It just checks for a directory first 
before calling file-executable-p and hence check_executable.

Message #14 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Achim Gratz <Stromeko <at> nexgo.de>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sat, 03 May 2014 11:35:01 +0200

Philip Hodges writes:
> Your example platform likely has euidaccess and so takes the effective
> user id and ACL into account. Thanks for checking. I'm glad it works.
>
> Otherwise fileio.c:check_executable calls access which ignores any
> effective id, or looks at bits returned by stat.

Cygwin also uses euidaccess (or should, anyway):
http://cygwin.com/ml/cygwin/2011-12/msg00364.html

But yes, you can totally have full access to a directory or file on a
Windows box while all permission flags show that you don't (I guess that
is also possible on UN*X, although I've never tried).  You'll never know
until you actually try.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

Factory and User Sound Singles for Waldorf Q+, Q and microQ:
http://Synth.Stromeko.net/Downloads.html#WaldorfSounds

Message #17 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Achim Gratz <Stromeko <at> nexgo.de>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Sat, 03 May 2014 16:26:28 +0300

> From: Achim Gratz <Stromeko <at> nexgo.de>
> Date: Sat, 03 May 2014 11:35:01 +0200
> 
> But yes, you can totally have full access to a directory or file on a
> Windows box while all permission flags show that you don't (I guess that
> is also possible on UN*X, although I've never tried).  You'll never know
> until you actually try.

That's true (on Windows; I don't think it's possible on Unix), but why
is that an Emacs problem?  I think this problem should be communicated
to the Cygwin maintainers: an accessible directory should return
success from the faccessat call, because otherwise Posix-originated
programs such as Emacs will misbehave.

If the Cygwin maintainers will decide that this specific situation
(whose particulars as far as Windows ACL data of the directory in
question was never shown by the OP, by the way), then this would mean
it's either a cockpit error (i.e. the user shoot himself in the foot
by creating a security descriptor he shouldn't have), or that Cygwin
does not intend to support such situations in the first place.  Either
way, Emacs is not to blame here.

Message #20 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Sat, 03 May 2014 16:40:40 +0300

> Date: Sat, 03 May 2014 11:17:43 +0200
> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> Cc: 17330 <at> debbugs.gnu.org
> 
> Your example platform likely has euidaccess and so takes the effective 
> user id and ACL into account. Thanks for checking. I'm glad it works.
> 
> Otherwise fileio.c:check_executable calls access which ignores any 
> effective id, or looks at bits returned by stat.

As Achim correctly points out, this doesn't matter, because on Windows
a directory can be accessible to you by virtue of your belonging to
certain user groups (like Power Users etc.), even though all the ACEs
in that directory's security descriptor deny you any access.

> Calling something like GetFileAttributes might give a truer picture

No, it won't: GetFileAttributes does not return any access rights,
only the file's attributes.  IOW, it could tell that the file is a
directory, but not if the directory is accessible to this or that
user.

> Would simply opening the directory, or starting a directory entries
> listing, be more reliable, or too slow?

If you try cd'ing, you might be unable to distinguish between an
existing file that is not a directory and an inaccessible directory.
Not sure if that is important here, I didn't analyze the users of
cd-absolute.  (Opening a directory is non-portable, and furthemore
will not necessarily tell you that it can be listed -- these are
different privileges on Windows.)

> Samba can be configured to allow or deny various kinds of access to 
> various users, and to map the permission bits in artificial ways. The 
> host access and client access permissions actually applied can differ 
> wildly from what stat returns. I dare say the configuration of my samba 
> share can be improved, but I think there will still be valid use cases 
> as well as misconfigurations where false negatives cannot be completely 
> ruled out.

Why not ask Cygwin maintainers to cover these situations in their
euidaccess and/or faccessat implementations?  Why should Emacs solve
this for you, when it works on any other Posix platforms (and on some
non-Posix ones)?  Unlike the native Windows port of Emacs, the Cygwin
build relies on a free library whose sources are freely available and
can be fixed if a bug there is found and reported.  So that's what I
suggest to do -- report this to the Cygwin maintainers, and ask them
to fix this.

> It's not the whole story. There are still circumstances where I get a 
> slightly different error message about no permission to set current 
> directory, after callproc.c:call-process or proc.c:start-process calls 
> file-accessible-directory-p. It is C code calling C code, I don't 
> suppose it can be intercepted in Lisp?

You will have endless such problems if accessible directories don't
pass the test by euidaccess or access.  This _must_ be fixed in the
Cygwin library, or by your changing the permissions of those
directories so that they are reported as executables.

> A process does not necessarily even always have or need a current 
> directory (for example MacOS before MacOSX).

It does in Emacs, because Emacs frequently (if not always) invokes
programs in the context of an Emacs buffer, which conceptually (from
the user POV) means the program should run in that buffer's directory.

> Why does a directory even have to exist to use it to expand a
> relative pathname?

It doesn't:

  (expand-file-name "../doesntexist/anotherfake/foobar")
    => "d:/gnu/bzr/emacs/doesntexist/anotherfake/foobar"

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Achim Gratz <Stromeko <at> nexgo.de>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Sat, 03 May 2014 15:58:05 +0200

Eli Zaretskii writes:
> If the Cygwin maintainers will decide that this specific situation
> (whose particulars as far as Windows ACL data of the directory in
> question was never shown by the OP, by the way), then this would mean
> it's either a cockpit error (i.e. the user shoot himself in the foot
> by creating a security descriptor he shouldn't have), or that Cygwin
> does not intend to support such situations in the first place.

Cygwin already has the "noacl" mount option which fakes the uid/gid and
permissions for that file system so that they always look sane from the
POSIX perspective and then you'll find out when you access the file
whether you have access.  While it's generally a sign of some
problematic configuration someplace else and introducing some other
problems when you have to use it, the OP could maybe tell us if that
makes a difference to the reported problem.


Regards,
Achim.
-- 
+<[Q+ Matrix-12 WAVE#46+305 Neuron microQkb Andromeda XTk Blofeld]>+

SD adaptations for KORG EX-800 and Poly-800MkII V0.9:
http://Synth.Stromeko.net/Downloads.html#KorgSDada

Message #26 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Achim Gratz <Stromeko <at> nexgo.de>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false
 negative	from	file-executable-p
Date: Sat, 03 May 2014 19:37:18 +0300

> From: Achim Gratz <Stromeko <at> nexgo.de>
> Date: Sat, 03 May 2014 15:58:05 +0200
> 
> Eli Zaretskii writes:
> > If the Cygwin maintainers will decide that this specific situation
> > (whose particulars as far as Windows ACL data of the directory in
> > question was never shown by the OP, by the way), then this would mean
> > it's either a cockpit error (i.e. the user shoot himself in the foot
> > by creating a security descriptor he shouldn't have), or that Cygwin
> > does not intend to support such situations in the first place.
> 
> Cygwin already has the "noacl" mount option which fakes the uid/gid and
> permissions for that file system so that they always look sane from the
> POSIX perspective and then you'll find out when you access the file
> whether you have access.

Yes, I know.  But it's not clear to me at this point that the only
reasonable way out of this is by using "noacl".

> While it's generally a sign of some problematic configuration
> someplace else and introducing some other problems when you have to
> use it, the OP could maybe tell us if that makes a difference to the
> reported problem.

Indeed, having more details about the ACLs of the offending
directories would be good.

Message #29 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: 17330 <at> debbugs.gnu.org
Subject: thanks for the comments
Date: Sun, 4 May 2014 00:43:25 +0200

thanks for all the discussion contributions.

I'm really happy about having my own personal same day lisp fix working for me to stop cd-absolute believing the false negative from the check_executable function. My best hope for the remaining subprocess cwd cases is to look at the samba configuration and see if I can get it improved, and maybe add a user to the cygwin passwd file.

Anything involving a rebuild and new release will take months before it is available for me to use where I need it.

For completeness, and in the spirit of DRY, whatever uses start-process and call-process *could* be refactored to share the same code to offer the same override as file-executable-p in the same situation. And while we are about it, can we please not ask if a file is executable when we really just want to know if it is a searchable directory suitable for cd.

I don't see how we can ever completely rule out false negatives, unless we are prepared to change check_executable to actually try to use the directory (cd to it, open it, list its entries, return t, whatever). Even the euidaccess man page warns against using it: "Generally, it is safer just to attempt the desired operation and handle any permission error that occurs". But I do accept that there may well be no consensus to follow through with more reliable or less gullible code. If it is just a few legacy platforms that lack euidaccess and fall back to checking the wrong uid with access, then never mind. If we can prove that the native and cygwin builds behave differently, then the offending library function can probably be fixed long before emacs can work around it. Thank you for pushing that suggestion.

My samba share is also afflicted with false negative writeable checks, as described here two years ago:
http://debbugs.gnu.org/cgi/bugreport.cgi?bug=10257

I connect to the samba filesystem directly in Windows as a user without local administrator rights. I'll let you know if mounting it from cygwin with special acl options confers more appropriate access permissions or even works at all.

Message #32 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 04 May 2014 00:16:17 -0400

I'm finding this rather abstract.
I think a reproducible example of the same kind of issue on a Unix
platform would be helpful.

Message #35 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Philip Hodges <philip.hodges <at> bluewin.ch>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 04 May 2014 09:01:12 -0400

> I'm finding this rather abstract.  I think a reproducible example of
> the same kind of issue on a Unix platform would be helpful.

It's probably somewhere between hard and impossible to reproduce on
a Unix platform.

The problem is that cd-absolute wants to signal an error if the
directory specified can't be used, whereas within Emacs any string can
be used for the "current directory" (it's just that some choices lead
to errors later on, such as when trying to get to a relative-named file
or when trying to spawn a process).

AFAIK, the only moment where Emacs cares whether the OS can use
something as a current directory (i.e. when it does "chdir") is when it
spawn a process via call-process or start-process.

So, we could solve this problem either by dropping this error, or by
using call-process instead of file-executable-p.  Yet another way would
be to provide a new subroutine that gives access to chdir somehow
(e.g. "file-chdirable-p").

I think the simplest solution for now is to turn the error into a warning.


        Stefan

Message #38 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: 17330 <at> debbugs.gnu.org
Subject: files.el cd-absolute overcome false negative from file-executable-p
Date: Sun, 4 May 2014 15:24:35 +0200

I came across a post that confirms my suspicions that:
- the _stat st_mode bits really cannot be trusted to give a meaningful answer, no matter whether it comes from Windows or cygwin.
- it is not worth calling GetFileSecurity (which is what I meant by "something like GetFileAttributes" *) and trying to reproduce the interpretation of the ACL information
- the best way is to find out if searching a directory will succeed really is to actually try it

http://stackoverflow.com/questions/3449465/find-the-permissions-of-a-file-in-windows

> ... somewhere between hard and impossible to reproduce ...
To reproduce on any platform, simply change check_executable to *always* return false. The easiest way to test what happens with false negatives is to have negatives all the time.

The more I read about it and think about it, the more I stand by my original change suggestion: tell the user that the directory might not be searchable, ask them if they want to go ahead and try it anyway. And also ask if they always want to do that, in case the question appears so often that it becomes annoying. I think we have to accept and work with what we have got instead of punishing users until such time as every possible combination of filesystem and library methods can be upgraded in a way that might not even be practical for many of them.

Surely the worst that can happen is that it becomes easier to end up with a buffer whose current directory really is not searchable, or file changes that cannot be saved in the original location. Good to avoid, but not vital. Rather that than being completely denied the use of a feature such as ediff-directories that would actually work.

Much of what was written for #10257 (writeable) applies to this #17330 (searchable directory) and vice versa. So if we are only comfortable with just treating a -1 uid or gid specially here too, well it is not a general solution, but better than no change at all. Having just the default entries in the passwd file hasn't been causing me any other noticeable trouble.

[*) If I have mixed up any other detail such as which conditional compiling macros are defined to choose between euidaccess and stat or _stat in which builds, or misreported something observed on a system that is not in front of me at that moment, please forgive me.]

Message #41 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: rgm <at> gnu.org, philip.hodges <at> bluewin.ch, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Sun, 04 May 2014 19:18:35 +0300

> From: Stefan Monnier <monnier <at> iro.umontreal.ca>
> Date: Sun, 04 May 2014 09:01:12 -0400
> Cc: Philip Hodges <philip.hodges <at> bluewin.ch>, 17330 <at> debbugs.gnu.org
> 
> So, we could solve this problem either by dropping this error, or by
> using call-process instead of file-executable-p.

Which program would you invoke for this?

Message #44 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Sun, 04 May 2014 19:24:39 +0300

> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> Date: Sun, 4 May 2014 15:24:35 +0200
> 
> I came across a post that confirms my suspicions that:
> - the _stat st_mode bits really cannot be trusted to give a meaningful answer, no matter whether it comes from Windows or cygwin.
> - it is not worth calling GetFileSecurity (which is what I meant by "something like GetFileAttributes" *) and trying to reproduce the interpretation of the ACL information
> - the best way is to find out if searching a directory will succeed really is to actually try it
> 
> http://stackoverflow.com/questions/3449465/find-the-permissions-of-a-file-in-windows

You still didn't provide any details about your particular situation
which triggers this problem.  Would you _please_ do that?

If is very hard to conduct a meaningful discussion without a clear
understanding of the problem.  Maybe you are right in your analysis,
but without actually seeing the ACLs of the related directories, we
cannot be sure that your analysis is correct and complete.  Given the
details, we could reason about the problem and maybe consult with some
experts if the knowledge we have here is insufficient.  That would
make this discussion much more productive, and a satisfactory solution
will probably be found quickly enough.

Thanks in advance.

Message #47 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Philip Hodges <philip.hodges <at> bluewin.ch>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 04 May 2014 14:07:50 -0400

Stefan Monnier wrote:

> So, we could solve this problem either by dropping this error, or by
> using call-process instead of file-executable-p.  Yet another way would
> be to provide a new subroutine that gives access to chdir somehow
> (e.g. "file-chdirable-p").

I'd still like to hear if file-accessible-directory-p does any better
than file-executable-p. At least on !DOS_NT, it seems to be something a
bit different.

> I think the simplest solution for now is to turn the error into a warning.

Sounds good, but might be annoying on Unix platforms; if as you say
this situation is impossible there an error might be clearer for them.

Message #50 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Philip Hodges <philip.hodges <at> bluewin.ch>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 04 May 2014 18:44:46 -0400

> I'd still like to hear if file-accessible-directory-p does any better
> than file-executable-p. At least on !DOS_NT, it seems to be something a
> bit different.

Indeed file-accessible-directory-p would probably be better than
file-executable-p.  Whether it's sufficiently better to keep the error,
I don't know.

>> I think the simplest solution for now is to turn the error into a warning.
> Sounds good, but might be annoying on Unix platforms; if as you say
> this situation is impossible there an error might be clearer for them.

Agreed.


        Stefan

Message #53 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Tue, 6 May 2014 00:43:27 +0200

After discovering that even C functions can be redefined, today I "activated an advice" so that all file-executable-p C code calls from Lisp return t.
No unexpected refusals, no noticeable downsides, no waiting months for C code changes to appear in a new official release. The solution is satisfactory in practice. So far as I am concerned, the case can be closed. Can we at least document the unreliability first though?

From the cygwin FAQ: "When working out the Unix-style attribute bits on a file, the library has to fill out some information not provided by the WIN32 API. It *guesses* ..."

I understand your being curious as to exactly why cygwin cannot guess correctly for this samba mount without going to an awful lot of trouble. But we do already have several statements confirming that it is not usual or practical to even try. These make sense to me. They explain and confirm what I am seeing. The analysis does not need to be complete. It just takes one reproducible false negative in a realistic scenario that is not going to go away anytime soon. At least one of the platform library functions called by file-executable-p sometimes cannot be trusted. That's enough for me. Let's stop trusting it at all.

We could soften all these file permission functions to ask the user if they want to try anyway. Then the next person won't have to figure out the checks are unreliable and how to override them. I'll try running with that for the next few days.

Message #56 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Tue, 06 May 2014 00:15:28 -0400

I would *still* like to hear if file-accessible-directory-p does any
better than file-executable-p. The result is only interesting in an
Emacs later than 24.3. Eg 24.3.90 pretest, or current trunk or emacs-24
branch. (Since we have no recipe to reproduce this problem, no-one else
can say.)

Message #59 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Tue, 06 May 2014 10:17:49 +0300

> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> Date: Tue, 6 May 2014 00:43:27 +0200
> Cc: 17330 <at> debbugs.gnu.org
> 
> After discovering that even C functions can be redefined, today I "activated an advice" so that all file-executable-p C code calls from Lisp return t.
> No unexpected refusals, no noticeable downsides, no waiting months for C code changes to appear in a new official release. The solution is satisfactory in practice. So far as I am concerned, the case can be closed. Can we at least document the unreliability first though?

I don't know what to document, since you never disclosed the details.
Good documentation should not include FUD, it should include details
that are understandable and actionable by users.  Writing such
documentation requires a good understanding of your situation,
something we don't have in this case.

> From the cygwin FAQ: "When working out the Unix-style attribute bits on a file, the library has to fill out some information not provided by the WIN32 API. It *guesses* ..."

That's not directly related to the case in point, AFAIK (and yes, I do
know what Cygwin does to emulate Posix permissions using Windows
ACLs).

> I understand your being curious as to exactly why cygwin cannot guess correctly for this samba mount without going to an awful lot of trouble. But we do already have several statements confirming that it is not usual or practical to even try. These make sense to me. They explain and confirm what I am seeing. The analysis does not need to be complete. It just takes one reproducible false negative in a realistic scenario that is not going to go away anytime soon. At least one of the platform library functions called by file-executable-p sometimes cannot be trusted. That's enough for me. Let's stop trusting it at all.

It is a pity that you don't tell the details.  The result will be that
the problem is perhaps solved for you (using a recipe that many Emacs
users will not know how to reproduce), but not for others.  I urge you
to reconsider.  After all, the amount of words we've wasted here is
surely large enough to describe your issue to the required depth.

Message #62 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Tue, 06 May 2014 08:46:27 -0400

> After discovering that even C functions can be redefined, today I "activated
> an advice" so that all file-executable-p C code calls from Lisp return t.
> No unexpected refusals, no noticeable downsides, no waiting months for
> C code changes to appear in a new official release.

The changes being discussed are to `cd-absolute' which is an Elisp
function, so no need for recompilation.

E.g. if you add

   (defun cd-absolute (dir)
     "Change current directory to given absolute file name DIR."
     ;; Put the name into directory syntax now,
     ;; because otherwise expand-file-name may give some bad results.
     (setq dir (file-name-as-directory dir))
     ;; We used to additionally call abbreviate-file-name here, for an
     ;; unknown reason.  Problem is that most buffers are setup
     ;; without going through cd-absolute and don't call
     ;; abbreviate-file-name on their default-directory, so the few that
     ;; do end up using a superficially different directory.
     (setq dir (expand-file-name dir))
     (if (not (file-directory-p dir))
         (if (file-exists-p dir)
   	  (error "%s is not a directory" dir)
   	(error "%s: no such directory" dir))
       (unless (file-accessible-directory-p dir)
         (error "Cannot cd to %s:  Permission denied" dir))
       (setq default-directory dir)
       (setq list-buffers-directory dir)))

to your .emacs, does it fix the problem for you?


        Stefan

Message #65 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Philip Hodges <philip.hodges <at> bluewin.ch>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative
 from	file-executable-p
Date: Tue, 06 May 2014 12:56:52 -0400

Stefan Monnier wrote:

> to your .emacs, does it fix the problem for you?

(In an Emacs later than 24.3; ie one where file-accessible-directory-p
is not just file-directory-p && file-executable-p.)

Message #68 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false
 negative	from	file-executable-p
Date: Wed, 07 May 2014 21:15:58 +0300

> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> Date: Wed, 7 May 2014 00:15:51 +0200
> 
> It's just a very ordinary samba mount from a UNIX box to Windows.
> It's not magic at all. There are no special options configured.
> With umask 027 most of the folders are 750 and regular files are 640.

Thanks, but this is not enough.  The problem is not in the (emulated)
Posix permissions, it's in the Windows ACLs.  E.g., what does icacls
report for that directory?

> The "documentation" that I had in mind was just something like:
> ... calls system functions that can conceivably produce race conditions and false negatives. In case Emacs will not let you even try to cd to the directory, or use packages like ediff that change to it or start a subprocess with it as current working directory, you can try to overcome that refusal with (defun file-executable-p (dir) "" t) or (defadvice file-executable-p ... activate ...).

I prefer to solve the problem rather than ask users work around it.

Message #71 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Thu, 08 May 2014 07:55:34 +0200

> I prefer to solve the problem rather than ask users work around it.

So when can we reasonably expect a guarantee of no more false negatives 
for users of 24.3 without having to inspect the fileio.c and files.el 
and reinvent an undocumented workaround?

It will be great if you really can *solve* the problem, even just for 
this one particular scenario. I already suggested a pathological 
counterexample. Other sources mentioned do indicate that it is 
impossible to solve it reliably in general. But perhaps it will be 
enough in practice.

Only the positive outcome of file-executable-p is documented as "this 
means you can access files in that directory". The negative outcome is 
not explicitly documented as meaning you cannot, yet that is how callers 
are interpreting it. So there is clearly scope for rewriting the 
documentation and changing the callers' logic to match.

Asking the user whether the directory is supposed to be searchable after 
all is not just a workaround. It may well be the only right thing to do. 
In this case he was the person whose umask set the file modes on the 
host system, and he can also inspect them on that system.

fgetacl showed no acl. I will try icacls but it may well be the same.

[I did take a look at fileio.c in trunk with a view to trying to build 
it native and in cygwin for windows. Instead of simply downloading a 
snapshot I ended up reading rather too much of a very long thread about 
whether bzr should be replaced with git, and wondering whether they 
aren't already coexisting as mirrors anyway. So right now I don't have 
much progress to report on trying out candidate preferred solutions.]

Message #74 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Thu, 08 May 2014 03:09:31 -0400

Look, if you are using Cygwin, just install 24.3.90 as referenced in
http://permalink.gmane.org/gmane.os.cygwin/146209

If you are using native MS Windows (or whatever they call it), download
24.3.90 from eg http://sourceforge.net/projects/emacs-bin/files/releases/ .

Then tell us if file-accessible-directory-p returns non-nil on this
problematic directory of yours.

Message #77 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Thu, 08 May 2014 19:18:13 +0300

> Date: Thu, 08 May 2014 07:55:34 +0200
> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> CC: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
> 
> > I prefer to solve the problem rather than ask users work around it.
> 
> So when can we reasonably expect a guarantee of no more false negatives 
> for users of 24.3 without having to inspect the fileio.c and files.el 
> and reinvent an undocumented workaround?

Emacs 24.3 was released more than a year ago, so fixing this in that
version might be possible only by some suitable change to the
directory's security descriptor outside of Emacs (if such a change is
possible).

But we can hope to fix this in future versions of Emacs.

> It will be great if you really can *solve* the problem, even just for 
> this one particular scenario. I already suggested a pathological 
> counterexample. Other sources mentioned do indicate that it is 
> impossible to solve it reliably in general. But perhaps it will be 
> enough in practice.

If we understand the problem in enough detail, we might find a
solution of some sort.

> Only the positive outcome of file-executable-p is documented as "this 
> means you can access files in that directory". The negative outcome is 
> not explicitly documented as meaning you cannot, yet that is how callers 
> are interpreting it. So there is clearly scope for rewriting the 
> documentation and changing the callers' logic to match.

That is a different, although related discussion.  Arguably, if a
directory is not accessible by me, Emacs had better not attempt that,
even if it might succeed, and instead leave it for the user to fix the
access rights by other means.

But even if we accept your views on this, it is better to try to solve
the problem than work around it.

Message #80 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Fri, 09 May 2014 02:54:22 -0400

>> I'd still like to hear if file-accessible-directory-p does any better
>> than file-executable-p.

I got tired of asking, so I simply changed it to use the former anyway,
since it can't be any worse than the latter and might be better.

Message #83 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 12:46:27 +0200

So I started going through my platform collection setting up smb shares,
so that I can connect to them, and open from native and cygwin builds.
It's harder to provoke a false negative on more recent systems. Usually 
it's either no connection at all, because I misconfigured something, or 
the owner and group are interpreted as known, resulting in a rash of 
positives, a few of which may be false.

Eventually I got a false negative result, sharing from Solaris 11.2.

The credentials given for the network connection are for the owner of 
the share, who has full rwx 7 access to all these folders and files. 
These messages are from cygwin emacs-w32 24.3.90.1:

; ediff-directories on my 700 and 750 folders *does* work:
Comparing '//192.168.0.18/myshare/smb/700/600' and 
'//192.168.0.18/myshare/smb/750/600' modulo 'nil'
Comparing files... Done

; cd to the 700 folder does not, this is clearly a false negative:
cd-absolute: Cannot cd to //192.168.0.18/myshare/smb/700/:  Permission 
denied

; in a shell cd to the same 700 folder works fine
$ cd //192.168.0.18/myshare/smb/700
$ ls
600  640  644  win
$ icacls .
. S-1-5-32-766:(OI)(CI)(RX,W,WDAC,WO,DC)
  S-1-5-32-767:(OI)(CI)(Rc,S,REA,RA)
  Everyone:(OI)(CI)(Rc,S,REA,RA)
Successfully processed 1 files; Failed processing 0 files

cygwin emacs-w32 emacs-version "24.3.90.1"
 (file-executable-p "//192.168.0.18/myshare/smb/700")
nil

Native builds do not seem to be affected:
native emacs-version "24.3.1"
 (file-executable-p "//192.168.0.18/myshare/smb/700")
t

native emacs-version "24.3.90.1"
 (file-executable-p "//192.168.0.18/myshare/smb/700")
t

Here are some other unexpected messages encountered on the way in no 
particular order. Maybe you can eliminate or mitigate some of them.

The credentials given for the network connection here are for another 
user in my group, which causes false positives for 600 and 700 modes,
resulting in noisy messages when it fails to work after all.
I would so much rather put up with messages like these from a false 
positive, than be prevented from using the file due to a false negative.

Error reading dir-locals: (file-error "Opening input file" "not a 
directory" "//mini2012/smb/640/.dir-locals.el")
; 640 is a regular file, not a directory, but why dir-locals.el ?
Error reading dir-locals: (file-error "Opening input file" "permission 
denied" "//mini2012/smb/700/.dir-locals.el")
; 700 is not group read and searchable, but why dir-locals.el ?
Error: (file-error "Searching for program" "no such file or directory" 
"bzr")
; why does emacs think I want to use bzr with this file?
Falling back on "slow" status detection ((file-error "Opening input 
file" "not a directory" "//mini2012/smb/640/.bzr/checkout/dirstate"))
File exists, but cannot be read
insert-directory: Listing directory failed but `access-file' worked
; which means?
find-file-noselect-1: Wrong type argument: arrayp, nil

I'm having a hard time understanding why you want to put so much faith 
in functions that are not reliable now, and will be quite hard or even 
genuinely impossible to make reliable in all of quite a large number of 
more or less realistic test scenarios.

; What is so wrong with:
(if (guess-this-will-work-p)
    (if (try-it-did-it-work-p)
        "worked as expected, you got lucky"
      "so many messages, how could it possibly not work")
  (if (you-want-to-try-it-anyway-be-it-on-your-own-head-p)
      (if (try-it-did-it-work-p)
          "so many messages: end of world, or never mind?"
        "whoo hoo, worked around it, so much for your stupid guess")))

; instead of
(if (guess-this-will-work-p)
    (if (try-it-did-it-work-p)
        "worked as expected, you got lucky"
      "so many messages, how could it possibly not work")
  "I will not let you even try, even though I sometimes guess wrong")))



On 2014-05-08 18:18, Eli Zaretskii wrote:
>> Date: Thu, 08 May 2014 07:55:34 +0200
>> From: Philip Hodges <philip.hodges <at> bluewin.ch>
>> CC: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
>>
>>> I prefer to solve the problem rather than ask users work around it.
>>
>> So when can we reasonably expect a guarantee of no more false negatives
>> for users of 24.3 without having to inspect the fileio.c and files.el
>> and reinvent an undocumented workaround?
>
> Emacs 24.3 was released more than a year ago, so fixing this in that
> version might be possible only by some suitable change to the
> directory's security descriptor outside of Emacs (if such a change is
> possible).
>
> But we can hope to fix this in future versions of Emacs.
>
>> It will be great if you really can *solve* the problem, even just for
>> this one particular scenario. I already suggested a pathological
>> counterexample. Other sources mentioned do indicate that it is
>> impossible to solve it reliably in general. But perhaps it will be
>> enough in practice.
>
> If we understand the problem in enough detail, we might find a
> solution of some sort.
>
>> Only the positive outcome of file-executable-p is documented as "this
>> means you can access files in that directory". The negative outcome is
>> not explicitly documented as meaning you cannot, yet that is how callers
>> are interpreting it. So there is clearly scope for rewriting the
>> documentation and changing the callers' logic to match.
>
> That is a different, although related discussion.  Arguably, if a
> directory is not accessible by me, Emacs had better not attempt that,
> even if it might succeed, and instead leave it for the user to fix the
> access rights by other means.
>
> But even if we accept your views on this, it is better to try to solve
> the problem than work around it.
>

Message #86 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Glenn Morris <rgm <at> gnu.org>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 15:55:11 +0200

On 2014-05-06 18:56, Glenn Morris wrote:
> Stefan Monnier wrote:
>
>> to your .emacs, does it fix the problem for you?
>
> (In an Emacs later than 24.3; ie one where file-accessible-directory-p
> is not just file-directory-p && file-executable-p.)

I can confirm that cd-absolute calling file-accessible-directory-p in 
emacs-w32 24.3.90 permits cd to a 700 folder on my Solaris 11.2 share.

Is there a way to have the process functions call it without rebuilding?

Message #89 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 20:00:48 +0300

> Date: Sun, 11 May 2014 12:46:27 +0200
> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> CC: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
> 
> ; cd to the 700 folder does not, this is clearly a false negative:
> cd-absolute: Cannot cd to //192.168.0.18/myshare/smb/700/:  Permission 
> denied

Why is that a false negative?  According to this:

> $ icacls .
> . S-1-5-32-766:(OI)(CI)(RX,W,WDAC,WO,DC)
>    S-1-5-32-767:(OI)(CI)(Rc,S,REA,RA)
>    Everyone:(OI)(CI)(Rc,S,REA,RA)  <<<<<<<<<<<<<<<<<<<<

you indeed have no "execute/traverse" rights to this directory.  It is
possible that Cygwin doesn't DTRT with the S-1-5-32-766 well-known
SID, which is NT_BUILTIN_CURRENT_OWNER SID.  Perhaps it would be a
good idea to describe all this on the Cygwin mailing list.

> ; in a shell cd to the same 700 folder works fine
> $ cd //192.168.0.18/myshare/smb/700
> $ ls
> 600  640  644  win

So the shell doesn't check, while Emacs does, so what?

> $ icacls .
> . S-1-5-32-766:(OI)(CI)(RX,W,WDAC,WO,DC)
>    S-1-5-32-767:(OI)(CI)(Rc,S,REA,RA)
>    Everyone:(OI)(CI)(Rc,S,REA,RA)
> Successfully processed 1 files; Failed processing 0 files
> 
> cygwin emacs-w32 emacs-version "24.3.90.1"
>   (file-executable-p "//192.168.0.18/myshare/smb/700")
> nil
> 
> Native builds do not seem to be affected:
> native emacs-version "24.3.1"
>   (file-executable-p "//192.168.0.18/myshare/smb/700")
> t

Irrelevant: native Windows Emacs doesn't examine the ACLs, so it
simply has no way of knowing this.

> Error reading dir-locals: (file-error "Opening input file" "not a 
> directory" "//mini2012/smb/640/.dir-locals.el")
> ; 640 is a regular file, not a directory, but why dir-locals.el ?
> Error reading dir-locals: (file-error "Opening input file" "permission 
> denied" "//mini2012/smb/700/.dir-locals.el")
> ; 700 is not group read and searchable, but why dir-locals.el ?
> Error: (file-error "Searching for program" "no such file or directory" 
> "bzr")
> ; why does emacs think I want to use bzr with this file?

See the relevant functions in files.el.  I see no problems here, all
of this is expected AFAIU.

> Falling back on "slow" status detection ((file-error "Opening input 
> file" "not a directory" "//mini2012/smb/640/.bzr/checkout/dirstate"))
> File exists, but cannot be read

As expected, given the ACLs.

> insert-directory: Listing directory failed but `access-file' worked
> ; which means?

It means what it says: you cannot list the directory (i.e. use
opendir/readdir APIs), but can successfully call 'access' on it.

> I'm having a hard time understanding why you want to put so much faith 
> in functions that are not reliable now, and will be quite hard or even 
> genuinely impossible to make reliable in all of quite a large number of 
> more or less realistic test scenarios.

The functions are reliable.  It's just that you have some obscure
situation with the share owner, file/directory owner, and network
connection, and this combination bites you.  It might also be a Cygwin
issue.

But I'm tired of wading through all this, so if
file-accessible-directory-p does the trick for you, let's forget about
the rest.

Message #92 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 20:26:10 +0200

>> ; cd to the 700 folder does not, this is clearly a false negative:
>> cd-absolute: Cannot cd to //192.168.0.18/myshare/smb/700/:  Permission
>> denied
>
> Why is that a false negative?  According to this:
>
>> $ icacls .
>> . S-1-5-32-766:(OI)(CI)(RX,W,WDAC,WO,DC)
>>     S-1-5-32-767:(OI)(CI)(Rc,S,REA,RA)
>>     Everyone:(OI)(CI)(Rc,S,REA,RA)  <<<<<<<<<<<<<<<<<<<<
>
> you indeed have no "execute/traverse" rights to this directory.  It is
> possible that Cygwin doesn't DTRT with the S-1-5-32-766 well-known
> SID, which is NT_BUILTIN_CURRENT_OWNER SID.  Perhaps it would be a
> good idea to describe all this on the Cygwin mailing list.

It's a false negative because the folder belongs to the host user I 
connected as, and that user happens to be able to do anything with it, 
including cd, if only cd-absolute would actually try it. I'm not just 
"everyone", I also happen to be the owner. Of course in the general case 
there might be host groups involved too, or even host ACLs.

>> ; in a shell cd to the same 700 folder works fine
>> $ cd //192.168.0.18/myshare/smb/700
>> $ ls
>> 600  640  644  win
>
> So the shell doesn't check, while Emacs does, so what?

Because cd and ls in that directory work in a cygwin shell, but opening 
and changing to that same directory do not work in cygwin emacs-w32.

>> $ icacls .
>> . S-1-5-32-766:(OI)(CI)(RX,W,WDAC,WO,DC)
>>     S-1-5-32-767:(OI)(CI)(Rc,S,REA,RA)
>>     Everyone:(OI)(CI)(Rc,S,REA,RA)
>> Successfully processed 1 files; Failed processing 0 files
>>
>> cygwin emacs-w32 emacs-version "24.3.90.1"
>>    (file-executable-p "//192.168.0.18/myshare/smb/700")
>> nil
>>
>> Native builds do not seem to be affected:
>> native emacs-version "24.3.1"
>>    (file-executable-p "//192.168.0.18/myshare/smb/700")
>> t
>
> Irrelevant: native Windows Emacs doesn't examine the ACLs, so it
> simply has no way of knowing this.

However it got there, whether by guesswork or sheer optimism, it 
happened to come up with the right answer.

>> ...

>> I'm having a hard time understanding why you want to put so much faith
>> in functions that are not reliable now, and will be quite hard or even
>> genuinely impossible to make reliable in all of quite a large number of
>> more or less realistic test scenarios.
>
> The functions are reliable.  It's just that you have some obscure
> situation with the share owner, file/directory owner, and network
> connection, and this combination bites you.  It might also be a Cygwin
> issue.

They are subject to race conditions, false positives and false 
negatives. They are reliable only in the sense that they generally do 
return (unless the network hangs, is there any way to stay responsive 
when that happens?) and the answer is quite often a true positive or 
true negative.

> But I'm tired of wading through all this, so if
> file-accessible-directory-p does the trick for you, let's forget about
> the rest.

I just skimmed through yet another tiring article about how there are 
fundamental reasons why cygwin can't always get permissions and ACLs 
exactly right, even without specifically mentioning remote SMB servers. 
I'm quite convinced the cygwin folks would have already done it if it 
was actually possible.

But I'm also curious about why different SMB implementations make a 
difference. If it was affecting Samba (recent with SMB2?) on GNU/Linux 
or Apple's own new SMB in MacOSX 10.9 (which defaults to SMB2.x) instead 
of just Oracle's Solaris (likely still SMB1) then would you still write 
it off as obscure? I didn't try a GNU/Linux yet by the way, it took me 
long enough just to find out how to configure a smb share on Solaris 
(with zfs commands on 11.2) and have virtualbox make it visible on my 
network (bridge).

I do appreciate your patience and tenacity. This thread just grew and 
grew longer against my expectations. I wish now that I had done more 
research and testing up front. And taken more care to note down or 
remember the getfacl command correctly, instead of recalling it as the 
typo fgetacl.

Message #95 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: Eli Zaretskii <eliz <at> gnu.org>, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 14:38:14 -0400

I already changed cd-absolute to use file-accessible-directory-p for
the next release, and documented that f-a-d-p is preferable to
file-executable-p for such purposes.

If you think anything else need changing, please say so.
Otherwise perhaps this is done.

Message #98 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 21:43:32 +0300

> Date: Sun, 11 May 2014 20:26:10 +0200
> From: Philip Hodges <philip.hodges <at> bluewin.ch>
> CC: rgm <at> gnu.org, 17330 <at> debbugs.gnu.org
> 
> >> I'm having a hard time understanding why you want to put so much faith
> >> in functions that are not reliable now, and will be quite hard or even
> >> genuinely impossible to make reliable in all of quite a large number of
> >> more or less realistic test scenarios.
> >
> > The functions are reliable.  It's just that you have some obscure
> > situation with the share owner, file/directory owner, and network
> > connection, and this combination bites you.  It might also be a Cygwin
> > issue.
> 
> They are subject to race conditions, false positives and false 
> negatives. They are reliable only in the sense that they generally do 
> return (unless the network hangs, is there any way to stay responsive 
> when that happens?) and the answer is quite often a true positive or 
> true negative.

I disagree with this, obviously.  Perfectly logical and systematic
behavior can appear random and "unreliable" to an observer who does
not understand that internal logic.

> I just skimmed through yet another tiring article about how there are 
> fundamental reasons why cygwin can't always get permissions and ACLs 
> exactly right, even without specifically mentioning remote SMB servers. 

Those articles are mostly trash, written by people who didn't bother
to learn the subject, and instead spread FUD.

> I'm quite convinced the cygwin folks would have already done it if it 
> was actually possible.

Maybe they don't know about this.  Which is why I think telling them
about the problem should be a good idea.  They do fix new problems
they encounter; e.g., they've just learned about a problem with
Microsoft Accounts, and mostly fixed it.

> If it was affecting Samba (recent with SMB2?) on GNU/Linux or
> Apple's own new SMB in MacOSX 10.9 (which defaults to SMB2.x)
> instead of just Oracle's Solaris (likely still SMB1) then would you
> still write it off as obscure?

Yes.  Again, it could also be a Cygwin issue, perhaps due to something
that rarely happens or something new.

Message #101 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Glenn Morris <rgm <at> gnu.org>
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Sun, 11 May 2014 23:59:12 +0200

> I already changed cd-absolute to use file-accessible-directory-p for
> the next release, and documented that f-a-d-p is preferable to
> file-executable-p for such purposes.
> 
> If you think anything else need changing, please say so.
> Otherwise perhaps this is done.

The new f-a-d-p completely ignores permissions, and it
looks like it is called in all directory check contexts before
changing directory or running a process with it as cwd, so
I don't see any way it can refuse to try to cd to a directory.

As for "anything else": I wonder if it will tell me I cannot
start a process to execute a regular file on my share...

Thanks.

Message #104 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Glenn Morris <rgm <at> gnu.org>
To: Philip Hodges <philip.hodges <at> bluewin.ch>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Mon, 12 May 2014 03:10:27 -0400

(Please keep 17330 <at> debbugs cc'd)

Philip Hodges wrote:

> The new f-a-d-p completely ignores permissions

This is trivially falsifiable:

ls -ld foo
drwx------ 2 root root 4096 May 12 00:04 foo/

as normal user:
(file-accessible-directory-p "foo")  ; -> nil

> , and it looks like it is called in all directory check contexts
> before changing directory or running a process with it as cwd, so I
> don't see any way it can refuse to try to cd to a directory.

M-x cd foo RET
cd-absolute: Cannot cd to /home/gm/foo/:  Permission denied

> As for "anything else": I wonder if it will tell me I cannot
> start a process to execute a regular file on my share...

I give up. Whatever you say.

Message #107 received at 17330 <at> debbugs.gnu.org (full text, mbox):

From: Philip Hodges <philip.hodges <at> bluewin.ch>
To: Glenn Morris <rgm <at> gnu.org>
Cc: 17330 <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Mon, 12 May 2014 09:26:21 +0200

> The new f-a-d-p completely ignores permissions

Sorry, I wasn't completely clear. I did not mean that it overcomes permissions to give information about the existence of directories that it cannot even see.

Depending on platform the code looks like it checks for a directory (using faccessat or stat) or checks existence of the . directory within it. It appears to be making checks that depend on certain permissions (in other words actually trying it) instead of inspecting permissions (and being misled by them).

Message #112 received at 17330-done <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Glenn Morris <rgm <at> gnu.org>
Cc: Eli Zaretskii <eliz <at> gnu.org>, Philip Hodges <philip.hodges <at> bluewin.ch>,
 17330-done <at> debbugs.gnu.org
Subject: Re: bug#17330: files.el cd-absolute overcome false negative from
 file-executable-p
Date: Fri, 22 Oct 2021 22:09:12 -0700

Glenn Morris <rgm <at> gnu.org> writes:

> I already changed cd-absolute to use file-accessible-directory-p for
> the next release, and documented that f-a-d-p is preferable to
> file-executable-p for such purposes.
>
> If you think anything else need changing, please say so.
> Otherwise perhaps this is done.

This was a sprawling discussion that ranged from not getting a recipe to
not getting a fix tested.  It seems like the conclusion was that this
was fixed, and we've heard nothing more in 7.5 years.  I'm therefore
closing this bug report.

If this conclusion is incorrect and this is still an issue, please reply
to this email (use "Reply to all" in your email client) and we can
reopen the bug report.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.