GNU bug report logs - #78209
30.1; project-query-replace-regexp fails on a symlink to a directory

Previous Next

To reply to this bug, email your comments to 78209 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Łukasz Wojniłowicz <lukasz.wojnilowicz <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 30.1; project-query-replace-regexp fails on a symlink to a directory
Date: Fri, 2 May 2025 11:35:06 +0200

Executing project-query-replace-regexp on a git repository like below
-----------------------
.
├── ./a
│   └── ./a/file.txt
└── ./link-to-a -> a

3 directories, 1 file
-----------------------

on Linux fails with

"fileloop-next-file: Read error: Is a directory, /absolute/path/to/project-query-replace-regexp/link-to-a"

Could you fix that?

Message #8 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dominik Schrempf <dominik.schrempf <at> gmail.com>
To: 78209 <at> debbugs.gnu.org
Subject: 30.1; project-query-replace-regexp fails on a symlink to a directory
Date: Wed, 12 Nov 2025 21:00:00 +0100

Hello!

I am also running into this quite frequently.

In particular, when executing `project-query-replace-regexp`, I get

apply: Read error: Is a directory, /path/to/project/.direnv/flake-inputs/a2awwif8wvk1jcgjqk104025g6lzr1jd-source

The file is a symlink, and the directory is ignored by git. (Should we replace also in git-ignored files)?

Best,
Dominik

Message #11 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Sean Whitton <spwhitton <at> spwhitton.name>
To: Dominik Schrempf <dominik.schrempf <at> gmail.com>
Cc: 78209 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a
 symlink to a directory
Date: Tue, 18 Nov 2025 15:11:58 +0000

Hello,

On Wed 12 Nov 2025 at 09:00pm +01, Dominik Schrempf wrote:

> Hello!
>
> I am also running into this quite frequently.
>
> In particular, when executing `project-query-replace-regexp`, I get
>
> apply: Read error: Is a directory,
> /path/to/project/.direnv/flake-inputs/a2awwif8wvk1jcgjqk104025g6lzr1jd-source
>
> The file is a symlink, and the directory is ignored by git. (Should we replace
> also in git-ignored files)?

Thanks.  Adding Stefan, who authored fileloop.

-- 
Sean Whitton

Message #14 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Sean Whitton <spwhitton <at> spwhitton.name>
Cc: Dominik Schrempf <dominik.schrempf <at> gmail.com>, 78209 <at> debbugs.gnu.org
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a
 symlink to a directory
Date: Tue, 18 Nov 2025 10:40:46 -0500

> Thanks.  Adding Stefan, who authored fileloop.

My involvement was mostly in refactoring that code.
IIRC fileloop is not responsible for the choice of which files to visit,
this is delegated to the user of fileloop.  IOW, I suspect the problem
is in `project.el`, more specifically `project-files` or
`project-query-replace-regexp` which should skip symlinks that don't point
to a file, or maybe even skip all symlinks (after all, if it points
outside of the project then we shouldn't visit it, and if it points
inside the project, then we'll visit it via its non-symlink name
already).


        Stefan

Message #17 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Dominik Schrempf <dominik.schrempf <at> gmail.com>, 78209 <at> debbugs.gnu.org
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a symlink
 to a directory
Date: Wed, 19 Nov 2025 04:22:54 +0200

Hi!

On 12/11/2025 22:00, Dominik Schrempf wrote:
> The file is a symlink, and the directory is ignored by git. (Should we replace also in git-ignored files)?

This is an important point.

Could you describe the corresponding .gitignore entry in more detail? Or 
show a self-contained example.

At least in basic testing, I see such an entry being honored, with 
project-files not including the (ignored) symlink in its return value.

Message #20 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>,
 Sean Whitton <spwhitton <at> spwhitton.name>
Cc: Dominik Schrempf <dominik.schrempf <at> gmail.com>, 78209 <at> debbugs.gnu.org
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a symlink
 to a directory
Date: Wed, 19 Nov 2025 04:26:44 +0200

On 18/11/2025 17:40, Stefan Monnier via Bug reports for GNU Emacs, the 
Swiss army knife of text editors wrote:
> IOW, I suspect the problem
> is in `project.el`, more specifically `project-files` or
> `project-query-replace-regexp` which should skip symlinks that don't point
> to a file, or maybe even skip all symlinks (after all, if it points
> outside of the project then we shouldn't visit it, and if it points
> inside the project, then we'll visit it via its non-symlink name
> already).

It seems like it can be expensive to do that inside 'project-files' 
(e.g. calling file-regular-p on every item in the returned list, eagerly).

And it looks like now, with bug#79356 resolved, at least in Emacs 31+ 
symlinks won't stop the replacement in all other, regular files. They'll 
just show up in the *Warnings* buffer.

But note my previous email, regarding symlinks being in .gitignore.

Message #23 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: Dominik Schrempf <dominik.schrempf <at> gmail.com>, 78209 <at> debbugs.gnu.org,
 Sean Whitton <spwhitton <at> spwhitton.name>
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a
 symlink to a directory
Date: Wed, 19 Nov 2025 08:39:14 -0500

>> IOW, I suspect the problem is in `project.el`, more specifically
>> `project-files` or `project-query-replace-regexp` which should skip
>> symlinks that don't point to a file, or maybe even skip all symlinks
>> (after all, if it points outside of the project then we shouldn't
>> visit it, and if it points inside the project, then we'll visit it
>> via its non-symlink name already).
>
> It seems like it can be expensive to do that inside 'project-files'
> (e.g. calling file-regular-p on every item in the returned list, eagerly).

Indeed, I don't think all callers of `project-files` would like that, so
it would have to be optional.  the case of
`project-query-replace-regexp`, the operation needs to do a lot more
than test the symlink-ness of every file anyway, so an extra
`file-regular-p` or `file-symlink-p` should be lost in the noise.

At most, one might need to do the test lazily rather than test all the
files before even calling `fileloop-initialize`, but other than that the
performance impact should be negligible.

> And it looks like now, with bug#79356 resolved, at least in Emacs 31+
> symlinks won't stop the replacement in all other, regular files. They'll
> just show up in the *Warnings* buffer.

Good.

> But note my previous email, regarding symlinks being in .gitignore.

Indeed, my understanding is that `project-files` should not include the
ignored files at least when called for `project-query-replace-regexp`.
The docstring clearly says:

    PROJECT is used to find the list of ignores for each directory.


- Stefan

Message #26 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dominik Schrempf <dominik.schrempf <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 78209 <at> debbugs.gnu.org
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a
 symlink to a directory
Date: Thu, 20 Nov 2025 17:22:26 +0100

Hi!

On Wed 19 Nov 2025 04:22:54 AM +02, Dmitry Gutov wrote:

> Hi!
>
> On 12/11/2025 22:00, Dominik Schrempf wrote:
>> The file is a symlink, and the directory is ignored by git. (Should we replace also in git-ignored files)?
>
> This is an important point.
>
> Could you describe the corresponding .gitignore entry in more detail? Or show a
> self-contained example.

I apologize for being so quiet, but I didn't observe the bug for a few
days now. This is a bit suspicious. Maybe it depends on the order how
the files are traversed?

For your reference, I have a project setup equivalent to

    $ tree -a
    .
    ├── .direnv
    │   ├── link-to-some-file-outside-the-project-dir-1
    │   └── link-to-some-file-outside-the-project-dir-2
    └── .gitignore

    2 directories, 3 fil

    $ cat .gitignore
    /.direnv/

Best,
Dominik

>
> At least in basic testing, I see such an entry being honored, with project-files
> not including the (ignored) symlink in its return value.

Message #29 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Dominik Schrempf <dominik.schrempf <at> gmail.com>
Cc: 78209 <at> debbugs.gnu.org
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a symlink
 to a directory
Date: Fri, 21 Nov 2025 04:33:16 +0200

On 20/11/2025 18:22, Dominik Schrempf wrote:

>> This is an important point.
>>
>> Could you describe the corresponding .gitignore entry in more detail? Or show a
>> self-contained example.
> 
> I apologize for being so quiet, but I didn't observe the bug for a few
> days now. This is a bit suspicious. Maybe it depends on the order how
> the files are traversed?

I don't think so. At least if you traverse the file list to the end, 
which you could do quickly, for example, by entering a search string 
that has no matches anywhere.

The latest Emacs did have turned the error into a warning, so if you're 
following the master branch, you could be enjoying the benefits of that. 
But still it should pop up in the *Warnings* buffer.

> For your reference, I have a project setup equivalent to
> 
>      $ tree -a
>      .
>      ├── .direnv
>      │   ├── link-to-some-file-outside-the-project-dir-1
>      │   └── link-to-some-file-outside-the-project-dir-2
>      └── .gitignore
> 
>      2 directories, 3 fil
> 
>      $ cat .gitignore
>      /.direnv/

I've tried that setup, and when such an entry is present in .gitignore, 
project-files correctly omits those symlinked files.

You can also test this without project-query-replace-regexp but just 
using project-find-file (bound to 'C-x p f'): the files in question, 
being ignored, should not be in the list of completions.

If you encounter the problem again, perhaps you could reduce it to a 
small directory, zip it up and upload somewhere. Or maybe you find more 
clues, that'll help.

Message #32 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dmitry Gutov <dmitry <at> gutov.dev>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Dominik Schrempf <dominik.schrempf <at> gmail.com>, 78209 <at> debbugs.gnu.org,
 Sean Whitton <spwhitton <at> spwhitton.name>
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a symlink
 to a directory
Date: Fri, 21 Nov 2025 04:39:07 +0200

On 19/11/2025 15:39, Stefan Monnier wrote:
>>> IOW, I suspect the problem is in `project.el`, more specifically
>>> `project-files` or `project-query-replace-regexp` which should skip
>>> symlinks that don't point to a file, or maybe even skip all symlinks
>>> (after all, if it points outside of the project then we shouldn't
>>> visit it, and if it points inside the project, then we'll visit it
>>> via its non-symlink name already).
>>
>> It seems like it can be expensive to do that inside 'project-files'
>> (e.g. calling file-regular-p on every item in the returned list, eagerly).
> 
> Indeed, I don't think all callers of `project-files` would like that, so
> it would have to be optional.  the case of
> `project-query-replace-regexp`, the operation needs to do a lot more
> than test the symlink-ness of every file anyway, so an extra
> `file-regular-p` or `file-symlink-p` should be lost in the noise.
> 
> At most, one might need to do the test lazily rather than test all the
> files before even calling `fileloop-initialize`, but other than that the
> performance impact should be negligible.

Right, but the existing helpers fileloop-initialize-search and 
fileloop-initialize-replace don't have a per-file argument (yet?). We 
could do that using lower level functions from fileloop, probably.

Anyway, that is if we decide to support symlinked files or directories 
that don't have corresponding .gitignore entries. But in that case their 
status is a little dubious, as you noted.

>> But note my previous email, regarding symlinks being in .gitignore.
> 
> Indeed, my understanding is that `project-files` should not include the
> ignored files at least when called for `project-query-replace-regexp`.
> The docstring clearly says:
> 
>      PROJECT is used to find the list of ignores for each directory.

Yep yep.

Message #35 received at 78209 <at> debbugs.gnu.org (full text, mbox):

From: Dominik Schrempf <dominik.schrempf <at> gmail.com>
To: Dmitry Gutov <dmitry <at> gutov.dev>
Cc: 78209 <at> debbugs.gnu.org
Subject: Re: bug#78209: 30.1; project-query-replace-regexp fails on a
 symlink to a directory
Date: Fri, 21 Nov 2025 08:27:45 +0100

On Fri 21 Nov 2025 04:33:16 AM +02, Dmitry Gutov wrote:

> On 20/11/2025 18:22, Dominik Schrempf wrote:
>
>>> This is an important point.
>>>
>>> Could you describe the corresponding .gitignore entry in more detail? Or show a
>>> self-contained example.
>> I apologize for being so quiet, but I didn't observe the bug for a few
>> days now. This is a bit suspicious. Maybe it depends on the order how
>> the files are traversed?
>
> I don't think so. At least if you traverse the file list to the end, which you
> could do quickly, for example, by entering a search string that has no matches
> anywhere.
>
> The latest Emacs did have turned the error into a warning, so if you're
> following the master branch, you could be enjoying the benefits of that. But
> still it should pop up in the *Warnings* buffer.

Thanks a lot for your patience.

No, I am not on master.

>
>> For your reference, I have a project setup equivalent to
>>      $ tree -a
>>      .
>>      ├── .direnv
>>      │   ├── link-to-some-file-outside-the-project-dir-1
>>      │   └── link-to-some-file-outside-the-project-dir-2
>>      └── .gitignore
>>      2 directories, 3 fil
>>      $ cat .gitignore
>>      /.direnv/
>
> I've tried that setup, and when such an entry is present in .gitignore,
> project-files correctly omits those symlinked files.
>
> You can also test this without project-query-replace-regexp but just using
> project-find-file (bound to 'C-x p f'): the files in question, being ignored,
> should not be in the list of completions.
>
> If you encounter the problem again, perhaps you could reduce it to a small
> directory, zip it up and upload somewhere. Or maybe you find more clues, that'll
> help.

I managed to reproduce the error in the original repository (and I can
do so reliably by using a unique string, see backtrace below), but I can
not create a minimal example. The backtrace is

Debugger entered--Lisp error: (file-error "Read error" "Is a directory" "/path/to/proj/examples/botan/botan/")
  insert-file-contents("/path/to/proj/examples/botan/botan/./" nil)
  fileloop-next-file(t)
  fileloop-continue()
  project-query-replace-regexp("alskdjflasjdlfkjasdf" "")
  funcall-interactively(project-query-replace-regexp "alskdjflasjdlfkjasdf" "")
  command-execute(project-query-replace-regexp)

'examples/botan/botan' is a git submodule (and it is empty in my case, I
haven't checked it out)! (But in a minimal example I tried to come up
with, that git-submodule fact alone didn't trigger the error). What is
also a bit surprising is this './' at the end of the argument to
`insert-file-contents'.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.