GNU bug report logs - #61846
Shell pathnames incorrectly handled, mostly due to hash characters

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 61846 in the body.
You can then email your comments to 61846 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: John Cummings <john <at> rootabega.net>
To: "bug-hyperbole <at> gnu.org" <bug-hyperbole <at> gnu.org>
Subject: Shell pathnames incorrectly handled, mostly due to hash characters
Date: Mon, 27 Feb 2023 16:50:16 +0000

[Message part 1 (text/plain, inline)]

I use:  Editor:      GNU Emacs 28.2
        Hyperbole:   9.0.0
        Sys Type:    x86_64-pc-linux-gnu
        OS Type:     gnu/linux
        Window Sys:  None
        News Reader: Gnus v5.13
        Install:     elpa-devel, 8.0.1pre0.20230220.191327


Background:

This is usage of the Hyperbole action key on implicit buttons in a
shell buffer directory listing (bash/ls). Some file or directory names
appear to have unexpected results.


Expected Behavior (General):

Each file or directory in the listing will be interpreted as a
pathname implicit button (regardless of whether the name could also
match some other implicit button type in another context), and using
the action key on it will visit that file or directory. This will also
be reflected in the hkey-help output.


Actual Behavior (General):

Certain characters or patterns in names, and/or positioning of the
names in the listing, cause the names to be interpreted
incorrectly. Some names are interpreted incorrectly due to patterns in
their own name, and some due to patterns in other names in the
listing. That is, Hyperbole seems to be interpreting several names in
the listing as one composite name.

This can cause the Hyperbole action key to perform the wrong action,
like visitng the wrong file or directory, or not acting at all.

In some cases, the Hyperbole action key DOES visit the expected
location, but the hkey-help output still appears wrong.

The unexpected behaviors I saw were mostly related to the hash
character in names, which is not uncommon since Emacs creates those
files automatically for autosaving. But the behaviors are also
influenced by name positions relative to each other, meaning that
differently-sized windows could produce different behaviors. In this
test, I limited my window with to 80 characters, but the results would
be slightly different if the window was wider.

The main misinterpretations are:

- interpreting names with a hash character as a social media reference

- Names mistakenly interpreted as a subordinate of some other
  name in the listing, like it's a file in a subdirectory, or a link
  anchor in a pathname. This also appears to be linked to the
  appearance of the hash character in the name of the false parent.

- names not having any implicit button context, and no results when
  pressing the Hyperbole action key




Test Data:

I'll attempt to attach the test data directory as a tar archive, and
screenshots for more context. I'll also include the hkey-debug log
messages for all the action key presses.

The names I tested were shaped like this in my shell buffer:
'#buttons-9.txt#'   foofoofoo                      sources
 buttons-9.txt      hyperbole.el                   test.el
 buttons-9.txt~    '#*message*-20230220-192738#'   test.html
 buttons.txt       '#*message*-20230220-205601#'   test.html~
 buttons.txt~      '#*message*-20230220-210029#'   test.kotl
 EXAMPLE.kotl       source.el                      test.kotl~
 foo                source.el~
'#foofoofoo#'       source.elc



Specific Results:

(starting at the top left, going across to the right, and then
repeating for each row) Unless otherwise noted, the hkey-help output
was consistent with the action key behavior, whether the action key
behavior was right or wrong.

'#buttons-9.txt#' - Appears to be a non-functional social media
reference.

foofoofoo - Appears to be a child/anchor of '#buttons-9.txt' which
does not exist. So action key visits the data directory instead.

sources - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

buttons-9.txt - Same as foofoofoo above. However, the file being on the next
line appears to make the (incorrect) relationship of this file to its
false parent more interesting, inferring an underscore character.

hyperbole.el - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

test.el - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

buttons-9.txt~ - The action key opens this file as expected. However,
the hkey-help shows the incorrect lbl-key, just like buttons-9.txt. It
appears to interpret this name as a child/anchor of '#buttons-9.txt',
as well as adding an underscore

'#*message*-20230220-192738#' - Action key results in an error:
'(hpath:find): "#*message*-20230220-192738#" does not exist'. It
appears that the current directory is not applied to the find action.

test.html - Both hkey-help and pressing the action key say that there
is no matching context.

buttons.txt - Same as buttons-9.txt~ - Action key works, but hkey-help
is wrong, showing file as child/anchor of another name.

'#*message*-20230220-205601#' - same as previous mail message file -
does not include the current directory in the find action, so the find
raises an error.

test.html~ - Same as test.html above - no hkey-help or action button
context found.

buttons.txt~ - Same as buttons-9.txt~ - Action key works, but hkey-help
is wrong, showing file as child/anchor of another name.

'#*message*-20230220-210029#' - same as previous mail message file -
does not include the current directory in the find action, so the find
raises an error.

test.kotl - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

EXAMPLE.kotl - Same as buttons-9.txt above - child/anchor of
'#buttons-9.txt', with added underscore

source.el - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

test.kotl~ - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

foo - Same as buttons-9.txt above - child/anchor of '#buttons-9.txt',
with added underscore

source.el~ - Same as foofoofoo above - child/anchor of '#buttons-9.txt'

'#foofoofoo#' - Same as '#buttons-9.txt#' above - nonfunctional social
media reference.

source.elc - Same as foofoofoo above - child/anchor of '#buttons-9.txt'



HyDebug messages from test session:
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "#buttons-9.txt" lbl-start 36 lbl-end 50 categ ibtypes::social-reference loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype social-reference args ("" "#" "buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/foofoofoo" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/sources" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/_buttons-9.txt" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/hyperbole.el" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/test.el" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/_buttons-9.txt~" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/buttons-9.txt~    "))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "#*message*-20230220-192738#" lbl-start 173 lbl-end 200 categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("#*message*-20230220-192738#"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/_buttons.txt" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/buttons.txt       "))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "#*message*-20230220-205601#" lbl-start 234 lbl-end 261 categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("#*message*-20230220-205601#"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/_buttons.txt~" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/buttons.txt~      "))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "#*message*-20230220-210029#" lbl-start 296 lbl-end 323 categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("#*message*-20230220-210029#"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/test.kotl" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/_EXAMPLE.kotl" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/source.el" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/test.kotl~" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/_foo" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/source.el~" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "#foofoofoo" lbl-start 431 lbl-end 441 categ ibtypes::social-reference loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype social-reference args ("" "#" "foofoofoo"))
(HyDebug) ButProps: (variable-documentation "The currently selected Hyperbole button.  Available to action routines." lbl-key "/home/emacs/shellbug/data/#buttons-9.txt#/source.elc" categ ibtypes::pathname loc #<buffer *shell*> dir "/home/emacs/shellbug/data/" actype actypes::link-to-file args ("/home/emacs/shellbug/data/#buttons-9.txt"))

[test-data.tar (application/x-tar, attachment)]

[screenshots.tar.xz (application/x-xz, attachment)]

Message #8 received at 61846 <at> debbugs.gnu.org (full text, mbox):

From: Robert Weiner <rsw <at> gnu.org>
To: John Cummings <john <at> rootabega.net>
Cc: 61846 <at> debbugs.gnu.org
Subject: Re: bug#61846: Shell pathnames incorrectly handled, mostly due to
 hash characters
Date: Tue, 28 Feb 2023 22:57:21 -0500

Hi John:

This issue has been resolved in the latest pre-release of Hyperbole
available from the elpa-devel package archive.  Thanks for reporting
it.  -- rsw


John Cummings <john <at> rootabega.net> writes:

> I use:  Editor:      GNU Emacs 28.2
>         Hyperbole:   9.0.0
>         Sys Type:    x86_64-pc-linux-gnu
>         OS Type:     gnu/linux
>         Window Sys:  None
>         News Reader: Gnus v5.13
>         Install:     elpa-devel, 8.0.1pre0.20230220.191327
>
>
> Background:
>
> This is usage of the Hyperbole action key on implicit buttons in a
> shell buffer directory listing (bash/ls). Some file or directory names
> appear to have unexpected results.
>
>
> Expected Behavior (General):
>
> Each file or directory in the listing will be interpreted as a
> pathname implicit button (regardless of whether the name could also
> match some other implicit button type in another context), and using
> the action key on it will visit that file or directory. This will also
> be reflected in the hkey-help output.
>
>
> Actual Behavior (General):
>
> Certain characters or patterns in names, and/or positioning of the
> names in the listing, cause the names to be interpreted
> incorrectly. Some names are interpreted incorrectly due to patterns in
> their own name, and some due to patterns in other names in the
> listing. That is, Hyperbole seems to be interpreting several names in
> the listing as one composite name.
>
> This can cause the Hyperbole action key to perform the wrong action,
> like visitng the wrong file or directory, or not acting at all.

Message #13 received at 61846 <at> debbugs.gnu.org (full text, mbox):

From: John Cummings <john <at> rootabega.net>
To: Robert Weiner <rsw <at> gnu.org>
Cc: 61846 <at> debbugs.gnu.org
Subject: Re: bug#61846: Shell pathnames incorrectly handled,
 mostly due to hash characters
Date: Wed, 01 Mar 2023 11:59:02 +0000

Thanks Bob, looks like the problem of interpreting files as children
of '#buttons-9.txt#' is fixed. But most/all of the other problems
are still there. Are you still looking at those and/or do you want me
to create a bug for each problem type?




------- Original Message -------
On Tuesday, February 28th, 2023 at 10:57 PM, Robert Weiner <rsw <at> gnu.org> wrote:

> 
> 
> Hi John:
> 
> This issue has been resolved in the latest pre-release of Hyperbole
> available from the elpa-devel package archive. Thanks for reporting
> it. -- rsw
> 
> 
> John Cummings john <at> rootabega.net writes:
> 
> > I use: Editor: GNU Emacs 28.2
> > Hyperbole: 9.0.0
> > Sys Type: x86_64-pc-linux-gnu
> > OS Type: gnu/linux
> > Window Sys: None
> > News Reader: Gnus v5.13
> > Install: elpa-devel, 8.0.1pre0.20230220.191327
> > 
> > Background:
> > 
> > This is usage of the Hyperbole action key on implicit buttons in a
> > shell buffer directory listing (bash/ls). Some file or directory names
> > appear to have unexpected results.
> > 
> > Expected Behavior (General):
> > 
> > Each file or directory in the listing will be interpreted as a
> > pathname implicit button (regardless of whether the name could also
> > match some other implicit button type in another context), and using
> > the action key on it will visit that file or directory. This will also
> > be reflected in the hkey-help output.
> > 
> > Actual Behavior (General):
> > 
> > Certain characters or patterns in names, and/or positioning of the
> > names in the listing, cause the names to be interpreted
> > incorrectly. Some names are interpreted incorrectly due to patterns in
> > their own name, and some due to patterns in other names in the
> > listing. That is, Hyperbole seems to be interpreting several names in
> > the listing as one composite name.
> > 
> > This can cause the Hyperbole action key to perform the wrong action,
> > like visitng the wrong file or directory, or not acting at all.

Message #16 received at 61846 <at> debbugs.gnu.org (full text, mbox):

From: Robert Weiner <rsw <at> gnu.org>
To: John Cummings <john <at> rootabega.net>
Cc: 61846 <at> debbugs.gnu.org
Subject: Re: bug#61846: Shell pathnames incorrectly handled, mostly due to
 hash characters
Date: Wed, 1 Mar 2023 19:59:23 -0500

[Message part 1 (text/plain, inline)]

John if you could do an ls in the Hyperbole distribution directory and
summarize what happens when you use {M-RET} on specific filenames, that
would help us evaluate the issue.  Thanks.

On Wed, Mar 1, 2023 at 6:59 AM John Cummings <john <at> rootabega.net> wrote:

> Thanks Bob, looks like the problem of interpreting files as children
> of '#buttons-9.txt#' is fixed. But most/all of the other problems
> are still there. Are you still looking at those and/or do you want me
> to create a bug for each problem type?
>
>
>
>
> ------- Original Message -------
> On Tuesday, February 28th, 2023 at 10:57 PM, Robert Weiner <rsw <at> gnu.org>
> wrote:
>
> >
> >
> > Hi John:
> >
> > This issue has been resolved in the latest pre-release of Hyperbole
> > available from the elpa-devel package archive. Thanks for reporting
> > it. -- rsw
> >
> >
> > John Cummings john <at> rootabega.net writes:
> >
> > > I use: Editor: GNU Emacs 28.2
> > > Hyperbole: 9.0.0
> > > Sys Type: x86_64-pc-linux-gnu
> > > OS Type: gnu/linux
> > > Window Sys: None
> > > News Reader: Gnus v5.13
> > > Install: elpa-devel, 8.0.1pre0.20230220.191327
> > >
> > > Background:
> > >
> > > This is usage of the Hyperbole action key on implicit buttons in a
> > > shell buffer directory listing (bash/ls). Some file or directory names
> > > appear to have unexpected results.
> > >
> > > Expected Behavior (General):
> > >
> > > Each file or directory in the listing will be interpreted as a
> > > pathname implicit button (regardless of whether the name could also
> > > match some other implicit button type in another context), and using
> > > the action key on it will visit that file or directory. This will also
> > > be reflected in the hkey-help output.
> > >
> > > Actual Behavior (General):
> > >
> > > Certain characters or patterns in names, and/or positioning of the
> > > names in the listing, cause the names to be interpreted
> > > incorrectly. Some names are interpreted incorrectly due to patterns in
> > > their own name, and some due to patterns in other names in the
> > > listing. That is, Hyperbole seems to be interpreting several names in
> > > the listing as one composite name.
> > >
> > > This can cause the Hyperbole action key to perform the wrong action,
> > > like visitng the wrong file or directory, or not acting at all.
>

[Message part 2 (text/html, inline)]

Message #19 received at 61846 <at> debbugs.gnu.org (full text, mbox):

From: John Cummings <john <at> rootabega.net>
To: rswgnu <at> gmail.com
Cc: 61846 <at> debbugs.gnu.org
Subject: Re: bug#61846: Shell pathnames incorrectly handled,
 mostly due to hash characters
Date: Thu, 02 Mar 2023 15:55:38 +0000

[Message part 1 (text/plain, inline)]

I want to make sure I don't drift too far, so here's my current map of
the situation:

One of the issues I reported originally has been fixed. That is where
the first name in the ls output is mistakenly interpreted as the
parent of other names in the listing. As far as I can tell, this was
happening because the regex was consuming the next newline when you
just ran ls without any filename arguments, and Hyperbole was
interpreting the first name in the listing as an argument to ls. Since
that was the most common issue in my original report, I think it makes
sense to treat this bug, #61846, as tracking that exact behavior, and
I agree that it appears fixed.

The other problems that I reported originally are still occurring as
described in my original report. I'll report a bug for each
problematic behavior.

Most of the filenames in the Hyperbole distribution directory seem
pretty ordinary. I sampled a few of them and found no issues. The ones
with interesting characters like . or _ also didn't cause any
problems. But this would not negate any of the problems I originally
reported, anyway.

------- Original Message -------
On Wednesday, March 1st, 2023 at 7:59 PM, Robert Weiner <rsw <at> gnu.org> wrote:

> John if you could do an ls in the Hyperbole distribution directory and summarize what happens when you use {M-RET} on specific filenames, that would help us evaluate the issue. Thanks.
>
> On Wed, Mar 1, 2023 at 6:59 AM John Cummings <john <at> rootabega.net> wrote:
>
>> Thanks Bob, looks like the problem of interpreting files as children
>> of '#buttons-9.txt#' is fixed. But most/all of the other problems
>> are still there. Are you still looking at those and/or do you want me
>> to create a bug for each problem type?
>>
>> ------- Original Message -------
>> On Tuesday, February 28th, 2023 at 10:57 PM, Robert Weiner <rsw <at> gnu.org> wrote:
>>
>>>
>>>
>>> Hi John:
>>>
>>> This issue has been resolved in the latest pre-release of Hyperbole
>>> available from the elpa-devel package archive. Thanks for reporting
>>> it. -- rsw
>>>
>>>
>>> John Cummings john <at> rootabega.net writes:
>>>
>>> > I use: Editor: GNU Emacs 28.2
>>> > Hyperbole: 9.0.0
>>> > Sys Type: x86_64-pc-linux-gnu
>>> > OS Type: gnu/linux
>>> > Window Sys: None
>>> > News Reader: Gnus v5.13
>>> > Install: elpa-devel, 8.0.1pre0.20230220.191327
>>> >
>>> > Background:
>>> >
>>> > This is usage of the Hyperbole action key on implicit buttons in a
>>> > shell buffer directory listing (bash/ls). Some file or directory names
>>> > appear to have unexpected results.
>>> >
>>> > Expected Behavior (General):
>>> >
>>> > Each file or directory in the listing will be interpreted as a
>>> > pathname implicit button (regardless of whether the name could also
>>> > match some other implicit button type in another context), and using
>>> > the action key on it will visit that file or directory. This will also
>>> > be reflected in the hkey-help output.
>>> >
>>> > Actual Behavior (General):
>>> >
>>> > Certain characters or patterns in names, and/or positioning of the
>>> > names in the listing, cause the names to be interpreted
>>> > incorrectly. Some names are interpreted incorrectly due to patterns in
>>> > their own name, and some due to patterns in other names in the
>>> > listing. That is, Hyperbole seems to be interpreting several names in
>>> > the listing as one composite name.
>>> >
>>> > This can cause the Hyperbole action key to perform the wrong action,
>>> > like visitng the wrong file or directory, or not acting at all.

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.