GNU bug report logs - #45333
complex command history should not save optional nil parameters

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45333 in the body.
You can then email your comments to 45333 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: novim <laszlomail <at> protonmail.com>
To: "bug-gnu-emacs <at> gnu.org" <bug-gnu-emacs <at> gnu.org>
Subject: complex command history should not save optional nil parameters
Date: Sun, 20 Dec 2020 09:01:27 +0000

[Message part 1 (text/plain, inline)]

I often use complex command history and I often copy commands from it to my notes. For some commands the expression saved in this history is more complex than necessary.

Here's an example with a simple command:

I do a query replace then go to complex command history which shows:

(query-replace-regexp "a" "b" nil nil nil nil nil)

If I save this command then I delete the nils, because they are unnecessary there and make the expression noisy in my notes, so I delete them manually:

(query-replace-regexp "a" "b")

This is the same command, since the rest of the params are optional:

(query-replace FROM-STRING TO-STRING &optional DELIMITED START END BACKWARD REGION-NONCONTIGUOUS-P)

When commands are saved in the complex history they should be saved in a format which skips the rest of the arguments if they are optional and nil.

[Message part 2 (text/html, inline)]

Message #8 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: novim <laszlomail <at> protonmail.com>
Cc: 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 07 Jun 2022 13:57:36 +0200

novim <laszlomail <at> protonmail.com> writes:

> I  often use complex command history and I often copy commands from it to my
> notes. For some commands the expression saved in this history is more complex than
> necessary.
>
> Here's an example with a simple command:
>
> I do a query replace then go to complex command history which shows:

(I'm going through old bug reports that unfortunately weren't resolved
at the time.)

I'm unfamiliar with the term "complex command history" -- how do you see
this history?  (`M-x apropos RET complex.*command RET' didn't tell me
anything more.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #13 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 07 Jun 2022 16:15:47 +0300

> Cc: 45333 <at> debbugs.gnu.org
> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Tue, 07 Jun 2022 13:57:36 +0200
> 
> novim <laszlomail <at> protonmail.com> writes:
> 
> > I  often use complex command history and I often copy commands from it to my
> > notes. For some commands the expression saved in this history is more complex than
> > necessary.
> >
> > Here's an example with a simple command:
> >
> > I do a query replace then go to complex command history which shows:
> 
> (I'm going through old bug reports that unfortunately weren't resolved
> at the time.)
> 
> I'm unfamiliar with the term "complex command history" -- how do you see
> this history?

I think like this:

  M-x repeat-complex-command RET
  M-n
  M-n
  ...

Message #16 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: larsi <at> gnus.org, laszlomail <at> protonmail.com
Cc: 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 07 Jun 2022 16:20:15 +0300

> Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
> Date: Tue, 07 Jun 2022 16:15:47 +0300
> From: Eli Zaretskii <eliz <at> gnu.org>
> 
>   M-x repeat-complex-command RET
>   M-n
>   M-n
>   ...

Sorry, I meant M-p, of course...

Message #19 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Wed, 08 Jun 2022 13:43:50 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> > I  often use complex command history and I often copy commands from it to my
>> > notes. For some commands the expression saved in this history is
>> > more complex than
>> > necessary.
>> >
>> > Here's an example with a simple command:
>> >
>> > I do a query replace then go to complex command history which shows:

[...]

>> I'm unfamiliar with the term "complex command history" -- how do you see
>> this history?
>
> I think like this:
>
>   M-x repeat-complex-command RET
>   M-n
>   M-n
>   ...

Ah; thanks.  Yes, we land things like

(replace-string "foo" "bar" nil nil nil nil nil)

in `command-history' -- where all those nils are optional.  We could
pretty easily filter those out by looking at the signature of the
function and peeling off trailing nils, I think?  Would there be any
disadvantages to doing so?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #22 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Wed, 08 Jun 2022 16:57:11 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: laszlomail <at> protonmail.com,  45333 <at> debbugs.gnu.org
> Date: Wed, 08 Jun 2022 13:43:50 +0200
> 
> >   M-x repeat-complex-command RET
> >   M-n
> >   M-n
> >   ...
> 
> Ah; thanks.  Yes, we land things like
> 
> (replace-string "foo" "bar" nil nil nil nil nil)
> 
> in `command-history' -- where all those nils are optional.  We could
> pretty easily filter those out by looking at the signature of the
> function and peeling off trailing nils, I think?  Would there be any
> disadvantages to doing so?

I don't see any advantages, FWIW.

Message #25 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 09 Jun 2022 12:22:52 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> (replace-string "foo" "bar" nil nil nil nil nil)
>> 
>> in `command-history' -- where all those nils are optional.  We could
>> pretty easily filter those out by looking at the signature of the
>> function and peeling off trailing nils, I think?  Would there be any
>> disadvantages to doing so?
>
> I don't see any advantages, FWIW.

The advantage would be that the history is easier to navigate, because
you don't have to stare at all those nils when going through the list
looking for the command to execute.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #28 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 09 Jun 2022 13:33:14 +0300

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Cc: laszlomail <at> protonmail.com,  45333 <at> debbugs.gnu.org
> Date: Thu, 09 Jun 2022 12:22:52 +0200
> 
> Eli Zaretskii <eliz <at> gnu.org> writes:
> 
> >> (replace-string "foo" "bar" nil nil nil nil nil)
> >> 
> >> in `command-history' -- where all those nils are optional.  We could
> >> pretty easily filter those out by looking at the signature of the
> >> function and peeling off trailing nils, I think?  Would there be any
> >> disadvantages to doing so?
> >
> > I don't see any advantages, FWIW.
> 
> The advantage would be that the history is easier to navigate, because
> you don't have to stare at all those nils when going through the list
> looking for the command to execute.

OTOH, you get to see all the optional arguments, which is also useful.

It's a minor issue, to be sure.

Message #31 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Eli Zaretskii <eliz <at> gnu.org>
Cc: laszlomail <at> protonmail.com, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Fri, 10 Jun 2022 10:52:59 +0200

Eli Zaretskii <eliz <at> gnu.org> writes:

>> The advantage would be that the history is easier to navigate, because
>> you don't have to stare at all those nils when going through the list
>> looking for the command to execute.
>
> OTOH, you get to see all the optional arguments, which is also useful.
>
> It's a minor issue, to be sure.

Very minor, but on the whole, I think it makes for a slightly better
`command-history', so I've now pushed this to Emacs 29.  We'll see
whether there's any problems in this area...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #36 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 16:16:58 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Ah; thanks.  Yes, we land things like
>
> (replace-string "foo" "bar" nil nil nil nil nil)
>
> in `command-history' -- where all those nils are optional.  We could
> pretty easily filter those out by looking at the signature of the
> function and peeling off trailing nils, I think?  Would there be any
> disadvantages to doing so?

Note that there are only such `nil`s when the evaluation of the
interactive spec returned them.  It's somewhat consistent if the command
history includes the argument list exactly as returned by `interactive'.

The more general problem is to provide interactive forms that work
nicely with `repeat-complex-command' - something programmers are often
not aware of when writing commands.

Anyway, trying of prettify away those `nil`s seems like overkill to me.
In theory even an optional `nil' can be significant.

Michael.

Message #39 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Michael Heerdegen <michael_heerdegen <at> web.de>, Lars Ingebrigtsen
 <larsi <at> gnus.org>
Cc: "laszlomail <at> protonmail.com" <laszlomail <at> protonmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>, "45333 <at> debbugs.gnu.org" <45333 <at> debbugs.gnu.org>
Subject: RE: [External] : bug#45333: complex command history should not save
 optional nil parameters
Date: Tue, 5 Jul 2022 14:53:50 +0000

> > (replace-string "foo" "bar" nil nil nil nil nil)
> >
> > in `command-history' -- where all those nils are optional.  We could
> > pretty easily filter those out by looking at the signature of the
> > function and peeling off trailing nils, I think?  Would there be any
> > disadvantages to doing so?
> 
> Note that there are only such `nil`s when the evaluation of the
> interactive spec returned them.  It's somewhat consistent if the
> command history includes the argument list exactly as returned by
> `interactive'...
> 
> Anyway, trying of prettify away those `nil`s seems like overkill to me.
> In theory even an optional `nil' can be significant.

+1.

Among other uses, `repeat-complex-command' shows
a user just what args were passed to the function.

Message #42 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 18:41:23 +0200

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> Anyway, trying of prettify away those `nil`s seems like overkill to me.
> In theory even an optional `nil' can be significant.

They can be if the signature is a &rest that parses the parameters by
hand, but are there other cases where a nil optional argument is
significant?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #45 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 20:45:11 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Michael Heerdegen <michael_heerdegen <at> web.de> writes:
>
> > Anyway, trying of prettify away those `nil`s seems like overkill to me.
> > In theory even an optional `nil' can be significant.
>
> They can be if the signature is a &rest that parses the parameters by
> hand, but are there other cases where a nil optional argument is
> significant?

`cl-defun's?  Also advises have access to the actually passed parameter
list.  Dunno if there are other cases.

Michael.

Message #48 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 20:48:46 +0200

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> `cl-defun's?

I've always assumed that those were really &rest, but I haven't actually
checked.

> Also advises have access to the actually passed parameter
> list.  Dunno if there are other cases.

Those are also detectable (and are probably &rest under the hood, I
guess?).

The proposal is to only peel the nils of the end if they are indeed
&optional.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #51 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 21:06:59 +0200

Lars,

everything depends on how you define "signature" and where get it from.
Does adding an :around advice change the signature of a command?

Michael.

Message #54 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 21:09:50 +0200

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> Lars,
>
> everything depends on how you define "signature" and where get it from.
> Does adding an :around advice change the signature of a command?

This isn't a language design issue.  We're talking about making the
history prettier where it's 100% safe to do so, and nothing more than
that.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #57 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Michael Heerdegen <michael_heerdegen <at> web.de>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 21:43:13 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> > everything depends on how you define "signature" and where get it from.
> > Does adding an :around advice change the signature of a command?
>
> This isn't a language design issue.  We're talking about making the
> history prettier where it's 100% safe to do so, and nothing more than
> that.

My question is if 100% safe cases exist?  For example:

#+begin_src emacs-lisp
(defun test (&optional x)
  (interactive)
  x)

(advice-add 'test :around (defun test-ad (f &rest args) args))
#+end_src

the signature of `test' is still `(test &optional X)` as reported by C-h
f although the function is able to distinguish a specified nil from a
not specified optional argument.

What you suggest might still be doable by obtaining the effective
signature differently... but I still don't like the idea.

Michael.

Message #60 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 21:50:25 +0200

Michael Heerdegen <michael_heerdegen <at> web.de> writes:

> (advice-add 'test :around (defun test-ad (f &rest args) args))
> #+end_src
>
> the signature of `test' is still `(test &optional X)` as reported by C-h
> f although the function is able to distinguish a specified nil from a
> not specified optional argument.

`C-h f' lies, of course.  :-)  That is, `help-function-arglist'.  (Which
reminds me -- I've always found it odd that that function lives in
help.el, and is in Emacs Lisp instead of C.  It is, after all, used by
eldoc etc, and it'd probably be nice if it were more efficient.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #63 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>, Michael Heerdegen
 <michael_heerdegen <at> web.de>
Cc: "laszlomail <at> protonmail.com" <laszlomail <at> protonmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>, "45333 <at> debbugs.gnu.org" <45333 <at> debbugs.gnu.org>
Subject: RE: [External] : bug#45333: complex command history should not save
 optional nil parameters
Date: Tue, 5 Jul 2022 20:00:03 +0000

 
> > (advice-add 'test :around (defun test-ad (f &rest args) args))
> > #+end_src
> >
> > the signature of `test' is still `(test &optional X)` as reported by
> C-h
> > f although the function is able to distinguish a specified nil from a
> > not specified optional argument.
> 
> `C-h f' lies, of course.  :-)  That is, `help-function-arglist'.
> (Which reminds me -- I've always found it odd that that function lives in
> help.el, and is in Emacs Lisp instead of C.  It is, after all, used by
> eldoc etc, and it'd probably be nice if it were more efficient.)

Please don't move it to C.
___

Advice isn't supported very well by Help (or vice versa).
(Perhaps "integrated with" instead of "supported by".)

The old advice system used to let you add to or replace
the doc-string text directly, so you saw it directly in
`*Help*'.  The new advice system tells you the function
is advised and gives you a link to the doc provided by
the advice.
___

And yeah, the Lisp calling sequence / signature is also
not supported/integrated.

Message #66 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>, Michael Heerdegen
 <michael_heerdegen <at> web.de>
Cc: "laszlomail <at> protonmail.com" <laszlomail <at> protonmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>, "45333 <at> debbugs.gnu.org" <45333 <at> debbugs.gnu.org>
Subject: RE: [External] : bug#45333: complex command history should not save
 optional nil parameters
Date: Tue, 5 Jul 2022 20:15:10 +0000

> The proposal is to only peel the nils of the end if
> they are indeed &optional.

But why?  You're already looking at the doc, which
tells you which ones are optional.  What's gained?

Anyway, that's not what the OP use case was.  This
enhancement request is for someone saving a Lisp
command call to a file of _notes_.

  If I save this command then I delete the nils,
  because they are unnecessary there and make the
  expression noisy in my notes, so I delete them
  manually:
  (query-replace-regexp "a" "b")

Does that sound like a frequent or important need?

Seeing what `repeat-extended-command' is going to
reinvoke by default is important interactively.
Including for editing.  Better to show all of the
args, IMO.

The save-to-my-notes use case is not what the UI
of `repeat-extended-command' is, or should be,
designed for.

Just one opinion.

Message #69 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Michael Heerdegen <michael_heerdegen <at> web.de>
Cc: laszlomail <at> protonmail.com, Eli Zaretskii <eliz <at> gnu.org>,
 45333 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Tue, 05 Jul 2022 22:49:57 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> (Which reminds me -- I've always found it odd that that function lives
> in help.el, and is in Emacs Lisp instead of C.  It is, after all, used
> by eldoc etc, and it'd probably be nice if it were more efficient.)

So I started taking a stab at this to see whether there's any gains to
be had, and I've now basically rewritten it in C.  And, yes, I think it
makes sense, because we can avoid at least some garbage generating
functions without contorting ourselves too much.

One thing, though -- I'm not sure how to do the oclosure stuff from C,
so I've added Stefan to the CCs.  It's this bit:

  ;; Advice wrappers have "catch all" args, so fetch the actual underlying
  ;; function to find the real arguments.
  (setq def (advice--cd*r def))

This is an oclosure these days.

(While poking at this, if we want to do the proposed history shortening,
we don't need to look at the arglist at all -- we can just call
Ffunc_arity, which will return Qmany for &rest, so this
help-function-arglist rewrite is totally irrelevant for this bug
report.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #72 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Drew Adams <drew.adams <at> oracle.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>, Michael Heerdegen
 <michael_heerdegen <at> web.de>
Cc: "laszlomail <at> protonmail.com" <laszlomail <at> protonmail.com>,
 Eli Zaretskii <eliz <at> gnu.org>, "45333 <at> debbugs.gnu.org" <45333 <at> debbugs.gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: RE: [External] : bug#45333: complex command history should not save
 optional nil parameters
Date: Tue, 5 Jul 2022 22:37:28 +0000

> > (Which reminds me -- I've always found it odd that that function
> > lives in help.el, and is in Emacs Lisp instead of C.

I don't care where it lives, but why should you
even consider moving it to C?  Is this to solve
some performance problem?  Leave it in Lisp, so
users can modify or otherwise improve it.

More should be moved from C to Lisp, not vice versa.

> > It is, after all, used by eldoc etc,

So move it somewhere else in Lisp.

> and it'd probably be nice if it were more efficient.)

That's never been a good enough reason to move something
to C.  Not unless there's a real performance problem
that needs to be solved by doing so (i.e., can't be
solved some other, better way).

> So I started taking a stab at this to see whether there's 
> any gains to be had, and I've now basically rewritten it in C.

Gag.

> And, yes, I think it makes sense, because we can avoid
> at least some garbage generating functions without
> contorting ourselves too much.

Stats?  The problem is ________.  ?

> this help-function-arglist rewrite is totally irrelevant
> for this bug report.

For what problem do you think it _is_ relevant?  Why?

Message #75 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Wed, 06 Jul 2022 14:41:19 -0400

>> (Which reminds me -- I've always found it odd that that function lives
>> in help.el, and is in Emacs Lisp instead of C.  It is, after all, used
>> by eldoc etc, and it'd probably be nice if it were more efficient.)
> So I started taking a stab at this to see whether there's any gains to
> be had, and I've now basically rewritten it in C.

IIUC you're talking about `help-function-arglist`?

> And, yes, I think it makes sense, because we can avoid at least some
> garbage generating functions without contorting ourselves too much.

I can't think of any reason we'd ever want to use that function anywhere
near performance critical code, so I'd rather keep it in ELisp if
possible.  `help.el` may not be the best choice for it, admittely.
Over the course of my hacking on OClosures, I had moved that code (and
the accompanying fundoc extraction/addition) to `subr.el` where I think
it might make more sense (together with a bit of renaming to take it out
of the `help-` namespace).

> One thing, though -- I'm not sure how to do the oclosure stuff from C,
> so I've added Stefan to the CCs.  It's this bit:
>
>   ;; Advice wrappers have "catch all" args, so fetch the actual underlying
>   ;; function to find the real arguments.
>   (setq def (advice--cd*r def))

OClosure or not doesn't make much of a difference here, I think (the
previous version of the code would also be quite hideous in C).
Why not just call `advice--cd*r`?


        Stefan

Message #78 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 09:29:22 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> I can't think of any reason we'd ever want to use that function anywhere
> near performance critical code, so I'd rather keep it in ELisp if
> possible.

I think it'd be nice to have eldoc generate less garbage, though.

> OClosure or not doesn't make much of a difference here, I think (the
> previous version of the code would also be quite hideous in C).
> Why not just call `advice--cd*r`?

Sure, that's possible, but just seems a bit backwards.

Anyway, looking at the functionality in this area, things just seem a
bit messy overall, and I wonder whether there's any scope here for
making things better.

For instance, this code:

(help-function-arglist 'car t)
=> (list)

What it does is call `documentation', find the "(fn LIST)" at the end,
do a `downcase' and `read-from-string' which seems to suggest that
upcasing the arguments in the first place before putting them into
etc/DOC was the wrong thing to do.  Not only do we lose the
capitalisation -- if the argument was really `List', that's lost.
(Perhaps fortunately, some might say -- we don't really use case much in
symbols in Emacs Lisp.)

Moreover, looking at the code in make-docfile, it seems to be upcasing
wrong, so if the argument had been `lïst', we'd put (fn LïST) into
etc/DOC.  (And again, it doesn't really matter that much, because we
don't really use anything but ASCII.)

And even more moreover, if the doc string had (fn [FOO]),
`help-function-arglist' would just ignore that and return (arg1), but if
it has (fn foo bar zot), it'll return (foo bar zot).

(Some of the same issues are with doc strings/arg lists in .elc files.)

So two issues:

1) If we have an advertised calling convention, the original arg list can't
be recoved.  2) We can't recover the original case of the arguments at
all.  (Except for lambda/closure forms.)

For 2), I think we could stop upcasing arguments and adjust callers --
but old .elc files would still have upcased arguments until recompiled.

For 1), we could also stash the actual arglist in addition to the
advertised calling convention into .elc/DOC (i.e., two (fn ...) forms,
for instance).

We could then have

  (function-arglist FUNC &optional ADVERTISED)

that would return either form.

(And there's also `advertised-signature-table', which perhaps we could
then get rid of?)

Of course, I don't really know whether it's worth doing anything about
any of this -- but it just seems a bit messy at the moment, and not very
satisfactory.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #81 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 11:45:05 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Of course, I don't really know whether it's worth doing anything about
> any of this -- but it just seems a bit messy at the moment, and not very
> satisfactory.

If we wanted to get more ambitious here, we could get rid of the (fn
...) nonsense altogether.

That is, we could store the arg list explicitly, and we could introduce
a (declare (calling-convention ...)) form.

This would lead to increased memory usage (as we'd probably end up with
the arglists being stored in the symbol plist or the like), but it
wouldn't be very noticeable, I think.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #84 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>,
 Peter Dean <laszlomail <at> protonmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 7 Jul 2022 12:00:40 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Of course, I don't really know whether it's worth doing anything about
> any of this -- but it just seems a bit messy at the moment, and not very
> satisfactory.

Are we still planning to get rid of DOC at some point?

Message #87 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>,
 Peter Dean <laszlomail <at> protonmail.com>, Eli Zaretskii <eliz <at> gnu.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 12:06:31 +0200

Stefan Kangas <stefan <at> marxist.se> writes:

> Are we still planning to get rid of DOC at some point?

That would be my preference, but I think we're a ways off -- we'd need
to figure out how to compile loaddefs files, and there's still the issue
of doc strings from DEFUNs in C files.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #90 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 09:54:06 -0400

> For 2), I think we could stop upcasing arguments and adjust callers --
> but old .elc files would still have upcased arguments until recompiled.

`C-h o` wants them upcased, eldoc wants them downcased.
So far we have prioritized `C-h o`.  Maybe eldoc shouldn't downcase :-)

> For 1), we could also stash the actual arglist in addition to the
> advertised calling convention into .elc/DOC (i.e., two (fn ...) forms,
> for instance).

Why/when do we need the non-advertized arglist?


        Stefan

Message #93 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 09:58:29 -0400

>> OClosure or not doesn't make much of a difference here, I think (the
>> previous version of the code would also be quite hideous in C).
>> Why not just call `advice--cd*r`?
> Sure, that's possible, but just seems a bit backwards.

Don't know about backward, but if you don't like that dependency, you
could break it using a hook, like a `function-unwrap-function` whose
default value is `identity` and which `nadvice.el` could change to
`advice-cd*r`.


        Stefan

Message #96 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 16:10:58 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

>> For 1), we could also stash the actual arglist in addition to the
>> advertised calling convention into .elc/DOC (i.e., two (fn ...) forms,
>> for instance).
>
> Why/when do we need the non-advertized arglist?

Because (foo [bar zot] ...) isn't very helpful when doing low level
stuff.  So `help-function-arglist' returns a non-name-preserving list in
that case.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #99 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 10:49:20 -0400

Lars Ingebrigtsen [2022-07-07 16:10:58] wrote:
> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>>> For 1), we could also stash the actual arglist in addition to the
>>> advertised calling convention into .elc/DOC (i.e., two (fn ...) forms,
>>> for instance).
>> Why/when do we need the non-advertized arglist?
> Because (foo [bar zot] ...) isn't very helpful when doing low level
> stuff.

Sorry, both (foo [bar zot] ...) and "doing low level stuff" sound a bit
too hypothetical for me to get a sense of what we're talking about.

More specifically I don't know of any use of advertized signature that
presents the args with a [...] notation, and I can't remember ever
needing a pretty arglist when working on "low-level stuff".


        Stefan

Message #102 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 16:52:05 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> Sorry, both (foo [bar zot] ...) and "doing low level stuff" sound a bit
> too hypothetical for me to get a sense of what we're talking about.
>
> More specifically I don't know of any use of advertized signature that
> presents the args with a [...] notation, 

---
setq is a special form in src/eval.c.

(setq [SYM VAL]...)
---

> and I can't remember ever needing a pretty arglist when working on
> "low-level stuff".

You don't need the pretty arglist, but it can be informative.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #105 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 11:05:02 -0400

Lars Ingebrigtsen [2022-07-07 16:52:05] wrote:
> Stefan Monnier <monnier <at> iro.umontreal.ca> writes:
>> Sorry, both (foo [bar zot] ...) and "doing low level stuff" sound a bit
>> too hypothetical for me to get a sense of what we're talking about.
>>
>> More specifically I don't know of any use of advertized signature that
>> presents the args with a [...] notation, 
>
> ---
> setq is a special form in src/eval.c.
>
> (setq [SYM VAL]...)
> ---

Ah, so you're not talking about advertised-calling-convention!?

>> and I can't remember ever needing a pretty arglist when working on
>> "low-level stuff".
> You don't need the pretty arglist, but it can be informative.

Usually in those cases I look at the source, which is even
more informative.

Clearly, there is some info we "throw away" or make harder to get to,
but to the extent that you can get it back by visiting the source code,
I don't see a good reason to spend much effort trying to cater to those
very rare needs.

Basically, if we want to go there, we start to want to distinguish
between:
- the "final and pretty" advertized arglist
- the "final and pretty" arglist but disregarding advertised-calling-convention
- the original arglist rather than the one stashed in the docstring.
- the raw arglist seen by the interpreter.
I doubt very many chunks of code will know which level to choose (and
even fewer will have a good justification for that choice).


        Stefan

Message #108 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 17:21:26 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> Ah, so you're not talking about advertised-calling-convention!?

No, I'm talking about the...  advertised calling convention.

> Basically, if we want to go there, we start to want to distinguish
> between:
> - the "final and pretty" advertized arglist
> - the "final and pretty" arglist but disregarding advertised-calling-convention
> - the original arglist rather than the one stashed in the docstring.
> - the raw arglist seen by the interpreter.
> I doubt very many chunks of code will know which level to choose (and
> even fewer will have a good justification for that choice).

I think there's just two levels that are interesting.  We have the er
advertised calling convention (whether from
advertised-calling-convention or "(fn ...)"), and then the real, actual
for sure arglist.

The former is what `C-h f' and eldoc wants, and particularly for the
latter, it's a shame that it's so hard to get at -- i.e., opening .elc
files and chopping up doc strings etc.

The actual for sure arglist is needed when you want to redefine
functions and the like, and need to actually retain the interface in all
details, and today you have to look at the source code to do that, which
a self-documenting editor shouldn't need.  (But it's pretty rare to need
this, so it doesn't matter that it's slow.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #111 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 11:30:56 -0400

> I think there's just two levels that are interesting.  We have the er
> advertised calling convention (whether from
> advertised-calling-convention or "(fn ...)"), and then the real, actual
> for sure arglist.

The "actual for sure arglist" is the one that sometimes comes with
bogus names.

> The actual for sure arglist is needed when you want to redefine
> functions and the like, and need to actually retain the interface in all
> details, and today you have to look at the source code to do that, which
> a self-documenting editor shouldn't need.  (But it's pretty rare to need
> this, so it doesn't matter that it's slow.)

In my experience once you get to places where you need to care about
"the actual for sure arglist" the real needs depend a lot on the
specifics, so it's not even clear what would be a good answer that works
across the board (e.g. in the presence of CL's &key arguments).
The answer you get from `help-function-arglist` without the
`preserve-names` arg should be good enough for most cases.

I do think the current situation kinda sucks because stashing arglists
inside docstrings is a PITA, but it does have its advantages and
I haven't seen a good alternative yet.


        Stefan

Message #114 received at 45333 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: Michael Heerdegen <michael_heerdegen <at> web.de>, laszlomail <at> protonmail.com,
 Eli Zaretskii <eliz <at> gnu.org>, 45333 <at> debbugs.gnu.org
Subject: Re: bug#45333: complex command history should not save optional nil
 parameters
Date: Thu, 07 Jul 2022 20:00:02 +0200

Stefan Monnier <monnier <at> iro.umontreal.ca> writes:

> The "actual for sure arglist" is the one that sometimes comes with
> bogus names.

Yup.

> In my experience once you get to places where you need to care about
> "the actual for sure arglist" the real needs depend a lot on the
> specifics, so it's not even clear what would be a good answer that works
> across the board (e.g. in the presence of CL's &key arguments).
> The answer you get from `help-function-arglist` without the
> `preserve-names` arg should be good enough for most cases.

That's basically just a long-winded expansion of `func-arity'.

> I do think the current situation kinda sucks because stashing arglists
> inside docstrings is a PITA, but it does have its advantages and
> I haven't seen a good alternative yet.

Well, one easy thing we can do is (pseudo-)deprecate stuff like this:

(defun desktop-read (&optional dirname ask)
  "Read and process the desktop file in directory DIRNAME.
[...]
\n(fn DIRNAME)"

and use (declare (advertised-calling-convention ...)) for those -- then
we'd have both the real and the advertised calling convention for all
those functions.  We can do the same for "(fn [FOO BAR] ..."), but stash
it somewhere else than advertised-signature-table since we can't use it
the same way for warnings.

That fixes either 1) or 2); I forget which way I was counting.

The other issue was to make this stuff faster --
`advertised-calling-convention' puts stuff into a hash table, and that's
fine.  We can do the same with the real arglist, or we can put it in the
symbol plist.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.