GNU bug report logs - #27748
26.0.50; doc strings should be in DOC file

Previous Next

Package: emacs;

Reported by: Ken Raeburn <raeburn <at> raeburn.org>

Date: Tue, 18 Jul 2017 06:48:02 UTC

Severity: normal

Tags: patch, wontfix

Found in version 26.0.50

Done: Lars Ingebrigtsen <larsi <at> gnus.org>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 27748 in the body.
You can then email your comments to 27748 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Tue, 18 Jul 2017 06:48:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Ken Raeburn <raeburn <at> raeburn.org>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs <at> gnu.org. (Tue, 18 Jul 2017 06:48:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> raeburn.org>
To: Bug-Gnu-Emacs <bug-gnu-emacs <at> gnu.org>
Subject: 26.0.50; doc strings should be in DOC file
Date: Tue, 18 Jul 2017 02:47:09 -0400
There are a bunch of doc strings from the preloaded Lisp files that do not wind up in the DOC file.  Presumably this means they wind up in the emacs executable image itself, saved away as extra string objects that GC needs to track.  (In the big-elc-file work Stefan started and I'm experimenting with, these doc strings wind up in the dumped Lisp environment file, and need to be parsed and saved away at load time.)

1. defcustom doc strings from files compiled with lexical binding.

   For example, files.el (lexical bindings) defines
   delete-auto-save-files but it doesn't show up in the DOC file;
   files.elc starts with an initial byte-code blob which includes the
   symbol delete-auto-save-files and its doc string in the constants
   array.

   On the other hand, custom.el (dynamic bindings) declares
   custom-theme-directory, the .elc file dumps out the doc string in a
   #@... block before a separate top-level call to
   custom-declare-variable, and since this is what make-docfile looks
   for, the doc string winds up in the DOC file.

2. In isearch, CL macro expansion results in symbols like
   isearch--state-forward--cmacro having function definitions that
   include the two doc strings "Access slot \"forward\" of
   `(isearch--state (:constructor nil) (:copier nil) (:constructor
   isearch--get-state (&aux (string isearch-string) (message
   isearch-message) (point (point)) (success isearch-success) (forward
   isearch-forward) (other-end isearch-other-end) (word
   isearch-regexp-function) (error isearch-error) (wrapped
   isearch-wrapped) (barrier isearch-barrier) (case-fold-search
   isearch-case-fold-search) (pop-fun (if isearch-push-state-function
   (funcall isearch-push-state-function))))))' struct CL-X." and
   "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC (with
   "\n\n(fn CL-X)" appended) as the documentation for the function
   isearch--state-forward.

   It appears that, as far as the Emacs help system is concerned,
   isearch--state-*--cmacro functions are undocumented.

   The --cmacro functions generate cl-block forms that include the
   original doc string, since it's treated as part of the body, but it's
   not clear to me whether it has any use there at all, or if it could
   just be discarded, or perhaps looked up via the non --cmacro symbols
   at run time if it is of use.

3. Undocumented functions, strange as it sounds... in files.el, function
   file-name-non-special is defined with no documentation.  In
   files.elc, the bytecode object constructed has a doc string "\n\n(fn
   OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
   separate string that make-docfile can pick up.

To “reproduce”…

Delete or “chmod 0” the DOC file before starting Emacs.  Use ielm to look at the symbol property lists of delete-auto-save-files and custom-theme-directory; the former has a string for its variable-documentation property, and the latter has a number.  Look at the function definitions of isearch--state-forward--cmacro and file-name-non-special and see the doc strings in them.  Examine the .elc files and DOC.






In GNU Emacs 26.0.50 (build 2, x86_64-apple-darwin15.6.0, NS appkit-1404.47 Version 10.11.6 (Build 15G1217))
 of 2017-07-18 built on bang.local
Repository revision: 0083123499cc29e301c197218d3809b225675e57
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
Loading ~/lib/elisp/sue.elc...done
Loading desktop...done
Warning: desktop file appears to be in use by PID 83193.
Using it may cause conflicts.  Use it anyway? (y or n) n
Desktop file in use; not loaded.
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured features:
RSVG IMAGEMAGICK DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  desktop-save-mode: t
  global-hi-lock-mode: t
  hi-lock-mode: t
  which-function-mode: t
  icomplete-mode: t
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr warnings emacsbug message subr-x puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader add-log desktop frameset cus-start
cus-load kr-defs hi-lock which-func imenu icomplete iso-transl
smart-quotes easy-mmode tramp tramp-compat tramp-loaddefs trampver shell
pcomplete comint ansi-color ring parse-time format-spec advice cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs server time sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils finder-inf package easymenu epg-config
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib time-date tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 261240 12410)
 (symbols 48 26239 2)
 (miscs 40 57 285)
 (strings 32 47062 2197)
 (string-bytes 1 1448035)
 (vectors 16 42658)
 (vector-slots 8 784892 12266)
 (floats 8 70 90)
 (intervals 56 241 0)
 (buffers 992 12))





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sun, 06 Aug 2017 00:08:02 GMT) Full text and rfc822 format available.

Message #8 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Ken Raeburn <raeburn <at> raeburn.org>
Cc: 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sat, 05 Aug 2017 20:09:16 -0400
[Message part 1 (text/plain, inline)]
Ken Raeburn <raeburn <at> raeburn.org> writes:

>
> 1. defcustom doc strings from files compiled with lexical binding.
>
>    For example, files.el (lexical bindings) defines
>    delete-auto-save-files but it doesn't show up in the DOC file;
>    files.elc starts with an initial byte-code blob which includes the
>    symbol delete-auto-save-files and its doc string in the constants
>    array.

Actually, it's not only about lexical binding, the following file:

    ;;; -*- lexical-binding: nil -*-

    (defcustom custom-foo nil
      "a custom variable"
      :type 'boolean
      :group 'foo-group)

    ;; (defun foo ()
    ;;   t)

    (defcustom custom-bar nil
      "another custom variable"
      :type 'boolean
      :group 'foo-group)

produces (with prologue removed, and reformatted for readability)

    (byte-code "\300\301\302\303\304\305\306\307&\210\300\310\302\311\304\305\306\307&\207"
               [custom-declare-variable custom-foo nil "a custom variable" :type boolean :group foo-group
                                        custom-bar "another custom variable"]
               8)

Uncommenting the (defun foo...) produces:

    #@19 a custom variable
    (custom-declare-variable 'custom-foo nil '(#$ . 411) :type 'boolean :group 'foo-group)
    (defalias 'foo #[nil "\300\207" [t] 1])
    #@25 another custom variable
    (custom-declare-variable 'custom-bar nil '(#$ . 562) :type 'boolean :group 'foo-group)

Then changing to lexical binding produces:

    (byte-code "\300\301\302\303\304DD\305\306\307\310\311&\207"
               [custom-declare-variable
                custom-foo funcall function
                #[0 "\300\207" [nil] 1] "a custom variable"
                :type boolean :group foo-group]
               8)
    (defalias 'foo #[0 "\300\207" [t] 1])
    (byte-code "\300\301\302\303\304DD\305\306\307\310\311&\207"
               [custom-declare-variable
                custom-bar funcall function
                #[0 "\300\207" [nil] 1]
                "another custom variable" :type boolean :group foo-group]
               8)

As far as I can tell, the problem is that the
byte-compile-dynamic-docstrings feature (that's the #@19 thing) relies
on `byte-compile-out-toplevel' to decompile "trivial" functions back
into source code.  So having lexical binding set, or 2 defcustoms in a
row produces "non-trivial" code which is not decompiled, and therefore
not recognized in byte-compile-output-file-form as something which
should be used with byte-compile-output-docform.

    (defun byte-compile-out-toplevel (&optional for-effect output-type)
      ...
      ;; Decompile trivial functions:
      ...
        (cond
         ;; #### This should be split out into byte-compile-nontrivial-function-p.
         ((or (eq output-type 'lambda)
          (nthcdr (if (eq output-type 'file) 50 8) byte-compile-output)
          ...
            (while
                    (cond
                     ((memq (car (car rest)) '(byte-varref byte-constant))
                     ...
                     ((and maycall
                           ;; Allow a funcall if at most one atom follows it.
                      ...
                      (setq maycall nil)	; Only allow one real function call.
                      ...
                      (or (eq output-type 'file)
                          (not (delq nil (mapcar 'consp (cdr (car body))))))))
            ...
        (list 'byte-code (byte-compile-lapcode byte-compile-output)
              byte-compile-vector byte-compile-maxdepth)))
         ;; it's a trivial function
         ((cdr body) (cons 'progn (nreverse body)))
         ((car body))))

    (defun byte-compile-output-file-form (form)
      ;; Write the given form to the output buffer, being careful of docstrings
      ;; in defvar, defvaralias, defconst, autoload and
      ;; custom-declare-variable because make-docfile is so amazingly stupid.
      ...
        (if (and (memq (car-safe form) '(defvar defvaralias defconst
                                          autoload custom-declare-variable))
                 (stringp (nth 3 form)))
            (byte-compile-output-docform nil nil '("\n(" 3 ")") form nil
                                         (memq (car form)
                                               '(defvaralias autoload
                                                  custom-declare-variable)))
          (princ "\n" byte-compile--outbuffer)
          (prin1 form byte-compile--outbuffer)
          nil)))

The following patch prevents custom-declare-variable from being compiled
and lets the docstrings get printed properly.  Probably needs a bit more
refinement though.

[0001-lisp-custom.el-custom-declare-variable-Don-t-compile.patch (text/x-diff, inline)]
From 4cb45936966de76f91b95971c886599a24361c5b Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs <at> gmail.com>
Date: Sat, 5 Aug 2017 20:02:19 -0400
Subject: [PATCH] * lisp/custom.el (custom-declare-variable): Don't compile
 (Bug#27748).

---
 lisp/custom.el | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lisp/custom.el b/lisp/custom.el
index ecfa34db5b..5876d3fd56 100644
--- a/lisp/custom.el
+++ b/lisp/custom.el
@@ -144,6 +144,9 @@ (defun custom-declare-variable (symbol default doc &rest args)
 DEFAULT is stored as SYMBOL's standard value, in SYMBOL's property
 `standard-value'.  At the same time, SYMBOL's property `force-value' is
 set to nil, as the value is no longer rogue."
+  (declare (compiler-macro
+            (lambda (form)
+              `(eval ',form lexical-binding))))
   (put symbol 'standard-value (purecopy (list default)))
   ;; Maybe this option was rogue in an earlier version.  It no longer is.
   (when (get symbol 'force-value)
-- 
2.11.1


Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Tue, 08 Aug 2017 01:03:01 GMT) Full text and rfc822 format available.

Message #11 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Ken Raeburn <raeburn <at> raeburn.org>
Cc: 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Mon, 07 Aug 2017 21:03:47 -0400
npostavs <at> users.sourceforge.net writes:

> The following patch prevents custom-declare-variable from being compiled
> and lets the docstrings get printed properly.  Probably needs a bit more
> refinement though.

Hmm, this approach might not work at all, it causes a bazillion "free
variable reference" warnings, one for each reference to a defcustom
variable.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sun, 13 Aug 2017 18:04:01 GMT) Full text and rfc822 format available.

Message #14 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Ken Raeburn <raeburn <at> raeburn.org>
Cc: 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sun, 13 Aug 2017 14:04:48 -0400
[Message part 1 (text/plain, inline)]
npostavs <at> users.sourceforge.net writes:

> npostavs <at> users.sourceforge.net writes:
>
>> The following patch prevents custom-declare-variable from being compiled
>> and lets the docstrings get printed properly.  Probably needs a bit more
>> refinement though.
>
> Hmm, this approach might not work at all, it causes a bazillion "free
> variable reference" warnings, one for each reference to a defcustom
> variable.

Okay, here is an alternate approach which decouples the docstring
production from decompilation (note this isn't finished yet, I only
implemented it for defcustom, so applying this patch currently prevents
make-doc from finishing successfully).

[0001-WIP-Produce-dynamic-docstrings-for-bytecode-Bug-2774.patch (text/plain, attachment)]

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sun, 20 Aug 2017 22:04:01 GMT) Full text and rfc822 format available.

Message #17 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Ken Raeburn <raeburn <at> raeburn.org>
Cc: 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sun, 20 Aug 2017 18:05:07 -0400
[Message part 1 (text/plain, inline)]
tags 27748 + patch
quit

Ken Raeburn <raeburn <at> raeburn.org> writes:

> 1. defcustom doc strings from files compiled with lexical binding.
>
>    For example, files.el (lexical bindings) defines
>    delete-auto-save-files but it doesn't show up in the DOC file;
>    files.elc starts with an initial byte-code blob which includes the
>    symbol delete-auto-save-files and its doc string in the constants
>    array.
>
>    On the other hand, custom.el (dynamic bindings) declares
>    custom-theme-directory, the .elc file dumps out the doc string in a
>    #@... block before a separate top-level call to
>    custom-declare-variable, and since this is what make-docfile looks
>    for, the doc string winds up in the DOC file.

With patch 0001 defcustoms which are compiled to bytecode now produce
dynamic docstrings which make-doc can digest (note that I had to change
make-doc a bit for this, but the .elc format remains the same as far as
the Emacs loading it is concerned.  See the commit message for details).

[0001-Produce-dynamic-docstrings-for-bytecode-Bug-27748.patch (text/plain, attachment)]
[Message part 3 (text/plain, inline)]
> 2. In isearch, CL macro expansion results in symbols like
>    isearch--state-forward--cmacro having function definitions that
>    include the two doc strings "Access slot \"forward\" of
>    `(isearch--state (:constructor nil) [...]" and
>    "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC (with
>    "\n\n(fn CL-X)" appended) as the documentation for the function
>    isearch--state-forward.
>
>    It appears that, as far as the Emacs help system is concerned,
>    isearch--state-*--cmacro functions are undocumented.
>
>    The --cmacro functions generate cl-block forms that include the
>    original doc string, since it's treated as part of the body, but it's
>    not clear to me whether it has any use there at all, or if it could
>    just be discarded, or perhaps looked up via the non --cmacro symbols
>    at run time if it is of use.

I think it is just an oversight, since the string was put inside the
cl-block it is not recognized as a docstring at all.  Patch 0002 drops
the function's docstring from compiler-macro and adds a simple
"compiler-macro for inlining `NAME'" instead.

[0002-Drop-docstrings-from-cl-defsubst-produced-inline-bod.patch (text/plain, attachment)]
[Message part 5 (text/plain, inline)]
> 3. Undocumented functions, strange as it sounds... in files.el, function
>    file-name-non-special is defined with no documentation.  In
>    files.elc, the bytecode object constructed has a doc string "\n\n(fn
>    OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
>    separate string that make-docfile can pick up.

Looks fairly trivial to fix, see patch 0003.

[0003-Support-lazy-loading-for-autogenerated-usage-docstri.patch (text/plain, attachment)]

Added tag(s) patch. Request was from npostavs <at> users.sourceforge.net to control <at> debbugs.gnu.org. (Sun, 20 Aug 2017 22:04:02 GMT) Full text and rfc822 format available.

Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Tue, 29 Aug 2017 10:10:01 GMT) Full text and rfc822 format available.

Message #22 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Ken Raeburn <raeburn <at> raeburn.org>
To: npostavs <at> users.sourceforge.net
Cc: 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Tue, 29 Aug 2017 06:09:09 -0400
Sorry it’s taken me a while to get to testing these out…

On Aug 20, 2017, at 18:05, npostavs <at> users.sourceforge.net wrote:
> 
>> 1. defcustom doc strings from files compiled with lexical binding.

> With patch 0001 defcustoms which are compiled to bytecode now produce
> dynamic docstrings which make-doc can digest (note that I had to change
> make-doc a bit for this, but the .elc format remains the same as far as
> the Emacs loading it is concerned.  See the commit message for details).

I think I like the new format.  It’s a little bit bigger, but it may load faster, as we can do one big fseek at the beginning of the file (thus not even loading a lot of those pages) rather than lots of small ones as we go along.

Will this new make-docfile play nicely with files compiled with byte-compile-dynamic, where byte code is mixed in with the usual doc strings?  Or if we decide to make lambdas (which have “(fn…)” doc strings by default but have no names to associate with them in DOC) load their doc strings dynamically from the .elc file?

>> 2. In isearch, CL macro expansion results in symbols like
>>   isearch--state-forward--cmacro having function definitions that
>>   include the two doc strings "Access slot \"forward\" of
>>   `(isearch--state (:constructor nil) [...]" and
>>   "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC (with
>>   "\n\n(fn CL-X)" appended) as the documentation for the function
>>   isearch--state-forward.

> I think it is just an oversight, since the string was put inside the
> cl-block it is not recognized as a docstring at all.  Patch 0002 drops
> the function's docstring from compiler-macro and adds a simple
> "compiler-macro for inlining `NAME'" instead.

I’m seeing the shorter string, *and* it’s stored in the DOC file.

>> 3. Undocumented functions, strange as it sounds... in files.el, function
>>   file-name-non-special is defined with no documentation.  In
>>   files.elc, the bytecode object constructed has a doc string "\n\n(fn
>>   OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
>>   separate string that make-docfile can pick up.
> 
> Looks fairly trivial to fix, see patch 0003.

This one seems to be working well too.

I did a few spot checks looking at what wound up in the DOC file, and checking the ability to load the documentation in Emacs, and  things look good.

Between these three changes, the byte count for lisp/*.elc grew by 2-3%.  The DOC file is about 6% bigger, due to the strings that weren’t being picked up before.  Curiously, the generated emacs executable was just a little bit bigger, less than 1%, but from the numbers returned by (garbage-collect) I think that’s probably freed space.

I tested the changes out against the scratch/raeburn-startup branch as well.  In that case, the saved Lisp environment (with the more costly parsing process than etc/DOC) shrinks by about 5%, which will probably speed up the startup load time a little bit. There are still some doc strings left, but I’ll look into those later.

Thanks for digging into this!

Ken



Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Thu, 31 Aug 2017 00:50:02 GMT) Full text and rfc822 format available.

Message #25 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: npostavs <at> users.sourceforge.net
To: Ken Raeburn <raeburn <at> raeburn.org>
Cc: 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Wed, 30 Aug 2017 20:50:32 -0400
Ken Raeburn <raeburn <at> raeburn.org> writes:

> Sorry it’s taken me a while to get to testing these out…

Hah, no problem.  I confess it's been on my todo list to test out your
scratch/raeburn-startup branch for an even longer while...

> On Aug 20, 2017, at 18:05, npostavs <at> users.sourceforge.net wrote:
>> 
>>> 1. defcustom doc strings from files compiled with lexical binding.
>
>> With patch 0001 defcustoms which are compiled to bytecode now produce
>> dynamic docstrings which make-doc can digest (note that I had to change
>> make-doc a bit for this, but the .elc format remains the same as far as
>> the Emacs loading it is concerned.  See the commit message for details).
>
> I think I like the new format.  It’s a little bit bigger, but it may
> load faster, as we can do one big fseek at the beginning of the file
> (thus not even loading a lot of those pages) rather than lots of small
> ones as we go along.

Indeed, that was my thought too.  I haven't measured anything though.

> Will this new make-docfile play nicely with files compiled with
> byte-compile-dynamic, where byte code is mixed in with the usual doc
> strings?  Or if we decide to make lambdas (which have “(fn…)” doc
> strings by default but have no names to associate with them in DOC)
> load their doc strings dynamically from the .elc file?

Hmm, it will not.  We would have to add a "nameless" type I guess,
something like ^_A^_anonymous docstring here...^_.

I pushed patches [2: bc5d96a0b2] and [3: 160295867d] to master, since
they are pretty straightforward bugfixes.

[2: bc5d96a0b2]: 2017-08-30 20:07:39 -0400
  Drop docstrings from cl-defsubst produced inline bodies (Bug#27748)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=bc5d96a0b2a1dccf7eeeec459e40d21b54c977f4>

[3: 160295867d]: 2017-08-30 20:07:39 -0400
  Support lazy loading for autogenerated usage docstrings too (Bug#27748)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=160295867de98241a16f2ede93da7e825ed4406b  





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Mon, 24 Jun 2019 22:39:02 GMT) Full text and rfc822 format available.

Message #28 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: npostavs <at> users.sourceforge.net
Cc: Ken Raeburn <raeburn <at> raeburn.org>, 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Tue, 25 Jun 2019 00:38:01 +0200
npostavs <at> users.sourceforge.net writes:

> Okay, here is an alternate approach which decouples the docstring
> production from decompilation (note this isn't finished yet, I only
> implemented it for defcustom, so applying this patch currently prevents
> make-doc from finishing successfully).

These strings still aren't in the DOC file, but I'm not sure whether
that's a problem or not -- what are the practical effects of the doc
string of `delete-auto-save-files' not being in that file?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Mon, 24 Jun 2019 23:01:02 GMT) Full text and rfc822 format available.

Message #31 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Ken Raeburn <raeburn <at> raeburn.org>, 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Mon, 24 Jun 2019 19:00:52 -0400
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> npostavs <at> users.sourceforge.net writes:
>
>> Okay, here is an alternate approach which decouples the docstring
>> production from decompilation (note this isn't finished yet, I only
>> implemented it for defcustom, so applying this patch currently prevents
>> make-doc from finishing successfully).
>
> These strings still aren't in the DOC file, but I'm not sure whether
> that's a problem or not -- what are the practical effects of the doc
> string of `delete-auto-save-files' not being in that file?

It's mostly just an optimization thing, so not hugely important (at the
time, Ken was working on a pure Lisp dumping strategy, so shrinking the
preloaded code was important for that, but we've gone with the pdumper
instead).  However, I seem to recall that applying something like this
will make it possible to solve Bug#4845, because the docstring loading
mechanism will no longer be reliant on finding "(defun foo" in the .elc
file.  So that might be nifty.





Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Mon, 10 Aug 2020 15:01:02 GMT) Full text and rfc822 format available.

Message #34 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: Ken Raeburn <raeburn <at> raeburn.org>, 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Mon, 10 Aug 2020 17:00:43 +0200
Noam Postavsky <npostavs <at> gmail.com> writes:

> It's mostly just an optimization thing, so not hugely important (at the
> time, Ken was working on a pure Lisp dumping strategy, so shrinking the
> preloaded code was important for that, but we've gone with the pdumper
> instead).  However, I seem to recall that applying something like this
> will make it possible to solve Bug#4845, because the docstring loading
> mechanism will no longer be reliant on finding "(defun foo" in the .elc
> file.  So that might be nifty.

Hm...  4845 is about leaking uninterned symbols, so that seems
unrelated?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Mon, 10 May 2021 12:10:01 GMT) Full text and rfc822 format available.

Message #37 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: Ken Raeburn <raeburn <at> raeburn.org>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Mon, 10 May 2021 14:09:00 +0200
Noam Postavsky <npostavs <at> gmail.com> writes:

>> These strings still aren't in the DOC file, but I'm not sure whether
>> that's a problem or not -- what are the practical effects of the doc
>> string of `delete-auto-save-files' not being in that file?
>
> It's mostly just an optimization thing, so not hugely important (at the
> time, Ken was working on a pure Lisp dumping strategy, so shrinking the
> preloaded code was important for that, but we've gone with the pdumper
> instead).  However, I seem to recall that applying something like this
> will make it possible to solve Bug#4845, because the docstring loading
> mechanism will no longer be reliant on finding "(defun foo" in the .elc
> file.  So that might be nifty.

Stefan M recently suggested getting rid of the DOC file entirely, so I
wonder whether he has any comments here.  (Added to the CCs.)

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sat, 25 Sep 2021 15:42:01 GMT) Full text and rfc822 format available.

Message #40 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 27748 <at> debbugs.gnu.org, Ken Raeburn <raeburn <at> raeburn.org>,
 Noam Postavsky <npostavs <at> gmail.com>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sat, 25 Sep 2021 08:41:17 -0700
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Noam Postavsky <npostavs <at> gmail.com> writes:
>
>>> These strings still aren't in the DOC file, but I'm not sure whether
>>> that's a problem or not -- what are the practical effects of the doc
>>> string of `delete-auto-save-files' not being in that file?
>>
>> It's mostly just an optimization thing, so not hugely important (at the
>> time, Ken was working on a pure Lisp dumping strategy, so shrinking the
>> preloaded code was important for that, but we've gone with the pdumper
>> instead).  However, I seem to recall that applying something like this
>> will make it possible to solve Bug#4845, because the docstring loading
>> mechanism will no longer be reliant on finding "(defun foo" in the .elc
>> file.  So that might be nifty.
>
> Stefan M recently suggested getting rid of the DOC file entirely, so I
> wonder whether he has any comments here.  (Added to the CCs.)

FWIW, I think we should get rid of it.  Stefan M did an analysis of this
and the amount of saved memory was very small, from bytecomp.el:

  ;; For the compilation itself, we could largely get rid of this hunk-handler,
  ;; if it weren't for the fact that we need to figure out when a defalias
  ;; defines a macro, so as to add it to byte-compile-macro-environment.
  ;;
  ;; FIXME: we also use this hunk-handler to implement the function's dynamic
  ;; docstring feature.  We could actually implement it more elegantly in
  ;; byte-compile-lambda so it applies to all lambdas, but the problem is that
  ;; the resulting .elc format will not be recognized by make-docfile, so
  ;; either we stop using DOC for the docstrings of preloaded elc files (at the
  ;; cost of around 24KB on 32bit hosts, double on 64bit hosts) or we need to
  ;; build DOC in a more clever way (e.g. handle anonymous elements).




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sun, 26 Sep 2021 05:34:01 GMT) Full text and rfc822 format available.

Message #43 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: 27748 <at> debbugs.gnu.org, Ken Raeburn <raeburn <at> raeburn.org>,
 Noam Postavsky <npostavs <at> gmail.com>, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sun, 26 Sep 2021 07:32:56 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> FWIW, I think we should get rid of it.  Stefan M did an analysis of this
> and the amount of saved memory was very small, from bytecomp.el:

[...]

>   ;; (at the cost of around 24KB on 32bit hosts, double on 64bit
>   ;; hosts)

Yes, that seems pretty insignificant.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sat, 23 Oct 2021 17:33:02 GMT) Full text and rfc822 format available.

Message #46 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Kangas <stefan <at> marxist.se>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: Ken Raeburn <raeburn <at> raeburn.org>, Noam Postavsky <npostavs <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sat, 23 Oct 2021 10:32:22 -0700
Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Stefan Kangas <stefan <at> marxist.se> writes:
>
>> FWIW, I think we should get rid of it.  Stefan M did an analysis of this
>> and the amount of saved memory was very small, from bytecomp.el:
>
> [...]
>
>>   ;; (at the cost of around 24KB on 32bit hosts, double on 64bit
>>   ;; hosts)
>
> Yes, that seems pretty insignificant.

If getting rid of DOC is what we want to eventually do, we should
probably just close this as wontfix.  It hardly seems worth fixing such
problems in an area that we basically want to get rid of anyway.




Information forwarded to bug-gnu-emacs <at> gnu.org:
bug#27748; Package emacs. (Sun, 24 Oct 2021 13:00:02 GMT) Full text and rfc822 format available.

Message #49 received at 27748 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Stefan Kangas <stefan <at> marxist.se>
Cc: Ken Raeburn <raeburn <at> raeburn.org>, Noam Postavsky <npostavs <at> gmail.com>,
 Stefan Monnier <monnier <at> iro.umontreal.ca>, 27748 <at> debbugs.gnu.org
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
Date: Sun, 24 Oct 2021 14:59:34 +0200
Stefan Kangas <stefan <at> marxist.se> writes:

> If getting rid of DOC is what we want to eventually do, we should
> probably just close this as wontfix.  It hardly seems worth fixing such
> problems in an area that we basically want to get rid of anyway.

Yup.  So I'm closing this bug report.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no




Added tag(s) wontfix. Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Sun, 24 Oct 2021 13:00:03 GMT) Full text and rfc822 format available.

bug closed, send any further explanations to 27748 <at> debbugs.gnu.org and Ken Raeburn <raeburn <at> raeburn.org> Request was from Lars Ingebrigtsen <larsi <at> gnus.org> to control <at> debbugs.gnu.org. (Sun, 24 Oct 2021 13:00:03 GMT) Full text and rfc822 format available.

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Mon, 22 Nov 2021 12:24:08 GMT) Full text and rfc822 format available.

This bug report was last modified 2 years and 127 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.