GNU bug report logs - #27748
26.0.50; doc strings should be in DOC file

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: emacs; Reported by: Ken Raeburn <raeburn@HIDDEN>; Keywords: patch; dated Tue, 18 Jul 2017 06:48:02 UTC; Maintainer for emacs is bug-gnu-emacs@HIDDEN.

Message received at 27748 <at> debbugs.gnu.org:


Received: (at 27748) by debbugs.gnu.org; 31 Aug 2017 00:49:02 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Aug 30 20:49:02 2017
Received: from localhost ([127.0.0.1]:35489 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1dnDfK-0000aw-8y
	for submit <at> debbugs.gnu.org; Wed, 30 Aug 2017 20:49:02 -0400
Received: from mail-io0-f177.google.com ([209.85.223.177]:33543)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1dnDfI-0000aS-JZ
 for 27748 <at> debbugs.gnu.org; Wed, 30 Aug 2017 20:49:01 -0400
Received: by mail-io0-f177.google.com with SMTP id s101so13725680ioe.0
 for <27748 <at> debbugs.gnu.org>; Wed, 30 Aug 2017 17:49:00 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=QvGKzsmxUj4cwVXiJxEJ97GQ1yiDpN2GAmzTxM9HCEk=;
 b=dDntG5EFwc68NA7q4Qq9xh3ZKLdsCgIJdws3EgOtpB/plzYmLlDlkkvAyM/XdTjVYp
 +qHmSH8pUE12/PvKR92RXRIjE7fL6yHoxVYwMpO2JHrlyYTb2lTzj05k0xMt1NOaLKGB
 HhzLWWFFb8Z46Ie6Y3kghygMxIC9eLJS4seO5cYE0W9S4OIuvSgeXkWb96xkpvYVhFwa
 +sFKXcnJAgHr36xF3MU3u0OCrVoD7FQZo2GgXGnl6DwNZiSMRvGoa29GPsVtBkHVgQPN
 8UOh5AHyAp4jd/TCDyWFtYoRexu0TkIxEi53rCbN4CNTIqHKfsyIXlRw57zbavOAK4vo
 WCWA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:from:to:cc:subject:references:date
 :in-reply-to:message-id:user-agent:mime-version
 :content-transfer-encoding;
 bh=QvGKzsmxUj4cwVXiJxEJ97GQ1yiDpN2GAmzTxM9HCEk=;
 b=FZgjgepj5eKq1H114glxT7U14YvshNuk6OtddpL1yhNH1JFTv1BNIUKYFhFziVQkOh
 VqxBSx2Viqv9SDhyEXrk5g7JFOnuoqBnto7ZSwNOws4mY889zgs8TADzzatme3Nev9+/
 GVJ5QvXxxOUXuSx+UQ+PrCT67E03hd1DkVVC6NHviZ4Zfq6Htddmvw7Oid0VV+L0CU/f
 ktOXs6ts9FThG+4f5qlmqt3tMqa/upr3JwC61mZbxOePf+xyCFPKWRagLkaxSLi4RkuJ
 oWI9JVF7lIAwoa8UDlMBm7myFZm0zptGKrOj1fKdxL6eSu/6TS5KdPKBGd9R8qIDx3ZK
 2SNw==
X-Gm-Message-State: AHPjjUjFMKcMPLYv5xiBc9o5BKPE0JcMkbfWDjYqoLC9HI8bFRwAlpPX
 eyOLw2Cakv7sGWXZ
X-Received: by 10.107.156.74 with SMTP id f71mr3687573ioe.230.1504140534592;
 Wed, 30 Aug 2017 17:48:54 -0700 (PDT)
Received: from zony ([45.2.119.49])
 by smtp.googlemail.com with ESMTPSA id y202sm1497999itc.2.2017.08.30.17.48.50
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Wed, 30 Aug 2017 17:48:51 -0700 (PDT)
From: npostavs@HIDDEN
To: Ken Raeburn <raeburn@HIDDEN>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
References: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
 <87pobpaoa4.fsf@HIDDEN>
 <BDAD29A9-7B3E-4DF4-8A1C-026270E71DD6@HIDDEN>
Date: Wed, 30 Aug 2017 20:50:32 -0400
In-Reply-To: <BDAD29A9-7B3E-4DF4-8A1C-026270E71DD6@HIDDEN> (Ken Raeburn's
 message of "Tue, 29 Aug 2017 06:09:09 -0400")
Message-ID: <87val47e7b.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 27748
Cc: 27748 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

Ken Raeburn <raeburn@HIDDEN> writes:

> Sorry it=E2=80=99s taken me a while to get to testing these out=E2=80=A6

Hah, no problem.  I confess it's been on my todo list to test out your
scratch/raeburn-startup branch for an even longer while...

> On Aug 20, 2017, at 18:05, npostavs@HIDDEN wrote:
>>=20
>>> 1. defcustom doc strings from files compiled with lexical binding.
>
>> With patch 0001 defcustoms which are compiled to bytecode now produce
>> dynamic docstrings which make-doc can digest (note that I had to change
>> make-doc a bit for this, but the .elc format remains the same as far as
>> the Emacs loading it is concerned.  See the commit message for details).
>
> I think I like the new format.  It=E2=80=99s a little bit bigger, but it =
may
> load faster, as we can do one big fseek at the beginning of the file
> (thus not even loading a lot of those pages) rather than lots of small
> ones as we go along.

Indeed, that was my thought too.  I haven't measured anything though.

> Will this new make-docfile play nicely with files compiled with
> byte-compile-dynamic, where byte code is mixed in with the usual doc
> strings?  Or if we decide to make lambdas (which have =E2=80=9C(fn=E2=80=
=A6)=E2=80=9D doc
> strings by default but have no names to associate with them in DOC)
> load their doc strings dynamically from the .elc file?

Hmm, it will not.  We would have to add a "nameless" type I guess,
something like ^_A^_anonymous docstring here...^_.

I pushed patches [2: bc5d96a0b2] and [3: 160295867d] to master, since
they are pretty straightforward bugfixes.

[2: bc5d96a0b2]: 2017-08-30 20:07:39 -0400
  Drop docstrings from cl-defsubst produced inline bodies (Bug#27748)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=3Dbc5d96a0b2a1dccf7=
eeeec459e40d21b54c977f4>

[3: 160295867d]: 2017-08-30 20:07:39 -0400
  Support lazy loading for autogenerated usage docstrings too (Bug#27748)
  http://git.savannah.gnu.org/cgit/emacs.git/commit/?id=3D160295867de98241a=
16f2ede93da7e825ed4406b=20=20





Information forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.

Message received at 27748 <at> debbugs.gnu.org:


Received: (at 27748) by debbugs.gnu.org; 29 Aug 2017 10:09:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Aug 29 06:09:20 2017
Received: from localhost ([127.0.0.1]:60132 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1dmdSR-0007dn-No
	for submit <at> debbugs.gnu.org; Tue, 29 Aug 2017 06:09:19 -0400
Received: from mail-qk0-f179.google.com ([209.85.220.179]:37249)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raeburn@HIDDEN>) id 1dmdSP-0007dR-3t
 for 27748 <at> debbugs.gnu.org; Tue, 29 Aug 2017 06:09:17 -0400
Received: by mail-qk0-f179.google.com with SMTP id k126so12990299qkb.4
 for <27748 <at> debbugs.gnu.org>; Tue, 29 Aug 2017 03:09:17 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=raeburn-org.20150623.gappssmtp.com; s=20150623;
 h=mime-version:subject:from:in-reply-to:date:cc
 :content-transfer-encoding:message-id:references:to;
 bh=zlHvIFLlj/SfOMcnaso0ApqivT0F2ghdgzB4XVRKrEw=;
 b=pYA4wiuyDtjWtWurSKuyHFYvyo4Kjj6C2Wnjjet69INXarvYNIzLqP+Sw+E0tHNVbL
 2csAlwIK5WBD75HPUS4Ne12UjRIRcU4ChcCzXSAK1/ZUVMKyFS/nJz3qtnQPoLBiysp0
 DF9B4GwWaDb1cRl0X5ojufUJNkh1Pt8jFe0M7YKfJWN39zkBlUMbfn2wBdD2EjG8YKXQ
 kC2O9+BVj0GcKKrDc0lj8kpHB0B4OvnAJ2ODpi+454C30briqum/2lYCjPGKoeqkYHwh
 lfI4glO3ooH0U6b63jrrR9XPvmyHzXFool4qtsDME/NNR5J228ixDmBNlhQn3zV71MQM
 iD3g==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:mime-version:subject:from:in-reply-to:date:cc
 :content-transfer-encoding:message-id:references:to;
 bh=zlHvIFLlj/SfOMcnaso0ApqivT0F2ghdgzB4XVRKrEw=;
 b=pVtoSMtds6hdUhWPTbUh5wFI1PuLqv6zeiURGzI1US8yU1AF2iRmg1W4TzV0sb+pNV
 Zi3keQPxf9fgoXC52JcOd+XsTj/nfy7InKnNbX8dZuIN/dhf3Ar4d5hiDZdPBdrzwy+a
 Fc4dwHGVgM0zqVJYCbPbZtkJCVebwvln+dY5bwivDmd2GaE8D7UPoX0qPPtI33hTHU1a
 4kYujHcmfRvgjyabLDmj9siBrBIglT4BtI3gKkSAiWdu0A06XmzZ4kQMWHDvWxH8WjxJ
 WjuuIVLcl9/8aVWOR5c6VKeAlBPE9SzQmNdp8PZbeyH7eK7fk3fCfxMECu0cyNQhA7YO
 07Bw==
X-Gm-Message-State: AHYfb5h4MZTBBf4NtZY6xu3b3S6ZfgKTtYloxwEMQGJ+pd24J0e07i38
 pLn7N6QG5wC4yWOo
X-Received: by 10.55.49.10 with SMTP id x10mr5334985qkx.97.1504001351467;
 Tue, 29 Aug 2017 03:09:11 -0700 (PDT)
Received: from [192.168.23.135] (c-73-253-167-23.hsd1.ma.comcast.net.
 [73.253.167.23])
 by smtp.gmail.com with ESMTPSA id h89sm1620719qtd.81.2017.08.29.03.09.10
 (version=TLS1_2 cipher=ECDHE-RSA-AES128-GCM-SHA256 bits=128/128);
 Tue, 29 Aug 2017 03:09:10 -0700 (PDT)
Content-Type: text/plain; charset=utf-8
Mime-Version: 1.0 (Mac OS X Mail 10.3 \(3273\))
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
From: Ken Raeburn <raeburn@HIDDEN>
In-Reply-To: <87pobpaoa4.fsf@HIDDEN>
Date: Tue, 29 Aug 2017 06:09:09 -0400
Content-Transfer-Encoding: quoted-printable
Message-Id: <BDAD29A9-7B3E-4DF4-8A1C-026270E71DD6@HIDDEN>
References: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
 <87pobpaoa4.fsf@HIDDEN>
To: npostavs@HIDDEN
X-Mailer: Apple Mail (2.3273)
X-Spam-Score: 0.5 (/)
X-Debbugs-Envelope-To: 27748
Cc: 27748 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.5 (/)

Sorry it=E2=80=99s taken me a while to get to testing these out=E2=80=A6

On Aug 20, 2017, at 18:05, npostavs@HIDDEN wrote:
>=20
>> 1. defcustom doc strings from files compiled with lexical binding.

> With patch 0001 defcustoms which are compiled to bytecode now produce
> dynamic docstrings which make-doc can digest (note that I had to =
change
> make-doc a bit for this, but the .elc format remains the same as far =
as
> the Emacs loading it is concerned.  See the commit message for =
details).

I think I like the new format.  It=E2=80=99s a little bit bigger, but it =
may load faster, as we can do one big fseek at the beginning of the file =
(thus not even loading a lot of those pages) rather than lots of small =
ones as we go along.

Will this new make-docfile play nicely with files compiled with =
byte-compile-dynamic, where byte code is mixed in with the usual doc =
strings?  Or if we decide to make lambdas (which have =E2=80=9C(fn=E2=80=A6=
)=E2=80=9D doc strings by default but have no names to associate with =
them in DOC) load their doc strings dynamically from the .elc file?

>> 2. In isearch, CL macro expansion results in symbols like
>>   isearch--state-forward--cmacro having function definitions that
>>   include the two doc strings "Access slot \"forward\" of
>>   `(isearch--state (:constructor nil) [...]" and
>>   "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC =
(with
>>   "\n\n(fn CL-X)" appended) as the documentation for the function
>>   isearch--state-forward.

> I think it is just an oversight, since the string was put inside the
> cl-block it is not recognized as a docstring at all.  Patch 0002 drops
> the function's docstring from compiler-macro and adds a simple
> "compiler-macro for inlining `NAME'" instead.

I=E2=80=99m seeing the shorter string, *and* it=E2=80=99s stored in the =
DOC file.

>> 3. Undocumented functions, strange as it sounds... in files.el, =
function
>>   file-name-non-special is defined with no documentation.  In
>>   files.elc, the bytecode object constructed has a doc string =
"\n\n(fn
>>   OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
>>   separate string that make-docfile can pick up.
>=20
> Looks fairly trivial to fix, see patch 0003.

This one seems to be working well too.

I did a few spot checks looking at what wound up in the DOC file, and =
checking the ability to load the documentation in Emacs, and  things =
look good.

Between these three changes, the byte count for lisp/*.elc grew by 2-3%. =
 The DOC file is about 6% bigger, due to the strings that weren=E2=80=99t =
being picked up before.  Curiously, the generated emacs executable was =
just a little bit bigger, less than 1%, but from the numbers returned by =
(garbage-collect) I think that=E2=80=99s probably freed space.

I tested the changes out against the scratch/raeburn-startup branch as =
well.  In that case, the saved Lisp environment (with the more costly =
parsing process than etc/DOC) shrinks by about 5%, which will probably =
speed up the startup load time a little bit. There are still some doc =
strings left, but I=E2=80=99ll look into those later.

Thanks for digging into this!

Ken=




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.
Added tag(s) patch. Request was from npostavs@HIDDEN to control <at> debbugs.gnu.org. Full text available.

Message received at 27748 <at> debbugs.gnu.org:


Received: (at 27748) by debbugs.gnu.org; 20 Aug 2017 22:03:40 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 20 18:03:40 2017
Received: from localhost ([127.0.0.1]:47035 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1djYJn-0003cd-Vo
	for submit <at> debbugs.gnu.org; Sun, 20 Aug 2017 18:03:40 -0400
Received: from mail-io0-f182.google.com ([209.85.223.182]:33581)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>)
 id 1djYJk-0003cL-CK; Sun, 20 Aug 2017 18:03:37 -0400
Received: by mail-io0-f182.google.com with SMTP id o196so996698ioe.0;
 Sun, 20 Aug 2017 15:03:36 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=PR9QXi0algEg54ZzuHhsbdnhp0+ikoee85Bv+ZB27/s=;
 b=vNHtAqbd29QwmyOY602eVJCF2csi6XvQIB+1CIVdGdKgCeQ99HeBJWGkn5QcSRPTNm
 qi8IMBkIce5LJ4gQ2JYAtOS3UTEHHNg56IZJXSw0NDoW6z4X0gzwow6evlFgBuFPhYuJ
 z2PodQiuf3rh3ac38PfDs1jdSBvCI7KeB7w4N9o8kytQQIXy5//DeLb9jEqqW2MlOe+g
 3bMZjuYk69Os6zA0EKsQ2/OsU1W7aQsXoA8csFPO3Z584A7lUqzK9IQqfmyArRB5Bnvg
 dsVIXdmCLq9ULVCbC56k8EXGuKipIJCyQ3VQ6zdLAaUOY5q+XvVLfOyCdVq9JuAYnXgQ
 K4yg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:from:to:cc:subject:references:date
 :in-reply-to:message-id:user-agent:mime-version;
 bh=PR9QXi0algEg54ZzuHhsbdnhp0+ikoee85Bv+ZB27/s=;
 b=SMc81fbDUo/jScv8m1U0RUjP0J4aKAoz8EOftUyc6oVkQ1GsVoMSwCy1zhMAaAOxUH
 vpexZ8pvyAJlye5kP2+Y+keUHjb5VtIKrxpTl62rMpOG17YKM1j4HiVk0RYhHEpExESQ
 c8jZlptlG2jC6Q8m+xaOKflOBVQbVMZuLP6eTfHPjgVpMduXVYhd1FgojgD3r31aGaKk
 eTNlKfINWVQnEgvg2Aku2pXhmqZ98Drs4LtX3ajmIrUp1qrMyR2WOc9hnkJMvcxkPJh2
 wpJQht+kOrx36XCOyxsy4W4oxQmmtVEJJwkTxqumiNkWDydXNhMFe94XACSR6buSM2iZ
 HmAg==
X-Gm-Message-State: AHYfb5ja6RCZyziBAiPjsv0bFtHYgF4oo/U8RptieZRD+qRjvN+oEvy/
 SDQvvAXDTFCZnmUm
X-Received: by 10.107.131.197 with SMTP id n66mr863699ioi.269.1503266610576;
 Sun, 20 Aug 2017 15:03:30 -0700 (PDT)
Received: from zony ([45.2.119.49])
 by smtp.googlemail.com with ESMTPSA id n12sm3271655itn.16.2017.08.20.15.03.29
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sun, 20 Aug 2017 15:03:29 -0700 (PDT)
From: npostavs@HIDDEN
To: Ken Raeburn <raeburn@HIDDEN>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
References: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
Date: Sun, 20 Aug 2017 18:05:07 -0400
In-Reply-To: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN> (Ken Raeburn's
 message of "Tue, 18 Jul 2017 02:47:09 -0400")
Message-ID: <87pobpaoa4.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 27748
Cc: 27748 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

--=-=-=
Content-Type: text/plain

tags 27748 + patch
quit

Ken Raeburn <raeburn@HIDDEN> writes:

> 1. defcustom doc strings from files compiled with lexical binding.
>
>    For example, files.el (lexical bindings) defines
>    delete-auto-save-files but it doesn't show up in the DOC file;
>    files.elc starts with an initial byte-code blob which includes the
>    symbol delete-auto-save-files and its doc string in the constants
>    array.
>
>    On the other hand, custom.el (dynamic bindings) declares
>    custom-theme-directory, the .elc file dumps out the doc string in a
>    #@... block before a separate top-level call to
>    custom-declare-variable, and since this is what make-docfile looks
>    for, the doc string winds up in the DOC file.

With patch 0001 defcustoms which are compiled to bytecode now produce
dynamic docstrings which make-doc can digest (note that I had to change
make-doc a bit for this, but the .elc format remains the same as far as
the Emacs loading it is concerned.  See the commit message for details).


--=-=-=
Content-Type: text/plain
Content-Disposition: attachment;
 filename=0001-Produce-dynamic-docstrings-for-bytecode-Bug-27748.patch
Content-Description: patch

From 73c753f07c21ad2fe32fac124b7287bd8b6ab01b Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@HIDDEN>
Date: Sun, 13 Aug 2017 13:15:10 -0400
Subject: [PATCH 1/3] Produce dynamic docstrings for bytecode (Bug#27748)

Instead of relying on decompilation to create source forms that we can
easily extract docstrings from, record the docstrings as we compile
and then print out all the docstrings with their corresponding symbol
names in a single comment at the top.

Old format:

    #@ nnn Docstring of var1^_
    (defvar var1 (init-expression) (#$ . nn))
    #@ nnn Docstring of var2^_
    (defvar var2 (init-expression) (#$ . nn))

New format:

    #@ nnnnnn ^_Vvar1^_Docstring of var1^_
    ^_Vvar2^_Docstring of var2^_
    (defvar var1 (init-expression) (#$ . nn))
    (defvar var2 (init-expression) (#$ . nn))

(Where "^_" represents the character \037, aka "Unit Separator".)

The new format can still be loaded by older Emacs versions since the
bytecode loader only requires that dynamic docstrings be at the right
file offset and preceded with "^_" or "#@ ".  It cannot be used by
older make-doc versions.

* lisp/emacs-lisp/bytecomp.el (byte-compile-docstring-handler): New
function, push a special kind of constant onto
`byte-compile-constants' for the given docstring.
(byte-compile-file-form-defvar-function): Use it on the docstring.
(byte-compile--docstring-constants): New variable.
(byte-compile-constants-vector): Use it to rememember the indices of
the special docstring constants.
(byte-compile-top-level, byte-compile-flush-pending): Let-bind
byte-compile--docstring-constants to nil.

(byte-compile--docstring-marker): New variable.
(byte-compile-from-buffer): Let-bind it to nil.
(byte-compile-insert-header): Set it to a pair markers pointing to the
end of the header.
(byte-compile-output-as-comment): Write the docstrings collected into
byte-compile--docstring-constants to the second marker in
byte-compile--docstring-marker.  When writing docstrings (as opposed
to lazy loaded bytecode), also print V<symbol> or F<symbol> prior to the
docstring.
(byte-compile-output-file-form): When writing out the constants
vector, use the (#$ . %d) format instead of the string itself.
(byte-compile-escape-docstring): New function, extracted from
`byte-compile-output-as-comment'.
(byte-compile-fix-header-docstring-comment): New function, comment out
the docstrings at the top of the file with a #@N kind of comment.
Delete semicolons from the header as needed to preserve offsets.
(byte-compile-fix-header-multibyte): Rename from
byte-compile-fix-header.
(byte-compile-fix-header): Call both
`byte-compile-fix-header-multibyte' and
`byte-compile-fix-header-docstring-comment'.

* lib-src/make-docfile.c (scan_lisp_file): Update for new format,
collect all of symbol type, name, and docstring directly from #@N
comments.
---
 lib-src/make-docfile.c      |  61 ++++++++++-------
 lisp/emacs-lisp/bytecomp.el | 158 ++++++++++++++++++++++++++++++--------------
 2 files changed, 147 insertions(+), 72 deletions(-)

diff --git a/lib-src/make-docfile.c b/lib-src/make-docfile.c
index ecd6447ab7..8daca9aba2 100644
--- a/lib-src/make-docfile.c
+++ b/lib-src/make-docfile.c
@@ -1258,7 +1258,8 @@ read_lisp_symbol (FILE *infile, char *buffer)
       c = getc (infile);
       if (c == '\\')
 	*(++fillp) = getc (infile);
-      else if (c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '(' || c == ')')
+      else if (c == ' ' || c == '\t' || c == '\n' || c == '\r' ||
+               c == '(' || c == ')' || c == '\037')
 	{
 	  ungetc (c, infile);
 	  *fillp = 0;
@@ -1367,14 +1368,13 @@ scan_lisp_file (const char *filename, const char *mode)
       /* Skip the line break.  */
       while (c == '\n' || c == '\r')
 	c = getc (infile);
-      /* Detect a dynamic doc string and save it for the next expression.  */
+      /* Detect the dynamic string block.  */
       if (c == '#')
 	{
 	  c = getc (infile);
 	  if (c == '@')
 	    {
 	      ptrdiff_t length = 0;
-	      ptrdiff_t i;
 
 	      /* Read the length.  */
 	      while ((c = getc (infile),
@@ -1387,31 +1387,46 @@ scan_lisp_file (const char *filename, const char *mode)
 		}
 
 	      if (length <= 1)
-		fatal ("invalid dynamic doc string length");
+		fatal ("%s: invalid dynamic doc string length", filename);
+
+              /* We expect one newline character following the
+                 comment.  */
+              ptrdiff_t end_offset = ftell (infile) + length + 1;
 
 	      if (c != ' ')
 		fatal ("space not found after dynamic doc string length");
 
-	      /* The next character is a space that is counted in the length
-		 but not part of the doc string.
-		 We already read it, so just ignore it.  */
-	      length--;
-
 	      /* Read in the contents.  */
-	      free (saved_string);
-	      saved_string = xmalloc (length);
-	      for (i = 0; i < length; i++)
-		saved_string[i] = getc (infile);
-	      /* The last character is a ^_.
-		 That is needed in the .elc file
-		 but it is redundant in DOC.  So get rid of it here.  */
-	      saved_string[length - 1] = 0;
-	      /* Skip the line break.  */
-	      while (c == '\n' || c == '\r')
-		c = getc (infile);
-	      /* Skip the following line.  */
-	      while (c != '\n' && c != '\r')
-		c = getc (infile);
+              for (;;)
+                {
+                  c = getc (infile);
+                  if (c != '\037') break;
+                  type = getc (infile);
+                  if (type != 'V' && type != 'F')
+                    fatal ("%s: 'V' or 'F' not found before symbol name (%c)\n", filename, c);
+                  read_lisp_symbol (infile, buffer);
+                  c = getc (infile);
+                  if (c != '\037')
+                    fatal ("\\037 not found after symbol name");
+
+                  printf ("\037%c%s\n", type, buffer);
+                  for (;;)
+                    {
+                      c = getc (infile);
+                      if (c == '\037')
+                        {
+                          if ('\n' != getc (infile))
+                            fatal ("newline not found after dynamic doc string\n");
+                          break;
+                        }
+                      putc (c, stdout);
+                    }
+                }
+              /* All dynamic strings should be in that block.  */
+              if (ftell (infile) != end_offset)
+                fatal ("%s: wrong dynamic doc string length (%ld != %ld)",
+                       filename, ftell (infile), end_offset);
+              break;
 	    }
 	  continue;
 	}
diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index cf06c0c8ef..d2768a159b 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -1976,6 +1976,8 @@ byte-compile-from-buffer
 	;; Simulate entry to byte-compile-top-level
         (byte-compile-jump-tables nil)
         (byte-compile-constants nil)
+        (byte-compile--docstring-constants nil)
+        (byte-compile--docstring-marker nil)
 	(byte-compile-variables nil)
 	(byte-compile-tag-number 0)
 	(byte-compile-depth 0)
@@ -2050,6 +2052,22 @@ byte-compile-from-buffer
      byte-compile--outbuffer)))
 
 (defun byte-compile-fix-header (_filename)
+  (byte-compile-fix-header-multibyte)
+  (byte-compile-fix-header-docstring-comment))
+
+(defun byte-compile-fix-header-docstring-comment ()
+  (pcase byte-compile--docstring-marker
+    (`(,beg . ,end)
+     (let* ((bytes (- (position-bytes end) (position-bytes beg)))
+            (comment-beg (format "#@%d " bytes)))
+       (when (> bytes 0)
+         (goto-char (point-min))
+         (search-forward ";;;;;;;;;;" beg)
+         (beginning-of-line)
+         (delete-char (length comment-beg))
+         (princ comment-beg beg))))))
+
+(defun byte-compile-fix-header-multibyte ()
   "If the current buffer has any multibyte characters, insert a version test."
   (when (< (point-max) (position-bytes (point-max)))
     (goto-char (point-min))
@@ -2127,7 +2145,9 @@ byte-compile-insert-header
        ;; can delete them so as to keep the buffer positions
        ;; constant for the actual compiled code.
        ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n"
-       ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n"))))
+       ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n")
+      (setq byte-compile--docstring-marker
+            (cons (point-marker) (point-marker))))))
 
 (defun byte-compile-output-file-form (form)
   ;; Write the given form to the output buffer, being careful of docstrings
@@ -2151,7 +2171,29 @@ byte-compile-output-file-form
                                            '(defvaralias autoload
                                               custom-declare-variable)))
       (princ "\n" byte-compile--outbuffer)
-      (prin1 form byte-compile--outbuffer)
+      (pcase form
+        ((and (guard byte-compile--docstring-constants)
+              (guard byte-compile--docstring-marker)
+              `(byte-code ,bytestr ,constants ,maxdepth))
+         (princ "(byte-code " byte-compile--outbuffer)
+         (prin1 bytestr byte-compile--outbuffer)
+         (princ " [" byte-compile--outbuffer)
+         (cl-callf cl-sort byte-compile--docstring-constants #'< :key #'car)
+         (let ((docs-head byte-compile--docstring-constants))
+           (dotimes (i (length constants))
+             (if (or (null docs-head) (/= i (caar docs-head)))
+                 (prin1 (aref constants i) byte-compile--outbuffer)
+               (pcase-let* ((`(,_i ,symtype ,sym) (car docs-head)))
+                 (princ (format "(#$ . %d)"
+                                (byte-compile-output-as-comment
+                                 (aref constants i) nil symtype sym))
+                        byte-compile--outbuffer))
+               (pop docs-head))
+             (princ " " byte-compile--outbuffer)))
+         (princ "] " byte-compile--outbuffer)
+         (prin1 maxdepth byte-compile--outbuffer)
+         (princ ")" byte-compile--outbuffer))
+        (_ (prin1 form byte-compile--outbuffer)))
       nil)))
 
 (defvar byte-compile--for-effect)
@@ -2172,17 +2214,15 @@ byte-compile-output-docform
   (let ((dynamic-docstrings byte-compile-dynamic-docstrings))
     (with-current-buffer byte-compile--outbuffer
       (let (position)
-
         ;; Insert the doc string, and make it a comment with #@LENGTH.
         (and (>= (nth 1 info) 0)
              dynamic-docstrings
-             (progn
-               ;; Make the doc string start at beginning of line
-               ;; for make-docfile's sake.
-               (insert "\n")
-               (setq position
-                     (byte-compile-output-as-comment
-                      (nth (nth 1 info) form) nil))
+             (pcase-let* (((or `',sym sym) (or name (nth 1 form)))
+                          (symtype (if (if preface (string-match-p "defalias" preface)
+                                         (memq (car form) '(autoload defalias)))
+                                       ?F ?V)))
+               (setq position (byte-compile-output-as-comment
+                               (nth (nth 1 info) form) nil symtype sym))
                ;; If the doc string starts with * (a user variable),
                ;; negate POSITION.
                (if (and (stringp (nth (nth 1 info) form))
@@ -2227,10 +2267,9 @@ byte-compile-output-docform
                           (not non-nil)))
                    ;; Output the byte code and constants specially
                    ;; for lazy dynamic loading.
-                   (let ((position
-                          (byte-compile-output-as-comment
-                           (cons (car form) (nth 1 form))
-                           t)))
+                   (let* ((position (byte-compile-output-as-comment
+                                     (cons (car form) (nth 1 form))
+                                     t nil nil)))
                      (princ (format "(#$ . %d) nil" position)
                             byte-compile--outbuffer)
                      (setq form (cdr form))
@@ -2275,6 +2314,7 @@ byte-compile-flush-pending
 	      (form
 	       (byte-compile-output-file-form form)))
 	(setq byte-compile-constants nil
+              byte-compile--docstring-constants nil
 	      byte-compile-variables nil
 	      byte-compile-depth 0
 	      byte-compile-maxdepth 0
@@ -2389,8 +2429,11 @@ byte-compile-file-form-defvar
 (put 'defvaralias 'byte-hunk-handler 'byte-compile-file-form-defvar-function)
 
 (defun byte-compile-file-form-defvar-function (form)
-  (pcase-let (((or `',name (let name nil)) (nth 1 form)))
-    (if name (byte-compile--declare-var name)))
+  (pcase-let (((or `',name (let name nil)) (nth 1 form))
+              (docstr (nth 3 form)))
+    (if name (byte-compile--declare-var name))
+    (when (stringp docstr)
+      (setf (nth 3 form) `(byte-compile-docstring ,docstr ?V ,name))))
   (byte-compile-keep-pending form))
 
 (put 'custom-declare-variable 'byte-hunk-handler
@@ -2578,42 +2621,43 @@ byte-compile-file-form-defmumble
           (princ ")" byte-compile--outbuffer)
           t)))))
 
-(defun byte-compile-output-as-comment (exp quoted)
-  "Print Lisp object EXP in the output file, inside a comment,
+(defun byte-compile-escape-docstring (beg &optional end)
+  "Quote characters in the range BEG to END for `get_doc_string'."
+  (save-excursion
+    (goto-char beg)
+    (while (search-forward "\^A" end t)
+      (replace-match "\^A\^A" t t))
+    (goto-char beg)
+    (while (search-forward "\000" end t)
+      (replace-match "\^A0" t t))
+    (goto-char beg)
+    (while (search-forward "\037" end t)
+      (replace-match "\^A_" t t))))
+
+(defun byte-compile-output-as-comment (exp quoted symtype sym)
+  "Print Lisp object EXP to the output file's header comment,
 and return the file (byte) position it will have.
-If QUOTED is non-nil, print with quoting; otherwise, print without quoting."
+The header lies between the markers in
+`byte-compile--docstring-marker'.
+If QUOTED is non-nil, print with quoting; otherwise, print without quoting.
+If SYMTYPE is a character, print it and SYM before EXP."
   (with-current-buffer byte-compile--outbuffer
-    (let ((position (point)))
-
-      ;; Insert EXP, and make it a comment with #@LENGTH.
-      (insert " ")
+    (let* ((doc-marker (cdr byte-compile--docstring-marker))
+           (position (progn (when (characterp symtype)
+                              (write-char ?\037 doc-marker)
+                              (write-char symtype doc-marker)
+                              (princ sym doc-marker))
+                            (write-char ?\037 doc-marker)
+                            (marker-position doc-marker))))
       (if quoted
-          (prin1 exp byte-compile--outbuffer)
-        (princ exp byte-compile--outbuffer))
-      (goto-char position)
-      ;; Quote certain special characters as needed.
-      ;; get_doc_string in doc.c does the unquoting.
-      (while (search-forward "\^A" nil t)
-        (replace-match "\^A\^A" t t))
-      (goto-char position)
-      (while (search-forward "\000" nil t)
-        (replace-match "\^A0" t t))
-      (goto-char position)
-      (while (search-forward "\037" nil t)
-        (replace-match "\^A_" t t))
-      (goto-char (point-max))
-      (insert "\037")
-      (goto-char position)
-      (insert "#@" (format "%d" (- (position-bytes (point-max))
-                                   (position-bytes position))))
-
+          (prin1 exp doc-marker)
+        (princ exp doc-marker))
+      (byte-compile-escape-docstring position doc-marker)
+      (princ "\037\n" doc-marker)
       ;; Save the file position of the object.
-      ;; Note we add 1 to skip the space that we inserted before the actual doc
-      ;; string, and subtract point-min to convert from an 1-origin Emacs
-      ;; position to a file position.
-      (prog1
-          (- (position-bytes (point)) (point-min) -1)
-        (goto-char (point-max))))))
+      ;; Note we subtract point-min to convert from an 1-origin Emacs
+      ;; position to a 0-origin file offset.
+      (- (position-bytes position) (point-min)))))
 
 (defun byte-compile--reify-function (fun)
   "Return an expression which will evaluate to a function value FUN.
@@ -2838,6 +2882,8 @@ byte-compile-lambda
                   (list (nth 1 int))))))))
 
 (defvar byte-compile-reserved-constants 0)
+(defvar byte-compile--docstring-constants nil)
+(defvar byte-compile--docstring-marker nil)
 
 (defun byte-compile-constants-vector ()
   ;; Builds the constants-vector from the current variables and constants.
@@ -2866,7 +2912,11 @@ byte-compile-constants-vector
 	 ((setq tmp (assq (car (car rest)) ret))
 	  (setcdr (car rest) (cdr tmp)))
 	 (t
-	  (setcdr (car rest) (setq i (1+ i)))
+          (setq i (1+ i))
+          (pcase (car rest)
+            (`(,_docstr docstring ,symtype ,sym)
+             (push (list i symtype sym) byte-compile--docstring-constants)))
+          (setcdr (car rest) i)
 	  (setq ret (cons (car rest) ret))))
 	(setq rest (cdr rest)))
       (setq limits (cdr limits)         ;Step
@@ -2885,6 +2935,7 @@ byte-compile-top-level
   ;;	'file		-> used at file-level.
   (let ((byte-compile--for-effect for-effect)
         (byte-compile-constants nil)
+        (byte-compile--docstring-constants nil)
 	(byte-compile-variables nil)
 	(byte-compile-tag-number 0)
 	(byte-compile-depth 0)
@@ -4578,6 +4629,15 @@ byte-compile-make-obsolete-variable
     (push (nth 1 (nth 1 form)) byte-compile-global-not-obsolete-vars))
   (byte-compile-normal-call form))
 
+
+(byte-defop-compiler
+ (byte-compile-docstring nil) byte-compile-docstring-handler)
+(defun byte-compile-docstring-handler (form)
+  ;; FORM = (byte-compile-docstring DOCSTR ?V NAME)
+  (byte-compile-out 'byte-constant
+                    (car (push (cl-list* (cadr form) 'docstring (cddr form))
+                               byte-compile-constants))))
+
 (defconst byte-compile-tmp-var (make-symbol "def-tmp-var"))
 
 (defun byte-compile-defvar (form)
-- 
2.14.1


--=-=-=
Content-Type: text/plain


> 2. In isearch, CL macro expansion results in symbols like
>    isearch--state-forward--cmacro having function definitions that
>    include the two doc strings "Access slot \"forward\" of
>    `(isearch--state (:constructor nil) [...]" and
>    "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC (with
>    "\n\n(fn CL-X)" appended) as the documentation for the function
>    isearch--state-forward.
>
>    It appears that, as far as the Emacs help system is concerned,
>    isearch--state-*--cmacro functions are undocumented.
>
>    The --cmacro functions generate cl-block forms that include the
>    original doc string, since it's treated as part of the body, but it's
>    not clear to me whether it has any use there at all, or if it could
>    just be discarded, or perhaps looked up via the non --cmacro symbols
>    at run time if it is of use.

I think it is just an oversight, since the string was put inside the
cl-block it is not recognized as a docstring at all.  Patch 0002 drops
the function's docstring from compiler-macro and adds a simple
"compiler-macro for inlining `NAME'" instead.


--=-=-=
Content-Type: text/plain
Content-Disposition: attachment;
 filename=0002-Drop-docstrings-from-cl-defsubst-produced-inline-bod.patch
Content-Description: patch

From 3e9846d2fb9f4857f9bde152227ebc28f6faccc7 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@HIDDEN>
Date: Fri, 18 Aug 2017 08:15:25 -0400
Subject: [PATCH 2/3] Drop docstrings from cl-defsubst produced inline bodies
 (Bug#27748)

* lisp/emacs-lisp/cl-macs.el (cl-defsubst): Use macroexp-parse-progn
to drop the docstring.  Add a simple docstring to the compiler-macro.
---
 lisp/emacs-lisp/cl-macs.el | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/cl-macs.el b/lisp/emacs-lisp/cl-macs.el
index 451e6490d7..6c69f16a20 100644
--- a/lisp/emacs-lisp/cl-macs.el
+++ b/lisp/emacs-lisp/cl-macs.el
@@ -2468,8 +2468,9 @@ cl-defsubst
              ,(if (memq '&key args)
                   `(&whole cl-whole &cl-quote ,@args)
                 (cons '&cl-quote args))
+             ,(format "compiler-macro for inlining `%s'." name)
              (cl--defsubst-expand
-              ',argns '(cl-block ,name ,@body)
+              ',argns '(cl-block ,name ,@(cdr (macroexp-parse-body body)))
               ;; We used to pass `simple' as
               ;; (not (or unsafe (cl-expr-access-order pbody argns)))
               ;; But this is much too simplistic since it
-- 
2.14.1


--=-=-=
Content-Type: text/plain


> 3. Undocumented functions, strange as it sounds... in files.el, function
>    file-name-non-special is defined with no documentation.  In
>    files.elc, the bytecode object constructed has a doc string "\n\n(fn
>    OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
>    separate string that make-docfile can pick up.

Looks fairly trivial to fix, see patch 0003.


--=-=-=
Content-Type: text/plain
Content-Disposition: attachment;
 filename=0003-Support-lazy-loading-for-autogenerated-usage-docstri.patch
Content-Description: patch

From 50113c164038f63bec1ba65ac21f9c6f41bd1f16 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@HIDDEN>
Date: Sat, 19 Aug 2017 10:29:05 -0400
Subject: [PATCH 3/3] Support lazy loading for autogenerated usage docstrings
 too (Bug#27748)

* lisp/emacs-lisp/bytecomp.el (byte-compile-file-form-defmumble):
Consider any documentation that ended up in code as a docstring (e.g.,
autogenerated (fn ARG1 ARG2) type things), not just what the user
passed.
---
 lisp/emacs-lisp/bytecomp.el | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index d2768a159b..b11c49f230 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -2607,7 +2607,7 @@ byte-compile-file-form-defmumble
           (let ((index
                  ;; If there's no doc string, provide -1 as the "doc string
                  ;; index" so that no element will be treated as a doc string.
-                 (if (not (stringp (car body))) -1 4)))
+                 (if (not (stringp (documentation code t))) -1 4)))
             ;; Output the form by hand, that's much simpler than having
             ;; b-c-output-file-form analyze the defalias.
             (byte-compile-output-docform
-- 
2.14.1


--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.

Message received at 27748 <at> debbugs.gnu.org:


Received: (at 27748) by debbugs.gnu.org; 13 Aug 2017 18:03:22 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Aug 13 14:03:22 2017
Received: from localhost ([127.0.0.1]:60316 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1dgxEQ-0007P1-4S
	for submit <at> debbugs.gnu.org; Sun, 13 Aug 2017 14:03:22 -0400
Received: from mail-io0-f194.google.com ([209.85.223.194]:38458)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1dgxEO-0007Ov-8q
 for 27748 <at> debbugs.gnu.org; Sun, 13 Aug 2017 14:03:20 -0400
Received: by mail-io0-f194.google.com with SMTP id o9so6366551iod.5
 for <27748 <at> debbugs.gnu.org>; Sun, 13 Aug 2017 11:03:20 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=LvTszTodpeI2iN5wOPWzrzDVKvJPiABF4wSOzC3OaBI=;
 b=dzjP/wstMPJbkGoj9Ad5FPuwUHxhjdSpBxPBdFE/QceTvfyuz45qJyW2A5OD3HqD69
 GPCGShL2i9Lj3lpAMKIglH+CA75iNqKVb4YcM2q87Ge4Ww/FmA7YPtJaur4m3cz4zFY1
 xkrIsGHN/1CV0WBbqEvV5GWv6577SF9/XvRrtW8JiTocVxsL7bOuoFv6OFetZpSD8Z6d
 5NYavhGCIcsCJPEp/TDeTntWIjjKXwXDQ0wYf05Xw0d4PVGHg3n7Clf0aS5glpJcIJLd
 iuJ8iIAsEFUSi4fr6MaPYSFlyXV8uUtPmcD8dcAsJKA5Qrg6F631y0oPqC5ZbB+16jXv
 byQA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:from:to:cc:subject:references:date
 :in-reply-to:message-id:user-agent:mime-version;
 bh=LvTszTodpeI2iN5wOPWzrzDVKvJPiABF4wSOzC3OaBI=;
 b=pTm24UZxz7OBst0Qx97JlLPU4q7qzUcX/fU6s94LHrM3rajCY54hvcBhpYERo3B9ok
 c0dp/yXt60RGwvmqBb/JwIpVT84aQeweXrsH8OhuIekBpaaKRhsho110nrrIqkMDrHq9
 /P6NjKrvoQuFpIzJSzQyGEiw3PXA8iWwswj69HB6B42cRSFsGPlDeksvPbl8LVRCJp1y
 AwRwoCwqZqWfanCK74NS19sZLkAXlztdDzYu3DVQgmebSw/u89j6N+8GTI25OS5rvOWa
 pPyDTIIsWE+4LT5uvDT+89O2+/F9ztNRKnOyyCrSEIYEh8puclE+BkOMLhvV+r4uoRK4
 CrMA==
X-Gm-Message-State: AHYfb5iUaQk8mg+b3toJiZU6pvtLAcCgcED+xp+WV8YGq0uvIWxwW62R
 a+mYovy5GwWHG/zu
X-Received: by 10.107.11.72 with SMTP id v69mr17002648ioi.302.1502647394539;
 Sun, 13 Aug 2017 11:03:14 -0700 (PDT)
Received: from zony ([45.2.119.49])
 by smtp.googlemail.com with ESMTPSA id s76sm1980217itb.9.2017.08.13.11.03.09
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sun, 13 Aug 2017 11:03:12 -0700 (PDT)
From: npostavs@HIDDEN
To: Ken Raeburn <raeburn@HIDDEN>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
References: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
 <87a83d5zgz.fsf@HIDDEN>
 <87h8xi50r0.fsf@HIDDEN>
Date: Sun, 13 Aug 2017 14:04:48 -0400
In-Reply-To: <87h8xi50r0.fsf@HIDDEN>
 (npostavs@HIDDEN's message of "Mon, 07 Aug 2017 21:03:47
 -0400")
Message-ID: <874ltb1gzj.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.6 (--)
X-Debbugs-Envelope-To: 27748
Cc: 27748 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.6 (--)

--=-=-=
Content-Type: text/plain

npostavs@HIDDEN writes:

> npostavs@HIDDEN writes:
>
>> The following patch prevents custom-declare-variable from being compiled
>> and lets the docstrings get printed properly.  Probably needs a bit more
>> refinement though.
>
> Hmm, this approach might not work at all, it causes a bazillion "free
> variable reference" warnings, one for each reference to a defcustom
> variable.

Okay, here is an alternate approach which decouples the docstring
production from decompilation (note this isn't finished yet, I only
implemented it for defcustom, so applying this patch currently prevents
make-doc from finishing successfully).


--=-=-=
Content-Type: text/plain
Content-Disposition: attachment;
 filename=0001-WIP-Produce-dynamic-docstrings-for-bytecode-Bug-2774.patch
Content-Description: patch

From 0fbe96c0add052338d68453de5fb3486201e61d0 Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@HIDDEN>
Date: Sun, 13 Aug 2017 13:15:10 -0400
Subject: [PATCH] [WIP] Produce dynamic docstrings for bytecode (Bug#27748)

Instead of relying on decompilation to create source forms that we can
easily extract docstrings from, record the docstrings as we compile
and then print out all the docstrings with their corresponding symbol
names in a single comment at the top.

NOTE: Currently only updates recording docstrings for defcustom
leaving other forms to produce the old format, but make-docfile is
updated for the new format, so this prevents successful building of
emacs.

* lisp/emacs-lisp/bytecomp.el (byte-compile-docstring-handler): New
function, push a special kind of constant onto
`byte-compile-constants' for the given docstring.
(byte-compile-file-form-defvar-function): Use it on the docstring.
(byte-compile--docstring-constants): New variable.
(byte-compile-constants-vector): Use it to rememember the indices of
the special docstring constants.
(byte-compile-top-level, byte-compile-flush-pending): Let-bind
byte-compile--docstring-constants to nil.

(byte-compile--docstring-marker): New variable.
(byte-compile-from-buffer): Let-bind it to nil.
(byte-compile-insert-header): Set it to a pair markers pointing to the
end of the header.
(byte-compile-output-file-form): Write the docstrings collected into
byte-compile--docstring-constants to the second marker in
byte-compile--docstring-marker.  When writing out the constants
vector, use the (#$ . %d) format instead of the string itself.
(byte-compile-escape-docstring): New function, extracted from
`byte-compile-output-as-comment'.
(byte-compile-fix-header-docstring-comment): New function, comment out
the docstrings at the top of the file with a #@N kind of comment.
Delete semicolons from the header as needed to preserve offsets.
(byte-compile-fix-header-multibyte): Rename from
byte-compile-fix-header.
(byte-compile-fix-header): Call both
`byte-compile-fix-header-multibyte' and
`byte-compile-fix-header-docstring-comment'.

* lib-src/make-docfile.c (scan_lisp_file): Update for new format,
collect all of symbol type, name, and docstring directly from #@N
comments.
---
 lib-src/make-docfile.c      |  43 +++++++++++-------
 lisp/emacs-lisp/bytecomp.el | 104 +++++++++++++++++++++++++++++++++++++-------
 2 files changed, 115 insertions(+), 32 deletions(-)

diff --git a/lib-src/make-docfile.c b/lib-src/make-docfile.c
index ecd6447ab7..06377c5fda 100644
--- a/lib-src/make-docfile.c
+++ b/lib-src/make-docfile.c
@@ -1258,7 +1258,8 @@ read_lisp_symbol (FILE *infile, char *buffer)
       c = getc (infile);
       if (c == '\\')
 	*(++fillp) = getc (infile);
-      else if (c == ' ' || c == '\t' || c == '\n' || c == '\r' || c == '(' || c == ')')
+      else if (c == ' ' || c == '\t' || c == '\n' || c == '\r' ||
+               c == '(' || c == ')' || c == '\037')
 	{
 	  ungetc (c, infile);
 	  *fillp = 0;
@@ -1374,7 +1375,6 @@ scan_lisp_file (const char *filename, const char *mode)
 	  if (c == '@')
 	    {
 	      ptrdiff_t length = 0;
-	      ptrdiff_t i;
 
 	      /* Read the length.  */
 	      while ((c = getc (infile),
@@ -1398,20 +1398,31 @@ scan_lisp_file (const char *filename, const char *mode)
 	      length--;
 
 	      /* Read in the contents.  */
-	      free (saved_string);
-	      saved_string = xmalloc (length);
-	      for (i = 0; i < length; i++)
-		saved_string[i] = getc (infile);
-	      /* The last character is a ^_.
-		 That is needed in the .elc file
-		 but it is redundant in DOC.  So get rid of it here.  */
-	      saved_string[length - 1] = 0;
-	      /* Skip the line break.  */
-	      while (c == '\n' || c == '\r')
-		c = getc (infile);
-	      /* Skip the following line.  */
-	      while (c != '\n' && c != '\r')
-		c = getc (infile);
+              for (;;)
+                {
+                  c = getc (infile);
+                  if (c != '\037') break;
+                  type = getc (infile);
+                  if (type != 'V' && type != 'F')
+                    fatal ("'V' or 'F' not found before symbol name (%c)\n", c);
+                  read_lisp_symbol (infile, buffer);
+                  c = getc (infile);
+                  if (c != '\037')
+                    fatal ("\\037 not found after symbol name");
+
+                  printf ("\037%c%s\n", type, buffer);
+                  for (;;)
+                    {
+                      c = getc (infile);
+                      if (c == '\037')
+                        {
+                          if ('\n' != getc (infile))
+                            fatal ("newline not found after dynamic doc string\n");
+                          break;
+                        }
+                      putc (c, stdout);
+                    }
+                }
 	    }
 	  continue;
 	}
diff --git a/lisp/emacs-lisp/bytecomp.el b/lisp/emacs-lisp/bytecomp.el
index d82b0385b1..28fd2b9cdd 100644
--- a/lisp/emacs-lisp/bytecomp.el
+++ b/lisp/emacs-lisp/bytecomp.el
@@ -1976,6 +1976,8 @@ byte-compile-from-buffer
 	;; Simulate entry to byte-compile-top-level
         (byte-compile-jump-tables nil)
         (byte-compile-constants nil)
+        (byte-compile--docstring-constants nil)
+        (byte-compile--docstring-marker nil)
 	(byte-compile-variables nil)
 	(byte-compile-tag-number 0)
 	(byte-compile-depth 0)
@@ -2050,6 +2052,21 @@ byte-compile-from-buffer
      byte-compile--outbuffer)))
 
 (defun byte-compile-fix-header (_filename)
+  (byte-compile-fix-header-multibyte)
+  (byte-compile-fix-header-docstring-comment))
+
+(defun byte-compile-fix-header-docstring-comment ()
+  (pcase byte-compile--docstring-marker
+    (`(,beg . ,end)
+     (let ((comment-beg (format "#@%d " (- (position-bytes end)
+                                           (position-bytes beg)))))
+       (goto-char (point-min))
+       (search-forward ";;;;;;;;;;" beg)
+       (beginning-of-line)
+       (delete-char (length comment-beg))
+       (princ comment-beg beg)))))
+
+(defun byte-compile-fix-header-multibyte ()
   "If the current buffer has any multibyte characters, insert a version test."
   (when (< (point-max) (position-bytes (point-max)))
     (goto-char (point-min))
@@ -2127,7 +2144,9 @@ byte-compile-insert-header
        ;; can delete them so as to keep the buffer positions
        ;; constant for the actual compiled code.
        ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n"
-       ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n"))))
+       ";;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;\n\n")
+      (setq byte-compile--docstring-marker
+            (cons (point-marker) (point-marker))))))
 
 (defun byte-compile-output-file-form (form)
   ;; Write the given form to the output buffer, being careful of docstrings
@@ -2151,7 +2170,39 @@ byte-compile-output-file-form
                                            '(defvaralias autoload
                                               custom-declare-variable)))
       (princ "\n" byte-compile--outbuffer)
-      (prin1 form byte-compile--outbuffer)
+      (pcase form
+        ((and (guard byte-compile--docstring-constants)
+              (guard byte-compile--docstring-marker)
+              `(byte-code ,bytestr ,constants ,maxdepth))
+         (princ "(byte-code " byte-compile--outbuffer)
+         (prin1 bytestr byte-compile--outbuffer)
+         (princ " [" byte-compile--outbuffer)
+         (cl-callf cl-sort byte-compile--docstring-constants #'< :key #'car)
+         (let ((docs-head byte-compile--docstring-constants))
+           (dotimes (i (length constants))
+             (if (/= i (caar docs-head))
+                 (prin1 (aref constants i) byte-compile--outbuffer)
+               (pcase-let* ((doc-marker (cdr byte-compile--docstring-marker))
+                            (start (marker-position doc-marker))
+                            (`(,_i ,symtype ,sym) (car docs-head)))
+                 (princ (format "(#$ . %d)"
+                                (with-current-buffer byte-compile--outbuffer
+                                  (- (position-bytes start) (point-min))))
+                        byte-compile--outbuffer)
+                 (write-char ?\037 doc-marker)
+                 (write-char symtype doc-marker)
+                 (princ sym doc-marker)
+                 (write-char ?\037 doc-marker)
+                 (princ (aref constants i) doc-marker)
+                 (princ "\037\n" doc-marker)
+                 (byte-compile-escape-docstring
+                  start doc-marker))
+               (pop docs-head))
+             (princ " " byte-compile--outbuffer)))
+         (princ "] " byte-compile--outbuffer)
+         (prin1 maxdepth byte-compile--outbuffer)
+         (princ ")" byte-compile--outbuffer))
+        (_ (prin1 form byte-compile--outbuffer)))
       nil)))
 
 (defvar byte-compile--for-effect)
@@ -2275,6 +2326,7 @@ byte-compile-flush-pending
 	      (form
 	       (byte-compile-output-file-form form)))
 	(setq byte-compile-constants nil
+              byte-compile--docstring-constants nil
 	      byte-compile-variables nil
 	      byte-compile-depth 0
 	      byte-compile-maxdepth 0
@@ -2389,10 +2441,18 @@ byte-compile-file-form-defvar
 (put 'defvaralias 'byte-hunk-handler 'byte-compile-file-form-defvar-function)
 
 (defun byte-compile-file-form-defvar-function (form)
-  (pcase-let (((or `',name (let name nil)) (nth 1 form)))
-    (if name (byte-compile--declare-var name)))
+  (pcase-let (((or `',name (let name nil)) (nth 1 form))
+              (docstr (nth 3 form)))
+    (if name (byte-compile--declare-var name))
+    (when (stringp docstr)
+      (setf (nth 3 form) `(byte-compile-docstring ,docstr ?V ,name))))
   (byte-compile-keep-pending form))
 
+(defun byte-compile-docstring-handler (form)
+  (byte-compile-out 'byte-constant
+                    (car (push (cl-list* (cadr form) 'docstring (cddr form))
+                               byte-compile-constants))))
+
 (put 'custom-declare-variable 'byte-hunk-handler
      'byte-compile-file-form-custom-declare-variable)
 (defun byte-compile-file-form-custom-declare-variable (form)
@@ -2578,6 +2638,18 @@ byte-compile-file-form-defmumble
           (princ ")" byte-compile--outbuffer)
           t)))))
 
+(defun byte-compile-escape-docstring (beg &optional end)
+  "Quote characters in the range BEG to END for `get_doc_string'."
+  (goto-char beg)
+  (while (search-forward "\^A" end t)
+    (replace-match "\^A\^A" t t))
+  (goto-char beg)
+  (while (search-forward "\000" end t)
+    (replace-match "\^A0" t t))
+  (goto-char beg)
+  (while (search-forward "\037" end t)
+    (replace-match "\^A_" t t)))
+
 (defun byte-compile-output-as-comment (exp quoted)
   "Print Lisp object EXP in the output file, inside a comment,
 and return the file (byte) position it will have.
@@ -2590,17 +2662,7 @@ byte-compile-output-as-comment
       (if quoted
           (prin1 exp byte-compile--outbuffer)
         (princ exp byte-compile--outbuffer))
-      (goto-char position)
-      ;; Quote certain special characters as needed.
-      ;; get_doc_string in doc.c does the unquoting.
-      (while (search-forward "\^A" nil t)
-        (replace-match "\^A\^A" t t))
-      (goto-char position)
-      (while (search-forward "\000" nil t)
-        (replace-match "\^A0" t t))
-      (goto-char position)
-      (while (search-forward "\037" nil t)
-        (replace-match "\^A_" t t))
+      (byte-compile-escape-docstring position)
       (goto-char (point-max))
       (insert "\037")
       (goto-char position)
@@ -2838,6 +2900,8 @@ byte-compile-lambda
                   (list (nth 1 int))))))))
 
 (defvar byte-compile-reserved-constants 0)
+(defvar byte-compile--docstring-constants nil)
+(defvar byte-compile--docstring-marker nil)
 
 (defun byte-compile-constants-vector ()
   ;; Builds the constants-vector from the current variables and constants.
@@ -2866,7 +2930,11 @@ byte-compile-constants-vector
 	 ((setq tmp (assq (car (car rest)) ret))
 	  (setcdr (car rest) (cdr tmp)))
 	 (t
-	  (setcdr (car rest) (setq i (1+ i)))
+          (setq i (1+ i))
+          (pcase (car rest)
+            (`(,_docstr docstring ,symtype ,sym)
+             (push (list i symtype sym) byte-compile--docstring-constants)))
+	  (setcdr (car rest) i)
 	  (setq ret (cons (car rest) ret))))
 	(setq rest (cdr rest)))
       (setq limits (cdr limits)         ;Step
@@ -2885,6 +2953,7 @@ byte-compile-top-level
   ;;	'file		-> used at file-level.
   (let ((byte-compile--for-effect for-effect)
         (byte-compile-constants nil)
+        (byte-compile--docstring-constants nil)
 	(byte-compile-variables nil)
 	(byte-compile-tag-number 0)
 	(byte-compile-depth 0)
@@ -3493,6 +3562,9 @@ byte-defop-compiler-1
 ;;####(byte-defop-compiler move-to-column	1)
 (byte-defop-compiler-1 interactive byte-compile-noop)
 
+(byte-defop-compiler (byte-compile-docstring nil)
+                     byte-compile-docstring-handler)
+
 
 (defun byte-compile-subr-wrong-args (form n)
   (byte-compile-set-symbol-position (car form))
-- 
2.14.1


--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.

Message received at 27748 <at> debbugs.gnu.org:


Received: (at 27748) by debbugs.gnu.org; 8 Aug 2017 01:02:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 07 21:02:19 2017
Received: from localhost ([127.0.0.1]:50901 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1desuZ-0007WH-Ps
	for submit <at> debbugs.gnu.org; Mon, 07 Aug 2017 21:02:19 -0400
Received: from mail-io0-f171.google.com ([209.85.223.171]:34358)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1desuY-0007W4-Hx
 for 27748 <at> debbugs.gnu.org; Mon, 07 Aug 2017 21:02:18 -0400
Received: by mail-io0-f171.google.com with SMTP id o9so8490896iod.1
 for <27748 <at> debbugs.gnu.org>; Mon, 07 Aug 2017 18:02:18 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=nwqWaWv89AMX/WK5X73fkgnSx8ipJWxeV8o6eKkUh+s=;
 b=AA/MAvIbQ/cuEBuJ6DNczJQj3HiBRL7uyRHpcyzZ2H+8zElsG3JZraS7S7oV9KSpZS
 KlMgElHr0PxaTdPzJp6rtkC5MAuAG0cVEp8f34/2B3IY/gAfQZ96Fyo6EeauwNt8ibOM
 f88KZYVOIrFVk0rSMz8oJuIGcqsMlk4X1ZZPJYHf51GM/3qYI3zBD+p25vsASjT/v5K4
 mC3elfsGFCQuUyNceHCxxBtvFIewfM+Z08AQOBnbhaTDh9Ujvua03I6r5/Zk6YdJJ6Th
 MzWG9uXucWnDGz8XAS8VkxYUnaDA/LpxntUTbJgl9/5rZEl4RNO8tzD5q40C1BQImqn5
 GgnQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:from:to:cc:subject:references:date
 :in-reply-to:message-id:user-agent:mime-version;
 bh=nwqWaWv89AMX/WK5X73fkgnSx8ipJWxeV8o6eKkUh+s=;
 b=IYUFprzP1+JxLwl/3ysRSVuNXkMnZ8lrhRGLan15ei04fSmSLi6KyAk569JeUJAYGP
 mIDeUkqtdsYWhAJo2fCg3WO8tlMDzPPQfZsrOYMd2+OB7APTJc9l+wMxiGK9ll2Olxe2
 L29QjEH+wENt1dY7e0GCV0oPvjYnWDJj64bAPx6F6aInhCyIlFDh7UrLhuMpJsh0RfrV
 hT58TwGA9neP82IAB1i/54d8MngNcrTRboZEW3qNvpsa2NHRP+Cbt3kJesHYVE9DIS6y
 n5QqXkxoAloO4xSFXtrOym3vxuS6Kt+dDioXPWOSzwZtpOfZrFjCmmfXDXTX8vPj7PY5
 GmcA==
X-Gm-Message-State: AHYfb5jz+uoIIFBeE56BhEGf2qK3hNHywuRkkd7IJtE0tDU12dO/YYBg
 USaytON/3kALh7wO
X-Received: by 10.107.201.65 with SMTP id z62mr2505506iof.74.1502154132411;
 Mon, 07 Aug 2017 18:02:12 -0700 (PDT)
Received: from zony ([45.2.119.49])
 by smtp.googlemail.com with ESMTPSA id e92sm3912381itd.0.2017.08.07.18.02.09
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Mon, 07 Aug 2017 18:02:09 -0700 (PDT)
From: npostavs@HIDDEN
To: Ken Raeburn <raeburn@HIDDEN>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
References: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
 <87a83d5zgz.fsf@HIDDEN>
Date: Mon, 07 Aug 2017 21:03:47 -0400
In-Reply-To: <87a83d5zgz.fsf@HIDDEN>
 (npostavs@HIDDEN's message of "Sat, 05 Aug 2017 20:09:16
 -0400")
Message-ID: <87h8xi50r0.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 27748
Cc: 27748 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

npostavs@HIDDEN writes:

> The following patch prevents custom-declare-variable from being compiled
> and lets the docstrings get printed properly.  Probably needs a bit more
> refinement though.

Hmm, this approach might not work at all, it causes a bazillion "free
variable reference" warnings, one for each reference to a defcustom
variable.




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.

Message received at 27748 <at> debbugs.gnu.org:


Received: (at 27748) by debbugs.gnu.org; 6 Aug 2017 00:07:46 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Aug 05 20:07:46 2017
Received: from localhost ([127.0.0.1]:43876 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1de96f-0000WT-NB
	for submit <at> debbugs.gnu.org; Sat, 05 Aug 2017 20:07:45 -0400
Received: from mail-io0-f196.google.com ([209.85.223.196]:32998)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <npostavs@HIDDEN>) id 1de96d-0000WG-VI
 for 27748 <at> debbugs.gnu.org; Sat, 05 Aug 2017 20:07:44 -0400
Received: by mail-io0-f196.google.com with SMTP id q64so3335004ioi.0
 for <27748 <at> debbugs.gnu.org>; Sat, 05 Aug 2017 17:07:43 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=sender:from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=OiOVSFeNOVy5vn/LjLnXvZOa4fC/50O9C6/jNtcGjZw=;
 b=WZnyHMp46ykTIM/50fsoom6a+S/W6ZanWYFgKbheSsTEAhBfPwgZl56Mde5V1JcI+W
 MdSnzuz5uvxX3uNHKEzEPonNIo2ZknCbveNqD7mhGJJkU4nthlNpx7DBek2ewBEhoUdN
 4eFrImF+8tIcY4ddZDZ6QabyOjnbYDdtve9zThCgIfpviRh7DyO2at5EruMJvqtYX85C
 luiyfhmslZnXe0y4kXknNJ6wxtEPLHcyRAJb0qgKC5ZrigK9iOV7Ome0oOgSBOAiWZKZ
 4socgg2O8I0oQyECm5mwxg0cqc+FCIfkOgEn0RFStDTwbXkcqKUno7k6ukxdpVSTnrUj
 rGig==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:sender:from:to:cc:subject:references:date
 :in-reply-to:message-id:user-agent:mime-version;
 bh=OiOVSFeNOVy5vn/LjLnXvZOa4fC/50O9C6/jNtcGjZw=;
 b=LMdhM70VKgbR8aQ/Rwebde13WEAdbmPRhvLnCKAFz+smRclNY/7pW3QiJBLiPzv6Mw
 oMAu3odBUMEi8PuUaqA6yJVBbi1XeBMwEHl9dm1LkjwDWDCBLOIE1OqsVGo3qmVFs2LC
 O2nnpbCOW2vOFWTtLsSifX+YyXO5yS9YxsVZ2MJrlRZhxtjA9mJ2HYD/k5VQPCDk+WL2
 J+xjK7cHMBm9+51BKGJfGTA8ShnNGIj5a1Lms7PDJTrJ/mJYqT1+zpLlTJUmsl6pFqgk
 cwcuRQ+jv2luE/MzR4QahkcPsRs09etAZbZ98IC8HAhmMImDzKOTt2OW+t4weWFXKBm+
 Z2+w==
X-Gm-Message-State: AHYfb5gavKtAyw0vS/pdlTJvf6vGEcIcWV6DFvIVVIFUuz2MkHq3G7SX
 jUccSQ4fn/DmBn3I
X-Received: by 10.107.164.98 with SMTP id n95mr7072939ioe.38.1501978058182;
 Sat, 05 Aug 2017 17:07:38 -0700 (PDT)
Received: from zony ([45.2.119.48])
 by smtp.googlemail.com with ESMTPSA id z75sm767349ita.18.2017.08.05.17.07.35
 (version=TLS1_2 cipher=ECDHE-RSA-CHACHA20-POLY1305 bits=256/256);
 Sat, 05 Aug 2017 17:07:35 -0700 (PDT)
From: npostavs@HIDDEN
To: Ken Raeburn <raeburn@HIDDEN>
Subject: Re: bug#27748: 26.0.50; doc strings should be in DOC file
References: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
Date: Sat, 05 Aug 2017 20:09:16 -0400
In-Reply-To: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN> (Ken Raeburn's
 message of "Tue, 18 Jul 2017 02:47:09 -0400")
Message-ID: <87a83d5zgz.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/25.2.50 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -2.1 (--)
X-Debbugs-Envelope-To: 27748
Cc: 27748 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.1 (--)

--=-=-=
Content-Type: text/plain

Ken Raeburn <raeburn@HIDDEN> writes:

>
> 1. defcustom doc strings from files compiled with lexical binding.
>
>    For example, files.el (lexical bindings) defines
>    delete-auto-save-files but it doesn't show up in the DOC file;
>    files.elc starts with an initial byte-code blob which includes the
>    symbol delete-auto-save-files and its doc string in the constants
>    array.

Actually, it's not only about lexical binding, the following file:

    ;;; -*- lexical-binding: nil -*-

    (defcustom custom-foo nil
      "a custom variable"
      :type 'boolean
      :group 'foo-group)

    ;; (defun foo ()
    ;;   t)

    (defcustom custom-bar nil
      "another custom variable"
      :type 'boolean
      :group 'foo-group)

produces (with prologue removed, and reformatted for readability)

    (byte-code "\300\301\302\303\304\305\306\307&\210\300\310\302\311\304\305\306\307&\207"
               [custom-declare-variable custom-foo nil "a custom variable" :type boolean :group foo-group
                                        custom-bar "another custom variable"]
               8)

Uncommenting the (defun foo...) produces:

    #@19 a custom variable
    (custom-declare-variable 'custom-foo nil '(#$ . 411) :type 'boolean :group 'foo-group)
    (defalias 'foo #[nil "\300\207" [t] 1])
    #@25 another custom variable
    (custom-declare-variable 'custom-bar nil '(#$ . 562) :type 'boolean :group 'foo-group)

Then changing to lexical binding produces:

    (byte-code "\300\301\302\303\304DD\305\306\307\310\311&\207"
               [custom-declare-variable
                custom-foo funcall function
                #[0 "\300\207" [nil] 1] "a custom variable"
                :type boolean :group foo-group]
               8)
    (defalias 'foo #[0 "\300\207" [t] 1])
    (byte-code "\300\301\302\303\304DD\305\306\307\310\311&\207"
               [custom-declare-variable
                custom-bar funcall function
                #[0 "\300\207" [nil] 1]
                "another custom variable" :type boolean :group foo-group]
               8)

As far as I can tell, the problem is that the
byte-compile-dynamic-docstrings feature (that's the #@19 thing) relies
on `byte-compile-out-toplevel' to decompile "trivial" functions back
into source code.  So having lexical binding set, or 2 defcustoms in a
row produces "non-trivial" code which is not decompiled, and therefore
not recognized in byte-compile-output-file-form as something which
should be used with byte-compile-output-docform.

    (defun byte-compile-out-toplevel (&optional for-effect output-type)
      ...
      ;; Decompile trivial functions:
      ...
        (cond
         ;; #### This should be split out into byte-compile-nontrivial-function-p.
         ((or (eq output-type 'lambda)
          (nthcdr (if (eq output-type 'file) 50 8) byte-compile-output)
          ...
            (while
                    (cond
                     ((memq (car (car rest)) '(byte-varref byte-constant))
                     ...
                     ((and maycall
                           ;; Allow a funcall if at most one atom follows it.
                      ...
                      (setq maycall nil)	; Only allow one real function call.
                      ...
                      (or (eq output-type 'file)
                          (not (delq nil (mapcar 'consp (cdr (car body))))))))
            ...
        (list 'byte-code (byte-compile-lapcode byte-compile-output)
              byte-compile-vector byte-compile-maxdepth)))
         ;; it's a trivial function
         ((cdr body) (cons 'progn (nreverse body)))
         ((car body))))

    (defun byte-compile-output-file-form (form)
      ;; Write the given form to the output buffer, being careful of docstrings
      ;; in defvar, defvaralias, defconst, autoload and
      ;; custom-declare-variable because make-docfile is so amazingly stupid.
      ...
        (if (and (memq (car-safe form) '(defvar defvaralias defconst
                                          autoload custom-declare-variable))
                 (stringp (nth 3 form)))
            (byte-compile-output-docform nil nil '("\n(" 3 ")") form nil
                                         (memq (car form)
                                               '(defvaralias autoload
                                                  custom-declare-variable)))
          (princ "\n" byte-compile--outbuffer)
          (prin1 form byte-compile--outbuffer)
          nil)))

The following patch prevents custom-declare-variable from being compiled
and lets the docstrings get printed properly.  Probably needs a bit more
refinement though.


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-lisp-custom.el-custom-declare-variable-Don-t-compile.patch
Content-Description: patch

From 4cb45936966de76f91b95971c886599a24361c5b Mon Sep 17 00:00:00 2001
From: Noam Postavsky <npostavs@HIDDEN>
Date: Sat, 5 Aug 2017 20:02:19 -0400
Subject: [PATCH] * lisp/custom.el (custom-declare-variable): Don't compile
 (Bug#27748).

---
 lisp/custom.el | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/lisp/custom.el b/lisp/custom.el
index ecfa34db5b..5876d3fd56 100644
--- a/lisp/custom.el
+++ b/lisp/custom.el
@@ -144,6 +144,9 @@ (defun custom-declare-variable (symbol default doc &rest args)
 DEFAULT is stored as SYMBOL's standard value, in SYMBOL's property
 `standard-value'.  At the same time, SYMBOL's property `force-value' is
 set to nil, as the value is no longer rogue."
+  (declare (compiler-macro
+            (lambda (form)
+              `(eval ',form lexical-binding))))
   (put symbol 'standard-value (purecopy (list default)))
   ;; Maybe this option was rogue in an earlier version.  It no longer is.
   (when (get symbol 'force-value)
-- 
2.11.1


--=-=-=--




Information forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 18 Jul 2017 06:47:29 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jul 18 02:47:29 2017
Received: from localhost ([127.0.0.1]:45247 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1dXMI4-00044T-Mc
	for submit <at> debbugs.gnu.org; Tue, 18 Jul 2017 02:47:29 -0400
Received: from eggs.gnu.org ([208.118.235.92]:48376)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <raeburn@HIDDEN>) id 1dXMI2-00044F-H7
 for submit <at> debbugs.gnu.org; Tue, 18 Jul 2017 02:47:26 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <raeburn@HIDDEN>) id 1dXMHv-0000Y1-RM
 for submit <at> debbugs.gnu.org; Tue, 18 Jul 2017 02:47:21 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=0.8 required=5.0 tests=BAYES_50,T_DKIM_INVALID
 autolearn=disabled version=3.3.2
Received: from lists.gnu.org ([2001:4830:134:3::11]:33252)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32)
 (Exim 4.71) (envelope-from <raeburn@HIDDEN>) id 1dXMHv-0000Xv-O7
 for submit <at> debbugs.gnu.org; Tue, 18 Jul 2017 02:47:19 -0400
Received: from eggs.gnu.org ([2001:4830:134:3::10]:39343)
 by lists.gnu.org with esmtp (Exim 4.71)
 (envelope-from <raeburn@HIDDEN>) id 1dXMHu-0005MB-1K
 for bug-gnu-emacs@HIDDEN; Tue, 18 Jul 2017 02:47:19 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <raeburn@HIDDEN>) id 1dXMHq-0000T5-4B
 for bug-gnu-emacs@HIDDEN; Tue, 18 Jul 2017 02:47:18 -0400
Received: from mail-qk0-x22e.google.com ([2607:f8b0:400d:c09::22e]:34103)
 by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_128_CBC_SHA1:16)
 (Exim 4.71) (envelope-from <raeburn@HIDDEN>) id 1dXMHp-0000RD-S7
 for bug-gnu-emacs@HIDDEN; Tue, 18 Jul 2017 02:47:14 -0400
Received: by mail-qk0-x22e.google.com with SMTP id t2so6091492qkc.1
 for <bug-gnu-emacs@HIDDEN>; Mon, 17 Jul 2017 23:47:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=raeburn-org.20150623.gappssmtp.com; s=20150623;
 h=from:content-transfer-encoding:date:subject:to:message-id
 :mime-version; bh=QOW1tGbepHh+9EgkB7CuM27HYu9yC7MJBoDAlkESmV8=;
 b=k8YfqFyhbrtyx6LGK04dCpc7ZCOXS6XDJiy/tLGSl6EUyw+LM7VAC/IpLhYc5PkGXC
 IeI/lRnU55cAvNERgV0TUNFfc4s5fTOr5sLobuJx4bzhZFFdUpoU8lXk3A8CRPpIIojE
 s8PCkiyfgOalsst33z/IUvK5QBFidy2skjnOWuQr3ZntamrAeUuGnFSAHBoBUinEDDOB
 3/YazQdB+qV4k6jX11QksAXoR780NH71+9WVG5jIQML7Ti5In9dXpMOLxAQ1SlA+sJ5T
 mdDRLeI4CZ873wYuxHf++/Ya0pNRez65csoJC2uJ2GYLipXlkkVnQpdHj+P6vF84YiQz
 xwJg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:content-transfer-encoding:date:subject:to
 :message-id:mime-version;
 bh=QOW1tGbepHh+9EgkB7CuM27HYu9yC7MJBoDAlkESmV8=;
 b=WfqjVe6NKhBkUNEdnYxk5Px0uWifzC+pXO3S2bMtigFwzZmK8luApDq7ivHuzvZ6Va
 zWbR/4C/6pli42bD0qo7VQkv+xHXe58m76YaM0iU6XABAu+r49f+3QY55tO6wIGH4Xpw
 NXcrd8gia65mgP9tuTBSl/6ypvgyMs5EFsr2a8L1JxLqOMp5NzuRTK0VrrYcZi5MgfXK
 5NmPieUwOpeOKKKx3QnNxJ6XvPk4UjRc3aYn5eXXVWBTtzJlCg/OX6L6iWQWgZ0QF5pn
 TpvNlC0wiPFWCxSB4q7T+wWe85WCgio3Sbd4CNli2rht9QY+g20ZISidlsXETug9jZe5
 n7Ug==
X-Gm-Message-State: AIVw111u4Sp0Xt2J3Ry5ZFJ8TGrfdQIdcWcVUu5EVz16elfPDnavpW2Q
 Bgd3jXBkYlkkXT8f2o1cxQ==
X-Received: by 10.55.187.134 with SMTP id l128mr231678qkf.90.1500360431264;
 Mon, 17 Jul 2017 23:47:11 -0700 (PDT)
Received: from [192.168.23.52] (c-73-253-167-23.hsd1.ma.comcast.net.
 [73.253.167.23])
 by smtp.gmail.com with ESMTPSA id g204sm990574qkb.50.2017.07.17.23.47.10
 (version=TLS1 cipher=ECDHE-RSA-AES128-SHA bits=128/128);
 Mon, 17 Jul 2017 23:47:10 -0700 (PDT)
From: Ken Raeburn <raeburn@HIDDEN>
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Date: Tue, 18 Jul 2017 02:47:09 -0400
Subject: 26.0.50; doc strings should be in DOC file
To: Bug-Gnu-Emacs <bug-gnu-emacs@HIDDEN>
Message-Id: <11A6DB23-F538-4A59-AC90-7CF5040FA496@HIDDEN>
Mime-Version: 1.0 (Mac OS X Mail 9.3 \(3124\))
X-Mailer: Apple Mail (2.3124)
X-detected-operating-system: by eggs.gnu.org: Genre and OS details not
 recognized.
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x
X-Received-From: 2001:4830:134:3::11
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

There are a bunch of doc strings from the preloaded Lisp files that do =
not wind up in the DOC file.  Presumably this means they wind up in the =
emacs executable image itself, saved away as extra string objects that =
GC needs to track.  (In the big-elc-file work Stefan started and I'm =
experimenting with, these doc strings wind up in the dumped Lisp =
environment file, and need to be parsed and saved away at load time.)

1. defcustom doc strings from files compiled with lexical binding.

   For example, files.el (lexical bindings) defines
   delete-auto-save-files but it doesn't show up in the DOC file;
   files.elc starts with an initial byte-code blob which includes the
   symbol delete-auto-save-files and its doc string in the constants
   array.

   On the other hand, custom.el (dynamic bindings) declares
   custom-theme-directory, the .elc file dumps out the doc string in a
   #@... block before a separate top-level call to
   custom-declare-variable, and since this is what make-docfile looks
   for, the doc string winds up in the DOC file.

2. In isearch, CL macro expansion results in symbols like
   isearch--state-forward--cmacro having function definitions that
   include the two doc strings "Access slot \"forward\" of
   `(isearch--state (:constructor nil) (:copier nil) (:constructor
   isearch--get-state (&aux (string isearch-string) (message
   isearch-message) (point (point)) (success isearch-success) (forward
   isearch-forward) (other-end isearch-other-end) (word
   isearch-regexp-function) (error isearch-error) (wrapped
   isearch-wrapped) (barrier isearch-barrier) (case-fold-search
   isearch-case-fold-search) (pop-fun (if isearch-push-state-function
   (funcall isearch-push-state-function))))))' struct CL-X." and
   "\n\n(fn CL-WHOLE-ARG CL-X)".  The former string is also in DOC (with
   "\n\n(fn CL-X)" appended) as the documentation for the function
   isearch--state-forward.

   It appears that, as far as the Emacs help system is concerned,
   isearch--state-*--cmacro functions are undocumented.

   The --cmacro functions generate cl-block forms that include the
   original doc string, since it's treated as part of the body, but it's
   not clear to me whether it has any use there at all, or if it could
   just be discarded, or perhaps looked up via the non --cmacro symbols
   at run time if it is of use.

3. Undocumented functions, strange as it sounds... in files.el, function
   file-name-non-special is defined with no documentation.  In
   files.elc, the bytecode object constructed has a doc string "\n\n(fn
   OPERATION &rest ARGUMENTS)" instead of a (#$ . NNN) reference to a
   separate string that make-docfile can pick up.

To =E2=80=9Creproduce=E2=80=9D=E2=80=A6

Delete or =E2=80=9Cchmod 0=E2=80=9D the DOC file before starting Emacs.  =
Use ielm to look at the symbol property lists of delete-auto-save-files =
and custom-theme-directory; the former has a string for its =
variable-documentation property, and the latter has a number.  Look at =
the function definitions of isearch--state-forward--cmacro and =
file-name-non-special and see the doc strings in them.  Examine the .elc =
files and DOC.






In GNU Emacs 26.0.50 (build 2, x86_64-apple-darwin15.6.0, NS =
appkit-1404.47 Version 10.11.6 (Build 15G1217))
 of 2017-07-18 built on bang.local
Repository revision: 0083123499cc29e301c197218d3809b225675e57
Windowing system distributor 'Apple', version 10.3.1404
Recent messages:
Loading ~/lib/elisp/sue.elc...done
Loading desktop...done
Warning: desktop file appears to be in use by PID 83193.
Using it may cause conflicts.  Use it anyway? (y or n) n
Desktop file in use; not loaded.
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured features:
RSVG IMAGEMAGICK DBUS NOTIFY ACL GNUTLS LIBXML2 ZLIB TOOLKIT_SCROLL_BARS
NS

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  desktop-save-mode: t
  global-hi-lock-mode: t
  hi-lock-mode: t
  which-function-mode: t
  icomplete-mode: t
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr warnings emacsbug message subr-x puny dired
dired-loaddefs rfc822 mml mml-sec epa derived epg gnus-util rmail
rmail-loaddefs mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader add-log desktop frameset cus-start
cus-load kr-defs hi-lock which-func imenu icomplete iso-transl
smart-quotes easy-mmode tramp tramp-compat tramp-loaddefs trampver shell
pcomplete comint ansi-color ring parse-time format-spec advice cc-mode
cc-fonts cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars
cc-defs server time sendmail rfc2047 rfc2045 ietf-drums mm-util
mail-prsvr mail-utils finder-inf package easymenu epg-config
url-handlers url-parse auth-source cl-seq eieio eieio-core cl-macs
eieio-loaddefs password-cache url-vars seq byte-opt gv bytecomp
byte-compile cconv cl-loaddefs cl-lib time-date tooltip eldoc electric
uniquify ediff-hook vc-hooks lisp-float-type mwheel term/ns-win ns-win
ucs-normalize mule-util term/common-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode elisp-mode
lisp-mode prog-mode register page menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame cl-generic cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms cp51932
hebrew greek romanian slovak czech european ethiopic indian cyrillic
chinese composite charscript charprop case-table epa-hook jka-cmpr-hook
help simple abbrev obarray minibuffer cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote dbusbind kqueue cocoa ns multi-tty make-network-process emacs)

Memory information:
((conses 16 261240 12410)
 (symbols 48 26239 2)
 (miscs 40 57 285)
 (strings 32 47062 2197)
 (string-bytes 1 1448035)
 (vectors 16 42658)
 (vector-slots 8 784892 12266)
 (floats 8 70 90)
 (intervals 56 241 0)
 (buffers 992 12))





Acknowledgement sent to Ken Raeburn <raeburn@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-gnu-emacs@HIDDEN. Full text available.
Report forwarded to bug-gnu-emacs@HIDDEN:
bug#27748; Package emacs. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 31 Aug 2017 01:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.