GNU bug report logs - #42701
28.0.50; Duplicate Edebug instrumentation of lambda form in some cases

Previous Next

To reply to this bug, email your comments to 42701 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 28.0.50; Duplicate Edebug instrumentation of lambda form in some cases
Date: Mon, 03 Aug 2020 18:47:28 +0200

Create a file /tmp/edebug.el:

$ cat /tmp/edebug.el 
(defun f () (if-let (x (funcall (lambda (y) 1) 2)) 3 4))

Visit the file:

$ emacs -Q -l subr-x -l edebug /tmp/edebug.el

Now instrument the function `f' using C-u C-M-x.  The *Messages* buffer
will now contain

Edebug: edebug-anon0 [2 times]
Edebug: f

The [2 times] indicates the problem: Edebug has instrumented two
definitions with the same (generated) symbol.  This is a problem when
using Edebug for e.g. coverage instrumentation, as the coverage
information is attached to the symbol itself (as a symbol property), and
duplicate symbols when instrumenting code lead to subtle errors such as
mismatching vector lengths for the position of the breakpoints and the
hit counts.

I'd speculate that this issue is similar to Bug#41988 in that Edebug
defines instrumented symbols even when backtracking later.  In this
case, Edebug backtracks to the legacy (SYM VAL) form, but has already
partially matched the ((SYM VAL)) form, including instrumenting the
lambda form therein.  I guess Edebug should perform some kind of
two-phase instrumentation and instrument subforms only when a form has
been chosen after backtracking.  Since this is somewhat difficult to
implement without rewriting larger parts of Edebug, it might be more
feasible to regenerate anonymous symbols after a failed match.



In GNU Emacs 28.0.50 (build 72, x86_64-pc-linux-gnu, GTK+ Version 3.24.18, cairo version 1.16.0)
 of 2020-08-03
Repository revision: 16b7f413a9ff819c374e07ee927c1fd2b4138109
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
System Description: Debian GNU/Linux rodete

Recent messages:
For information about GNU Emacs and the GNU system, type C-h C-a.

Configured using:
 'configure --enable-gcc-warnings=warn-only
 --enable-gtk-deprecation-warnings --without-pop --with-mailutils
 --enable-checking=all --enable-check-lisp-object-type --with-modules
 'CFLAGS=-O1 -ggdb3 -fno-omit-frame-pointer -fsanitize=address
 -fsanitize=undefined -fsanitize=pointer-compare
 -fsanitize=pointer-subtract''

Configured features:
XPM JPEG TIFF GIF PNG CAIRO SOUND DBUS GSETTINGS GLIB NOTIFY INOTIFY
LIBSELINUX GNUTLS FREETYPE HARFBUZZ ZLIB TOOLKIT_SCROLL_BARS GTK3 X11
XDBE XIM MODULES THREADS LIBSYSTEMD JSON PDUMPER

Important settings:
  value of $LANG: en_US.utf8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: Lisp Interaction

Minor modes in effect:
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
None found.

Features:
(shadow sort mail-extr emacsbug message rmc dired dired-loaddefs rfc822
mml easymenu mml-sec epa epg epg-config gnus-util rmail rmail-loaddefs
text-property-search time-date mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils phst skeleton derived edmacro
kmacro pcase ffap thingatpt url url-proxy url-privacy url-expand
url-methods url-history url-cookie url-domsuf url-util url-parse
auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars mailcap subr-x rx gnutls puny seq
byte-opt gv bytecomp byte-compile cconv dbus xml compile comint
ansi-color ring cl-loaddefs cl-lib tooltip eldoc electric uniquify
ediff-hook vc-hooks lisp-float-type mwheel term/x-win x-win
term/common-win x-dnd tool-bar dnd fontset image regexp-opt fringe
tabulated-list replace newcomment text-mode elisp-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch timer
select scroll-bar mouse jit-lock font-lock syntax facemenu font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite charscript charprop case-table epa-hook
jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote threads dbusbind inotify dynamic-setting system-font-setting
font-render-setting cairo move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 68791 8062)
 (symbols 48 8629 1)
 (strings 32 23741 1958)
 (string-bytes 1 764239)
 (vectors 16 13112)
 (vector-slots 8 166367 5382)
 (floats 8 25 30)
 (intervals 56 221 0)
 (buffers 992 11))

-- 
Google Germany GmbH
Erika-Mann-Straße 33
80636 München

Geschäftsführer: Paul Manicle, Halimah DeLaine Prado
Registergericht und -nummer: Hamburg, HRB 86891
Sitz der Gesellschaft: Hamburg

Diese E-Mail ist vertraulich.  Falls Sie diese fälschlicherweise erhalten haben
sollten, leiten Sie diese bitte nicht an jemand anderes weiter, löschen Sie
alle Kopien und Anhänge davon und lassen Sie mich bitte wissen, dass die E-Mail
an die falsche Person gesendet wurde.

This e-mail is confidential.  If you received this communication by mistake,
please don’t forward it to anyone else, please erase all copies and
attachments, and please let me know that it has gone to the wrong person.

Message #8 received at 42701 <at> debbugs.gnu.org (full text, mbox):

From: Alan Mackenzie <acm <at> muc.de>
To: Philipp Stephani <p.stephani2 <at> gmail.com>
Cc: acm <at> muc.de, 42701 <at> debbugs.gnu.org
Subject: Re: bug#42701: 28.0.50;
 Duplicate Edebug instrumentation of lambda form in some cases
Date: 4 Aug 2020 20:12:58 -0000

Hello, Philipp.

In article <mailman.680.1596473284.2739.bug-gnu-emacs <at> gnu.org> you wrote:

> Create a file /tmp/edebug.el:

> $ cat /tmp/edebug.el 
> (defun f () (if-let (x (funcall (lambda (y) 1) 2)) 3 4))

> Visit the file:

> $ emacs -Q -l subr-x -l edebug /tmp/edebug.el

> Now instrument the function `f' using C-u C-M-x.  The *Messages* buffer
> will now contain

> Edebug: edebug-anon0 [2 times]
> Edebug: f

> The [2 times] indicates the problem: Edebug has instrumented two
> definitions with the same (generated) symbol.

I don't think you're correct, here.  I think it's instrumented the
lambda form twice, once for each arm of the edebug spec.  It discards
the first attempt, then succeeds at the second.

The pertinent edebug spec (from the if-let definition in subr-x.el)
looks like:

    ([&or (&rest [&or symbolp (symbolp form) (form)])
          (symbolp form)]
     form body)

.  In the outer &or sub-spec, the (&rest ....) bit doesn't match, but
the failure to match only happens after the "Edebug: edebug-anon0"
message has been output the first time.  Edebug then tries the (symbolp
form) alternative, which does match and outputs the "...-anon0" message
a second time.

> This is a problem when using Edebug for e.g. coverage instrumentation,
> as the coverage information is attached to the symbol itself (as a
> symbol property), and duplicate symbols when instrumenting code lead
> to subtle errors such as mismatching vector lengths for the position
> of the breakpoints and the hit counts.

I don't think this is happening (see above), but I admit not having
looked into it all that closely.  Do you have any further evidence of
two distinct functions being mapped onto the same generated symbol?

> I'd speculate that this issue is similar to Bug#41988 in that Edebug
> defines instrumented symbols even when backtracking later.  In this
> case, Edebug backtracks to the legacy (SYM VAL) form, but has already
> partially matched the ((SYM VAL)) form, including instrumenting the
> lambda form therein.  I guess Edebug should perform some kind of
> two-phase instrumentation and instrument subforms only when a form has
> been chosen after backtracking.  Since this is somewhat difficult to
> implement without rewriting larger parts of Edebug, it might be more
> feasible to regenerate anonymous symbols after a failed match.

I don't know why Edebug re-uses the symbol.  But beyond the misleading
double message in *Messages*, is there any harm done?  Would it be less
confusing if two distinct messages "Edebug: edebug-anon0" and "Edebug:
edebug-anon1" were to be output?

> In GNU Emacs 28.0.50 (build 72, x86_64-pc-linux-gnu, GTK+ Version 3.24.18, cairo version 1.16.0)
>  of 2020-08-03
> Repository revision: 16b7f413a9ff819c374e07ee927c1fd2b4138109
> Repository branch: master
> Windowing system distributor 'The X.Org Foundation', version 11.0.12008000
> System Description: Debian GNU/Linux rodete

[ .... ]

-- 
Alan Mackenzie (Nuremberg, Germany).

Message #11 received at 42701 <at> debbugs.gnu.org (full text, mbox):

From: Philipp Stephani <p.stephani2 <at> gmail.com>
To: Alan Mackenzie <acm <at> muc.de>
Cc: 42701 <at> debbugs.gnu.org
Subject: Re: bug#42701: 28.0.50; Duplicate Edebug instrumentation of lambda
 form in some cases
Date: Sun, 9 Aug 2020 16:59:57 +0200

)

Am Di., 4. Aug. 2020 um 22:12 Uhr schrieb Alan Mackenzie <acm <at> muc.de>:
>
> Hello, Philipp.
>
> In article <mailman.680.1596473284.2739.bug-gnu-emacs <at> gnu.org> you wrote:
>
> > Create a file /tmp/edebug.el:
>
> > $ cat /tmp/edebug.el
> > (defun f () (if-let (x (funcall (lambda (y) 1) 2)) 3 4))
>
> > Visit the file:
>
> > $ emacs -Q -l subr-x -l edebug /tmp/edebug.el
>
> > Now instrument the function `f' using C-u C-M-x.  The *Messages* buffer
> > will now contain
>
> > Edebug: edebug-anon0 [2 times]
> > Edebug: f
>
> > The [2 times] indicates the problem: Edebug has instrumented two
> > definitions with the same (generated) symbol.
>
> I don't think you're correct, here.  I think it's instrumented the
> lambda form twice, once for each arm of the edebug spec.  It discards
> the first attempt, then succeeds at the second.
>
> The pertinent edebug spec (from the if-let definition in subr-x.el)
> looks like:
>
>     ([&or (&rest [&or symbolp (symbolp form) (form)])
>           (symbolp form)]
>      form body)
>
> .  In the outer &or sub-spec, the (&rest ....) bit doesn't match, but
> the failure to match only happens after the "Edebug: edebug-anon0"
> message has been output the first time.  Edebug then tries the (symbolp
> form) alternative, which does match and outputs the "...-anon0" message
> a second time.

That is correct, but when the match fails, it's already too late:
Edebug (in `edebug-make-form-wrapper') has already performed some
global mutations such as modifying `edebug-form-data' of the `edebug'
property of the newly-generated symbol. These mutations aren't unwound
after the first failed match.

>
> > This is a problem when using Edebug for e.g. coverage instrumentation,
> > as the coverage information is attached to the symbol itself (as a
> > symbol property), and duplicate symbols when instrumenting code lead
> > to subtle errors such as mismatching vector lengths for the position
> > of the breakpoints and the hit counts.
>
> I don't think this is happening (see above), but I admit not having
> looked into it all that closely.  Do you have any further evidence of
> two distinct functions being mapped onto the same generated symbol?

I've tried to provide some context in
https://lists.gnu.org/archive/html/bug-gnu-emacs/2020-08/msg00574.html.
The most common symptom of such duplicate definitions (as defined by
"calling `edebug-make-form-wrapper' twice with the same
`edebug-def-name'") is
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853.

>
> > I'd speculate that this issue is similar to Bug#41988 in that Edebug
> > defines instrumented symbols even when backtracking later.  In this
> > case, Edebug backtracks to the legacy (SYM VAL) form, but has already
> > partially matched the ((SYM VAL)) form, including instrumenting the
> > lambda form therein.  I guess Edebug should perform some kind of
> > two-phase instrumentation and instrument subforms only when a form has
> > been chosen after backtracking.  Since this is somewhat difficult to
> > implement without rewriting larger parts of Edebug, it might be more
> > feasible to regenerate anonymous symbols after a failed match.
>
> I don't know why Edebug re-uses the symbol.
.
My guess is that `edebug-old-def-name' becomes non-nil because the
form has already been instrumented
(https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/edebug.el?id=1a845a672dc73c8e98e6cb9bb734616e168e60ba#n1382).
It's then reused in
https://git.savannah.gnu.org/cgit/emacs.git/tree/lisp/emacs-lisp/edebug.el?id=1a845a672dc73c8e98e6cb9bb734616e168e60ba#n1250.

>  But beyond the misleading
> double message in *Messages*, is there any harm done?  Would it be less
> confusing if two distinct messages "Edebug: edebug-anon0" and "Edebug:
> edebug-anon1" were to be output?

The duplicate message is just a symptom of calling
`edebug-make-form-wrapper' twice with the same `edebug-def-name'
symbol, and attaching various instrumentation data to the symbol.
Depending on whether the two definitions are compatible, this can be
harmless or lead to subtle issues like
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853. Therefore in my
coverage instrumentation driver I'm rejecting such duplicate
instrumentations
(https://github.com/phst/rules_elisp/commit/31d30e99c21027c5859782229aebb314ed3cb36c,
since it doesn't seem to be possible to predict whether the
redefinition would indeed be harmless or trigger
https://debbugs.gnu.org/cgi/bugreport.cgi?bug=41853.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.