GNU bug report logs - #12409
24.2.50; OS X Python completion leaks junk to inferior-python-mode buffer

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 12409 in the body.
You can then email your comments to 12409 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Dan Davison <dandavison7 <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 10 Sep 2012 22:22:02 -0700

1. emacs -nw -q
2. M-x run-python
3. At the prompt, enter "aaaa = 4" followed by RETURN
4. At the prompt, enter "aaaaaa = 6" followed by RETURN
5. At the prompt enter "aa" followed by TAB to request a completion

After step (4) the buffer contents look like:
>>> aaaa = 4
>>> aaaaaa = 6
>>>

After step (5) it looks like:

>>> aaaa = 4
>>> aaaaaa = 6
>>> 'aaaa;aaaaaa'
>>> aaaa

Notice that the string containing possible completions was leaked to the
shell buffer.

The bug occurs both when running emacs in a terminal with `emacs -nw -q`
and when running the emacs cocoa app. Some discussion of this bug can
be found at
https://github.com/fgallina/python.el/issues/92. It seems to be OS X-specific,
related to `accept-process-output`.

Dan


In GNU Emacs 24.2.50.1 (x86_64-apple-darwin10.8.0, NS apple-appkit-1038.36)
 of 2012-09-10 on cotinga.local
Windowing system distributor `Apple', version 10.3.1038
Configured using:
 `configure '--with-ns' '--with-png=no' '--with-gif=no''

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: Inferior Python

Minor modes in effect:
  compilation-shell-minor-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t
  transient-mark-mode: t

Recent input:
SPC ESC O A ESC O D ESC O D ESC O D ESC O D ESC O D
ESC O D ESC O D ESC O D RET ESC O B ESC O C ESC O C
ESC O C ESC O C ESC O C ESC O C ESC O C ESC O C ESC
O C RET C-x o ESC O A ESC O B ESC O B ESC O B C-x o
ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC
O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B
ESC O B ESC O B ESC O B ESC O B ESC O B ESC O B ESC
O B ESC O B ESC O A ESC O A ESC O A ESC O A ESC O A
ESC O A ESC O A ESC O B ESC O A ESC x r e p o TAB r
TAB TAB RET C-g C-x b * P y TAB RET C-h v m a j o TAB
RET C-x C-f C-g C-u C-g C-x C-f DEL DEL DEL DEL DEL
DEL DEL DEL DEL DEL DEL DEL DEL DEL DEL DEL DEL DEL
DEL DEL DEL DEL DEL d a v TAB n o n TAB p y t TAB .
TAB z TAB . p y RET d e f s DEL a TAB ESC O B TAB ESC
O B TAB ESC O B a TAB ESC O B DEL ESC x c o m p l TAB
i TAB a t TAB RET a ESC x ESC O A RET C-x b RET ESC
x r e p o r TAB RET

Recent messages:
Can't guess python-indent-offset, using defaults: 4
Loading vc-git...done
...on_work/python.el/z.py locked by davison <at> cotin... (pid 2599): (s, q, p, ?)?
Please type q, s, or p; or ? for help
...on_work/python.el/z.py locked by davison <at> cotin... (pid 2599): (s, q, p, ?)?
Please type q, s, or p; or ? for help
...on_work/python.el/z.py locked by davison <at> cotin... (pid 2599): (s, q, p, ?)?
byte-code: End of buffer [4 times]
Making completion list...
You can run the command `completion-at-point' with C-M-i [2 times]

Load-path shadows:
None found.

Features:
(shadow sort mail-extr vc-git pp browse-url wid-edit network-stream
starttls url-http tls url-gw url-cache url-auth url url-proxy
url-privacy url-expand url-methods url-history url-cookie url-domsuf
url-util url-parse auth-source eieio byte-opt bytecomp byte-compile
cconv gnus-util password-cache url-vars mailcap emacsbug message
format-spec rfc822 mml mml-sec mm-decode mm-bodies mm-encode mail-parse
rfc2231 mailabbrev gmm-utils mailheader sendmail rfc2047 rfc2045
ietf-drums mm-util mail-prsvr mail-utils compile python rx comint ring
ansi-color help-mode easymenu help-fns time-date tooltip ediff-hook
vc-hooks lisp-float-type mwheel ns-win tool-bar dnd fontset image
regexp-opt fringe tabulated-list newcomment lisp-mode register page
menu-bar rfn-eshadow timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew greek
romanian slovak czech european ethiopic indian cyrillic chinese
case-table epa-hook jka-cmpr-hook help simple abbrev minibuffer loaddefs
button faces cus-face macroexp files text-properties overlay sha1 md5
base64 format env code-pages mule custom widget hashtable-print-readable
backquote make-network-process ns multi-tty emacs)

Message #8 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: martin rudalics <rudalics <at> gmx.at>
To: Dan Davison <dandavison7 <at> gmail.com>
Cc: 12409 <at> debbugs.gnu.org
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Wed, 12 Sep 2012 10:09:26 +0200

> 1. emacs -nw -q
> 2. M-x run-python
> 3. At the prompt, enter "aaaa = 4" followed by RETURN
> 4. At the prompt, enter "aaaaaa = 6" followed by RETURN
> 5. At the prompt enter "aa" followed by TAB to request a completion
>
> After step (4) the buffer contents look like:
>>>> aaaa = 4
>>>> aaaaaa = 6
>>>>
>
> After step (5) it looks like:
>
>>>> aaaa = 4
>>>> aaaaaa = 6
>>>> 'aaaa;aaaaaa'
>>>> aaaa

This could be related to the behavior of `complete-with-action'.  See
also bug#10851.

martin

Message #11 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Fabián Ezequiel Gallina <fabian <at> anue.biz>
To: 12409 <at> debbugs.gnu.org
Subject: 24.2.50; OS X Python completion leaks junk to inferior-python-mode
	buffer
Date: Mon, 24 Sep 2012 15:32:45 -0300

I believe this is directly related to #10295.

Message #14 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Fabián Ezequiel Gallina <fabian <at> anue.biz>
To: 12409 <at> debbugs.gnu.org
Subject: 24.2.50; OS X Python completion leaks junk to inferior-python-mode
	buffer
Date: Sun, 30 Sep 2012 22:00:47 -0300

I just pushed in revno 110300 something that might fix this. I'm using 
the same strategy that gud-gdb has for grabbing output from the inferior 
process.

Confirmations are much appreciated.

Message #17 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Jan Djärv <jan.h.d <at> swipnet.se>
To: martin rudalics <rudalics <at> gmx.at>,
	Fabián Ezequiel Gallina <fabian <at> anue.biz>
Cc: 12409 <at> debbugs.gnu.org, Dan Davison <dandavison7 <at> gmail.com>
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Sun, 7 Oct 2012 20:05:56 +0200

Hello.

In the trunk, it does behave much better:

 >>> aaaa = 4
aaaa = 4
>>> aaaaaa = 6
aaaaaa = 6
>>> aaaa


The first TAB on aa completes to aaaa, no garbage characters.
But on the second TAB, the completion buffer says:

Click <mouse-2> on a completion to select it.
In this buffer, type RET to select the completion near point.

Possible completions are:
aaaa
aaaaaa'^M

i.e. a trailing '^M is shown.  That is a real carriage return, not ^ and M btw.

	Jan D.


12 sep 2012 kl. 10:09 skrev martin rudalics <rudalics <at> gmx.at>:

> > 1. emacs -nw -q
> > 2. M-x run-python
> > 3. At the prompt, enter "aaaa = 4" followed by RETURN
> > 4. At the prompt, enter "aaaaaa = 6" followed by RETURN
> > 5. At the prompt enter "aa" followed by TAB to request a completion
> >
> > After step (4) the buffer contents look like:
> >>>> aaaa = 4
> >>>> aaaaaa = 6
> >>>>
> >
> > After step (5) it looks like:
> >
> >>>> aaaa = 4
> >>>> aaaaaa = 6
> >>>> 'aaaa;aaaaaa'
> >>>> aaaa
> 
> This could be related to the behavior of `complete-with-action'.  See
> also bug#10851.
> 
> martin
> 
> 

Message #20 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Fabián Ezequiel Gallina <fabian <at> anue.biz>
To: 12409 <at> debbugs.gnu.org
Subject: 24.2.50; OS X Python completion leaks junk to inferior-python-mode
	buffer
Date: Sun, 07 Oct 2012 16:51:20 -0300

Does the attached patch fix that?


=== modified file 'lisp/progmodes/python.el'
*** lisp/progmodes/python.el    2012-10-07 19:37:37 +0000
--- lisp/progmodes/python.el    2012-10-07 19:47:52 +0000
***************
*** 1887,1893 ****
     python-shell-output-filter-buffer
     (concat python-shell-output-filter-buffer string))
    (when (string-match
!          (format "\n\\(?:%s\\|%s\\|%s\\)$"
                   python-shell-prompt-regexp
                   python-shell-prompt-block-regexp
                   python-shell-prompt-pdb-regexp)
--- 1887,1895 ----
     python-shell-output-filter-buffer
     (concat python-shell-output-filter-buffer string))
    (when (string-match
!          ;; It seems on OSX an extra carriage return might be attached
!          ;; to the end of output.
!          (format " ?\n\\(?:%s\\|%s\\|%s\\)$"
                   python-shell-prompt-regexp
                   python-shell-prompt-block-regexp
                   python-shell-prompt-pdb-regexp)

Message #23 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Jan Djärv <jan.h.d <at> swipnet.se>
To: Fabián Ezequiel Gallina <fabian <at> anue.biz>
Cc: 12409 <at> debbugs.gnu.org
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 8 Oct 2012 09:24:03 +0200

Hello.

7 okt 2012 kl. 21:51 skrev Fabián Ezequiel Gallina <fabian <at> anue.biz>:

> Does the attached patch fix that?
> 
> 

No, unfortunately it does not.

The value of  python-shell-output-filter-buffer is:

';'.join(__COMPLETER_all_completions('''aaaa'''))^M
'aaaa;aaaaaa'^M
>>>


	Jan D.

> === modified file 'lisp/progmodes/python.el'
> *** lisp/progmodes/python.el    2012-10-07 19:37:37 +0000
> --- lisp/progmodes/python.el    2012-10-07 19:47:52 +0000
> ***************
> *** 1887,1893 ****
>     python-shell-output-filter-buffer
>     (concat python-shell-output-filter-buffer string))
>    (when (string-match
> !          (format "\n\\(?:%s\\|%s\\|%s\\)$"
>                   python-shell-prompt-regexp
>                   python-shell-prompt-block-regexp
>                   python-shell-prompt-pdb-regexp)
> --- 1887,1895 ----
>     python-shell-output-filter-buffer
>     (concat python-shell-output-filter-buffer string))
>    (when (string-match
> !          ;; It seems on OSX an extra carriage return might be attached
> !          ;; to the end of output.
> !          (format " ?\n\\(?:%s\\|%s\\|%s\\)$"
>                   python-shell-prompt-regexp
>                   python-shell-prompt-block-regexp
>                   python-shell-prompt-pdb-regexp)
> 
> 
> 
> 

Message #26 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Fabián Ezequiel Gallina <fabian <at> anue.biz>
To: Jan Djärv <jan.h.d <at> swipnet.se>
Cc: 12409 <at> debbugs.gnu.org
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 08 Oct 2012 11:07:58 -0300

[Message part 1 (text/plain, inline)]

On 10/08/2012 04:24 AM, Jan Djärv wrote:
> (format " ?\n\\(?:%s\\|%s\\|%s\\)$"
Unfortunately it seems the patch I copied here lost the carriage return 
because of my email client or something, attaching the file instead. 
Please try this one and let me know.


Thanks!,
Fabián

[python-osx-output.diff (text/plain, attachment)]

Message #29 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Schwab <schwab <at> linux-m68k.org>
To: Fabián Ezequiel Gallina <fabian <at> anue.biz>
Cc: 12409 <at> debbugs.gnu.org, Jan Djärv <jan.h.d <at> swipnet.se>
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 08 Oct 2012 18:44:59 +0200

Fabián Ezequiel Gallina <fabian <at> anue.biz> writes:

> === modified file 'lisp/progmodes/python.el'
> --- lisp/progmodes/python.el	2012-10-07 19:37:37 +0000
> +++ lisp/progmodes/python.el	2012-10-07 19:47:52 +0000
> @@ -1887,7 +1887,9 @@
>     python-shell-output-filter-buffer
>     (concat python-shell-output-filter-buffer string))
>    (when (string-match
> -         (format "\n\\(?:%s\\|%s\\|%s\\)$"
> +         ;; It seems on OSX an extra carriage return might be attached
> +         ;; to the end of output.
> +         (format "?\n\\(?:%s\\|%s\\|%s\\)$"
                     \r

Andreas.

-- 
Andreas Schwab, schwab <at> linux-m68k.org
GPG Key fingerprint = 58CA 54C7 6D53 942B 1756  01D3 44D5 214B 8276 4ED5
"And now for something completely different."

Message #32 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Jan Djärv <jan.h.d <at> swipnet.se>
To: Fabián Ezequiel Gallina <fabian <at> anue.biz>
Cc: 12409 <at> debbugs.gnu.org
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 8 Oct 2012 20:46:10 +0200

8 okt 2012 kl. 16:07 skrev Fabián Ezequiel Gallina <fabian <at> anue.biz>:

> On 10/08/2012 04:24 AM, Jan Djärv wrote:
>> (format " ?\n\\(?:%s\\|%s\\|%s\\)$"
> Unfortunately it seems the patch I copied here lost the carriage return because of my email client or something, attaching the file instead. Please try this one and let me know.
> 
> 
> Thanks!,
> Fabián
> 
> <python-osx-output.diff>

(format "\r?\n\\(?:%s\\|%s\\|%s\\)$" works.  It is a bad idea to use literal ^M in files, so take Andreas advice and use \r.

Thanks,

	Jan D.

Message #35 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Stefan Monnier <monnier <at> iro.umontreal.ca>
To: Jan Djärv <jan.h.d <at> swipnet.se>
Cc: 12409 <at> debbugs.gnu.org,
	Fabián Ezequiel Gallina <fabian <at> anue.biz>
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 08 Oct 2012 15:07:57 -0400

> (format "\r?\n\\(?:%s\\|%s\\|%s\\)$" works.  It is a bad idea to use literal
> ^M in files, so take Andreas advice and use \r.

BTW, could someone investigate into the cause of these ^M?
IOW, we need to look at the actual bytes that Python sends to Emacs,
then figure out why Emacs's EOL-conversion doesn't handle them (are
the bytes wrong, or is the process's coding-system set incorrectly?).


        Stefan

Message #38 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Jan Djärv <jan.h.d <at> swipnet.se>
To: Stefan Monnier <monnier <at> iro.umontreal.ca>
Cc: 12409 <at> debbugs.gnu.org,
	Fabián Ezequiel Gallina <fabian <at> anue.biz>
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 8 Oct 2012 21:49:45 +0200

Hello.

Python does not write them, using dtruss i see:

write_nocancel(0x1, "'aaaa;aaaaaa'\n\0", 0xE)		 

No \r in sight.

	Jan D.

8 okt 2012 kl. 21:07 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:

>> (format "\r?\n\\(?:%s\\|%s\\|%s\\)$" works.  It is a bad idea to use literal
>> ^M in files, so take Andreas advice and use \r.
> 
> BTW, could someone investigate into the cause of these ^M?
> IOW, we need to look at the actual bytes that Python sends to Emacs,
> then figure out why Emacs's EOL-conversion doesn't handle them (are
> the bytes wrong, or is the process's coding-system set incorrectly?).
> 
> 
>        Stefan

Message #41 received at 12409 <at> debbugs.gnu.org (full text, mbox):

From: Fabián Ezequiel Gallina <fabian <at> anue.biz>
To: Jan Djärv <jan.h.d <at> swipnet.se>
Cc: 12409 <at> debbugs.gnu.org, Stefan Monnier <monnier <at> iro.umontreal.ca>
Subject: Re: bug#12409: 24.2.50;
	OS X Python completion leaks junk to inferior-python-mode buffer
Date: Mon, 08 Oct 2012 18:57:28 -0300

On 10/08/2012 04:49 PM, Jan Djärv wrote:
> Hello.
>
> Python does not write them, using dtruss i see:
>
> write_nocancel(0x1, "'aaaa;aaaaaa'\n\0", 0xE)		
>
> No \r in sight.
>
> 	Jan D.
>
> 8 okt 2012 kl. 21:07 skrev Stefan Monnier <monnier <at> iro.umontreal.ca>:
>
>>> (format "\r?\n\\(?:%s\\|%s\\|%s\\)$" works.  It is a bad idea to use literal
>>> ^M in files, so take Andreas advice and use \r.
>> BTW, could someone investigate into the cause of these ^M?
>> IOW, we need to look at the actual bytes that Python sends to Emacs,
>> then figure out why Emacs's EOL-conversion doesn't handle them (are
>> the bytes wrong, or is the process's coding-system set incorrectly?).
>>
>>
>>         Stefan

I just pushed that small fix in 110473. I think for future discussion 
the extra \r on OSX might have earned his own ticket. If everyone agrees 
we can close this one.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.