GNU bug report logs - #9521
24.0.50; rmail-forward

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 9521 in the body.
You can then email your comments to 9521 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Kenichi Handa <handa <at> m17n.org>
To: bug-gnu-emacs <at> gnu.org
Subject: 24.0.50; rmail-forward
Date: Fri, 16 Sep 2011 16:51:12 +0900

[Message part 1 (text/plain, inline)]

rmail-forward doesn't handle a attachment file
correctly. For instance, when I have a message something
like this in RMAIL:

  ------------------------------------------------------------
  [...]
  [1:text/plain Hide]
  
  test of attachment
  
  [2:application/pdf Show Save:temp.pdf (2kB)]
  ------------------------------------------------------------

Typing f inserts just this in "part" part of *unsent mail to
...* buffer:

  ------------------------------------------------------------
  From: Kenichi Handa <handa <at> m17n.org>
  To: handa <at> m17n.org
  Subject: test from shatin
  Date: Thu, 15 Sep 2011 14:14:58 +0900
  Message-ID: <87aaa6xu7h.fsf <at> m17n.org>
  Content-Type: multipart/mixed; boundary="=-=-="


  [1:text/plain Hide]

  test of attachment


  [2:application/pdf Show Save:temp.pdf (2kB)]
  ------------------------------------------------------------

It's the content of rmail-view buffer and thus the outgoing
mail doesn't contain the correct attachment.

In Emacs 23.3, the content of "part" part was the original
whole message, and thus the outgoing mail surely contains an
attachment in a correct MIME form.

I'm attaching the same sample file so that you can see what
I described by typing 'f' in RMAIL.

---
Kenichi Handa
handa <at> m17n.org



In GNU Emacs 24.0.50.2 (i686-pc-linux-gnu, GTK+ Version 2.20.1)
 of 2011-09-16 on etlken
Windowing system distributor `The X.Org Foundation', version 11.0.10706000
Important settings:
  value of $LC_ALL: nil
  value of $LC_COLLATE: nil
  value of $LC_CTYPE: nil
  value of $LC_MESSAGES: nil
  value of $LC_MONETARY: nil
  value of $LC_NUMERIC: nil
  value of $LC_TIME: nil
  value of $LANG: ja_JP.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix
  default enable-multibyte-characters: t

Major mode: RMAIL

Minor modes in effect:
  diff-auto-refine-mode: t
  shell-dirtrack-mode: t
  display-time-mode: t
  tooltip-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  line-number-mode: t

Recent input:
<backspace> <backspace> <backspace> C-\ c o m b <backspace> 
v e n i e n t <escape> $ 0 <escape> < C-c C-c h k <return> 
g n d d d d d d d d d d d d d d d d d d d d d d d d 
d d d d SPC <backspace> C-x o C-u C-v C-x o n d d d 
d d d d d d d s y C-x o C-h v r m a i l - m i <tab> 
<tab> <tab> <tab> C-g C-x C-f M-p <M-backspace> <M-backspace> 
<M-backspace> <M-backspace> <M-backspace> e m <tab> 
/ w o <tab> l i s p / m a i <tab> r m <tab> m <tab> 
m <tab> <return> C-s f o r w a r d C-w C-w C-s C-a 
C-n C-n C-n C-n C-n C-n C-n C-u C-v C-n C-n C-n C-n 
C-n C-n C-u C-v C-u C-v C-u C-v M-f M-f M-b M-b C-s 
C-w C-w C-w C-w C-w C-a C-x C-g <down-mouse-1> <mouse-1> 
C-a C-x b <return> C-x C-f M-p <return> C-M-a M-f M-f 
M-b C-s C-w C-w C-w C-w C-s C-s C-w C-a C-x C-f r m 
<tab> . <tab> <return> C-s C-s C-s C-w C-s C-a C-x 
b <return> C-r C-r C-a C-x k <return> C-x o j C-x o 
x C-g C-x o M-f M-f M-f M-f M-f M-f M-f M-f M-f M-b 
C-s C-w C-w C-r C-r C-a C-x o C-v C-p C-p C-p C-p C-p 
C-p C-n C-n C-n C-n C-n C-SPC C-v C-v C-n C-n C-n C-n 
C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n C-n 
C-n C-p C-p C-p <escape> w C-x C-x C-x C-x C-x C-g 
<escape> x r e p <tab> o <tab> r <tab> <return>

Recent messages:
Quit
Making completion list...
Mark saved where search started [2 times]
Mark set
Mark saved where search started [3 times]
Quit
Mark saved where search started
Mark set
Saved text from "rmail-forward still has a problem when a"
Making completion list... [2 times]

Load-path shadows:
/usr/local/share/emacs/site-lisp/evi-mule hides /usr/local/share/emacs/site-lisp/lookup/evi-mule
/usr/local/share/emacs/site-lisp/evi hides /usr/local/share/emacs/site-lisp/lookup/evi
/usr/local/share/emacs/site-lisp/anthy/anthy hides /usr/local/share/emacs/site-lisp/egg/egg/anthy
/usr/local/share/emacs/site-lisp/egg/its/thai hides /usr/local/work/emacs/stable/lisp/language/thai
/usr/local/share/emacs/site-lisp/egg/its/greek hides /usr/local/work/emacs/stable/lisp/language/greek
/usr/local/work/emacs/stable/lisp/textmodes/table hides ~/emacslisp/table
/usr/local/work/emacs/stable/lisp/language/thai-word hides ~/emacslisp/thai-word
/usr/local/work/emacs/stable/lisp/progmodes/prolog hides ~/emacslisp/prolog
/usr/local/work/emacs/stable/lisp/emacs-lisp/syntax hides ~/emacslisp/syntax
/usr/local/work/emacs/stable/lisp/textmodes/tex-mode hides ~/emacslisp/tex-mode

Features:
(iso-transl parse-time vc-cvs edmacro kmacro rect dabbrev
find-func etags warnings compile info diff-mode diff
thingatpt browse-url ind-util sh-script executable tar-mode
pcmpl-gnu pcmpl-unix ispell shadow emacsbug doc-view
image-mode dired nxml-uchnm rng-xsd xsd-regexp rng-cmpct
rng-nxml rng-valid rng-loc rng-uri rng-parse nxml-parse
rng-match rng-dt rng-util rng-pttrn nxml-ns nxml-mode
nxml-outln nxml-rap nxml-util nxml-glyph nxml-enc xmltok
help-fns ansi-color shell pcomplete comint ring add-log
vc-bzr pp wid-edit descr-text network-stream starttls tls
mailalias smtpmail auth-source eieio byte-opt bytecomp
byte-compile cconv macroexp assoc password-cache sendmail
regexp-opt jka-compr sort mailcap newcomment ja-dic
mule-util kkc ja-dic-utl quail help-mode view supercite
easy-mmode regi gnus-util mail-extr multi-isearch qp
rmailkwd rmailmm message format-spec rfc822 mml easymenu
mml-sec mm-decode mm-bodies mm-encode mailabbrev gmm-utils
mailheader mail-parse rfc2231 js2-mode-autoloads package
tabulated-list rmail-parse-url time rmail-sa rmailsum rmail
rfc2047 rfc2045 ietf-drums mm-util mail-prsvr mail-utils
time-date japan-util tooltip ediff-hook vc-hooks
lisp-float-type mwheel x-win x-dnd tool-bar dnd fontset
image fringe lisp-mode register page menu-bar rfn-eshadow
timer select scroll-bar mouse jit-lock font-lock syntax
facemenu font-core frame cham georgian utf-8-lang misc-lang
vietnamese tibetan thai tai-viet lao korean japanese hebrew
greek romanian slovak czech european ethiopic indian
cyrillic chinese case-table epa-hook jka-cmpr-hook help
simple abbrev minibuffer loaddefs button faces cus-face
files text-properties overlay sha1 md5 base64 format env
code-pages mule custom widget hashtable-print-readable
backquote make-network-process dbusbind dynamic-setting
system-font-setting font-render-setting move-toolbar gtk
x-toolkit x multi-tty emacs)

[temp.pdf (application/pdf, attachment)]

Message #10 received at 9521 <at> debbugs.gnu.org (full text, mbox):

From: Mark Lillibridge <mdl <at> alum.mit.edu>
To: 9521 <at> debbugs.gnu.org
Subject: This bug is still present in version 24.2;
	it prevents forwarding correctly essentially any MINE message
Date: Fri, 28 Dec 2012 11:24:31 -0800

    In particular, it removes the MIME headers from the forwarded
message, making it undecodable.  This bug was not present in version
23.3.  It also forwards incorrectly non-MIME messages, leaving out
filtered headers, which may be wanted for diagnosing message bounces and
the like.

- Mark

Message #13 received at 9521 <at> debbugs.gnu.org (full text, mbox):

From: Mark Lillibridge <mdl <at> alum.mit.edu>
To: 9521 <at> debbugs.gnu.org, 9766 <at> debbugs.gnu.org
Subject: PATCH for bug #9521, *not* bug #9766
Date: Fri, 28 Dec 2012 12:46:33 -0800

[Message part 1 (text/plain, inline)]

    This bug (#9521) was easy to fix.  The problem was with the
rmail-insert-mime-forwarded-message function in rmailmm.el:1355:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
  	       forward-buffer
  	     rmail-view-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    This does exactly the wrong thing by inserting the decoded version
of the message.  Swapping the two buffers (forward-buffer,
rmail-view-buffer) in the if expression fixes this:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
> 	       rmail-view-buffer
> 	     forward-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    Note that this does not fix bug #9766, which was incorrectly merged
with bug #9521.  The problem there (#9766) is that many email clients
including in particular, the iPad email app, do not properly display
RFC822 attachments or do not show it inline.  Fixing that problem
requires substantial work, including on the design front.

    One idea would be to generate the RFC822 attachment as now, which
preserves the full details of the message for competent email clients,
and also generate an abbreviated version in the message body for human
viewers of incompetent email clients.  The simplest approach would be to
just insert the Rmail decoded version:

    ====== forwarded message as seen by sender (full message attached) ====
    From: Kenichi Handa <handa <at> m17n.org>
    To: handa <at> m17n.org
    Subject: test from shatin
    Date: Thu, 15 Sep 2011 14:14:58 +0900
    Message-ID: <87aaa6xu7h.fsf <at> m17n.org>
    Content-Type: multipart/mixed; boundary="=-=-="

    [1:text/plain Hide]
    
    test of attachment
    
    
    [2:application/pdf Show Save:temp.pdf (2kB)]

Here, the message headers have been filtered by the users usual header
filtering rules and the body is as seen when the forwarding was done.
e.g., whatever message part toggling the user did is still visible.

    Drawbacks: the PDF attachment here is not accessible to the
incompetent email clients, this approach fails miserably for HTML-only
messages (distressingly common these days), and there is no way to
forward the HTML part instead of the text part inline (perhaps the HTML
part has the real content).  The first of these could be fixed by
attaching all of the original non-inline attachments to the new message;
this is what email clients like Outlook do when you forword a message.

    For the second and third parts, I have been experimenting with the
following:

    

[Message part 2 (text/html, inline)]

[Message part 3 (text/plain, inline)]

Note the need to use PRE or the equivalent to protect the headers.
Probably additional escaping of the headers are required and likely the
charset needs very careful handling.  Lots of corner cases here,
including what to do with multiple HTML parts.  


    At the moment, I have three versions of forward implemented, one
that sends the entire message as a RFC822 attachment, one that forwards
just the text part, and one that forwards just the HTML part.  The later
two work correctly when the entire message is one part.  I find each
useful in different circumstances so probably we don't want to support
only one of them.

- Mark

Message #16 received at 9521 <at> debbugs.gnu.org (full text, mbox):

From: Mark Lillibridge <mdl <at> alum.mit.edu>
To: 9521 <at> debbugs.gnu.org, 9766 <at> debbugs.gnu.org
Subject: [RESEND] PATCH for bug #9521, *not* bug #9766
Date: Fri, 28 Dec 2012 12:55:25 -0800

    This bug (#9521) was easy to fix.  The problem was with the
rmail-insert-mime-forwarded-message function in rmailmm.el:1355:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
  	       forward-buffer
  	     rmail-view-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    This does exactly the wrong thing by inserting the decoded version
of the message.  Swapping the two buffers (forward-buffer,
rmail-view-buffer) in the if expression fixes this:

  (defun rmail-insert-mime-forwarded-message (forward-buffer)
    "Insert the message in FORWARD-BUFFER as a forwarded message.
  This is the usual value of `rmail-insert-mime-forwarded-message-function'."
    (let ((message-buffer
  	 (with-current-buffer forward-buffer
  	   (if rmail-buffer-swapped
> 	       rmail-view-buffer
> 	     forward-buffer))))
      (save-restriction
        (narrow-to-region (point) (point))
        (message-forward-make-body-mime message-buffer))))


    Note that this does not fix bug #9766, which was incorrectly merged
with bug #9521.  The problem there (#9766) is that many email clients
including in particular, the iPad email app, do not properly display
RFC822 attachments or do not show it inline.  Fixing that problem
requires substantial work, including on the design front.

    One idea would be to generate the RFC822 attachment as now, which
preserves the full details of the message for competent email clients,
and also generate an abbreviated version in the message body for human
viewers of incompetent email clients.  The simplest approach would be to
just insert the Rmail decoded version:

    ====== forwarded message as seen by sender (full message attached) ====
    From: Kenichi Handa <handa <at> m17n.org>
    To: handa <at> m17n.org
    Subject: test from shatin
    Date: Thu, 15 Sep 2011 14:14:58 +0900
    Message-ID: <87aaa6xu7h.fsf <at> m17n.org>
    Content-Type: multipart/mixed; boundary="=-=-="

    [1:text/plain Hide]
    
    test of attachment
    
    
    [2:application/pdf Show Save:temp.pdf (2kB)]

Here, the message headers have been filtered by the users usual header
filtering rules and the body is as seen when the forwarding was done.
e.g., whatever message part toggling the user did is still visible.

    Drawbacks: the PDF attachment here is not accessible to the
incompetent email clients, this approach fails miserably for HTML-only
messages (distressingly common these days), and there is no way to
forward the HTML part instead of the text part inline (perhaps the HTML
part has the real content).  The first of these could be fixed by
attaching all of the original non-inline attachments to the new message;
this is what email clients like Outlook do when you forword a message.

    For the second and third parts, I have been experimenting with the
following (had to add " " after <'s to make < #parts escaped):

    < #part type=text/html disposition=inline raw=t>
    <PRE>
    ===== Forwarded message (HTML part only) follows =====
    Date: Fri, 28 Dec 2012 19:44:57 -0000
    From: "Hilton Hotels & Resorts" <hiltonhotels <at> h1.hilton.com>
    To: lillibridge <at> gmail.com
    Subject: Pick your paradise: choose from three unforgettable resort experiences
    Reply-To: "Hilton Hotels & Resorts" <support-b7j7kpqa20rdfhauv5u0bqcx2vkpsh <at> h1.hilton.com>
    </PRE>
    
    <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd"><html xmlns="http://www.w3.org/1999/xhtml"><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
    <title>Hilton Hotels &amp; Resorts</title><style type="text/css">
    body {
    	-webkit-text-size-adjust: none;
    }
    .ReadMsgBody {
    	width: 100%;
    }
    ...
    <img src="http://h1.hilton.com/a/hBQ3bP4AH$iOmB8v3UjNt2dv0P$/spacer.gif">
    </body></html>
    
    < #/part>

Note the need to use PRE or the equivalent to protect the headers.
Probably additional escaping of the headers are required and likely the
charset needs very careful handling.  Lots of corner cases here,
including what to do with multiple HTML parts.  


    At the moment, I have three versions of forward implemented, one
that sends the entire message as a RFC822 attachment, one that forwards
just the text part, and one that forwards just the HTML part.  The later
two work correctly when the entire message is one part.  I find each
useful in different circumstances so probably we don't want to support
only one of them.

- Mark

Message #19 received at 9521 <at> debbugs.gnu.org (full text, mbox):

From: Mark Lillibridge <mdl <at> alum.mit.edu>
To: 9521 <at> debbugs.gnu.org, 9766 <at> debbugs.gnu.org
Subject: workaround for 9766
Date: Fri, 28 Dec 2012 13:22:28 -0800

    It appears that resend (C-u f) does not have any of the problems
that forward does -- correctly resends the message in essentially its
current form so incompetent mail clients display it correctly -- so I
recommend using it plus a separate message telling the recipient that
you are resending the message and why for now.

- Mark

Message #26 received at 9521-done <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: mdl <at> alum.mit.edu
Cc: 9521-done <at> debbugs.gnu.org, 9766 <at> debbugs.gnu.org
Subject: Re: bug#9521: [RESEND] PATCH for bug #9521, *not* bug #9766
Date: Sat, 29 Dec 2012 10:53:00 +0200

> From: Mark Lillibridge <mdl <at> alum.mit.edu>
> Date: Fri, 28 Dec 2012 12:55:25 -0800
> 
> 
>     This bug (#9521) was easy to fix.  The problem was with the
> rmail-insert-mime-forwarded-message function in rmailmm.el:1355:
> 
>   (defun rmail-insert-mime-forwarded-message (forward-buffer)
>     "Insert the message in FORWARD-BUFFER as a forwarded message.
>   This is the usual value of `rmail-insert-mime-forwarded-message-function'."
>     (let ((message-buffer
>   	 (with-current-buffer forward-buffer
>   	   (if rmail-buffer-swapped
>   	       forward-buffer
>   	     rmail-view-buffer))))
>       (save-restriction
>         (narrow-to-region (point) (point))
>         (message-forward-make-body-mime message-buffer))))
> 
> 
>     This does exactly the wrong thing by inserting the decoded version
> of the message.  Swapping the two buffers (forward-buffer,
> rmail-view-buffer) in the if expression fixes this:
> 
>   (defun rmail-insert-mime-forwarded-message (forward-buffer)
>     "Insert the message in FORWARD-BUFFER as a forwarded message.
>   This is the usual value of `rmail-insert-mime-forwarded-message-function'."
>     (let ((message-buffer
>   	 (with-current-buffer forward-buffer
>   	   (if rmail-buffer-swapped
> > 	       rmail-view-buffer
> > 	     forward-buffer))))
>       (save-restriction
>         (narrow-to-region (point) (point))
>         (message-forward-make-body-mime message-buffer))))

Thanks, I installed this simple change on the emacs-24 branch.

>     Note that this does not fix bug #9766, which was incorrectly merged
> with bug #9521.  The problem there (#9766) is that many email clients
> including in particular, the iPad email app, do not properly display
> RFC822 attachments or do not show it inline.  Fixing that problem
> requires substantial work, including on the design front.

I think #9766 is about both problems.  But I unmerged it anyway, and
am leaving it open for now.

Message #29 received at 9521 <at> debbugs.gnu.org (full text, mbox):

From: Mark Lillibridge <mdl <at> alum.mit.edu>
To: 10080 <at> debbugs.gnu.org
Cc: 9521 <at> debbugs.gnu.org
Subject: this bug (#10080) also causes an extra blank line to be appended to
	forwarded messages
Date: Tue, 01 Jan 2013 14:10:46 -0800

[Message part 1 (text/plain, inline)]

For example, when I forward the example message using f I get:

|To: 
|Subject: [mdl <at> alum.mit.edu: no blank lines follow]
|From: Mark Lillibridge <lillibridgem <at> build-debian-1.u.hpl.hp.com>
|--text follows this line--
|
|
|

[Message part 2 (message/rfc822, inline)]

|this is the only line
|
|

[Message part 3 (text/plain, inline)]

Notice the extra blank line at the end of the forwarded message.

    Fixing #9521 properly (switching to forwarding raw instead of
decoded messages) should fix this.  At the moment (post my 9521 patch),
it also appends a blank line to the end of the forwarded message.
 
- Mark

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.