GNU bug report logs - #49115
Mumi inserts spurious underscore in bug title

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 49115 in the body.
You can then email your comments to 49115 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: Bug Guix <bug-guix <at> gnu.org>
Subject: Mumi inserts spurious underscore in bug title
Date: Sat, 19 Jun 2021 13:37:50 +0200

[Message part 1 (text/plain, inline)]

Guix,

See <http://issues.guix.gnu.org/49114>, which Mumi currently 
titles

 ‘guix_ lint’ should catch certificate validation exceptions

There's no underscore in the original subject.  It's also rendered 
correctly in the grey ‘mail view’ div:

 Tobias Geerinckx-Rice wrote seconds ago
 ‘guix lint’ should catch certificate validation exceptions

Perhaps due to my use of Unicode quotes.  There's nothing 
suspicious in the HTML.

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 49115 <at> debbugs.gnu.org, bug-guix <at> gnu.org
Subject: Re: bug#49115: Mumi inserts spurious underscore in bug title
Date: Sat, 19 Jun 2021 13:43:27 +0200

[Message part 1 (text/plain, inline)]

Tobias Geerinckx-Rice via Bug reports for GNU Guix 写道：
> It's also rendered correctly in the grey ‘mail view’ div:

Otherwise I'd suspect guile-email but now I'm not so sure.

Debbugs doesn't do this: 
<https://debbugs.gnu.org/cgi/bugreport.cgi?bug=49114>.

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 49115 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: 49115 <at> debbugs.gnu.org
Cc: guile-email <at> systemreboot.net
Subject: Mumi inserts spurious underscore in bug title
Date: Sat, 24 Dec 2022 23:53:02 +0100

> See http://issues.guix.gnu.org/49114, which Mumi currently 
> titles
>
>   ‘guix_ lint’ should catch certificate validation exceptions
>
> There's no underscore in the original subject.

The debbugs “.log” file for issue 49114 contains this line:

  Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions

This is MIME Q encoding (similar to “quoted printable” encoding), as
specified in RFC 2047.

The underscore is mentioned in 4.2 (2):

   (2) The 8-bit hexadecimal value 20 (e.g., ISO-8859-1 SPACE) may be
       represented as "_" (underscore, ASCII 95.).  (This character may
       not pass through some internetwork mail gateways, but its use
       will greatly enhance readability of "Q" encoded data with mail
       readers that do not support this encoding.)  Note that the "_"
       always represents hexadecimal 20, even if the SPACE character
       occupies a different code position in the character set in use.

So, the underscore above ought to be replaced with a space.  This seems
to be a bug in guile-email.

-- 
Ricardo

Message #17 received at 49115 <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Ricardo Wurmus <rekado <at> elephly.net>, 49115 <at> debbugs.gnu.org
Cc: guile-email <at> systemreboot.net
Subject: Re: Mumi inserts spurious underscore in bug title
Date: Thu, 29 Dec 2022 19:36:07 +0000

Hi Ricardo,

Thanks for the bug report!

It's very strange, but I'm not able to reproduce this. I tried the
following. bug_49114.mbox is downloaded from
https://debbugs.gnu.org/cgi/bugreport.cgi?mbox=yes;bug=49114

--8<---------------cut here---------------start------------->8---
(use-modules (email email))

(pk
 (parse-email-headers "Subject: bug#49114: =?UTF-8?Q?=E2=80=98guix_?= =?UTF-8?Q?lint=E2=80=99?= should catch certificate validation exceptions
"))

(for-each (lambda (bv)
            (pk (assoc-ref (email-headers (parse-email bv))
                           'subject)))
          (call-with-input-file "bug_49114.mbox"
            mbox->emails))
--8<---------------cut here---------------end--------------->8---

Even at https://issues.guix.gnu.org/49114 , only the "bug title" has the
spurious underscore. The subject of the first message does not. Is the
bug title something stored in the xapian index? Could it be that this
was an older bug that has corrupted the xapian index?

If I understand correctly, mumi does not rebuild its xapian index. I
think it should do so from time to time. It would help prevent old bugs
from getting persisted in storage.

Cheers, and wish you a Happy New Year! :-)
Arun

Message #22 received at 49115-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 49115-done <at> debbugs.gnu.org, guile-email <at> systemreboot.net
Subject: Re: Mumi inserts spurious underscore in bug title
Date: Sun, 01 Jan 2023 13:15:02 +0100

Hi Arun,

> Thanks for the bug report!
>
> It's very strange, but I'm not able to reproduce this.

Thank you for your analysis!

> Even at https://issues.guix.gnu.org/49114 , only the "bug title" has the
> spurious underscore. The subject of the first message does not. Is the
> bug title something stored in the xapian index? Could it be that this
> was an older bug that has corrupted the xapian index?

As part of #60410 I rebuilt the xapian database (deleted it and then ran
“mumi fetch”), so it’s not that.

[… time passes …]

I think I found the culprit in mumi/debbugs.scm:

    (define qp-pattern "=\\?UTF-8\\?Q\\?([^?]+)\\?=")
    …
    (let ((subject (or (assoc-ref properties "Subject") "")))
      (if (string-contains subject "=?UTF-8?Q?")
          (or (false-if-exception
               (utf8->string
                (quoted-printable-decode
                 (regexp-substitute/global #f qp-pattern
                                           subject 'pre 1 'post))))
              subject)
          subject))

I’m probably not using quoted-printable-decode correctly.  My apologies
for assuming a bug in Guile Email.  I changed this with commit
9d7eb3c5efe6427a1a89fb3e4c7165259edff4dd, using parse-email-headers.

> If I understand correctly, mumi does not rebuild its xapian index. I
> think it should do so from time to time. It would help prevent old bugs
> from getting persisted in storage.

True.  I’ll be sure to rebuild the index on the next upgrade.

> Cheers, and wish you a Happy New Year! :-)

Thank you, and a Happy New Year to you too!

-- 
Ricardo

Message #25 received at 49115-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Arun Isaac <arunisaac <at> systemreboot.net>, 49115-done <at> debbugs.gnu.org,
 guile-email <at> systemreboot.net
Subject: [PATCH 0/1] Do not check for MIME encoded words
Date: Mon,  2 Jan 2023 17:43:11 +0000

Hi Ricardo,

I have an additional small improvement. guile-email transparently
handles MIME encoded words. We need not check for them. Patch follows.

Regards,
Arun

Arun Isaac (1):
  debbugs: Do not check for MIME encoded words in subject.

 mumi/debbugs.scm | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

-- 
2.38.1

Message #28 received at 49115-done <at> debbugs.gnu.org (full text, mbox):

From: Arun Isaac <arunisaac <at> systemreboot.net>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: Arun Isaac <arunisaac <at> systemreboot.net>, 49115-done <at> debbugs.gnu.org,
 guile-email <at> systemreboot.net
Subject: [PATCH 1/1] debbugs: Do not check for MIME encoded words in subject.
Date: Mon,  2 Jan 2023 17:43:12 +0000

guile-email transparently handles MIME encoded words. We do not have
to check for them.

* mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
in subject.
---
 mumi/debbugs.scm | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)

diff --git a/mumi/debbugs.scm b/mumi/debbugs.scm
index 16bff8e..7e95ddf 100644
--- a/mumi/debbugs.scm
+++ b/mumi/debbugs.scm
@@ -1,5 +1,6 @@
 ;;; mumi -- Mediocre, uh, mail interface
 ;;; Copyright © 2020 Ricardo Wurmus <rekado <at> elephly.net>
+;;; Copyright © 2023 Arun Isaac <arunisaac <at> systemreboot.net>
 ;;;
 ;;; This program is free software: you can redistribute it and/or
 ;;; modify it under the terms of the GNU Affero General Public License
@@ -280,10 +281,10 @@ defaults to 30 days."
                      (assoc-ref properties "Submitter")
                      (assoc-ref properties "Owner")
                      (or (assoc-ref properties "Severity") "normal")
-                     (let ((subject (or (assoc-ref properties "Subject") "")))
-                       (if (string-contains subject "=?UTF-8?Q?")
-                           (match (parse-email-headers (string-append "Subject: " subject "\n"))
-                             ((('subject . sub) . rest) sub)
-                             (other subject))
-                           subject))
+                     (assq-ref
+                      (parse-email-headers
+                       (string-append "Subject: "
+                                      (or (assoc-ref properties "Subject") "")
+                                      "\n"))
+                      'subject)
                      (assoc-ref properties "Tags"))))
-- 
2.38.1

Message #31 received at 49115-done <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Arun Isaac <arunisaac <at> systemreboot.net>
Cc: 49115-done <at> debbugs.gnu.org, guile-email <at> systemreboot.net
Subject: Re: [PATCH 1/1] debbugs: Do not check for MIME encoded words in
 subject.
Date: Mon, 02 Jan 2023 20:34:52 +0100

Arun Isaac <arunisaac <at> systemreboot.net> writes:

> guile-email transparently handles MIME encoded words. We do not have
> to check for them.
>
> * mumi/debbugs.scm (bug-status): Do not check for MIME encoded words
> in subject.

Thank you, I applied it.

-- 
Ricardo

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.