GNU bug report logs - #40089
27.0.90; nnimap shows empty groups with Cyrus IMAP 3.0.11

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40089 in the body.
You can then email your comments to 40089 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Rainer Orth <ro <at> CeBiTec.Uni-Bielefeld.DE>
To: bug-gnu-emacs <at> gnu.org
Subject: 27.0.90; nnimap shows empty groups with Cyrus IMAP 3.0.11
Date: Mon, 16 Mar 2020 14:49:39 +0100

Our site recently switched from Cyrus IMAP 2.3.14 to 3.0.11.  The first
time I connected to the new server, everything seemed fine (i.e. both
the correct nnimap groups were listed as well as the messages in the
groups).  However, once I've quit a group and reenter it, at best newly
arrived messages are shown, even in groups that have several marked
messages and thus shouldn't come up empty.  When no new messages are
present, the groups comes up empty, and if I try to enter it with C-u =,
I just get e.g.

Group nnimap+cebitec:INBOX contains no messages

which is simply not true.  What is a bit weird that in the group view,
all groups containing marked messages still showed up with the leading
"*", despite the claim not to contain messages.

Other mail clients (e.g. Thunderbird 68.4.1) show no unexpected
behaviour, displaying the group contents as expected, and we also got no
complaints from our users after the server upgrade, which mildly
suggests an issue on the nnimap side.

I originally experienced this with Emacs 26.3, but after upgrading to
27.0.90, the problem remains.

In the imapd logs, I find many message like

Mar 12 12:04:02 imap cyrus/imaps[29978]: inefficient qresync (1880 > 1503) user.ro.llvm-project

for my mailboxes, which suggests something weird is going on.  I did
enable full session logging for my mailbox, but my IMAP foo is very weak
these days, having looked deeper into the protocol more than 15 years
ago.

Prompted by the "inefficient qresync" messages cited above and the fact
that the previous imapd version didn't support that extension, I tried
and disabled it in impad.conf (suppress_capabilities: QRESYNC) and
immediately, everything went back to normal: my nnimap groups show all
the expected contents and nothing vanishes.


In GNU Emacs 27.0.90 (build 1, x86_64-pc-solaris2.11, GTK+ Version 3.24.11)
 of 2020-03-12 built on manam
Windowing system distributor 'Oracle Corporation, based on X.Org Foundation sources', version 11.0.12006000
System Description: Oracle Solaris Trunk

Recent messages:
No news is good news
Saving /homes/ro/.newsrc.eld...
Saving file /homes/ro/.newsrc.eld...
Wrote /homes/ro/.newsrc.eld
Saving /homes/ro/.newsrc.eld...done
Checking new news...
Reading active file via nnnil...done
nnimap read 129k from imap.CeBiTec.Uni-Bielefeld.DE
Checking new news...done
No news is good news

Configured using:
 'configure 'CFLAGS=-g3 -O2 -m64' --prefix=/vol/gnu --without-gif
 PKG_CONFIG_PATH=/usr/lib/64/pkgconfig:/usr/lib/64/pkgconfig:/usr/share/pkgconfig'

Configured features:
XPM JPEG TIFF PNG RSVG DBUS GSETTINGS GLIB NOTIFY GFILENOTIFY ACL GNUTLS
LIBXML2 FREETYPE HARFBUZZ XFT ZLIB TOOLKIT_SCROLL_BARS GTK3 X11 XDBE XIM
MODULES THREADS JSON PDUMPER LCMS2 GMP

Important settings:
  value of $LANG: en_US.UTF-8
  locale-coding-system: utf-8-unix

Major mode: Group

Minor modes in effect:
  gnus-topic-mode: t
  cursor-sensor-mode: t
  gnus-undo-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  tool-bar-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  buffer-read-only: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
/homes/ro/.emacs.d/elpa/go-mode-1.5.0/go-mode hides /vol/gnu/share/emacs/site-lisp/go-mode
/vol/gnu/share/emacs/site-lisp/psgml/psgml-edit hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-edit
/vol/gnu/share/emacs/site-lisp/psgml/psgml-charent hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-charent
/vol/gnu/share/emacs/site-lisp/psgml/psgml-dtd hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-dtd
/vol/gnu/share/emacs/site-lisp/psgml/psgml-parse hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-parse
/vol/gnu/share/emacs/site-lisp/psgml/psgml-api hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-api
/vol/gnu/share/emacs/site-lisp/psgml/psgml-info hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-info
/vol/gnu/share/emacs/site-lisp/psgml/psgml-other hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml-other
/vol/gnu/share/emacs/site-lisp/psgml/psgml hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.0/psgml
/vol/gnu/share/emacs/site-lisp/psgml/psgml-other hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-other
/vol/gnu/share/emacs/site-lisp/psgml/psgml-edit hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-edit
/vol/gnu/share/emacs/site-lisp/psgml/psgml hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml
/vol/gnu/share/emacs/site-lisp/psgml/psgml-api hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-api
/vol/gnu/share/emacs/site-lisp/psgml/psgml-info hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-info
/vol/gnu/share/emacs/site-lisp/psgml/psgml-dtd hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-dtd
/vol/gnu/share/emacs/site-lisp/psgml/psgml-charent hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-charent
/vol/gnu/share/emacs/site-lisp/psgml/psgml-parse hides /vol/gnu/share/emacs/site-lisp/psgml/1.2.5/psgml-parse
/vol/gnu/share/emacs/site-lisp/info-look hides /vol/gnu/share/emacs/27.0.90/lisp/info-look

Features:
(shadow emacsbug tabify conf-mode shr-color color dired-aux vc-filewise
vc-rcs jka-compr misearch multi-isearch smerge-mode diff diff-mode
easy-mmode mailalias smtpmail pcase sendmail nnir gnus-draft flow-fill
mule-util mm-archive cl-extra help-mode sort smiley ansi-color gnus-cite
mail-extr gnus-async gnus-bcklg qp gnus-ml disp-table gnus-topic
cursor-sensor nndraft nnmh gnus-agent gnus-srvr gnus-score score-mode
nnvirtual gnus-msg gnus-art mm-uu mml2015 mm-view mml-smime smime dig
utf-7 rfc2104 gnutls network-stream nsm nnfolder gnus-cache gnus-sum url
url-proxy url-privacy url-expand url-methods url-history mailcap shr
url-cookie url-domsuf url-util svg xml dom nnnil gnus-demon nntp
gnus-group gnus-undo gnus-start gnus-cloud nnimap nnmail mail-source
utf7 netrc nnoo parse-time iso8601 gnus-spec gnus-int gnus-range message
rmc puny dired dired-loaddefs format-spec rfc822 mml mml-sec epa derived
epg epg-config mm-decode mm-bodies mm-encode mail-parse rfc2231
mailabbrev gmm-utils mailheader gnus-win gnus nnheader gnus-util rmail
rmail-loaddefs rfc2047 rfc2045 ietf-drums text-property-search time-date
mail-utils mm-util mail-prsvr wid-edit tex-site finder-inf desktop
frameset package easymenu browse-url url-handlers url-parse auth-source
cl-seq eieio eieio-core cl-macs eieio-loaddefs password-cache json
subr-x map url-vars seq byte-opt gv bytecomp byte-compile cconv
cl-loaddefs cl-lib tooltip eldoc electric uniquify ediff-hook vc-hooks
lisp-float-type mwheel term/x-win x-win term/common-win x-dnd tool-bar
dnd fontset image regexp-opt fringe tabulated-list replace newcomment
text-mode elisp-mode lisp-mode prog-mode register page tab-bar menu-bar
rfn-eshadow isearch timer select scroll-bar mouse jit-lock font-lock
syntax facemenu font-core term/tty-colors frame minibuffer cl-generic
cham georgian utf-8-lang misc-lang vietnamese tibetan thai tai-viet lao
korean japanese eucjp-ms cp51932 hebrew greek romanian slovak czech
european ethiopic indian cyrillic chinese composite charscript charprop
case-table epa-hook jka-cmpr-hook help simple abbrev obarray
cl-preloaded nadvice loaddefs button faces cus-face macroexp files
text-properties overlay sha1 md5 base64 format env code-pages mule
custom widget hashtable-print-readable backquote threads dbusbind
gfilenotify lcms2 dynamic-setting system-font-setting
font-render-setting move-toolbar gtk x-toolkit x multi-tty
make-network-process emacs)

Memory information:
((conses 16 779860 123163)
 (symbols 48 19770 17)
 (strings 32 85182 20497)
 (string-bytes 1 2781675)
 (vectors 16 42632)
 (vector-slots 8 1438075 147088)
 (floats 8 458 325)
 (intervals 56 6739 2126)
 (buffers 1000 102))

-- 
-----------------------------------------------------------------------------
Rainer Orth, Center for Biotechnology, Bielefeld University

Message #8 received at 40089 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Rainer Orth <ro <at> CeBiTec.Uni-Bielefeld.DE>
Cc: 40089 <at> debbugs.gnu.org
Subject: Re: bug#40089: 27.0.90; nnimap shows empty groups with Cyrus IMAP
 3.0.11
Date: Sun, 19 Jul 2020 06:13:14 +0200

Rainer Orth <ro <at> CeBiTec.Uni-Bielefeld.DE> writes:

> Other mail clients (e.g. Thunderbird 68.4.1) show no unexpected
> behaviour, displaying the group contents as expected, and we also got no
> complaints from our users after the server upgrade, which mildly
> suggests an issue on the nnimap side.

[...]

> In the imapd logs, I find many message like
>
> Mar 12 12:04:02 imap cyrus/imaps[29978]: inefficient qresync (1880 > 1503) user.ro.llvm-project

Perhaps these other clients don't use QRESYNC?  Is it possible to switch
on tracing to see whether they do or not?  Because if they don't:

> Prompted by the "inefficient qresync" messages cited above and the fact
> that the previous imapd version didn't support that extension, I tried
> and disabled it in impad.conf (suppress_capabilities: QRESYNC) and
> immediately, everything went back to normal: my nnimap groups show all
> the expected contents and nothing vanishes.

It sounds like that version of Cyrus has a buggy QRESYNC
implementation.  But it's definitely possible that nnimap does something
it doesn't expect QRESYNC-wise...

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #13 received at 40089 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Rainer Orth <ro <at> CeBiTec.Uni-Bielefeld.DE>
Cc: 40089 <at> debbugs.gnu.org
Subject: Re: bug#40089: 27.0.90; nnimap shows empty groups with Cyrus IMAP
 3.0.11
Date: Mon, 26 Oct 2020 16:47:13 +0100

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Perhaps these other clients don't use QRESYNC?  Is it possible to switch
> on tracing to see whether they do or not?  Because if they don't:

More information was requested three months ago, but was not given, so
I'm closing this bug report.  If further progress can be made here,
please respond to the debbugs address and we'll reopen.

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.