GNU bug report logs - #53527
29.0.50; package-reinstall fails to initialize (enough of) package.el

Message received at 53527 <at> debbugs.gnu.org:

Received: (at 53527) by debbugs.gnu.org; 25 Jan 2022 19:54:53 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 25 14:54:53 2022
Received: from localhost ([127.0.0.1]:50217 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nCRu1-00028K-Gk
	for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:54:53 -0500
Received: from relay5-d.mail.gandi.net ([217.70.183.197]:57129)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <matt@HIDDEN>) id 1nCRty-000283-7Z
 for 53527 <at> debbugs.gnu.org; Tue, 25 Jan 2022 14:54:52 -0500
Received: (Authenticated sender: matt@HIDDEN)
 by mail.gandi.net (Postfix) with ESMTPSA id 73DA51C0003
 for <53527 <at> debbugs.gnu.org>; Tue, 25 Jan 2022 19:54:43 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
 t=1643140483;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=3GgHUcialr50kODk+s+iwt1YaxnTwapMHHZN+JRLoAc=;
 b=AUE+y026HR1av755thMVHLcBfhjXydwFA7dM2kEvDNu04GqIpbUnUR3ZaVBJLFtjafWkwA
 gvpZBaajYasF3/Z6JhynLLEwUKDUE8Bu1PoOXkjOmbJtq2RarQ8Wo1VHgKoL2Cu8zXXcOF
 8PSwmm+TZ8wRF5JQls9Kl45Ukd73JtyrM49enNB9to9D2CYi17ZPuflploQQ+77VAOdFWu
 jkVvCTiDyhnuENccYrSRUCbLvhFnQ181aP6WCQpiNNi2YbBcp7wHAIJzaAIfysoUunN4Lt
 +VAN7JyQGl5x6L39sR9UzvW1BeFCbOyJMjbGyPo7rH9KrIDhSvJgGV4jGnZnhQ==
Received: from matt by naz with local (Exim 4.95)
 (envelope-from <matt@HIDDEN>) id 1nCRto-000ESY-9l
 for 53527 <at> debbugs.gnu.org; Tue, 25 Jan 2022 11:54:40 -0800
From: Matt Armstrong <matt@HIDDEN>
To: 53527 <at> debbugs.gnu.org
Subject: Re: bug#53527: 29.0.50; package-reinstall fails to initialize
 (enough of) package.el
In-Reply-To: <87zgnjzpe9.fsf@HIDDEN>
References: <87zgnjzpe9.fsf@HIDDEN>
Date: Tue, 25 Jan 2022 11:54:40 -0800
Message-ID: <87ee4vr3pb.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 53527
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

--=-=-=
Content-Type: text/plain

tags 53527 + patch
quit

This is one way to fix it.  Another would have been to call
`package--alist' instead of reference `package-alist' in
`package-reinstall'.  ...but that wouldn't have been an optimization,
since `package-install' inevitably calls `package--archives-initialize'
anyway.  This patch just makes it happen earlier.


--=-=-=
Content-Type: text/x-diff
Content-Disposition: inline;
 filename=0001-Fix-calling-package-reinstall-just-after-quick-initi.patch
Content-Description: Patch to fix Bug#53527

From fdf15b2985d237a70277362ad0b7d771f20c5f38 Mon Sep 17 00:00:00 2001
From: Matt Armstrong <matt@HIDDEN>
Date: Tue, 25 Jan 2022 11:22:10 -0800
Subject: [PATCH] Fix calling `package-reinstall' just after quick
 initialization.

* lisp/emacs-lisp/package.el (package-reinstall): call
`package--archives-initialize', just like `package-install' does.
This populates `package-alist', and so fixes calling
`package-reinstall' as the first thing done after package "quick
init".  (Bug#53527)
---
 lisp/emacs-lisp/package.el | 12 ++++++++----
 1 file changed, 8 insertions(+), 4 deletions(-)

diff --git a/lisp/emacs-lisp/package.el b/lisp/emacs-lisp/package.el
index 7679ba2fae..6a8692c983 100644
--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -2379,10 +2379,14 @@ package-reinstall
   "Reinstall package PKG.
 PKG should be either a symbol, the package name, or a `package-desc'
 object."
-  (interactive (list (intern (completing-read
-                              "Reinstall package: "
-                              (mapcar #'symbol-name
-                                      (mapcar #'car package-alist))))))
+  (interactive
+   (progn
+     (package--archives-initialize)
+     (list (intern (completing-read
+                    "Reinstall package: "
+                    (mapcar #'symbol-name
+                            (mapcar #'car package-alist)))))))
+  (package--archives-initialize)
   (package-delete
    (if (package-desc-p pkg) pkg (cadr (assq pkg package-alist)))
    'force 'nosave)
-- 
2.34.1


--=-=-=--

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 25 Jan 2022 17:39:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Jan 25 12:39:19 2022
Received: from localhost ([127.0.0.1]:49967 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1nCPmp-0002LZ-88
	for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 12:39:19 -0500
Received: from lists.gnu.org ([209.51.188.17]:34514)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <matt@HIDDEN>) id 1nCPmj-0002LE-U3
 for submit <at> debbugs.gnu.org; Tue, 25 Jan 2022 12:39:17 -0500
Received: from eggs.gnu.org ([209.51.188.92]:47712)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <matt@HIDDEN>) id 1nCPmj-0000fx-DY
 for bug-gnu-emacs@HIDDEN; Tue, 25 Jan 2022 12:39:13 -0500
Received: from relay4-d.mail.gandi.net ([217.70.183.196]:53185)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <matt@HIDDEN>) id 1nCPmg-0000np-E9
 for bug-gnu-emacs@HIDDEN; Tue, 25 Jan 2022 12:39:13 -0500
Received: (Authenticated sender: matt@HIDDEN)
 by mail.gandi.net (Postfix) with ESMTPSA id DD894E0009
 for <bug-gnu-emacs@HIDDEN>; Tue, 25 Jan 2022 17:39:01 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=rfc20.org; s=gm1;
 t=1643132343;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:mime-version:mime-version:content-type:content-type;
 bh=Jfe0eY5oAV2tEVlCLMkyL7A3JPkuiI5xI6XXhcvXCes=;
 b=bQ3gUSuqDnh8G7dwQ56hEHphWPkJ1gTTuUSVLW1GbM0X7OaAetnnNz1GpR/UJgWWUUqJ44
 F4u4754n+h9ULuLCBE8IBabGFvnetsv+FTXm3e4OPZH4HIgbtMRmrgj0Z9r5eQ5iXr10Ny
 +e1fgDOZ4rllEB4FsG9l74o6AXwIFEEVsM1gqPPyAjv9dLMh03YXJqXidX7EwtYeFTKUZx
 gnXWUIoOgoGuCMMxLrRBBy+x5UN5STyVmpgdFmVMO+o3AGaCGPcQOMu/VuHnWa3vrlayWp
 d4ziI54SAwG8EGhDVksOr16Es4R5PvJr+hNoEmDYrYg2RO7yt8hKeyb8eGeKHA==
Received: from matt by naz with local (Exim 4.95)
 (envelope-from <matt@HIDDEN>) id 1nCPmV-000CTy-At
 for bug-gnu-emacs@HIDDEN; Tue, 25 Jan 2022 09:38:59 -0800
From: Matt Armstrong <matt@HIDDEN>
To: bug-gnu-emacs@HIDDEN
Subject: 29.0.50; package-reinstall fails to initialize (enough of) package.el
Date: Tue, 25 Jan 2022 09:38:54 -0800
Message-ID: <87zgnjzpe9.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=217.70.183.196; envelope-from=matt@HIDDEN;
 helo=relay4-d.mail.gandi.net
X-Spam_score_int: -7
X-Spam_score: -0.8
X-Spam_bar: /
X-Spam_report: (-0.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, LONGWORDS=2.035,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.2 (/)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.8 (/)

The `package-reinstall' function is broken when called before package.el
is initialized, or more precisely, before `package-load-all-descriptors'
has been called.

It looks like most of the other top level functions in package.el
attempt to handle this case, and do it successfully.

I had trouble formulating an emacs -Q repro because Emacs must be in a
"quick start" state, which requires installation of at least one package
and then restarting Emacs.  I could not figure out how to do that simply
and hermetically without a lot of infrastructure (like a separate user
on a Linux system, or a VM, or https://github.com/plexus/chemacs2).

Stefan's commit dde33727b2ace3ce417d97475d074f0a82b7c2b8 on 2019-01-22
introduced this function and called it in several places but not
`package-reinstall'.

+(defun package--alist ()
+  "Return `package-alist', after computing it if needed."
+  (or package-alist
+      (progn (package-load-all-descriptors)
+             package-alist)))

I found this bug after removing a long standing call to
`package-initialize` from my init.el, then calling this function of
mine:

(defun my-reinstall-packages ()
  "Re-install all Emacs packages that are installed presently."
  (interactive)
  (dolist (package-name package-activated-list)
    (message (format "Reinstalling %s" package-name))
    (package-reinstall package-name)))


I confirmed this bug is in Emacs 27 through master.


In GNU Emacs 29.0.50 (build 1, x86_64-pc-linux-gnu, GTK+ Version 3.24.31, cairo version 1.16.0)
 of 2022-01-13 built on naz
Repository revision: 53dba3202205b5fab078a47da77be4140b7f93b1
Repository branch: master
Windowing system distributor 'The X.Org Foundation', version 11.0.12101004
System Description: Debian GNU/Linux bookworm/sid

Configured using:
 'configure --with-native-compilation'

Configured features:
ACL CAIRO DBUS FREETYPE GIF GLIB GMP GNUTLS GPM GSETTINGS HARFBUZZ JPEG
JSON LCMS2 LIBOTF LIBSELINUX LIBSYSTEMD LIBXML2 M17N_FLT MODULES
NATIVE_COMP NOTIFY INOTIFY PDUMPER PNG RSVG SECCOMP SOUND SQLITE3
THREADS TIFF TOOLKIT_SCROLL_BARS X11 XDBE XIM XPM GTK3 ZLIB

Important settings:
  value of $LANG: en_US.UTF-8
  value of $XMODIFIERS: @im=ibus
  locale-coding-system: utf-8-unix

Major mode: ELisp/l

Minor modes in effect:
  bug-reference-prog-mode: t
  global-git-commit-mode: t
  magit-auto-revert-mode: t
  auto-revert-mode: t
  pyvenv-mode: t
  msb-mode: t
  display-time-mode: t
  shell-dirtrack-mode: t
  auto-insert-mode: t
  keyfreq-autosave-mode: t
  keyfreq-mode: t
  icomplete-mode: t
  savehist-mode: t
  vertico-mode: t
  which-key-mode: t
  electric-pair-mode: t
  override-global-mode: t
  tooltip-mode: t
  global-eldoc-mode: t
  eldoc-mode: t
  show-paren-mode: t
  electric-indent-mode: t
  mouse-wheel-mode: t
  menu-bar-mode: t
  file-name-shadow-mode: t
  global-font-lock-mode: t
  font-lock-mode: t
  blink-cursor-mode: t
  auto-composition-mode: t
  auto-encryption-mode: t
  auto-compression-mode: t
  temp-buffer-resize-mode: t
  column-number-mode: t
  line-number-mode: t
  transient-mark-mode: t

Load-path shadows:
~/env/elisp/ol-notmuch hides /home/matt/.config/emacs/elpa/ol-notmuch-20210530.2054/ol-notmuch
/home/matt/.config/emacs/elpa/transient-20220121.1603/transient hides /home/matt/git/emacs-build/lisp/transient

Features:
(shadow sort mail-extr emacsbug sendmail notmuch notmuch-tree
notmuch-jump notmuch-hello notmuch-show notmuch-print notmuch-crypto
notmuch-mua notmuch-message notmuch-draft notmuch-maildir-fcc
notmuch-address notmuch-company notmuch-parser notmuch-wash coolj
notmuch-query icalendar diary-lib diary-loaddefs notmuch-tag notmuch-lib
notmuch-compat hl-line mm-view mml-smime smime dig goto-addr
bug-reference magit-extras magit-bookmark magit-submodule magit-obsolete
magit-blame magit-stash magit-reflog magit-bisect magit-push magit-pull
magit-fetch magit-clone magit-remote magit-commit magit-sequence
magit-notes magit-worktree magit-tag magit-merge magit-branch
magit-reset magit-files magit-refs magit-status magit magit-repos
magit-apply magit-wip magit-log which-func magit-diff smerge-mode diff
git-commit log-edit message yank-media rmc puny rfc822 mml mml-sec epa
derived epg rfc6068 epg-config gnus-util mm-decode mm-bodies mm-encode
mail-parse rfc2231 rfc2047 rfc2045 mm-util ietf-drums mail-prsvr
mailabbrev mail-utils gmm-utils mailheader pcvs-util add-log magit-core
magit-autorevert autorevert magit-margin magit-transient magit-process
with-editor magit-mode transient comp comp-cstr magit-git magit-section
meson-mode dabbrev markdown-mode color vc-mtn vc-hg vc-bzr vc-src
vc-sccs vc-svn vc-cvs vc-rcs vc eglot array filenotify jsonrpc ert ewoc
debug backtrace tabify imenu dired-aux dired dired-loaddefs mule-util
jka-compr magit-utils crm dash copyright sh-script smie executable
misearch multi-isearch pylint company-oddmuse company-keywords
company-etags company-gtags company-dabbrev-code company-dabbrev
company-files company-clang company-template company-cmake company-bbdb
yasnippet highlight-indentation flymake-proc flymake warnings thingatpt
company-capf company pcase help-fns radix-tree elpy elpy-rpc pyvenv
eshell esh-cmd esh-ext esh-opt esh-proc esh-io esh-arg esh-module
esh-groups esh-util elpy-shell elpy-profile elpy-django s elpy-refactor
ido grep compile etags fileloop generator xref project cus-edit
cus-start cus-load wid-edit make-mode man protbuf cc-mode cc-fonts
cc-guess cc-menus cc-cmds cc-styles cc-align cc-engine cc-vars cc-defs
vc-git diff-mode vc-dispatcher bookmark text-property-search pp
face-remap msb time clang-format xml server dirtrack ob-ruby ob-python
python tramp-sh tramp tramp-loaddefs trampver tramp-integration files-x
tramp-compat shell parse-time ls-lisp ob-dot org-protocol org ob
ob-tangle ob-ref ob-lob ob-table ob-exp org-macro org-footnote org-src
ob-comint org-pcomplete pcomplete comint ansi-color ring org-list
org-faces org-entities noutline outline org-version ob-emacs-lisp
ob-core ob-eval org-table oc-basic bibtex iso8601 time-date org-keys oc
org-loaddefs find-func cal-menu calendar cal-loaddefs ol-notmuch ol rx
org-compat org-macs format-spec skeleton autoinsert advice keyfreq
icomplete edmacro kmacro savehist vertico orderless modus-operandi-theme
modus-themes which-key cl-extra help-mode elec-pair use-package
use-package-ensure use-package-delight use-package-diminish
use-package-bind-key bind-key easy-mmode use-package-core finder-inf
package browse-url url url-proxy url-privacy url-expand url-methods
url-history url-cookie url-domsuf url-util mailcap url-handlers
url-parse auth-source cl-seq eieio eieio-core cl-macs eieio-loaddefs
password-cache json map url-vars seq gv subr-x byte-opt bytecomp
byte-compile cconv cl-loaddefs cl-lib info iso-transl tooltip eldoc
paren electric uniquify ediff-hook vc-hooks lisp-float-type elisp-mode
mwheel term/x-win x-win term/common-win x-dnd tool-bar dnd fontset image
regexp-opt fringe tabulated-list replace newcomment text-mode lisp-mode
prog-mode register page tab-bar menu-bar rfn-eshadow isearch easymenu
timer select scroll-bar mouse jit-lock font-lock syntax font-core
term/tty-colors frame minibuffer cl-generic cham georgian utf-8-lang
misc-lang vietnamese tibetan thai tai-viet lao korean japanese eucjp-ms
cp51932 hebrew greek romanian slovak czech european ethiopic indian
cyrillic chinese composite emoji-zwj charscript charprop case-table
epa-hook jka-cmpr-hook help simple abbrev obarray cl-preloaded nadvice
button loaddefs faces cus-face macroexp files window text-properties
overlay sha1 md5 base64 format env code-pages mule custom widget keymap
hashtable-print-readable backquote threads dbusbind inotify lcms2
dynamic-setting system-font-setting font-render-setting cairo
move-toolbar gtk x-toolkit x multi-tty make-network-process
native-compile emacs)

Memory information:
((conses 16 648085 59613)
 (symbols 48 38951 83)
 (strings 32 141858 8213)
 (string-bytes 1 5316259)
 (vectors 16 70636)
 (vector-slots 8 1253041 72408)
 (floats 8 437 990)
 (intervals 56 21885 480)
 (buffers 992 61))