Package: guix;
Reported by: Ulf Herrman <striness <at> tilde.club>
Date: Fri, 5 May 2023 20:34:01 UTC
Severity: normal
To reply to this bug, email your comments to 63319 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
bug-guix <at> gnu.org:bug#63319; Package guix.
(Fri, 05 May 2023 20:34:01 GMT) Full text and rfc822 format available.Ulf Herrman <striness <at> tilde.club>:bug-guix <at> gnu.org.
(Fri, 05 May 2023 20:34:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Ulf Herrman <striness <at> tilde.club> To: bug-guix <at> gnu.org Subject: Incorrect propagation chain reporting on profile collision Date: Fri, 05 May 2023 15:32:51 -0500
[Message part 1 (text/plain, inline)]
A <manifest-entry> has a `parent' field that contains a promise that returns either #f or another <manifest-entry>. This is somewhat incomplete information - a given entry can be a propagated-input to multiple entries, so there can actually be multiple parents. Given that this is only used for tracing a propagation chain to display to the user, though, that shouldn't be a problem. What is a problem, though, is that this field is determined when the manifest is read in and remains static thereafter. To demonstrate this, consider the following example: User installs package A1, which propagates B1. User also installs package C, which also propagates B1. Later the package definition for A is updated with a newer version, A2, and this propagates a newer version of B, B2. User runs 'guix pull', and subsequently runs 'guix package -u'. Note that C is unchanged and still propagates B1. Guix sees that the proposed new manifest contains A2, C, B1, and B2, and properly detects that B1 and B2 represent a collision. It traces B2 back to A2 correctly, and then goes to hunt down what propagated B1. What happens next depends somewhat on the arbitrary ordering of entries in the manifest, but it is entirely possible that the parent recorded for B1 is not C, but A1, even though the proposed new manifest doesn't even contain A1. So you end up with bizarre error messages like this: ----------------------------------------- $ guix package -u certbot The following package will be upgraded: certbot 1.28.0 → 2.3.0 guix package: error: profile contains conflicting entries for python-cryptography guix package: error: first entry: python-cryptography <at> 40.0.1 /gnu/store/bcixgv8vg9bz6gyvijavqiha77h17icv-python-cryptography-40.0.1 guix package: error: ... propagated from python-josepy <at> 1.13.0 guix package: error: ... propagated from python-acme <at> 2.3.0 guix package: error: ... propagated from certbot <at> 2.3.0 guix package: error: second entry: python-cryptography <at> 3.4.8 /gnu/store/2kz701xhwvcswdp4dj3fd7v86v2ra70x-python-cryptography-3.4.8 guix package: error: ... propagated from python-josepy <at> 1.13.0 guix package: error: ... propagated from python-acme <at> 1.28.0 guix package: error: ... propagated from certbot <at> 1.28.0 hint: You cannot have two different versions or variants of `certbot' in the same profile. ----------------------------------------- This happens because the collision check is performed on the new manifest, and the parent fields are leftover from the old manifest. Ultimately, the parent field is just an optimization - it can always be re-derived from a manifest, and indeed, when a <manifest-entry> is shared across multiple manifests, *must* be re-derived with respect to a given manifest in order for it to make sense. I propose that we remove the 'parent' field entirely and modify &profile-collision-error so that it also reports the manifest the collision occurred in. (guix ui) can then use this to provide more complete (and also accurate) error messages. Note that at present check-for-collisions actually can report manifest entries that aren't "part of" *any* manifest, since they're freshly created by lower-manifest-entry. Tangentially related: it's worth noting that only showing two propagation chains tends to lead to frustrating "whack-a-mole"-style package transactions, since a decision about which package to keep, remove, or upgrade has to be made before the next collision source can be found. We should probably add a profile mode to 'guix graph' that shows the propagation graph for a proposed new profile manifest, or at least let 'guix package' show all collisions when invoked with higher verbosity.
[signature.asc (application/pgp-signature, inline)]
bug-guix <at> gnu.org:bug#63319; Package guix.
(Mon, 08 May 2023 20:34:02 GMT) Full text and rfc822 format available.Message #8 received at 63319 <at> debbugs.gnu.org (full text, mbox):
From: Ulf Herrman <striness <at> tilde.club> To: 63319 <at> debbugs.gnu.org Subject: [PATCH 0/3] Date: Mon, 8 May 2023 15:33:32 -0500
This patch series fixes issue #63319, making propagation chains leading to profile collisions be reported properly. It does this by having 'check-for-collisions' fill in the &profile-collision-error with the manifest and the lowered versions of the entries in question, and (guix ui) derive parent information from the manifest. It then removes the now-unused 'parent' field of <manifest-entry>. Ulf Herrman (3): profiles: include non-lowered entries and manifest in collision error. ui: derive parents of profile collision entries from manifest. profiles: remove `parent' field. guix/inferior.scm | 36 ++++----- guix/profiles.scm | 183 +++++++++++++++++++++++++--------------------- guix/ui.scm | 32 ++++++-- 3 files changed, 142 insertions(+), 109 deletions(-) -- 2.39.1
bug-guix <at> gnu.org:bug#63319; Package guix.
(Mon, 08 May 2023 20:34:02 GMT) Full text and rfc822 format available.Message #11 received at 63319 <at> debbugs.gnu.org (full text, mbox):
From: Ulf Herrman <striness <at> tilde.club> To: 63319 <at> debbugs.gnu.org Subject: [PATCH 1/3] profiles: include non-lowered entries and manifest in collision error. Date: Mon, 8 May 2023 15:33:33 -0500
This provides the necessary information for (guix ui) to accurately determine
the actual entries causing the collision. The entries alone aren't enough,
since they inherit their parent (singular!) field from whatever it happened to
be before any manifest transaction was applied. The lowered variants are
included because (guix ui) needs them for reporting store paths, and the
non-lowered variants are included so that the proper parents can be derived
from the included manifest, which must contain them.
We also add and export a convenience procedure for finding the parents of menu
entries in a particular manifest.
* guix/profiles.scm (profile-collision-error-entry-lowered,
profile-collision-error-conflict-lowered, profile-collision-error-manifest):
new fields.
(check-for-collisions): populate them.
(manifest-entry->parents): new procedure.
* guix/ui.scm (call-with-error-handling): use lowered entries.
---
guix/profiles.scm | 60 ++++++++++++++++++++++++++++++++++++++---------
guix/ui.scm | 4 ++--
2 files changed, 51 insertions(+), 13 deletions(-)
diff --git a/guix/profiles.scm b/guix/profiles.scm
index 03333785f9..b812a6f7d9 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -64,7 +64,10 @@ (define-module (guix profiles)
&profile-collision-error
profile-collision-error?
profile-collision-error-entry
+ profile-collision-error-entry-lowered
profile-collision-error-conflict
+ profile-collision-error-conflict-lowered
+ profile-collision-error-manifest
&missing-generation-error
missing-generation-error?
missing-generation-error-generation
@@ -107,6 +110,7 @@ (define-module (guix profiles)
manifest-installed?
manifest-matching-entries
manifest-search-paths
+ manifest-entry->parents
check-for-collisions
manifest->code
@@ -186,7 +190,10 @@ (define-condition-type &profile-not-found-error &profile-error
(define-condition-type &profile-collision-error &error
profile-collision-error?
(entry profile-collision-error-entry) ;<manifest-entry>
- (conflict profile-collision-error-conflict)) ;<manifest-entry>
+ (conflict profile-collision-error-conflict) ;<manifest-entry>
+ (entry-lowered profile-collision-error-entry-lowered) ;<manifest-entry>
+ (conflict-lowered profile-collision-error-conflict-lowered) ;<manifest-entry>
+ (manifest profile-collision-error-manifest)) ;<manifest>
(define-condition-type &unmatched-pattern-error &error
unmatched-pattern-error?
@@ -329,6 +336,34 @@ (define (recurse entry)
(item (derivation->output-path drv output))
(dependencies dependencies)))))))
+(define (manifest-entry->parents manifest)
+ "Return a procedure that maps each <manifest-entry> in MANIFEST to the list
+of <manifest-entry>s in MANIFEST or their dependencies, recursively, that
+have the entry in question as a direct dependency."
+ (define (visit-entries entries mapping visited?)
+ (match entries
+ (((and entry ($ <manifest-entry> _ _ _ _ dependencies)) . rest)
+ (if (vhash-assq entry visited?)
+ (visit-entries rest mapping visited?)
+ (call-with-values
+ (lambda ()
+ (visit-entries dependencies
+ (fold (lambda (dependency mapping)
+ (vhash-consq dependency entry mapping))
+ mapping
+ dependencies)
+ (vhash-consq entry #t visited?)))
+ (lambda (mapping visited?)
+ (visit-entries rest mapping visited?)))))
+ (()
+ (values mapping visited?))))
+
+ (define mapping
+ (visit-entries (manifest-entries manifest) vlist-null vlist-null))
+
+ (lambda (entry)
+ (vhash-foldq* cons '() entry mapping)))
+
(define* (check-for-collisions manifest system #:key target)
"Check whether the entries of MANIFEST conflict with one another; raise a
'&profile-collision-error' when a conflict is encountered."
@@ -348,25 +383,28 @@ (define candidates
(define lower-pair
(match-lambda
((first second)
- (mlet %store-monad ((first (lower-manifest-entry first system
- #:target target))
- (second (lower-manifest-entry second system
- #:target target)))
- (return (list first second))))))
+ (mlet %store-monad ((first-low (lower-manifest-entry first system
+ #:target target))
+ (second-low (lower-manifest-entry second system
+ #:target target)))
+ (return (list first first-low second second-low))))))
;; Start by lowering CANDIDATES "in parallel".
- (mlet %store-monad ((lst (mapm/accumulate-builds lower-pair candidates)))
+ (mlet* %store-monad ((lst (mapm/accumulate-builds lower-pair candidates)))
(foldm %store-monad
(lambda (entries result)
(match entries
- ((first second)
- (if (string=? (manifest-entry-item first)
- (manifest-entry-item second))
+ ((first first-low second second-low)
+ (if (string=? (manifest-entry-item first-low)
+ (manifest-entry-item second-low))
(return result)
(raise (condition
(&profile-collision-error
(entry first)
- (conflict second))))))))
+ (entry-lowered first-low)
+ (conflict second)
+ (conflict-lowered second-low)
+ (manifest manifest))))))))
#t
lst)))
diff --git a/guix/ui.scm b/guix/ui.scm
index d75243458d..5d2ae23c25 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -750,8 +750,8 @@ (define (port-filename* port)
("out" #f)
(output output)))))
((profile-collision-error? c)
- (let ((entry (profile-collision-error-entry c))
- (conflict (profile-collision-error-conflict c)))
+ (let ((entry (profile-collision-error-entry-lowered c))
+ (conflict (profile-collision-error-conflict-lowered c)))
(define (report-parent-entries entry)
(let ((parent (force (manifest-entry-parent entry))))
(when (manifest-entry? parent)
--
2.39.1
bug-guix <at> gnu.org:bug#63319; Package guix.
(Mon, 08 May 2023 20:34:03 GMT) Full text and rfc822 format available.Message #14 received at 63319 <at> debbugs.gnu.org (full text, mbox):
From: Ulf Herrman <striness <at> tilde.club> To: 63319 <at> debbugs.gnu.org Subject: [PATCH 2/3] ui: derive parents of profile collision entries from manifest. Date: Mon, 8 May 2023 15:33:34 -0500
This fixes <https://issues.guix.gnu.org/63319>.
* guix/ui.scm (display-collision-resolution-hint, call-with-error-handling):
use manifest-entry->parents and the manifest included with the collision to
get the parents.
---
guix/ui.scm | 30 +++++++++++++++++++++++++-----
1 file changed, 25 insertions(+), 5 deletions(-)
diff --git a/guix/ui.scm b/guix/ui.scm
index 5d2ae23c25..71fe5caa72 100644
--- a/guix/ui.scm
+++ b/guix/ui.scm
@@ -674,9 +674,19 @@ (define unit
(define (display-collision-resolution-hint collision)
"Display hints on how to resolve COLLISION, a &profile-collistion-error."
+ (define manifest (profile-collision-error-manifest collision))
+
+ (define entry->parents
+ (manifest-entry->parents manifest))
+
+ (define (manifest-entry-parent entry)
+ (match (entry->parents entry)
+ (() #f)
+ ((x . _) x)))
+
(define (top-most-entry entry)
(let loop ((entry entry))
- (match (force (manifest-entry-parent entry))
+ (match (manifest-entry-parent entry)
(#f entry)
(parent (loop parent)))))
@@ -751,9 +761,19 @@ (define (port-filename* port)
(output output)))))
((profile-collision-error? c)
(let ((entry (profile-collision-error-entry-lowered c))
- (conflict (profile-collision-error-conflict-lowered c)))
+ (conflict (profile-collision-error-conflict-lowered c))
+ (manifest (profile-collision-error-manifest c)))
+
+ (define entry->parents
+ (manifest-entry->parents manifest))
+
+ (define (manifest-entry-parent entry)
+ (match (entry->parents entry)
+ (() #f)
+ ((x . rest) x)))
+
(define (report-parent-entries entry)
- (let ((parent (force (manifest-entry-parent entry))))
+ (let ((parent (manifest-entry-parent entry)))
(when (manifest-entry? parent)
(report-error (G_ " ... propagated from ~a@~a~%")
(manifest-entry-name parent)
@@ -773,13 +793,13 @@ (define (manifest-entry-output* entry)
(manifest-entry-version entry)
(manifest-entry-output* entry)
(manifest-entry-item entry))
- (report-parent-entries entry)
+ (report-parent-entries (profile-collision-error-entry c))
(report-error (G_ " second entry: ~a@~a~a ~a~%")
(manifest-entry-name conflict)
(manifest-entry-version conflict)
(manifest-entry-output* conflict)
(manifest-entry-item conflict))
- (report-parent-entries conflict)
+ (report-parent-entries (profile-collision-error-conflict c))
(display-collision-resolution-hint c)
(exit 1)))
((nar-error? c)
--
2.39.1
bug-guix <at> gnu.org:bug#63319; Package guix.
(Mon, 08 May 2023 20:34:03 GMT) Full text and rfc822 format available.Message #17 received at 63319 <at> debbugs.gnu.org (full text, mbox):
From: Ulf Herrman <striness <at> tilde.club> To: 63319 <at> debbugs.gnu.org Subject: [PATCH 3/3] profiles: remove `parent' field. Date: Mon, 8 May 2023 15:33:35 -0500
This field was only present for consumption by (guix ui) when reporting
propagation chains that lead to profile collision errors, but it is only valid
in general with respect to a single manifest. (guix ui) now derives parent
information by itself with respect to an explicit manifest, so this field is
no longer needed.
* guix/profiles.scm (manifest-entry-parent): remove field.
(package->manifest-entry, sexp->manifest): do not populate it.
(manifest->gexp): adjust match specifications to account for its absence.
* guix/inferior.scm (inferior-package->manifest-entry): do not populate
nonexistent parent field.
---
guix/inferior.scm | 36 ++++++--------
guix/profiles.scm | 123 +++++++++++++++++++---------------------------
2 files changed, 67 insertions(+), 92 deletions(-)
diff --git a/guix/inferior.scm b/guix/inferior.scm
index 5dfd30a6c8..4030640f6d 100644
--- a/guix/inferior.scm
+++ b/guix/inferior.scm
@@ -819,27 +819,23 @@ (define-syntax-rule (memoized package output exp)
result))))
(let loop ((package package)
- (output output)
- (parent (delay #f)))
+ (output output))
(memoized package output
- ;; For each dependency, keep a promise pointing to its "parent" entry.
- (letrec* ((deps (map (match-lambda
- ((label package)
- (loop package "out" (delay entry)))
- ((label package output)
- (loop package output (delay entry))))
- (inferior-package-propagated-inputs package)))
- (entry (manifest-entry
- (name (inferior-package-name package))
- (version (inferior-package-version package))
- (output output)
- (item package)
- (dependencies (delete-duplicates deps))
- (search-paths
- (inferior-package-transitive-native-search-paths package))
- (parent parent)
- (properties properties))))
- entry))))
+ (let ((deps (map (match-lambda
+ ((label package)
+ (loop package "out"))
+ ((label package output)
+ (loop package output)))
+ (inferior-package-propagated-inputs package))))
+ (manifest-entry
+ (name (inferior-package-name package))
+ (version (inferior-package-version package))
+ (output output)
+ (item package)
+ (dependencies (delete-duplicates deps))
+ (search-paths
+ (inferior-package-transitive-native-search-paths package))
+ (properties properties))))))
;;;
diff --git a/guix/profiles.scm b/guix/profiles.scm
index b812a6f7d9..0d22667362 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -90,7 +90,6 @@ (define-module (guix profiles)
manifest-entry-item
manifest-entry-dependencies
manifest-entry-search-paths
- manifest-entry-parent
manifest-entry-properties
lower-manifest-entry
@@ -229,8 +228,6 @@ (define-record-type* <manifest-entry> manifest-entry
(default '()))
(search-paths manifest-entry-search-paths ; search-path-specification*
(default '()))
- (parent manifest-entry-parent ; promise (#f | <manifest-entry>)
- (default (delay #f)))
(properties manifest-entry-properties ; list of symbol/value pairs
(default '())))
@@ -416,29 +413,23 @@ (define (default-properties package)
(transformations `((transformations . ,transformations)))))
(define* (package->manifest-entry package #:optional (output "out")
- #:key (parent (delay #f))
(properties (default-properties package)))
"Return a manifest entry for the OUTPUT of package PACKAGE."
- ;; For each dependency, keep a promise pointing to its "parent" entry.
- (letrec* ((deps (map (match-lambda
- ((label package)
- (package->manifest-entry package
- #:parent (delay entry)))
- ((label package output)
- (package->manifest-entry package output
- #:parent (delay entry))))
- (package-propagated-inputs package)))
- (entry (manifest-entry
- (name (package-name package))
- (version (package-version package))
- (output output)
- (item package)
- (dependencies (delete-duplicates deps))
- (search-paths
- (package-transitive-native-search-paths package))
- (parent parent)
- (properties properties))))
- entry))
+ (let ((deps (map (match-lambda
+ ((label package)
+ (package->manifest-entry package))
+ ((label package output)
+ (package->manifest-entry package output)))
+ (package-propagated-inputs package))))
+ (manifest-entry
+ (name (package-name package))
+ (version (package-version package))
+ (output output)
+ (item package)
+ (dependencies (delete-duplicates deps))
+ (search-paths
+ (package-transitive-native-search-paths package))
+ (properties properties))))
(define* (package->development-manifest package
#:optional
@@ -534,7 +525,7 @@ (define (entry->gexp entry)
(return
(match entry
(($ <manifest-entry> name version output (? string? path)
- (_ ...) (search-paths ...) _ (properties ...))
+ (_ ...) (search-paths ...) (properties ...))
#~(#$name #$version #$output #$path
#$@(optional 'propagated-inputs deps)
#$@(optional 'search-paths
@@ -542,7 +533,7 @@ (define (entry->gexp entry)
search-paths))
#$@(optional 'properties properties)))
(($ <manifest-entry> name version output package
- (_deps ...) (search-paths ...) _ (properties ...))
+ (_deps ...) (search-paths ...) (properties ...))
#~(#$name #$version #$output
(ungexp package (or output "out"))
#$@(optional 'propagated-inputs deps)
@@ -565,7 +556,7 @@ (define (entry->gexp entry)
(define (sexp->manifest sexp)
"Parse SEXP as a manifest."
- (define (infer-dependency item parent)
+ (define (infer-dependency item)
;; Return a <manifest-entry> for ITEM.
(let-values (((name version)
(package-name->name+version
@@ -573,31 +564,25 @@ (define (infer-dependency item parent)
(manifest-entry
(name name)
(version version)
- (item item)
- (parent parent))))
+ (item item))))
- (define* (sexp->manifest-entry/v3 sexp #:optional (parent (delay #f)))
+ (define* (sexp->manifest-entry/v3 sexp)
;; Read SEXP as a version 3 manifest entry.
(match sexp
((name version output path
('propagated-inputs deps)
('search-paths search-paths)
extra-stuff ...)
- ;; For each of DEPS, keep a promise pointing to ENTRY.
- (letrec* ((deps* (map (cut sexp->manifest-entry/v3 <> (delay entry))
- deps))
- (entry (manifest-entry
- (name name)
- (version version)
- (output output)
- (item path)
- (dependencies deps*)
- (search-paths (map sexp->search-path-specification
- search-paths))
- (parent parent)
- (properties (or (assoc-ref extra-stuff 'properties)
- '())))))
- entry))))
+ (manifest-entry
+ (name name)
+ (version version)
+ (output output)
+ (item path)
+ (dependencies (map sexp->manifest-entry/v3 deps))
+ (search-paths (map sexp->search-path-specification
+ search-paths))
+ (properties (or (assoc-ref extra-stuff 'properties)
+ '()))))))
(define-syntax let-fields
(syntax-rules ()
@@ -611,7 +596,7 @@ (define-syntax let-fields
((_ lst () body ...)
(begin body ...))))
- (define* (sexp->manifest-entry sexp #:optional (parent (delay #f)))
+ (define* (sexp->manifest-entry sexp)
(match sexp
(('repeated name version path)
;; This entry is the same as another one encountered earlier; look it
@@ -628,23 +613,20 @@ (define* (sexp->manifest-entry sexp #:optional (parent (delay #f)))
((name version output path fields ...)
(let-fields fields (propagated-inputs search-paths properties)
(mlet* %state-monad
- ((entry -> #f)
- (deps (mapm %state-monad
- (cut sexp->manifest-entry <> (delay entry))
+ ((deps (mapm %state-monad
+ sexp->manifest-entry
propagated-inputs))
+ (entry -> (manifest-entry
+ (name name)
+ (version version)
+ (output output)
+ (item path)
+ (dependencies deps)
+ (search-paths (map sexp->search-path-specification
+ search-paths))
+ (properties properties)))
(visited (current-state))
(key -> (list name version path)))
- (set! entry ;XXX: emulate 'letrec*'
- (manifest-entry
- (name name)
- (version version)
- (output output)
- (item path)
- (dependencies deps)
- (search-paths (map sexp->search-path-specification
- search-paths))
- (parent parent)
- (properties properties)))
(mbegin %state-monad
(set-current-state (vhash-cons key entry visited))
(return entry)))))))
@@ -661,18 +643,15 @@ (define* (sexp->manifest-entry sexp #:optional (parent (delay #f)))
...)))
(manifest
(map (lambda (name version output path deps search-paths)
- (letrec* ((deps* (map (cute infer-dependency <> (delay entry))
- deps))
- (entry (manifest-entry
- (name name)
- (version version)
- (output output)
- (item path)
- (dependencies deps*)
- (search-paths
- (map sexp->search-path-specification
- search-paths)))))
- entry))
+ (manifest-entry
+ (name name)
+ (version version)
+ (output output)
+ (item path)
+ (dependencies (map infer-dependency deps))
+ (search-paths
+ (map sexp->search-path-specification
+ search-paths))))
name version output path deps search-paths)))
;; Version 3 represents DEPS as full-blown manifest entries.
--
2.39.1
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.