GNU bug report logs - #43893
make update-guix-package produced an incorrect hash

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Reported by: Maxim Cournoyer <maxim.cournoyer@HIDDEN>; Keywords: patch; dated Fri, 9 Oct 2020 21:59:02 UTC; Maintainer for guix is bug-guix@HIDDEN.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 14 Oct 2020 04:10:20 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Oct 14 00:10:20 2020
Received: from localhost ([127.0.0.1]:49640 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kSY7H-0007VG-57
	for submit <at> debbugs.gnu.org; Wed, 14 Oct 2020 00:10:19 -0400
Received: from mail-il1-f193.google.com ([209.85.166.193]:43855)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kSY7E-0007Uu-4k
 for 43893 <at> debbugs.gnu.org; Wed, 14 Oct 2020 00:10:18 -0400
Received: by mail-il1-f193.google.com with SMTP id k1so384134ilc.10
 for <43893 <at> debbugs.gnu.org>; Tue, 13 Oct 2020 21:10:16 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=oRLRqkknBqtzicT9tbxXlf631yjjCOCpkiOGpUuwVRA=;
 b=X/1BPJdrvqmXx+VacwCIxjqzWVlFI/ZvZtWd5b2jb66RbSr7wf5welt9heRYXRjZq+
 /wrPJ5z8xuGMNuGjGfo6VaNpQ+JveaBfnYKYX8BTpXIFAaurP8V9+ns9MQE7ru+IFocy
 kq/tdF+dUx7EJZCU0LCNU9Bl86MfvslJwLUM/fEPYV8nImu2UXA5wbKxionIduJW2XjC
 dT4vdLR63b2T77BAY66/ITvztEmgMlgCZaqp0u3mEwFUiwQiXhLBv5jnbe4h+3+sz0CT
 5ij92HoYGfXbLae9XQjQ9hTKNzlB3QKVaasrGjhySrg/QzvU0H6rnksw2OV5lFxY1w0Q
 P3nw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=oRLRqkknBqtzicT9tbxXlf631yjjCOCpkiOGpUuwVRA=;
 b=iDWm/cuL6/nyUjIAB96wSsFUsHEtCrdCo6OjEKAuwLWq3/ueMZ9Squh9X6eeyEDhyC
 62WVxKTrGvpJu4fGgYmF+KLs/RkELcytHWod/aR+5Q+loNzzA9EeJEeH1GWHObYeV4TX
 Em/EZf+kRKt3SEfS51WHV19h/NUMBF8IUUh76hzrxMKZ2ezZSZ/GxGlA/RpkxjNTAuaY
 wLC09+KTN+kCtiLr5WKyOhaSSj9Jek8S7JcpnfrgRQEP+o1XQt4obwJeDyevvZVhUxua
 W5C48+c82dAbx/EmNhsO70DP4gPFJ+S0ahfcmr0s8tM9SlbIsC25cSojWmWxYfL7g2HU
 ClQQ==
X-Gm-Message-State: AOAM530FpdFrAK84UNoWz3vvOHGkGLha4+YWfbEfMXrNCdPKNr3QRSFr
 72rRJeSCaDR7A55RbwS6bojf/uFdqBYKSw==
X-Google-Smtp-Source: ABdhPJyQJT7Ryc3GXZrQWd5xpfrCjW7mnzzZpSu7cIpLFr3vVfMnxM8ScBv5jmSW4HHrOzJIowwgbg==
X-Received: by 2002:a05:6e02:e01:: with SMTP id
 a1mr2491346ilk.273.1602648610324; 
 Tue, 13 Oct 2020 21:10:10 -0700 (PDT)
Received: from hurd (dsl-155-33.b2b2c.ca. [66.158.155.33])
 by smtp.gmail.com with ESMTPSA id o17sm1769255ioa.3.2020.10.13.21.10.08
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Oct 2020 21:10:09 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Marius Bakke <marius@HIDDEN>
Subject: Re: bug#43893: [PATCH v2] maint: update-guix-package: Prevent
 accidentally breaking guix pull.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201011195740.1993-1-maxim.cournoyer@HIDDEN>
 <87imbedsko.fsf@HIDDEN>
Date: Wed, 14 Oct 2020 00:10:07 -0400
In-Reply-To: <87imbedsko.fsf@HIDDEN> (Marius Bakke's message of "Tue, 13 Oct
 2020 18:00:23 +0200")
Message-ID: <87imbd5ty8.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello Marius,

And thanks for the review!

[...]

>> +(define (git-add-worktree directory commit-ish)
>> +  "Create a new git worktree at DIRECTORY, detached on commit COMMIT-ISH."
>> +  (invoke "git" "worktree" "add" "--detach" directory commit-ish))
>
> Is it feasible to use Guile-Git here (given appropriate bindings)?

I had a cursory look at the guile-git sources, and it seems to miss at
least bindings for manipulating worktrees, and I'm not sure how we could
get a list of all the remotes and check their URLs to find the remote
used to push to Savannah.  Supposing we'd be able to get that remote, I
also don't know how we could query if its master branch contains a given
commit (I imagine it's doable, but it's not documented so it takes time
to figure it out :-).

[...]

>> -                              (string-append (getcwd) "/" root))
>> +(define (commit-already-pushed? remote commit)
>> +  "True if COMMIT is found in the REMOTE repository."
>> +  (not (string-null? (with-input-pipe-to-string
>> +                      "git" "branch" "-r" "--contains" commit
>> +                      (string-append remote "/master")))))
>
> ...because parsing git CLI output is error-prone and "ugly" (IMO).  But
> not a strong opinion.

I agree; but for the time being we don't have an another option.  I'd be
happy to be proven wrong.

[...]

>> +         (lambda ()
>> +           (invoke "git" "worktree" "prune")))))
>
> This is not great, because users (well, developers who run this script)
> may have worktrees that are temporarily inaccessible (e.g. on a USB
> drive or whatever).  Better to just leave the stale reference instead of
> potentially destroying users worktrees.

That's a good point.  I've improved the cleanup in v3 to only remove the
worktree it creates and no other.

> Perhaps the script could 'git clone --maxdepth=1' instead of creating
> a worktree?

I think you meant something like:

--8<---------------cut here---------------start------------->8---
$ git clone --branch the-branch --depth -1 %top-srcdir
--8<---------------cut here---------------end--------------->8---

That could work, but it's about 2x slower and more expensive than
creating a worktree (15975 syscalls vs 647, according to 'strace -c').

Thank you,

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 14 Oct 2020 03:18:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 13 23:18:04 2020
Received: from localhost ([127.0.0.1]:49547 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kSXIh-00042R-RS
	for submit <at> debbugs.gnu.org; Tue, 13 Oct 2020 23:18:04 -0400
Received: from mail-il1-f196.google.com ([209.85.166.196]:39448)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kSXIf-00041w-34
 for 43893 <at> debbugs.gnu.org; Tue, 13 Oct 2020 23:18:02 -0400
Received: by mail-il1-f196.google.com with SMTP id q1so3548012ilt.6
 for <43893 <at> debbugs.gnu.org>; Tue, 13 Oct 2020 20:18:01 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:date:message-id:in-reply-to:references
 :mime-version:content-transfer-encoding;
 bh=NpxD8fVmrvRyMrFF4/r3MAZoT01J8fBQGWi+bQMEcME=;
 b=T+K2eDQHtDTRk35MX1nnAA5U0HtlciFO0VGn3TrDK/xaY0K5IN8yLCzjYicJaDuRmV
 JXNzYc/Ckh8zbghvLSj11IvW1dfOx/YwPg8zien6Fz+h3BHhlJOGAVgDs2pZzKq95QEU
 svdgmmyqMnSwBRwrGQaJYDVSMx6K1JtA/vukYqIa9728VKVoCyTmNwo3LigyXIqM+jmc
 26OWcV6D/hbNU78v54khmJmwdvXGagP5HjxDvjrI2vwbFZV2rMWCuUNyn4nD9EjHr0pl
 uvYBV34MZUdgbTH2HRVUpCXwkI30YQ9CQppljSKz3VWn0FIiW9L3WpvdGLnJHDWVM5CD
 8o/A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:in-reply-to
 :references:mime-version:content-transfer-encoding;
 bh=NpxD8fVmrvRyMrFF4/r3MAZoT01J8fBQGWi+bQMEcME=;
 b=OLfFos8c8ZJNHkl2JoZWj6Kx/cf7iPYE3IPt/7K+sovl+5Lx29H+l70cO8719uEHeI
 othpRvFiQuuyaiiItvPqI8pIncfFrVr4FwD0Wh7TyWzsi0HEkc/iT8MjH4Ubd6G+WRNK
 KwrfOs/ETwVqcOzfcBza4lH17feNN3mhaK4k+OH3yIieHmaThywYQ+V3iUpaoe+4Khil
 g7eRdKM+oYkS2zrUAJnN5MiOtGAO/yQXx7VX1tOqkgvD3IOj6vy2c8PSwXdoQh5rO5HP
 qQXlBDbUrFFBSUing7tcoCdB5TSiiF+LoMnCea/e9E3hJoUJ+OUoNy0tioWsewhU9gt8
 z7Pg==
X-Gm-Message-State: AOAM530FAHG2DjdN0py4TDny3jZdKF0Dj69JQuwQXdJcP+ic7AJwz13I
 Vexax20TNZ2nsPvmnPrTln6KhwewVu6CPg==
X-Google-Smtp-Source: ABdhPJxbDU0D24P+6O81XetRg+6YfB/N7W/pSh7CjmcLhH4mM1Hu29n/TWgdPuvlg4kE46YwwZxJ2g==
X-Received: by 2002:a92:3608:: with SMTP id d8mr2294983ila.2.1602645475057;
 Tue, 13 Oct 2020 20:17:55 -0700 (PDT)
Received: from localhost.localdomain (dsl-155-33.b2b2c.ca. [66.158.155.33])
 by smtp.gmail.com with ESMTPSA id p23sm1682747ioo.10.2020.10.13.20.17.53
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Tue, 13 Oct 2020 20:17:54 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: 43893 <at> debbugs.gnu.org
Subject: [PATCH v3] maint: update-guix-package: Prevent accidentally breaking
 guix pull.
Date: Tue, 13 Oct 2020 23:17:05 -0400
Message-Id: <20201014031705.4516-1-maxim.cournoyer@HIDDEN>
X-Mailer: git-send-email 2.28.0
In-Reply-To: <87imbedsko.fsf@HIDDEN>
References: <87imbedsko.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=yes
Content-Transfer-Encoding: 8bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: Maxim Cournoyer <maxim.cournoyer@HIDDEN>, marius@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Fixes <https://issues.guix.gnu.org/43893>.

This changes the 'update-guix-package' tool so that it:

1. Always uses a clean checkout to compute the hash of the updated 'guix'
package.
2. Ensures the commit used in the updated 'guix' package definition has already
been pushed upstream.

* build-aux/update-guix-package.scm (%savannah-guix-git-repo-push-url): New
variable.
(with-input-pipe-to-string, with-temporary-git-worktree): New syntaxes.
(find-origin-remote, git-add-worktree): New procedures.
(commit-already-pushed?): New predicate.
(main): Check the commit used has already been pushed upstream and compute the
hash from a clean checkout.
* doc/contributing.texi (Updating the Guix Package): Document it.
* .dir-locals.el (scheme-mode): Fix indentation of with-temporary-git-worktree.
---
 .dir-locals.el                    |  1 +
 build-aux/update-guix-package.scm | 98 +++++++++++++++++++++----------
 doc/contributing.texi             | 43 ++++++++++++++
 3 files changed, 112 insertions(+), 30 deletions(-)

diff --git a/.dir-locals.el b/.dir-locals.el
index 7f310d2612..19f15b3e1a 100644
--- a/.dir-locals.el
+++ b/.dir-locals.el
@@ -103,6 +103,7 @@
    (eval . (put 'call-with-progress-reporter 'scheme-indent-function 1))
    (eval . (put 'with-repository 'scheme-indent-function 2))
    (eval . (put 'with-temporary-git-repository 'scheme-indent-function 2))
+   (eval . (put 'with-temporary-git-worktree 'scheme-indent-function 2))
    (eval . (put 'with-environment-variables 'scheme-indent-function 1))
    (eval . (put 'with-fresh-gnupg-setup 'scheme-indent-function 1))
 
diff --git a/build-aux/update-guix-package.scm b/build-aux/update-guix-package.scm
index f695e91cfd..9b03b06c7c 100644
--- a/build-aux/update-guix-package.scm
+++ b/build-aux/update-guix-package.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017, 2018 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -24,13 +25,20 @@
 ;;; Code:
 
 (use-modules (guix)
+             (guix ui)
              (guix git-download)
              (guix upstream)
              (guix utils)
              (guix base32)
              (guix build utils)
+             (guix scripts hash)
              (gnu packages package-management)
-             (ice-9 match))
+             (ice-9 match)
+             (ice-9 popen)
+             (ice-9 textual-ports)
+             (srfi srfi-1)
+             (srfi srfi-2)
+             (srfi srfi-26))
 
 (define %top-srcdir
   (string-append (current-source-directory) "/.."))
@@ -101,44 +109,74 @@ COMMIT."
       (exp
        (error "'guix' package definition is not as expected" exp)))))
 
+(define (git-add-worktree directory commit)
+  "Create a new git worktree at DIRECTORY, detached on commit COMMIT."
+  (invoke "git" "worktree" "add" "--detach" directory commit))
+
+(define-syntax-rule (with-temporary-git-worktree commit body ...)
+  "Execute BODY in the context of a temporary git worktree created from COMMIT."
+  (call-with-temporary-directory
+   (lambda (tmp-directory)
+     (dynamic-wind
+       (lambda ()
+         #t)
+       (lambda ()
+         (git-add-worktree tmp-directory commit)
+         (with-directory-excursion tmp-directory body ...))
+       (lambda ()
+         (invoke "git" "worktree" "remove" "--force" tmp-directory))))))
+
+(define %savannah-guix-git-repo-push-url
+  "git.savannah.gnu.org/srv/git/guix.git")
+
+(define-syntax-rule (with-input-pipe-to-string prog arg ...)
+  (let* ((input-pipe (open-pipe* OPEN_READ prog arg ...))
+	 (output (get-string-all input-pipe))
+	 (exit-val (status:exit-val (close-pipe input-pipe))))
+    (unless (zero? exit-val)
+      (error (format #f "Command ~s exited with non-zero exit status: ~s"
+                     (string-join (list prog arg ...)) exit-val)))
+    (string-trim-both output)))
+
+(define (find-origin-remote)
+  "Find the name of the git remote with the Savannah Guix git repo URL."
+  (and-let* ((remotes (string-split (with-input-pipe-to-string
+                                     "git" "remote" "-v")
+                                    #\newline))
+             (origin-entry (find (cut string-contains <>
+                                      (string-append
+                                       %savannah-guix-git-repo-push-url
+                                       " (push)"))
+                                 remotes)))
+    (first (string-split origin-entry #\tab))))
+
+(define (commit-already-pushed? remote commit)
+  "True if COMMIT is found in the REMOTE repository."
+  (not (string-null? (with-input-pipe-to-string
+                      "git" "branch" "-r" "--contains" commit
+                      (string-append remote "/master")))))
+
 
 (define (main . args)
   (match args
     ((commit version)
-     (with-store store
-       (let* ((source   (add-to-store store
-                                      "guix-checkout" ;dummy name
-                                      #t "sha256" %top-srcdir
-                                      #:select? version-controlled?))
-              (hash     (query-path-hash store source))
+     (with-directory-excursion %top-srcdir
+       (or (getenv "GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT")
+           (commit-already-pushed? (find-origin-remote) commit)
+           (leave (G_ "Commit ~a is not pushed upstream.  Aborting.~%") commit))
+       (let* ((hash (with-temporary-git-worktree commit
+                        (nix-base32-string->bytevector
+                         (string-trim-both
+                          (with-output-to-string
+		            (lambda ()
+		              (guix-hash "-rx" ".")))))))
               (location (package-definition-location))
               (old-hash (content-hash-value
-                          (origin-hash (package-source guix)))))
+                         (origin-hash (package-source guix)))))
          (edit-expression location
                           (update-definition commit hash
                                              #:old-hash old-hash
-                                             #:version version))
-
-         ;; Re-add SOURCE to the store, but this time under the real name used
-         ;; in the 'origin'.  This allows us to build the package without
-         ;; having to make a real checkout; thus, it also works when working
-         ;; on a private branch.
-         (reload-module
-          (resolve-module '(gnu packages package-management)))
-
-         (let* ((source (add-to-store store
-                                      (origin-file-name (package-source guix))
-                                      #t "sha256" source))
-                (root   (store-path-package-name source)))
-
-           ;; Add an indirect GC root for SOURCE in the current directory.
-           (false-if-exception (delete-file root))
-           (symlink source root)
-           (add-indirect-root store
-                              (string-append (getcwd) "/" root))
-
-           (format #t "source code for commit ~a: ~a (GC root: ~a)~%"
-                   commit source root)))))
+                                             #:version version)))))
     ((commit)
      ;; Automatically deduce the version and revision numbers.
      (main commit #f))))
diff --git a/doc/contributing.texi b/doc/contributing.texi
index af3601442e..11a932a9bf 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -28,6 +28,7 @@ choice.
 * Submitting Patches::          Share your work.
 * Tracking Bugs and Patches::   Using Debbugs.
 * Commit Access::               Pushing to the official repository.
+* Updating the Guix Package::   Updating the Guix package definition.
 @end menu
 
 @node Building from Git
@@ -1323,3 +1324,45 @@ only push their own awesome changes, but also offer some of their time
 @emph{reviewing} and pushing other people's changes.  As a committer,
 you're welcome to use your expertise and commit rights to help other
 contributors, too!
+
+@node Updating the Guix Package
+@section Updating the Guix Package
+
+@cindex update-guix-package, updating the guix package
+It is sometimes desirable to update the @code{guix} package itself (the
+package defined in @code{(gnu packages package-management)}), for
+example to make new daemon features available for use by the
+@code{guix-service-type} service type.  In order to simplify this task,
+the following command can be used:
+
+@example
+make update-guix-package
+@end example
+
+The @code{update-guix-package} make target will use the last known
+@emph{commit} corresponding to @code{HEAD} in your Guix checkout,
+compute the hash of the Guix sources corresponding to that commit and
+update the @code{commit}, @code{revision} and hash of the @code{guix}
+package definition.
+
+To validate that the updated @code{guix} package hashes are correct and
+that it can be built successfully, the following command can be run from
+the directory of your Guix checkout:
+
+@example
+./pre-inst-env guix build guix
+@end example
+
+To guard against accidentally updating the @code{guix} package to a
+commit that others can't refer to, a check is made that the commit used
+has already been pushed to the Savannah-hosted Guix git repository.
+
+This check can be disabled, @emph{at your own peril}, by setting the
+@code{GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT} environment variable.
+
+To build the resulting 'guix' package when using a private commit, the
+following command can be used:
+
+@example
+./pre-inst-env guix build guix --with-git-url=guix=$PWD
+@end example
-- 
2.28.0





Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 13 Oct 2020 16:01:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Oct 13 12:01:24 2020
Received: from localhost ([127.0.0.1]:48491 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kSMjr-0001vE-UH
	for submit <at> debbugs.gnu.org; Tue, 13 Oct 2020 12:01:24 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35416)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <marius@HIDDEN>) id 1kSMjq-0001v1-1j
 for 43893 <at> debbugs.gnu.org; Tue, 13 Oct 2020 12:01:22 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:37901)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <marius@HIDDEN>)
 id 1kSMjk-0007Ml-Ls; Tue, 13 Oct 2020 12:01:16 -0400
Received: from ti0006q161-1594.bb.online.no ([46.9.75.77]:35668 helo=localhost)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <marius@HIDDEN>)
 id 1kSMjh-0003ao-CT; Tue, 13 Oct 2020 12:01:15 -0400
From: Marius Bakke <marius@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>, 43893 <at> debbugs.gnu.org
Subject: Re: bug#43893: [PATCH v2] maint: update-guix-package: Prevent
 accidentally breaking guix pull.
In-Reply-To: <20201011195740.1993-1-maxim.cournoyer@HIDDEN>
References: <87eem7qcxc.fsf@HIDDEN>
 <20201011195740.1993-1-maxim.cournoyer@HIDDEN>
Date: Tue, 13 Oct 2020 18:00:23 +0200
Message-ID: <87imbedsko.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 43893
Cc: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Maxim Cournoyer <maxim.cournoyer@HIDDEN> writes:

> Fixes <https://issues.guix.gnu.org/43893>.
>
> This changes the 'update-guix-package' tool so that it:
>
> 1. Always uses a clean checkout to compute the hash of the updated 'guix'
> package.
> 2. Ensures the commit used in the updated 'guix' package definition has a=
lready
> been pushed upstream.
>
> * build-aux/update-guix-package.scm (%savannah-guix-git-repo-push-url): N=
ew
> variable.
> (with-input-pipe-to-string): New syntax.
> (find-origin-remote, git-add-worktree): New procedures.
> (commit-already-pushed?): New predicate.
> (main): Check the commit used has already been pushed upstream and comput=
e the
> hash from a clean checkout.
> * doc/contributing.texi (Updating the Guix Package): Document it.

[...]
=20=20
>  (define %top-srcdir
>    (string-append (current-source-directory) "/.."))
> @@ -101,44 +109,69 @@ COMMIT."
>        (exp
>         (error "'guix' package definition is not as expected" exp)))))
>=20=20
> -
> -(define (main . args)
> -  (match args
> -    ((commit version)
> -     (with-store store
> -       (let* ((source   (add-to-store store
> -                                      "guix-checkout" ;dummy name
> -                                      #t "sha256" %top-srcdir
> -                                      #:select? version-controlled?))
> -              (hash     (query-path-hash store source))
> -              (location (package-definition-location))
> -              (old-hash (content-hash-value
> -                          (origin-hash (package-source guix)))))
> -         (edit-expression location
> -                          (update-definition commit hash
> -                                             #:old-hash old-hash
> -                                             #:version version))
> +(define (git-add-worktree directory commit-ish)
> +  "Create a new git worktree at DIRECTORY, detached on commit COMMIT-ISH=
."
> +  (invoke "git" "worktree" "add" "--detach" directory commit-ish))

Is it feasible to use Guile-Git here (given appropriate bindings)?

> +(define %savannah-guix-git-repo-push-url
> +  "git.savannah.gnu.org/srv/git/guix.git")
>=20=20
> -         ;; Re-add SOURCE to the store, but this time under the real nam=
e used
> -         ;; in the 'origin'.  This allows us to build the package without
> -         ;; having to make a real checkout; thus, it also works when wor=
king
> -         ;; on a private branch.
> -         (reload-module
> -          (resolve-module '(gnu packages package-management)))
> +(define-syntax-rule (with-input-pipe-to-string prog arg ...)
> +  (let* ((input-pipe (open-pipe* OPEN_READ prog arg ...))
> +	 (output (get-string-all input-pipe))
> +	 (exit-val (status:exit-val (close-pipe input-pipe))))
> +    (unless (zero? exit-val)
> +      (error (format #f "Command ~s exited with non-zero exit status: ~s"
> +                     (string-join (list prog arg ...)) exit-val)))
> +    (string-trim-both output)))
>=20=20
> -         (let* ((source (add-to-store store
> -                                      (origin-file-name (package-source =
guix))
> -                                      #t "sha256" source))
> -                (root   (store-path-package-name source)))
> +(define (find-origin-remote)
> +  "Find the name of the git remote with the Savannah Guix git repo URL."
> +  (and-let* ((remotes (string-split (with-input-pipe-to-string
> +                                     "git" "remote" "-v")
> +                                    #\newline))
> +             (origin-entry (find (cut string-contains <>
> +                                      (string-append
> +                                       %savannah-guix-git-repo-push-url
> +                                       " (push)"))
> +                                 remotes)))
> +    (first (string-split origin-entry #\tab))))
>=20=20
> -           ;; Add an indirect GC root for SOURCE in the current director=
y.
> -           (false-if-exception (delete-file root))
> -           (symlink source root)
> -           (add-indirect-root store
> -                              (string-append (getcwd) "/" root))
> +(define (commit-already-pushed? remote commit)
> +  "True if COMMIT is found in the REMOTE repository."
> +  (not (string-null? (with-input-pipe-to-string
> +                      "git" "branch" "-r" "--contains" commit
> +                      (string-append remote "/master")))))

...because parsing git CLI output is error-prone and "ugly" (IMO).  But
not a strong opinion.

> -           (format #t "source code for commit ~a: ~a (GC root: ~a)~%"
> -                   commit source root)))))
> +
> +(define (main . args)
> +  (match args
> +    ((commit version)
> +     (with-directory-excursion %top-srcdir
> +       (or (getenv "GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT")
> +           (commit-already-pushed? (find-origin-remote) commit)
> +           (leave (G_ "Commit ~a is not pushed upstream.  Aborting.~%") =
commit))
> +       (dynamic-wind
> +         (lambda ()
> +           #t)
> +         (lambda ()
> +           (call-with-temporary-directory
> +            (lambda (tmp-directory)
> +              (let* ((dummy (git-add-worktree tmp-directory commit))
> +                     (hash (nix-base32-string->bytevector
> +                            (string-trim-both
> +                             (with-output-to-string
> +		               (lambda ()
> +		                 (guix-hash "-rx" tmp-directory))))))
> +                     (location (package-definition-location))
> +                     (old-hash (content-hash-value
> +                                (origin-hash (package-source guix)))))
> +                (edit-expression location
> +                                 (update-definition commit hash
> +                                                    #:old-hash old-hash
> +                                                    #:version version)))=
)))
> +         (lambda ()
> +           (invoke "git" "worktree" "prune")))))

This is not great, because users (well, developers who run this script)
may have worktrees that are temporarily inaccessible (e.g. on a USB
drive or whatever).  Better to just leave the stale reference instead of
potentially destroying users worktrees.

Perhaps the script could 'git clone --maxdepth=3D1' instead of creating a
worktree?

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEu7At3yzq9qgNHeZDoqBt8qM6VPoFAl+FzxcACgkQoqBt8qM6
VPr3UAf6Aq3Pf6Va1zEAP3DCV5RUFoYQSH2mnns+qI0cULZK1iShIvcwdC6X7u1J
1LMaerMRmzSTtjtO3csi62AMvXAr6rI5Ld4aON9NwJkmbnR0dKAvLfC/34Igcm9h
cJO2vp5L6tQag0xcKmQYdI6FcI+xKdh+uBPczVTofyyAl225CENMsyqdAUEmnVHR
KHo8v1Uyy+r4bvnQZnf+sJMGtv/RgbEmPMCk36tCrQYMvQ+2atdx/A5lQ5LpfDPv
OJLL9F06930dyJBxMEC2JKNbplKgVujue7Vyy9n9Mog80jM2Ok2nb69IcYqlm1J+
Gpw9LOmiueEiqldEJ5FeuR2cmu8Q8g==
=rNVH
-----END PGP SIGNATURE-----
--=-=-=--




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 13 Oct 2020 01:33:35 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 12 21:33:35 2020
Received: from localhost ([127.0.0.1]:44542 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kS9C2-0008SD-V8
	for submit <at> debbugs.gnu.org; Mon, 12 Oct 2020 21:33:35 -0400
Received: from mail-qk1-f196.google.com ([209.85.222.196]:37922)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kS9C1-0008Rz-9h
 for 43893 <at> debbugs.gnu.org; Mon, 12 Oct 2020 21:33:33 -0400
Received: by mail-qk1-f196.google.com with SMTP id f21so6619096qko.5
 for <43893 <at> debbugs.gnu.org>; Mon, 12 Oct 2020 18:33:33 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=9mPjx18hwIdDmWQzIlfiNTKBM70KlWbH2rNPzuWwsuE=;
 b=DtoPmsklsrTMQYY2fA82ZKg+UYFHFInwqg2EP+rUQY0gp8JZEiKmoc7CZMOhY9/FTT
 HY1bBqvPlPt4c0pDNYw/XgLAL48Fx8uwRLQFqPxHSzAwdP9cci0awVTLtzmbAokZP7gw
 kb/rXtwli5zlx3mjjhK8IG6qFZKztVEmSe59WCWzRrChfeCfN/nln7P9k+CUNo5XMkvI
 eUqqAhV5BixSigK4ZmFRx0ernXRzl3Z0XGapV8ebSwG9N25Hu4W1Ja7hZ6oMyGNLuesK
 B8Ccba2RHM9c/GYd7BnhJdRnPWRsKOQzLQjeXpWoaGryUIJtpBR90zGcKnfNyzJ4zlug
 gl1A==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=9mPjx18hwIdDmWQzIlfiNTKBM70KlWbH2rNPzuWwsuE=;
 b=pV45s+ZibBlDae1C7qsY4TjDqPoOet8ZG41weVAk8t2/J44cCZrEU104mcf3I1sK0A
 NC7M0aOXoBNFxF8xVS6FwSeExcmvPQETw6aSyynhHqDy7hwj1V1XwyqyulGl2cH0qq1B
 aSnJifWnsPKIQaKfYlo65zGAfZY+sIX56AK3eQBESwgke3UY+R1xlBqPsBllkrV3iY3L
 e9sdFITH/LSyIKFhkObR/hclXl8YwZaZHs10zswRrXV64uwnFrPB2kLoCYJuUUGFkt8f
 T0CMVQJMtGt2YnVA1eaMMIUAX85rJhDOaXKQEkMHELVWyW6J4wUIsk5mbsVZxRDzi0i7
 6hTw==
X-Gm-Message-State: AOAM5306RcqBRIZ6yPtx32qurY1z9gHd3HHZgdcoEJ0GhcpBAXmYk+Zr
 VJURBt7OqCEUmfg4RLCHSIaVQ/A9ADA=
X-Google-Smtp-Source: ABdhPJxcHyujzQfyvOL7jPfpMSLwMhW3D0PWTorMvIqBI623PS8ZesxqmDVB2w9F6xbdBU9LF28xcg==
X-Received: by 2002:a37:480f:: with SMTP id v15mr12328439qka.279.1602552807457; 
 Mon, 12 Oct 2020 18:33:27 -0700 (PDT)
Received: from hurd (dsl-10-136-39.b2b2c.ca. [72.10.136.39])
 by smtp.gmail.com with ESMTPSA id d129sm13940819qkg.127.2020.10.12.18.33.26
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Mon, 12 Oct 2020 18:33:26 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN> <875z7gtupi.fsf@HIDDEN>
 <87lfgbvky6.fsf@HIDDEN>
Date: Mon, 12 Oct 2020 21:33:26 -0400
In-Reply-To: <87lfgbvky6.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
 =?utf-8?Q?s?= message of "Mon, 12 Oct 2020 11:43:13 +0200")
Message-ID: <87blh69aft.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Ludovic,

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

> Hi,
>
> Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:
>
>> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>>
>> [...]
>>
>>>> * build-aux/update-guix-package.scm (git-add-worktree): New procedure.
>>>> (main): Use it to checkout a clean copy of the used commit, and comput=
e the
>>>> hash from it.  Print a user warning after completion.
>>>
>>> I=E2=80=99m not quite enthusiastic about the tool creating a worktree b=
ehind my
>>> back.
>>
>> May I ask why?  It's not something you'd (need to) be aware of (it
>> doesn't leave traces in my v2 as 'git worktree prune' clears up the
>> temporary worktree entries), and it's quite cheap to create.
>
> To me, this is all my workspace, and I generally assume I=E2=80=99m the o=
nly one
> touching it; it=E2=80=99s more about the mental model, I guess.

I see. I understand your position, but also feel it's not a big deal
here, given you wouldn't be able to notice a new worktree unless you
watch the output of 'git worktree list' in a loop while updating the
guix package ;-).

I think the change brings good guarantees to guard against breaking
'guix pull' with a 'make update-guix-package' as it is, and also
documents the thing.  If it's OK with you I'd like to merge it in the
coming days.

Thanks,

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 12 Oct 2020 14:19:00 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 12 10:19:00 2020
Received: from localhost ([127.0.0.1]:43596 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRyfE-0001hU-JB
	for submit <at> debbugs.gnu.org; Mon, 12 Oct 2020 10:19:00 -0400
Received: from dd26836.kasserver.com ([85.13.145.193]:35540)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dannym@HIDDEN>) id 1kRyfC-0001hK-K9
 for 43893 <at> debbugs.gnu.org; Mon, 12 Oct 2020 10:18:59 -0400
Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at
 [80.110.126.103])
 by dd26836.kasserver.com (Postfix) with ESMTPSA id C46383361011;
 Mon, 12 Oct 2020 16:18:56 +0200 (CEST)
Date: Mon, 12 Oct 2020 16:18:53 +0200
From: Danny Milosavljevic <dannym@HIDDEN>
To: Ludovic =?ISO-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
Message-ID: <20201012161853.0ae2308f@HIDDEN>
In-Reply-To: <87r1q3vl1z.fsf@HIDDEN>
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN> <20201010231416.6322dea1@HIDDEN>
 <87r1q3vl1z.fsf@HIDDEN>
X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/RDN9vV_pCnL4nhQNDSbpVTn";
 protocol="application/pgp-signature"; micalg=pgp-sha512
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

--Sig_/RDN9vV_pCnL4nhQNDSbpVTn
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi Ludo,

On Mon, 12 Oct 2020 11:40:56 +0200
Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> Danny Milosavljevic <dannym@HIDDEN> skribis:
>=20
> > Doesn't work for me on x86_64, using a checkout of guix at commit 93d3c=
fec32bbbe1dfbe0be686b371973545b35b8. =20
>=20
> Oh I see, you=E2=80=99re hitting a test failure; I thought you were sayin=
g that
> =E2=80=98--with-git-url=E2=80=99 didn=E2=80=99t work.

I wasn't saying the latter.  That was Maxim.

It just happened that I also had a similar problem, and provided more detai=
l.
But whether it was the same I don't know (it probably was, though).

(also, it was not possible to distinguish whether the test failure was caus=
ed
by "--with-git-url")

--Sig_/RDN9vV_pCnL4nhQNDSbpVTn
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl+EZc0ACgkQ5xo1VCww
uqU39Af5AZ8Gy4GNsL/+/4W+jV+jVCMXU2EX5+hWhA6Pe0QfmT1Dw+WoKPRE28Be
2sEbnpHLm6GNLaRxugZRLwMFvLHShd+aO0vQS5/q+WLIO6tEufQMOeVPsqSoGQJz
9XlR8nl/uYcVsS0aqYiD+fKpUmfOy3fv3XySZgoQyVXT3+r44INmpHKHLfef0hA6
8ffCu/46j5OAeEFJo7KokNZXf4XqIVQ3IuGM9Pwsexs7w4YfC1CI+j0OvKdBejE+
IxnHhnJdkF8gb4rdqVvNvjh+aYK7jxmPoRt1X4W3RGRmXCHsmH/4obrUYiFEW7F/
A9GgbCyF3/n8YydlsI0MgzuMFISurg==
=FKY2
-----END PGP SIGNATURE-----

--Sig_/RDN9vV_pCnL4nhQNDSbpVTn--




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 12 Oct 2020 09:43:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 12 05:43:23 2020
Received: from localhost ([127.0.0.1]:41747 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRuMU-0000eM-PJ
	for submit <at> debbugs.gnu.org; Mon, 12 Oct 2020 05:43:23 -0400
Received: from eggs.gnu.org ([209.51.188.92]:35110)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1kRuMT-0000e9-1y
 for 43893 <at> debbugs.gnu.org; Mon, 12 Oct 2020 05:43:21 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:40139)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1kRuMN-0000jc-PA; Mon, 12 Oct 2020 05:43:15 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=54030 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1kRuMM-0002sv-JT; Mon, 12 Oct 2020 05:43:15 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN> <875z7gtupi.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 21 =?utf-8?Q?Vend=C3=A9miaire?= an 229 de la
 =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 12 Oct 2020 11:43:13 +0200
In-Reply-To: <875z7gtupi.fsf@HIDDEN> (Maxim Cournoyer's message of "Sun, 11
 Oct 2020 15:43:05 -0400")
Message-ID: <87lfgbvky6.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

> Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
>
> [...]
>
>>> * build-aux/update-guix-package.scm (git-add-worktree): New procedure.
>>> (main): Use it to checkout a clean copy of the used commit, and compute=
 the
>>> hash from it.  Print a user warning after completion.
>>
>> I=E2=80=99m not quite enthusiastic about the tool creating a worktree be=
hind my
>> back.
>
> May I ask why?  It's not something you'd (need to) be aware of (it
> doesn't leave traces in my v2 as 'git worktree prune' clears up the
> temporary worktree entries), and it's quite cheap to create.

To me, this is all my workspace, and I generally assume I=E2=80=99m the onl=
y one
touching it; it=E2=80=99s more about the mental model, I guess.

Thanks,
Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 12 Oct 2020 09:41:06 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 12 05:41:06 2020
Received: from localhost ([127.0.0.1]:41742 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRuKI-0000al-EJ
	for submit <at> debbugs.gnu.org; Mon, 12 Oct 2020 05:41:06 -0400
Received: from eggs.gnu.org ([209.51.188.92]:34266)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1kRuKG-0000aB-Bh
 for 43893 <at> debbugs.gnu.org; Mon, 12 Oct 2020 05:41:04 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:40109)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1kRuKA-0000Rw-Ld; Mon, 12 Oct 2020 05:40:58 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=54028 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1kRuKA-0002hI-1W; Mon, 12 Oct 2020 05:40:58 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Danny Milosavljevic <dannym@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN> <20201010231416.6322dea1@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 21 =?utf-8?Q?Vend=C3=A9miaire?= an 229 de la
 =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 12 Oct 2020 11:40:56 +0200
In-Reply-To: <20201010231416.6322dea1@HIDDEN> (Danny Milosavljevic's
 message of "Sat, 10 Oct 2020 23:14:16 +0200")
Message-ID: <87r1q3vl1z.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Danny Milosavljevic <dannym@HIDDEN> skribis:

> Doesn't work for me on x86_64, using a checkout of guix at commit 93d3cfe=
c32bbbe1dfbe0be686b371973545b35b8.

Oh I see, you=E2=80=99re hitting a test failure; I thought you were saying =
that
=E2=80=98--with-git-url=E2=80=99 didn=E2=80=99t work.

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.
Added tag(s) patch. Request was from Maxim Cournoyer <maxim.cournoyer@HIDDEN> to control <at> debbugs.gnu.org. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 12 Oct 2020 04:40:49 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Oct 12 00:40:49 2020
Received: from localhost ([127.0.0.1]:41335 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRpdh-0007Li-19
	for submit <at> debbugs.gnu.org; Mon, 12 Oct 2020 00:40:49 -0400
Received: from mail-qk1-f195.google.com ([209.85.222.195]:38156)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kRpdf-0007LV-FT
 for 43893 <at> debbugs.gnu.org; Mon, 12 Oct 2020 00:40:47 -0400
Received: by mail-qk1-f195.google.com with SMTP id f21so3860540qko.5
 for <43893 <at> debbugs.gnu.org>; Sun, 11 Oct 2020 21:40:47 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=FKx0WnnZpyP8wflmhziVnkSaL2SQEjCe9/A8TIDKpJw=;
 b=Vw2ohINPPo25SjTdCK2SLJYyTDTQ8FFmrTCqS2YxNxu35hPNatEhEuTj3fbiW6i+//
 OMEMWjRbchB9wGCGVk3SkVr+jO6P46evZ+abMWQbg7AM3Rs3USUy+TmhpZWg2j6RMxu4
 T98wwM03z8IrEINY9wOCAGqkpPGtgqiawQSwG9GOw5m+A9sgu3du/X2IB5/cVSN4WXjK
 8vrvoOFRX50e0UeuTk77b7a4963PWEP80pppfI3ImWdRFemn4mnTkVf9KCybiFAqF9Yq
 dIzRF+7Vc5piiAqk83cmcSofbiIOPPPGIoBgyrjeMBfCiqSOxzJ4aV1z7hGsjjEXQXPZ
 gXEQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=FKx0WnnZpyP8wflmhziVnkSaL2SQEjCe9/A8TIDKpJw=;
 b=VNqodn/ohdFhfnrpMhi87vPKR8WTwqN73Key5cYW8kCMjPEVEXQ5e//RF3tS5vz/Re
 pik5hqTLWbbsOAYltD6Kvk71DLeKYo9QrFBeedH2XOs0cg6VCJEfj7dD2QWcNhS+qrPo
 gaDNjpbDEUnzbB+zb5leWlsVZo0rf1xxE2WzgosOuLwpK1TX96pRcu5H/8PNIpDMb+Eg
 zS3p2I+hJ3NUwCmY7kwq6/VZMxN6Sn3RVxKpjqk/byiGzUVQNY098l/MlR8fTIcFZP2L
 Tl+xs6Wwho8tktyorq8/gzIyFMrgY8cTd3DJz5x/S6XRreC2E4m8iLHGYpyvq25bgN9N
 /9bg==
X-Gm-Message-State: AOAM531EeJfWYWERRy1HI+ZY5VKZ8aNwRGTeEV/1cAseAmYxrU+K5wm4
 Ru3+h6PwEUD+6ntopfvrVOTGu0A0I1g=
X-Google-Smtp-Source: ABdhPJwjRdJoq3UYjwoM4oco30WUNYiZLj1OYyRVSx109c5T024TdZdR2QQw30eHR9FYBuFKhKkPqw==
X-Received: by 2002:a37:9a46:: with SMTP id c67mr8215543qke.292.1602477641788; 
 Sun, 11 Oct 2020 21:40:41 -0700 (PDT)
Received: from hurd (dsl-10-136-39.b2b2c.ca. [72.10.136.39])
 by smtp.gmail.com with ESMTPSA id i5sm8728947qte.34.2020.10.11.21.40.40
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 11 Oct 2020 21:40:41 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Danny Milosavljevic <dannym@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN> <20201010231416.6322dea1@HIDDEN>
Date: Mon, 12 Oct 2020 00:40:40 -0400
In-Reply-To: <20201010231416.6322dea1@HIDDEN> (Danny Milosavljevic's
 message of "Sat, 10 Oct 2020 23:14:16 +0200")
Message-ID: <87r1q4rr93.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello Danny,

Danny Milosavljevic <dannym@HIDDEN> writes:

> Hi Ludo,

[...]

> Doesn't work for me on x86_64, using a checkout of guix at commit 93d3cfec32bbbe1dfbe0be686b371973545b35b8.
>
> $ guix environment --pure guix --ad-hoc git guile-readline guile-json
> nano guile-zlib guile-lzlib bash -- ./pre-inst-env guix build guix
> --with-git-url=guix=file://$PWD
>
> I get:
>
> [...]
> updating checkout of 'file:///home/dannym/src/guix-master/guix'...
> retrieved commit 93d3cfec32bbbe1dfbe0be686b371973545b35b8
> [...]
> test-name: channel-instances->manifest
> location: /tmp/guix-build-guix-1.1.0-30.5918cb5.drv-0/source/tests/channels.scm:233
> source:
> + (test-assert
> +   "channel-instances->manifest"

[...]

> actual-value: #f
> actual-error:
> + (wrong-type-arg
> +   "struct-vtable"
> +   "Wrong type argument in position 1 (expecting struct): ~S"
> +   (#f)
> +   (#f))
> result: FAIL
> [...]
> command "make" "check" failed with status 2
> builder for `/gnu/store/csagsyh01rq7ilqkcpaa2d7vp1bn41w3-guix-1.1.0-30.5918cb5.drv' failed with exit code 1
> build of /gnu/store/csagsyh01rq7ilqkcpaa2d7vp1bn41w3-guix-1.1.0-30.5918cb5.drv failed
> View build log at '/var/log/guix/drvs/cs/agsyh01rq7ilqkcpaa2d7vp1bn41w3-guix-1.1.0-30.5918cb5.drv.bz2'.
> guix build: error: build of `/gnu/store/csagsyh01rq7ilqkcpaa2d7vp1bn41w3-guix-1.1.0-30.5918cb5.drv' failed
>
> I didn't even RUN update-guix-package or apply this patch--and the above still fails.
>
> This is reproducible every time--both on my laptop and on a x86_64 build host.
>
> So I'm confused how there's a substitute for the package "guix" available.  How was it built? O_o.

You found a bug!  I've reported and investigated it at
https://issues.guix.gnu.org/43940 and marked the test as expected to
fail for now so that we can continue to update our Guix package.

The current guix package was using
4e3ed9bad9ed5758cdee6e636805f65e9ab710eb, which didn't had that problem.

Thanks,

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 11 Oct 2020 19:58:13 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 11 15:58:13 2020
Received: from localhost ([127.0.0.1]:41100 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRhTw-00016P-Qq
	for submit <at> debbugs.gnu.org; Sun, 11 Oct 2020 15:58:13 -0400
Received: from mail-qt1-f196.google.com ([209.85.160.196]:42694)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kRhTv-00016E-RY
 for 43893 <at> debbugs.gnu.org; Sun, 11 Oct 2020 15:58:12 -0400
Received: by mail-qt1-f196.google.com with SMTP id t9so11398000qtp.9
 for <43893 <at> debbugs.gnu.org>; Sun, 11 Oct 2020 12:58:11 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:date:message-id:mime-version
 :content-transfer-encoding;
 bh=eeKbl+OiwT4i7R1w2hHnqUmE/Ttx6jXVIv098rlk6OY=;
 b=H2fM+tARE/pizmiH+cvTFkdZL2EFXpLUVA21eewzBbqAjnXRCgPwBiunBfAK9ozQwC
 jhv1rgrALoxr9W3hUsXRDiUC3n/xtDE3gj5TXnl6EasebLFojmLc1RMkg1Kw0jtUELPE
 j3KEiDVMVnC1/JNp6LEI1uCYIG4zzj/3gD2j8ju7PssBofKOdavIOxhetprKA8DhvvDd
 3c8YCroLpZQ9GZXbpg+HvPS+aDsQv/ueGTU6aLm/Cb8DfmZgPZSHgWujy8cOHmJYnWCf
 E6oMsa95Eo7xAPQ9KuIxobrxK2PFVrRvoZDrv8YtnkF0DZhmAx2s8mBi8xbJmok82znD
 Wnrw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
 :content-transfer-encoding;
 bh=eeKbl+OiwT4i7R1w2hHnqUmE/Ttx6jXVIv098rlk6OY=;
 b=SNpGr2UZE4mo5zvjsGT9s6ZVzQZFcSx/p6NWAp5MjXHa9I5syJE8C33L5jiiv0lsR/
 gPjZkPjT31cqRhe5SeyDGYXahtRumM+Yen2ytAJs+iqoBoIJitNbM71FD09FHFvf5baw
 2Kjo9ylcd0iN9Lv7RgGB3rXoe5Tk9bB6tipEeMVSVKORXbzysJ1GZxzhTGkRry5a5GP4
 28euNSqIXH96ofo1HQ6sK53WXnqpl7SO5wguZQCkBNoqCz3OMvWCAsvYG/gWOJbzUZWM
 ZqN6PKDy/krWo9YKIACwtyKdlZlEIQmHgJcJnhMolbV3tATOhGWh9M8uqTvIKhh6HAX2
 5Mlg==
X-Gm-Message-State: AOAM5320yEa9PKw49bqq1fkaGy04BfYWZsn43/73FuX1Hwv6B766mttp
 YrN2xgENvfTcX5CuX0PG+c81Efvvw1Y=
X-Google-Smtp-Source: ABdhPJxqCfSXwOe19DVLt1LXD0zuUk/hRE3j1iDPY3mMwP7TMyZdUR2FWafUk0TeRc8Cm2z9/Hk2TQ==
X-Received: by 2002:aed:27dc:: with SMTP id m28mr7195636qtg.12.1602446285768; 
 Sun, 11 Oct 2020 12:58:05 -0700 (PDT)
Received: from localhost.localdomain (dsl-10-136-39.b2b2c.ca. [72.10.136.39])
 by smtp.gmail.com with ESMTPSA id
 l30sm11588584qta.73.2020.10.11.12.58.04
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 11 Oct 2020 12:58:05 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: 43893 <at> debbugs.gnu.org
Subject: [PATCH v2] maint: update-guix-package: Prevent accidentally breaking
 guix pull.
Date: Sun, 11 Oct 2020 15:57:40 -0400
Message-Id: <20201011195740.1993-1-maxim.cournoyer@HIDDEN>
X-Mailer: git-send-email 2.28.0
MIME-Version: 1.0
Content-Type: text/plain; charset=yes
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: dannym@HIDDEN, ludo@HIDDEN,
 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Fixes <https://issues.guix.gnu.org/43893>.

This changes the 'update-guix-package' tool so that it:

1. Always uses a clean checkout to compute the hash of the updated 'guix'
package.
2. Ensures the commit used in the updated 'guix' package definition has already
been pushed upstream.

* build-aux/update-guix-package.scm (%savannah-guix-git-repo-push-url): New
variable.
(with-input-pipe-to-string): New syntax.
(find-origin-remote, git-add-worktree): New procedures.
(commit-already-pushed?): New predicate.
(main): Check the commit used has already been pushed upstream and compute the
hash from a clean checkout.
* doc/contributing.texi (Updating the Guix Package): Document it.
---
 build-aux/update-guix-package.scm | 103 ++++++++++++++++++++----------
 doc/contributing.texi             |  43 +++++++++++++
 2 files changed, 111 insertions(+), 35 deletions(-)

diff --git a/build-aux/update-guix-package.scm b/build-aux/update-guix-package.scm
index f695e91cfd..397b404922 100644
--- a/build-aux/update-guix-package.scm
+++ b/build-aux/update-guix-package.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017, 2018 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -24,13 +25,20 @@
 ;;; Code:
 
 (use-modules (guix)
+             (guix ui)
              (guix git-download)
              (guix upstream)
              (guix utils)
              (guix base32)
              (guix build utils)
+             (guix scripts hash)
              (gnu packages package-management)
-             (ice-9 match))
+             (ice-9 match)
+             (ice-9 popen)
+             (ice-9 textual-ports)
+             (srfi srfi-1)
+             (srfi srfi-2)
+             (srfi srfi-26))
 
 (define %top-srcdir
   (string-append (current-source-directory) "/.."))
@@ -101,44 +109,69 @@ COMMIT."
       (exp
        (error "'guix' package definition is not as expected" exp)))))
 
-
-(define (main . args)
-  (match args
-    ((commit version)
-     (with-store store
-       (let* ((source   (add-to-store store
-                                      "guix-checkout" ;dummy name
-                                      #t "sha256" %top-srcdir
-                                      #:select? version-controlled?))
-              (hash     (query-path-hash store source))
-              (location (package-definition-location))
-              (old-hash (content-hash-value
-                          (origin-hash (package-source guix)))))
-         (edit-expression location
-                          (update-definition commit hash
-                                             #:old-hash old-hash
-                                             #:version version))
+(define (git-add-worktree directory commit-ish)
+  "Create a new git worktree at DIRECTORY, detached on commit COMMIT-ISH."
+  (invoke "git" "worktree" "add" "--detach" directory commit-ish))
+
+(define %savannah-guix-git-repo-push-url
+  "git.savannah.gnu.org/srv/git/guix.git")
 
-         ;; Re-add SOURCE to the store, but this time under the real name used
-         ;; in the 'origin'.  This allows us to build the package without
-         ;; having to make a real checkout; thus, it also works when working
-         ;; on a private branch.
-         (reload-module
-          (resolve-module '(gnu packages package-management)))
+(define-syntax-rule (with-input-pipe-to-string prog arg ...)
+  (let* ((input-pipe (open-pipe* OPEN_READ prog arg ...))
+	 (output (get-string-all input-pipe))
+	 (exit-val (status:exit-val (close-pipe input-pipe))))
+    (unless (zero? exit-val)
+      (error (format #f "Command ~s exited with non-zero exit status: ~s"
+                     (string-join (list prog arg ...)) exit-val)))
+    (string-trim-both output)))
 
-         (let* ((source (add-to-store store
-                                      (origin-file-name (package-source guix))
-                                      #t "sha256" source))
-                (root   (store-path-package-name source)))
+(define (find-origin-remote)
+  "Find the name of the git remote with the Savannah Guix git repo URL."
+  (and-let* ((remotes (string-split (with-input-pipe-to-string
+                                     "git" "remote" "-v")
+                                    #\newline))
+             (origin-entry (find (cut string-contains <>
+                                      (string-append
+                                       %savannah-guix-git-repo-push-url
+                                       " (push)"))
+                                 remotes)))
+    (first (string-split origin-entry #\tab))))
 
-           ;; Add an indirect GC root for SOURCE in the current directory.
-           (false-if-exception (delete-file root))
-           (symlink source root)
-           (add-indirect-root store
-                              (string-append (getcwd) "/" root))
+(define (commit-already-pushed? remote commit)
+  "True if COMMIT is found in the REMOTE repository."
+  (not (string-null? (with-input-pipe-to-string
+                      "git" "branch" "-r" "--contains" commit
+                      (string-append remote "/master")))))
 
-           (format #t "source code for commit ~a: ~a (GC root: ~a)~%"
-                   commit source root)))))
+
+(define (main . args)
+  (match args
+    ((commit version)
+     (with-directory-excursion %top-srcdir
+       (or (getenv "GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT")
+           (commit-already-pushed? (find-origin-remote) commit)
+           (leave (G_ "Commit ~a is not pushed upstream.  Aborting.~%") commit))
+       (dynamic-wind
+         (lambda ()
+           #t)
+         (lambda ()
+           (call-with-temporary-directory
+            (lambda (tmp-directory)
+              (let* ((dummy (git-add-worktree tmp-directory commit))
+                     (hash (nix-base32-string->bytevector
+                            (string-trim-both
+                             (with-output-to-string
+		               (lambda ()
+		                 (guix-hash "-rx" tmp-directory))))))
+                     (location (package-definition-location))
+                     (old-hash (content-hash-value
+                                (origin-hash (package-source guix)))))
+                (edit-expression location
+                                 (update-definition commit hash
+                                                    #:old-hash old-hash
+                                                    #:version version))))))
+         (lambda ()
+           (invoke "git" "worktree" "prune")))))
     ((commit)
      ;; Automatically deduce the version and revision numbers.
      (main commit #f))))
diff --git a/doc/contributing.texi b/doc/contributing.texi
index af3601442e..11a932a9bf 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -28,6 +28,7 @@ choice.
 * Submitting Patches::          Share your work.
 * Tracking Bugs and Patches::   Using Debbugs.
 * Commit Access::               Pushing to the official repository.
+* Updating the Guix Package::   Updating the Guix package definition.
 @end menu
 
 @node Building from Git
@@ -1323,3 +1324,45 @@ only push their own awesome changes, but also offer some of their time
 @emph{reviewing} and pushing other people's changes.  As a committer,
 you're welcome to use your expertise and commit rights to help other
 contributors, too!
+
+@node Updating the Guix Package
+@section Updating the Guix Package
+
+@cindex update-guix-package, updating the guix package
+It is sometimes desirable to update the @code{guix} package itself (the
+package defined in @code{(gnu packages package-management)}), for
+example to make new daemon features available for use by the
+@code{guix-service-type} service type.  In order to simplify this task,
+the following command can be used:
+
+@example
+make update-guix-package
+@end example
+
+The @code{update-guix-package} make target will use the last known
+@emph{commit} corresponding to @code{HEAD} in your Guix checkout,
+compute the hash of the Guix sources corresponding to that commit and
+update the @code{commit}, @code{revision} and hash of the @code{guix}
+package definition.
+
+To validate that the updated @code{guix} package hashes are correct and
+that it can be built successfully, the following command can be run from
+the directory of your Guix checkout:
+
+@example
+./pre-inst-env guix build guix
+@end example
+
+To guard against accidentally updating the @code{guix} package to a
+commit that others can't refer to, a check is made that the commit used
+has already been pushed to the Savannah-hosted Guix git repository.
+
+This check can be disabled, @emph{at your own peril}, by setting the
+@code{GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT} environment variable.
+
+To build the resulting 'guix' package when using a private commit, the
+following command can be used:
+
+@example
+./pre-inst-env guix build guix --with-git-url=guix=$PWD
+@end example
-- 
2.28.0





Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 11 Oct 2020 19:43:14 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Oct 11 15:43:14 2020
Received: from localhost ([127.0.0.1]:41074 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRhFS-0000jP-2q
	for submit <at> debbugs.gnu.org; Sun, 11 Oct 2020 15:43:14 -0400
Received: from mail-qt1-f194.google.com ([209.85.160.194]:37122)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kRhFQ-0000jA-MA
 for 43893 <at> debbugs.gnu.org; Sun, 11 Oct 2020 15:43:13 -0400
Received: by mail-qt1-f194.google.com with SMTP id h19so2125878qtq.4
 for <43893 <at> debbugs.gnu.org>; Sun, 11 Oct 2020 12:43:12 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version:content-transfer-encoding;
 bh=Sakn7RVGxxkBe6d1+xBbSld54ZPRyhpslk7w8zgcFIY=;
 b=I3EDsIz/znOqUyr4AKO8fFqJyDOwHKwVBs83Rq+MpYWvkQk3nz4XRkgVods7/VWxtO
 DA3fnPgj9/ICJj8u/42kzgtadOfBGioyhSQ8Lvx+FiT6qXubwNUpcuDSSYCshl58qcC2
 LOEtaFT3fVyJDIExb4rNpaQTFj0MlDC5/yFq9h8qfpL3ybhwxhz0QB6dp9aepempbg74
 Co2bv1xNd4YobI7TuSTJfleubji3J47OSw2jviNgTUBdHKtaJuMJZQKnqVFEBy+Z4Ns+
 01de94P15YAEkYzXA3VOBk0K0rbI6hYnrNAFnvXCkEgbBqRojQQ3vmIIutCLfEcxFunJ
 JEqg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version:content-transfer-encoding;
 bh=Sakn7RVGxxkBe6d1+xBbSld54ZPRyhpslk7w8zgcFIY=;
 b=oiT85uLayydUE+KLqr1TW+3ElejQNqUObgh5579cINlvXtGLuMqxvN3vrr5B+Sv5w4
 9DhrIcxMb3t6KPQ01UoQ2hwlBecBD2SBu0XOYXWlkVpaYdSC9O2xAUB0B71sfp7oRUfL
 66tDWfH96t780NpDKSs+diJq3LPTX03Q66/dIJO+/Wen4rd9E1emTnXCqOkCls+aeTOu
 3VErfdnDag7ej2B0UGmtFZ1uTGCyo42UAH1VoFRn4JHd01G0c6glXgnv8MVvLKjuZ2sq
 4V+c7d3q+ojDeFeDsVsEfI3uSKJm/OIlhBy9l90t/G8T5DoJcMpQhtbUwvbS8tycyotG
 AAqg==
X-Gm-Message-State: AOAM533UygeVMnsNibH90U9gG7n7BS+OrSSGW8ix38rJ2WQUp3BWbzhx
 ueUdOCV6NZOPwMKJX5EP48FC1ekA5dY=
X-Google-Smtp-Source: ABdhPJzAZwSgnEC8gsjaOcJgfY1VVFQ/KgJTlVFK+cLwTTqdmNdgbEvDdOGRFwhFOxkSEaxoSKpw8Q==
X-Received: by 2002:ac8:5649:: with SMTP id 9mr7071289qtt.109.1602445386824;
 Sun, 11 Oct 2020 12:43:06 -0700 (PDT)
Received: from hurd (dsl-10-136-39.b2b2c.ca. [72.10.136.39])
 by smtp.gmail.com with ESMTPSA id t2sm11325456qti.25.2020.10.11.12.43.05
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sun, 11 Oct 2020 12:43:06 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN>
Date: Sun, 11 Oct 2020 15:43:05 -0400
In-Reply-To: <871ri5j13g.fsf@HIDDEN> ("Ludovic =?utf-8?Q?Court=C3=A8s=22'?=
 =?utf-8?Q?s?= message of "Sat, 10 Oct 2020 22:08:19 +0200")
Message-ID: <875z7gtupi.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello!

Ludovic Court=C3=A8s <ludo@HIDDEN> writes:

[...]

>> * build-aux/update-guix-package.scm (git-add-worktree): New procedure.
>> (main): Use it to checkout a clean copy of the used commit, and compute =
the
>> hash from it.  Print a user warning after completion.
>
> I=E2=80=99m not quite enthusiastic about the tool creating a worktree beh=
ind my
> back.

May I ask why?  It's not something you'd (need to) be aware of (it
doesn't leave traces in my v2 as 'git worktree prune' clears up the
temporary worktree entries), and it's quite cheap to create.

>> -         ;; Re-add SOURCE to the store, but this time under the real na=
me used
>> -         ;; in the 'origin'.  This allows us to build the package witho=
ut
>> -         ;; having to make a real checkout; thus, it also works when wo=
rking
>> -         ;; on a private branch.
>
> So this preserves this possibility, right?

Yes.

[...]

> I could also suggest running =E2=80=98guix build guix --check=E2=80=99.
>
> Honestly, I would simply add this last message; better yet, we could use
> Guile-Git to (1) check whether we=E2=80=99re on a dirty tree and stop rig=
ht away
> if we are

Using the worktree approach, checking for local changes is made
unnecessary; as long as the tool clearly says which commit will be used
for the updated Guix package, I don't see why we should force the
developer to stash their changes (in the same way git checkout doesn't
forces you to do so in the absence of conflicts).

Also, this bug reported was triggered by something like:

--8<---------------cut here---------------start------------->8---
echo ";;some comment" >> .dir-locals \
     && git update-index --skip-worktree .dir-locals \
     && make update-guix-package)
--8<---------------cut here---------------end--------------->8---

leading 'update-guix-package' to produce a wrong hash.  Git has been
told to ignore changes on .dir-locals so checking for changes would not
have been sufficient (you'd also need to 'git ls-files -v' and interpret
the result).

> and (2) check whether the commit exists in the official Git
> repo and error out if it doesn=E2=80=99t, unless
> GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT is set.

That's a good idea; implemented in the v2 patch I'm about to send.

Thanks,

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 11 Oct 2020 02:35:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 10 22:35:31 2020
Received: from localhost ([127.0.0.1]:38929 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRRCt-0004Xr-KV
	for submit <at> debbugs.gnu.org; Sat, 10 Oct 2020 22:35:31 -0400
Received: from mail-qv1-f66.google.com ([209.85.219.66]:38267)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kRRCs-0004XZ-4P
 for 43893 <at> debbugs.gnu.org; Sat, 10 Oct 2020 22:35:30 -0400
Received: by mail-qv1-f66.google.com with SMTP id de3so6777442qvb.5
 for <43893 <at> debbugs.gnu.org>; Sat, 10 Oct 2020 19:35:30 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=jDxiFkyuV2PkdKRAgt22Iv/N4WQTcEEAXoLAa/f++fc=;
 b=EAP2G79v0HN2jEi2+wfAhDa7X+rWqAYKVT0mRFh94Z70ub6Qa5Zt0s8OLGS5MYHiGQ
 ZhjladXf5e3HiWeg4r2j+KUaaxCXJKGw7AdhOiro0kPy2wz9zdmFvmrrE5bT6lketcUf
 eqTdb4dPl7FaLHmxUtAxmeN0n5DJBHVEcT1FV1auXcUFzzjLsDJPpw2BP5ldHZszchnd
 jZyhxsONB4I2RbjvUipFB/SiVRZSO48HV9LUa1TYM5qBGPDIEJVyHb1qa0xqAbNxHFd2
 0CNGeENE6zEKK/Fs8Or/VuELxnJuTQdC7aaUju7p/8wg05JLdA8zlHGP3V087EZzGtRt
 4Gbg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=jDxiFkyuV2PkdKRAgt22Iv/N4WQTcEEAXoLAa/f++fc=;
 b=DCGVPgmvc//h5Ug0r6qOEH6Q27QtoYUy967t6BWrpFKMGj9fRFVZqKBrLxFwt2fxKz
 Qf0vhqYZZzNGlcmF4wqYVZMlhjwW2UMVU/f9pnTVzUls9Vv1gR8K3qPnp1VoS0MX8p/B
 Pr7eY7NiN0LK1oVEjQDUzbEknonOFaDnceDFLg8L3d78N81FiyxnmmXTScuounlHAl0q
 ER+IbdwVEKSjAohxIMXNuenaRryfg/LSjaC+P5YrLoqv89zZQwoih2FGPDH9a9jDcW1e
 viOn3mykige84WFHqiwSx2IpGtr/E39ac+13g75FKPyMKcNLCUFMYibAuJe2AjTI0eN0
 O81w==
X-Gm-Message-State: AOAM532XAZuXUlsJkG/kggXtZuQFNOa01e9HxkRqEZXTE22rte9aI5HN
 duhTFBndzXFW+aJ2Ayv7O/3CuS+QB24=
X-Google-Smtp-Source: ABdhPJxu/xzjHg6Gb4+1CB2P80gPn/XR6f/ClgqjHOClxHZLiU7jXsf/fznpo/TK4iFRpkSWTQeFAw==
X-Received: by 2002:ad4:4c4a:: with SMTP id cs10mr20354264qvb.48.1602383724448; 
 Sat, 10 Oct 2020 19:35:24 -0700 (PDT)
Received: from hurd (dsl-10-148-10.b2b2c.ca. [72.10.148.10])
 by smtp.gmail.com with ESMTPSA id p5sm8113685qtu.13.2020.10.10.19.35.23
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Sat, 10 Oct 2020 19:35:23 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Danny Milosavljevic <dannym@HIDDEN>, ludo@HIDDEN
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <20201010135913.6d0e02d5@HIDDEN>
Date: Sat, 10 Oct 2020 22:35:22 -0400
In-Reply-To: <20201010135913.6d0e02d5@HIDDEN> (Danny Milosavljevic's
 message of "Sat, 10 Oct 2020 13:59:13 +0200")
Message-ID: <87eem5trpx.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Danny,

Danny Milosavljevic <dannym@HIDDEN> writes:

> Hi Maxim,
>
> hmm, git worktree can fail if the commit already is checked out somewhere (for
> example if you invoke make update-guix-package twice in a row), or if the user
> used git worktree on that repo for that commit for other purposes.  That would
> mean that
>
>   make update-guix-package

The "--detached" option is the one allowing to have multiple checkouts of
the same commit.

> would fail in weird undocumented ways again.  Please please let's document
> stuff at least.
>
> Also, why not just fail when there's uncommitted stuff?

Well, if the tool says clearly it's going to update the guix package to
commit X, which is the latest commit in your tree, and doesn't have
technical reasons preventing it from doing so successfully, why enforce
a clean tree?

Using a worktree to setup a pristine checkout also covers for uses such
as 'git update-index --skip-worktree some-file-or-path' or 'git
update-index --assume-unchanged some-file-or-path', which would go
undetected by checking for modifications with 'git diff-index --quiet
HEAD'.  We'd also have to check for the output of

> This patch looks like it goes to quite some length to enable you to build a
> guix package of committed stuff only (which is NOT what your working copy is
> actually like).  Is there a use case for that?  Sounds weird to me.

That's exactly what the tool should do, as there's no point upgrading
the guix package definition to a state that doesn't exist as a commit.

> Even if there's a use case for that, please add a warning if there are
> uncommitted changes that are now not included in the "guix" package.

They were never included in the upgraded guix package (the one you'd
'git push' anyway), they were just put in your store for computing the
hash, which is problematic as we saw.

> Other than that, okay.
>
>>#FIXME: This doesn't work (recursion?)
>>./pre-inst-env guix build guix --with-git-url=guix=file://$PWD
>
> Why doesn't it work?  That sounds like a big limitation--that basically means
> you can't test with local-only commits, you'd always have to push.  We
> should find out why this doesn't work and fix it.

I've tested it again, and it worked... I think what got is surprising is
that the only output I had for multiple minutes was:

--8<---------------cut here---------------start------------->8---
updating checkout of 'file:///home/maxim/src/guix'...
--8<---------------cut here---------------end--------------->8---

And the CPU peaked and the RAM stayed around 1 GiB while it was doing
its magic.

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 21:14:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 10 17:14:25 2020
Received: from localhost ([127.0.0.1]:38774 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRMC8-0005F9-OT
	for submit <at> debbugs.gnu.org; Sat, 10 Oct 2020 17:14:25 -0400
Received: from dd26836.kasserver.com ([85.13.145.193]:41976)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dannym@HIDDEN>) id 1kRMC5-0005Ew-Dp
 for 43893 <at> debbugs.gnu.org; Sat, 10 Oct 2020 17:14:23 -0400
Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at
 [80.110.126.103])
 by dd26836.kasserver.com (Postfix) with ESMTPSA id AED3033612A7;
 Sat, 10 Oct 2020 23:14:19 +0200 (CEST)
Date: Sat, 10 Oct 2020 23:14:16 +0200
From: Danny Milosavljevic <dannym@HIDDEN>
To: Ludovic =?ISO-8859-1?Q?Court=E8s?= <ludo@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
Message-ID: <20201010231416.6322dea1@HIDDEN>
In-Reply-To: <871ri5j13g.fsf@HIDDEN>
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
 <871ri5j13g.fsf@HIDDEN>
X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/aPtWACLK8wDl8CqI/7E=.Qt";
 protocol="application/pgp-signature"; micalg=pgp-sha512
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

--Sig_/aPtWACLK8wDl8CqI/7E=.Qt
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi Ludo,

On Sat, 10 Oct 2020 22:08:19 +0200
Ludovic Court=C3=A8s <ludo@HIDDEN> wrote:

> Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:
>=20
> >  #FIXME: This doesn't work (recursion?)
> > ./pre-inst-env guix build guix --with-git-url=3Dguix=3Dfile://$PWD =20
>=20
> Works for me, please open a separate bug report.  :-)

Doesn't work for me on x86_64, using a checkout of guix at commit 93d3cfec3=
2bbbe1dfbe0be686b371973545b35b8.

$ guix environment --pure guix --ad-hoc git guile-readline guile-json nano =
guile-zlib guile-lzlib bash -- ./pre-inst-env guix build guix --with-git-ur=
l=3Dguix=3Dfile://$PWD

I get:

[...]
updating checkout of 'file:///home/dannym/src/guix-master/guix'...
retrieved commit 93d3cfec32bbbe1dfbe0be686b371973545b35b8
[...]
test-name: channel-instances->manifest
location: /tmp/guix-build-guix-1.1.0-30.5918cb5.drv-0/source/tests/channels=
.scm:233
source:
+ (test-assert
+   "channel-instances->manifest"
+   (let* ((spec (lambda deps
+                  `(channel
+                     (version 0)
+                     (dependencies
+                       ,@(map (lambda (dep)
+                                `(channel
+                                   (name ,dep)
+                                   (url "http://example.org")))
+                              deps)))))
+          (guix (make-instance #:name 'guix))
+          (instance0 (make-instance #:name 'a))
+          (instance1
+            (make-instance #:name 'b #:spec (spec 'a)))
+          (instance2
+            (make-instance #:name 'c #:spec (spec 'b)))
+          (instance3
+            (make-instance #:name 'd #:spec (spec 'c 'a))))
+     (%graft? #f)
+     (let ((source (channel-instance-checkout guix)))
+       (mkdir (string-append source "/build-aux"))
+       (call-with-output-file
+         (string-append
+           source
+           "/build-aux/build-self.scm")
+         (lambda (port)
+           (write '(begin
+                     (use-modules (guix) (gnu packages bootstrap))
+                     (lambda _ (package->derivation %bootstrap-guile)))
+                  port))))
+     (with-store
+       store
+       (let ()
+         (define manifest
+           (run-with-store
+             store
+             (channel-instances->manifest
+               (list guix
+                     instance0
+                     instance1
+                     instance2
+                     instance3))))
+         (define entries (manifest-entries manifest))
+         (define (depends? drv in out)
+           (let ((set (list->set
+                        (requisites
+                          store
+                          (list (derivation-file-name drv)))))
+                 (in (map derivation-file-name in))
+                 (out (map derivation-file-name out)))
+             (and (every (cut set-contains? set <>) in)
+                  (not (any (cut set-contains? set <>) out)))))
+         (define (lookup name)
+           (run-with-store
+             store
+             (lower-object
+               (manifest-entry-item
+                 (manifest-lookup
+                   manifest
+                   (manifest-pattern (name name)))))))
+         (let ((drv-guix (lookup "guix"))
+               (drv0 (lookup "a"))
+               (drv1 (lookup "b"))
+               (drv2 (lookup "c"))
+               (drv3 (lookup "d")))
+           (and (depends?
+                  drv-guix
+                  '()
+                  (list drv0 drv1 drv2 drv3))
+                (depends? drv0 (list) (list drv1 drv2 drv3))
+                (depends? drv1 (list drv0) (list drv2 drv3))
+                (depends? drv2 (list drv1) (list drv3))
+                (depends? drv3 (list drv2 drv0) (list))))))))
actual-value: #f
actual-error:
+ (wrong-type-arg
+   "struct-vtable"
+   "Wrong type argument in position 1 (expecting struct): ~S"
+   (#f)
+   (#f))
result: FAIL
[...]
command "make" "check" failed with status 2
builder for `/gnu/store/csagsyh01rq7ilqkcpaa2d7vp1bn41w3-guix-1.1.0-30.5918=
cb5.drv' failed with exit code 1
build of /gnu/store/csagsyh01rq7ilqkcpaa2d7vp1bn41w3-guix-1.1.0-30.5918cb5.=
drv failed
View build log at '/var/log/guix/drvs/cs/agsyh01rq7ilqkcpaa2d7vp1bn41w3-gui=
x-1.1.0-30.5918cb5.drv.bz2'.
guix build: error: build of `/gnu/store/csagsyh01rq7ilqkcpaa2d7vp1bn41w3-gu=
ix-1.1.0-30.5918cb5.drv' failed

I didn't even RUN update-guix-package or apply this patch--and the above st=
ill fails.

This is reproducible every time--both on my laptop and on a x86_64 build ho=
st.

So I'm confused how there's a substitute for the package "guix" available. =
 How was it built? O_o.

--Sig_/aPtWACLK8wDl8CqI/7E=.Qt
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl+CJCgACgkQ5xo1VCww
uqUvGgf9EY0PMR+deakwHrEkZprVMt2c5E2/robHJBxFn+FtYBCEw/SfUrkee/Nr
Tbr9jx1bZpsuP/mqZYgJ+ZJD51A067VM6gnGVs/02nsJmB8QgunyXDDL7BAjHN4C
eF/fglRcrhRF34/t6BA1ARdq2O2re+l09vuKiHgxXEcvkNacw9uMD5QkKrPJ7gaV
fLk0PI+34JjTd0FM+zz5nmrkDf9VW60qwS7kffP0hSyISJxBRNNGWny+xk9VGZYD
QjMctH9o+Y9jAKchyz+ex5mTrktdhSuYGQsio3UUHkdIy9SCu5cWdJyJiD6L2FHb
2wN2V8hUnl4coL4RKDqEarQ0gnoOgQ==
=fgYz
-----END PGP SIGNATURE-----

--Sig_/aPtWACLK8wDl8CqI/7E=.Qt--




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 20:08:28 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 10 16:08:28 2020
Received: from localhost ([127.0.0.1]:38627 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRLAK-0007il-A6
	for submit <at> debbugs.gnu.org; Sat, 10 Oct 2020 16:08:28 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39318)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1kRLAI-0007iV-Vk
 for 43893 <at> debbugs.gnu.org; Sat, 10 Oct 2020 16:08:27 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:41719)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1kRLAD-0003NU-PV; Sat, 10 Oct 2020 16:08:21 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=47060 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1kRLAD-0007BV-Bh; Sat, 10 Oct 2020 16:08:21 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
Date: Sat, 10 Oct 2020 22:08:19 +0200
In-Reply-To: <20201010050852.19975-1-maxim.cournoyer@HIDDEN> (Maxim
 Cournoyer's message of "Sat, 10 Oct 2020 01:08:52 -0400")
Message-ID: <871ri5j13g.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis:

> By using a fresh copy of the last commit, we ensure the computed hash is
> stable in the face of local edits.  This change also computes the hash
> externally from the store, which allows to verify that the hashes are val=
id
> using, e.g.:
>
>  #FIXME: This doesn't work (recursion?)
> ./pre-inst-env guix build guix --with-git-url=3Dguix=3Dfile://$PWD

Works for me, please open a separate bug report.  :-)

> * build-aux/update-guix-package.scm (git-add-worktree): New procedure.
> (main): Use it to checkout a clean copy of the used commit, and compute t=
he
> hash from it.  Print a user warning after completion.

I=E2=80=99m not quite enthusiastic about the tool creating a worktree behin=
d my
back.

> -         ;; Re-add SOURCE to the store, but this time under the real nam=
e used
> -         ;; in the 'origin'.  This allows us to build the package without
> -         ;; having to make a real checkout; thus, it also works when wor=
king
> -         ;; on a private branch.

So this preserves this possibility, right?

> +            (format #t "Updated Guix to commit ~s.  You must ensure this
> +commit hash exists in the public repository, else 'guix pull' will break.
> +Beware of 'git rebase'~%" commit))))))

I think this is the most important bit.  :-)

I could also suggest running =E2=80=98guix build guix --check=E2=80=99.

Honestly, I would simply add this last message; better yet, we could use
Guile-Git to (1) check whether we=E2=80=99re on a dirty tree and stop right=
 away
if we are, and (2) check whether the commit exists in the official Git
repo and error out if it doesn=E2=80=99t, unless
GUIX_ALLOW_ME_TO_USE_PRIVATE_COMMIT is set.

#2 could also be implemented by building the derivation of
(package-source guix) in =E2=80=98check=E2=80=99 mode (perhaps easier).

WDYT?

Thanks for looking into it, experience has shown that this really needs
to be addressed!

Ludo=E2=80=99.




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 11:59:25 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 10 07:59:25 2020
Received: from localhost ([127.0.0.1]:37065 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kRDX2-00042Q-MQ
	for submit <at> debbugs.gnu.org; Sat, 10 Oct 2020 07:59:25 -0400
Received: from dd26836.kasserver.com ([85.13.145.193]:50640)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dannym@HIDDEN>) id 1kRDWx-00042C-Uc
 for 43893 <at> debbugs.gnu.org; Sat, 10 Oct 2020 07:59:22 -0400
Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at
 [80.110.126.103])
 by dd26836.kasserver.com (Postfix) with ESMTPSA id 253743361700;
 Sat, 10 Oct 2020 13:59:17 +0200 (CEST)
Date: Sat, 10 Oct 2020 13:59:13 +0200
From: Danny Milosavljevic <dannym@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#43893: [PATCH] maint: update-guix-package: Ensure sources
 are clean.
Message-ID: <20201010135913.6d0e02d5@HIDDEN>
In-Reply-To: <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/zlSATVCbyCgHxzNW/Qe==Mr";
 protocol="application/pgp-signature"; micalg=pgp-sha512
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

--Sig_/zlSATVCbyCgHxzNW/Qe==Mr
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

Hi Maxim,

hmm, git worktree can fail if the commit already is checked out somewhere (=
for
example if you invoke make update-guix-package twice in a row), or if the u=
ser
used git worktree on that repo for that commit for other purposes.  That wo=
uld
mean that

  make update-guix-package

would fail in weird undocumented ways again.  Please please let's document
stuff at least.

Also, why not just fail when there's uncommitted stuff?

This patch looks like it goes to quite some length to enable you to build a
guix package of committed stuff only (which is NOT what your working copy is
actually like).  Is there a use case for that?  Sounds weird to me.

Even if there's a use case for that, please add a warning if there are
uncommitted changes that are now not included in the "guix" package.

Other than that, okay.

>#FIXME: This doesn't work (recursion?)
>./pre-inst-env guix build guix --with-git-url=3Dguix=3Dfile://$PWD

Why doesn't it work?  That sounds like a big limitation--that basically mea=
ns
you can't test with local-only commits, you'd always have to push.  We
should find out why this doesn't work and fix it.

--Sig_/zlSATVCbyCgHxzNW/Qe==Mr
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl+BohEACgkQ5xo1VCww
uqWhjQf/csexxtAxHPK5c56adbbkonAjGkOTJ+D9THjRt14TPiUH84HMYhhT02Sd
pLpl1I5ZQX/5sxsoXj5xWcMC2LEZGVP9wgqieXVTx9GA8i+HDatco7YVKKKW/22n
ZAAW49MFf2zT3rlOEedWeCFXWOk1B+jy+odSDp7KHceIsDvl7uBYTTvqqU5J4Qpc
NPmXXv4Cr/Itg84VlGkjfqmLUwbY4VSk3diexP/kApcb6p/e4d44Ld06LoNGGECR
+i5rRLvjYQ+4dGM5Xp61juNoAIciqcMBRUTUmGT2+YpIcJG8WfGIWx946lV5XPfh
i2EDgAMc9WJmwRczooyXR6jM7VIWcA==
=q0DZ
-----END PGP SIGNATURE-----

--Sig_/zlSATVCbyCgHxzNW/Qe==Mr--




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 05:09:37 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 10 01:09:37 2020
Received: from localhost ([127.0.0.1]:36780 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kR78T-0006Op-BO
	for submit <at> debbugs.gnu.org; Sat, 10 Oct 2020 01:09:37 -0400
Received: from mail-qk1-f181.google.com ([209.85.222.181]:33841)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kR78Q-0006OZ-NW
 for 43893 <at> debbugs.gnu.org; Sat, 10 Oct 2020 01:09:35 -0400
Received: by mail-qk1-f181.google.com with SMTP id x20so5904006qkn.1
 for <43893 <at> debbugs.gnu.org>; Fri, 09 Oct 2020 22:09:34 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:date:message-id:mime-version
 :content-transfer-encoding;
 bh=zoUo5lWBoSaqckErFru2+iG+WN65NMr6BV3I83e0GbI=;
 b=cp/HNy4LqhgOQ1A+QYK9uKtIF2zvdqLs6r8MMtHUPpg+TCXSUINycACWvt+9LVGg7j
 QhWAF4hHRa52vdrM2mnFZxN4raUp9Km4ycQ5LJa50+ASWFXpshZaCnaq4s1le6uLw6Gb
 DekwI5PRGKTThyv2tJAVYylaJBPNNyhTte5COK6zFqDmBEEiF+/zeXH5Cj830XQjIEdU
 QUFfO2VqL7fBx1IkE+spAAgT+CrM3VuD+vU7yWmsJVohjEcdxcQBUAKYv4NARAYGNwWr
 gmc5MOGoWug50UBEkKnbsSW9DWMbffqKxZ6G7uvU6cATp79/abU7zcvyBYmYR1EjdOVw
 AVsQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
 :content-transfer-encoding;
 bh=zoUo5lWBoSaqckErFru2+iG+WN65NMr6BV3I83e0GbI=;
 b=owKi0VDyJ+dzyQV0iUyM4+ydLf0MO6mKgT6btTNERmrL3U9GQKEfXpvLkiCoJMpH6V
 kWoQeJJ+vZskLd8JKWaoRYrf/0ibkN3/AoFKxzYriRXky+qxA/wVG85McDPztshGF94J
 g+QIzGcORPMGRLw78zHKtoDhKR4V0PzDQKTYmlHfMvCrmV6CW40eFzuap/CdDczIbSTf
 8ECvzo24zqIFFBiJRsiGbYGVkjnhFW05xkV9Rgw7B3ODR6DPbzM3+c0eC8Cwg8cdkZbl
 JejdAoVGSwPb24c+udmO6hmhbszHgwfvAOfuPwrN6bD29Z4COJp3t/9E27pQUKaH0VID
 YBdA==
X-Gm-Message-State: AOAM5325+hwD/Lk2Uf9/VkllBKOJonsQxG3jcPksWGstvd9L30dGcyhV
 i/sGINz6O7l7G5m42VMxfI2DIEiQ8Cd/LA==
X-Google-Smtp-Source: ABdhPJwxw4kFqFFB33VjFKSIKJv+Mj5kKGe/XuYDTqV8TLbYt0YC+xHFimYzSTH6y5piNGdVQvhfBQ==
X-Received: by 2002:a37:7385:: with SMTP id o127mr1140391qkc.235.1602306569018; 
 Fri, 09 Oct 2020 22:09:29 -0700 (PDT)
Received: from localhost.localdomain (dsl-10-148-10.b2b2c.ca. [72.10.148.10])
 by smtp.gmail.com with ESMTPSA id
 w4sm8204162qtb.0.2020.10.09.22.09.28
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Oct 2020 22:09:28 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: 43893 <at> debbugs.gnu.org
Subject: [PATCH] maint: update-guix-package: Ensure sources are clean.
Date: Sat, 10 Oct 2020 01:08:52 -0400
Message-Id: <20201010050852.19975-1-maxim.cournoyer@HIDDEN>
X-Mailer: git-send-email 2.28.0
MIME-Version: 1.0
Content-Type: text/plain; charset=yes
Content-Transfer-Encoding: 8bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

By using a fresh copy of the last commit, we ensure the computed hash is
stable in the face of local edits.  This change also computes the hash
externally from the store, which allows to verify that the hashes are valid
using, e.g.:

 #FIXME: This doesn't work (recursion?)
./pre-inst-env guix build guix --with-git-url=guix=file://$PWD

* build-aux/update-guix-package.scm (git-add-worktree): New procedure.
(main): Use it to checkout a clean copy of the used commit, and compute the
hash from it.  Print a user warning after completion.
---
 build-aux/update-guix-package.scm | 59 +++++++++++++------------------
 1 file changed, 25 insertions(+), 34 deletions(-)

diff --git a/build-aux/update-guix-package.scm b/build-aux/update-guix-package.scm
index f695e91cfd..b609e57b8f 100644
--- a/build-aux/update-guix-package.scm
+++ b/build-aux/update-guix-package.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright © 2017, 2018 Ludovic Courtès <ludo@HIDDEN>
+;;; Copyright © 2020 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -29,6 +30,7 @@
              (guix utils)
              (guix base32)
              (guix build utils)
+             (guix scripts hash)
              (gnu packages package-management)
              (ice-9 match))
 
@@ -101,44 +103,33 @@ COMMIT."
       (exp
        (error "'guix' package definition is not as expected" exp)))))
 
+(define (git-add-worktree directory commit-ish)
+  "Create a new git worktree at DIRECTORY, detached on commit COMMIT-ISH."
+  (invoke "git" "worktree" "add" "--detach" directory commit-ish))
+
 
 (define (main . args)
   (match args
     ((commit version)
-     (with-store store
-       (let* ((source   (add-to-store store
-                                      "guix-checkout" ;dummy name
-                                      #t "sha256" %top-srcdir
-                                      #:select? version-controlled?))
-              (hash     (query-path-hash store source))
-              (location (package-definition-location))
-              (old-hash (content-hash-value
-                          (origin-hash (package-source guix)))))
-         (edit-expression location
-                          (update-definition commit hash
-                                             #:old-hash old-hash
-                                             #:version version))
-
-         ;; Re-add SOURCE to the store, but this time under the real name used
-         ;; in the 'origin'.  This allows us to build the package without
-         ;; having to make a real checkout; thus, it also works when working
-         ;; on a private branch.
-         (reload-module
-          (resolve-module '(gnu packages package-management)))
-
-         (let* ((source (add-to-store store
-                                      (origin-file-name (package-source guix))
-                                      #t "sha256" source))
-                (root   (store-path-package-name source)))
-
-           ;; Add an indirect GC root for SOURCE in the current directory.
-           (false-if-exception (delete-file root))
-           (symlink source root)
-           (add-indirect-root store
-                              (string-append (getcwd) "/" root))
-
-           (format #t "source code for commit ~a: ~a (GC root: ~a)~%"
-                   commit source root)))))
+     (with-directory-excursion %top-srcdir
+       (call-with-temporary-directory
+        (lambda (tmp-directory)
+          (let* ((dummy (git-add-worktree tmp-directory commit))
+                 (hash (nix-base32-string->bytevector
+                        (string-trim-both
+                         (with-output-to-string
+		           (lambda ()
+		             (guix-hash "-rx" tmp-directory))))))
+                 (location (package-definition-location))
+                 (old-hash (content-hash-value
+                            (origin-hash (package-source guix)))))
+            (edit-expression location
+                             (update-definition commit hash
+                                                #:old-hash old-hash
+                                                #:version version))
+            (format #t "Updated Guix to commit ~s.  You must ensure this
+commit hash exists in the public repository, else 'guix pull' will break.
+Beware of 'git rebase'~%" commit))))))
     ((commit)
      ;; Automatically deduce the version and revision numbers.
      (main commit #f))))
-- 
2.28.0





Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 05:08:41 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Oct 10 01:08:41 2020
Received: from localhost ([127.0.0.1]:36776 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kR77Y-0006ND-RM
	for submit <at> debbugs.gnu.org; Sat, 10 Oct 2020 01:08:41 -0400
Received: from mail-qt1-f178.google.com ([209.85.160.178]:42611)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kR77T-0006Mq-Se
 for 43893 <at> debbugs.gnu.org; Sat, 10 Oct 2020 01:08:39 -0400
Received: by mail-qt1-f178.google.com with SMTP id t9so8964304qtp.9
 for <43893 <at> debbugs.gnu.org>; Fri, 09 Oct 2020 22:08:35 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:references:date:in-reply-to:message-id
 :user-agent:mime-version;
 bh=2rTk1Dfr7uSVsrTgPRhLiG3TO5OF4Se7SklVB6/EgrE=;
 b=B7PZVWGMpS9LGiX6OKcmR1mcYItmhw61yCoikc/iJcTmN4ldZNgGTL7c9EinHuRMew
 aAxsrK9nmdoWBs+ndhWX2BC5WRZaTCBdqLFpkIQ4jONAV9y6OrhcaD5WHshsv+tZ9hji
 ecRB3NszEmVd4Pa2jc8romh+fpw/EDaZeUn09KO+Fb3RFUKmvuYIA1ih6TluaiiPpiPd
 Cj6Gof1K+33i7sj4RPgxTp9EosoZY+mVMMVtOnUNaMrhqp326aQUWNKUVDUQcwov/QdT
 cDwutEa5jeCOPjtqFWQAdS7yHtqdj5Bs7OrxemJYL7mDGlOE8nqUOstodm6/H+27Uov5
 4zhA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
 :message-id:user-agent:mime-version;
 bh=2rTk1Dfr7uSVsrTgPRhLiG3TO5OF4Se7SklVB6/EgrE=;
 b=cFBrNGt8qxYzuQDFxdbKALMznJL81e21akl+gU1LrEmqV9EP4T/FqhpahNot4xp29T
 UnqjEbgd6KKCZpWTiKrYYGl7IWqEiF3Awg7mEm4M/isKJkThwpIvr3VI6zc2HKet7Ihv
 GH+V46/vRsU9v5R/Q+PqnxPzFNeo1T11mnZdAFRuHeIZVwAHQGw4i6oCiGXFGLMgs1RR
 FQczAO3uu8NbfwZkNkCRhasnhWyVv5A+yo3ZtbmC2nAvWs/A7lV50CJyEiy++7dTJqqa
 vk3g2uBAkwWHDkb1TBwRk/djlmMuiHnSd4jVT3GL1WMobHlFuvpcLi6sUEScHhLTfLQa
 Okuw==
X-Gm-Message-State: AOAM531sMN7qsk2WbRa5gfwdubHrdE3IdEpNc/0wiS1Ik0awHeXrM67j
 cGcfalLGXkjXGCF/xy7D/7su3N1/jMgjGw==
X-Google-Smtp-Source: ABdhPJx9ObHdD4PbmxH62JKa2geylv+yQiP4MjkYd1Fiu8VGkMHwL/SHgGOyyMoN/w/QocKucfPpQw==
X-Received: by 2002:ac8:3947:: with SMTP id t7mr1221566qtb.239.1602306510154; 
 Fri, 09 Oct 2020 22:08:30 -0700 (PDT)
Received: from hurd (dsl-10-148-10.b2b2c.ca. [72.10.148.10])
 by smtp.gmail.com with ESMTPSA id l13sm8192475qtv.82.2020.10.09.22.08.29
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Oct 2020 22:08:29 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Danny Milosavljevic <dannym@HIDDEN>
Subject: Re: bug#43893: make update-guix-package produced an incorrect hash
References: <87eem7qcxc.fsf@HIDDEN>
 <20201010020410.3a301654@HIDDEN>
Date: Sat, 10 Oct 2020 01:08:28 -0400
In-Reply-To: <20201010020410.3a301654@HIDDEN> (Danny Milosavljevic's
 message of "Sat, 10 Oct 2020 02:04:10 +0200")
Message-ID: <874kn2r7lf.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.1 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hello Danny,

Thanks for the quick investigation.

Danny Milosavljevic <dannym@HIDDEN> writes:

> I'm guessing it has something to do with update-guix-package using git-predicate
> to add only git-known (but not necessarily committed) files to the store and then
> calculating the checksum of that--but the git-fetch for the guix package not
> necessarily doing the same.

That's a good observation; it is indeed dangerous.  In my case, my tree
looks clean (no modified git-tracked files), but I had in fact
modifications made to .dir-locals that I've been testing and these were
hidden from the view by using:

$ git update-index --skip-worktree .dir-locals

But since the .dir-locals file is known to git, it was probably picked
up with my invisible changes, causing the hash mismatch.

> Then update-guix-package.scm does one worse and actively prevents guix from
> doing the checkout from git when building that "guix" package.  That means the
> person invoking update-guix-package.scm can't notice even when the sha256 hash
> is definitely wrong--because guix will have the source for package "guix" in
> the store already (a faked entry added by update-guix-package.scm) and thus
> won't fetch it again.
>
> Also, doesn't this entire approach have a problem?
>
> If you make a commit into the git repo of guix in order to update the
> package "guix" to commit A, at that point you can't know what commit hash
> commit A will have (since you haven't committed it yet) and yet you have
> to know the commit hash of commit A in order to write it into the package
> definition of package "guix".
>
> That cannot work.
> The only way it works, more or less by accident is that,
>
> (1) At first, update-guix-package.scm does NOT update the "guix" package
> inside, and calculates the hash of the working copy (hash A).
> (2) Then, it updates the "guix" package inside to refer to hash A and to a
> USER-SPECIFIED COMMIT HASH (the latter is determined by the user via
> git rev-parse HEAD).
> (3) Then, it commits that changed working copy as commit B.  Commit B is
> essentially not referred-to by anyone--it's just to make it to the
> git repository so guix pull can pick it up.

Yes, that's my understanding of how it works too.  This means you have
to be extra careful doing this while no-one else is commiting changes,
else you have to start over because rebasing is not an option (it'd
change the hashes, breaking the computed Guix hash).  That's how I broke
'guix pull' the first time I used 'make update-guix-package' :-).  But I
think it's inevitable, so perhaps the best we can do is documement it
well and print a warning when running the target.

> That works only as long as there will be no reference to a nested-nested "guix"
> package, by the eventual user.

What do you mean by nested-nested Guix? Are there valid uses of such a
thing?

> @Maxim: I think this entire thing has to assume that
>
>   git rev-parse HEAD
>
> (which it did at the very beginning of make update-guix-package) actually
> refers to a commit that is available on the guix git repository on savannah.
>
> That means as soon as you change anything (no matter what) (and not actually
> commit that) before invoking
>
>   make update-guix-package
>
> the commit it refers to in the "guix" package will be one which cannot be
> resolved by users.

Indeed.

[...]

> Long story short, we should make "make update-guix-package" check for
> uncommitted changes in the working copy, and fail if any such exist[1].
> There are no downsides that I can see.  Even building from local working
> copy still works then.

Yes, that's a good step.  Actually I just had an idea to use a clean
worktree to do the computation, because that's even safer as it prevents
subtle things like "git update-index --skip-worktree some/path" from
interacting with the computed hash too.

> Also, let's please document update-guix-package.

I'll send a first commit.  I haven't found a way to build it locally
with the command in the message; it seems to create a cycle.  Let me
know what you think.

Maxim




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at 43893 <at> debbugs.gnu.org:


Received: (at 43893) by debbugs.gnu.org; 10 Oct 2020 00:04:17 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 09 20:04:17 2020
Received: from localhost ([127.0.0.1]:36670 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kR2My-0007eL-NK
	for submit <at> debbugs.gnu.org; Fri, 09 Oct 2020 20:04:17 -0400
Received: from dd26836.kasserver.com ([85.13.145.193]:48150)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <dannym@HIDDEN>) id 1kR2Mv-0007eA-Uy
 for 43893 <at> debbugs.gnu.org; Fri, 09 Oct 2020 20:04:15 -0400
Received: from localhost (80-110-126-103.cgn.dynamic.surfer.at
 [80.110.126.103])
 by dd26836.kasserver.com (Postfix) with ESMTPSA id 1637A33624E6;
 Sat, 10 Oct 2020 02:04:12 +0200 (CEST)
Date: Sat, 10 Oct 2020 02:04:10 +0200
From: Danny Milosavljevic <dannym@HIDDEN>
To: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
Subject: Re: bug#43893: make update-guix-package produced an incorrect hash
Message-ID: <20201010020410.3a301654@HIDDEN>
In-Reply-To: <87eem7qcxc.fsf@HIDDEN>
References: <87eem7qcxc.fsf@HIDDEN>
X-Mailer: Claws Mail 3.17.5 (GTK+ 2.24.32; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="Sig_/UAIpur.K=Qc86uNelppw49/";
 protocol="application/pgp-signature"; micalg=pgp-sha512
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 43893
Cc: 43893 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

--Sig_/UAIpur.K=Qc86uNelppw49/
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: quoted-printable

I'm guessing it has something to do with update-guix-package using git-pred=
icate
to add only git-known (but not necessarily committed) files to the store an=
d then
calculating the checksum of that--but the git-fetch for the guix package not
necessarily doing the same.

Then update-guix-package.scm does one worse and actively prevents guix from
doing the checkout from git when building that "guix" package.  That means =
the
person invoking update-guix-package.scm can't notice even when the sha256 h=
ash
is definitely wrong--because guix will have the source for package "guix" in
the store already (a faked entry added by update-guix-package.scm) and thus
won't fetch it again.

Also, doesn't this entire approach have a problem?

If you make a commit into the git repo of guix in order to update the
package "guix" to commit A, at that point you can't know what commit hash
commit A will have (since you haven't committed it yet) and yet you have
to know the commit hash of commit A in order to write it into the package
definition of package "guix".

That cannot work.

The only way it works, more or less by accident is that,

(1) At first, update-guix-package.scm does NOT update the "guix" package
inside, and calculates the hash of the working copy (hash A).
(2) Then, it updates the "guix" package inside to refer to hash A and to a
USER-SPECIFIED COMMIT HASH (the latter is determined by the user via
git rev-parse HEAD).
(3) Then, it commits that changed working copy as commit B.  Commit B is
essentially not referred-to by anyone--it's just to make it to the
git repository so guix pull can pick it up.

That works only as long as there will be no reference to a nested-nested "g=
uix"
package, by the eventual user.

@Maxim: I think this entire thing has to assume that

  git rev-parse HEAD

(which it did at the very beginning of make update-guix-package) actually
refers to a commit that is available on the guix git repository on savannah.

That means as soon as you change anything (no matter what) (and not actually
commit that) before invoking

  make update-guix-package

the commit it refers to in the "guix" package will be one which cannot be
resolved by users.

Worse, if you change anything but not commit it (even locally), then that
surely counts as "part of the checkout" for make update-guix-package, so
the hash will be calculated including those change--but the changes are
not committed, so no one can build the resulting guix package (because
of a hash mismatch).  That can happen automatically very easily if "make"
updates po files.

An easy fix, also done by a lot of other such release tools, is to make

  make update-guix-package

first check whether there are any uncommitted changes.  If so, make it fail.

There's

  guix build guix --with-git-url=3Dguix=3D.

but even that won't work with (locally) uncommitted changes.

Note: uncommitted and unpushed are different.

It's totally fine to have UNPUSHED changes and then use

  ./pre-inst-env guix build guix --with-git-url=3Dguix=3D`pwd`

in order to build it anyway.

But it's not fine to do that with UNCOMMITTED changes--because the sha256
hash will include those, but the commit id won't.

Long story short, we should make "make update-guix-package" check for
uncommitted changes in the working copy, and fail if any such exist[1].
There are no downsides that I can see.  Even building from local working
copy still works then.

Also, let's please document update-guix-package.

[1] git diff-index --quiet HEAD || echo fail

--Sig_/UAIpur.K=Qc86uNelppw49/
Content-Type: application/pgp-signature
Content-Description: OpenPGP digital signature

-----BEGIN PGP SIGNATURE-----

iQEzBAEBCgAdFiEEds7GsXJ0tGXALbPZ5xo1VCwwuqUFAl+A+noACgkQ5xo1VCww
uqXeZgf+MqjmvegOlqH7NPQKPtzT5Yyc5EqplXtTY9pYnKKV5wZj1f2MHpTCLoTq
vQI1aiIEMYkLhmE+AXgh1nM0QZ+ENE7n8Gh/QpdY52Wg4IJ88xJzT2ifQ8z9qURm
VyxwhkGWb/XschF/GhQ6nX4pl2Tu7KR/fTDzS0ROtpcBhKTD7ckIKTAGUYWTRQ4S
u9Ungnkm7rfA7I8oj2cPjiIdj6gRniGLrdAfIDhQglYBtHSUabte2fO3crploVOj
y9FAxrG+ZXF3Er51+k95yJ28sz8gnsWKDCzs8GKxwyJYKOjllwq5oUkdJW+X9UpP
j3RxVrqqPVYsbIoZfxj/jKUvYqRehg==
=EbQx
-----END PGP SIGNATURE-----

--Sig_/UAIpur.K=Qc86uNelppw49/--




Information forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 9 Oct 2020 21:58:47 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Oct 09 17:58:47 2020
Received: from localhost ([127.0.0.1]:36451 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1kR0PX-0002LU-Jb
	for submit <at> debbugs.gnu.org; Fri, 09 Oct 2020 17:58:47 -0400
Received: from lists.gnu.org ([209.51.188.17]:39864)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maxim.cournoyer@HIDDEN>) id 1kR0PU-0002LL-IE
 for submit <at> debbugs.gnu.org; Fri, 09 Oct 2020 17:58:46 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:50624)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <maxim.cournoyer@HIDDEN>)
 id 1kR0PT-00064l-MK
 for bug-guix@HIDDEN; Fri, 09 Oct 2020 17:58:44 -0400
Received: from mail-qv1-xf2e.google.com ([2607:f8b0:4864:20::f2e]:43564)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <maxim.cournoyer@HIDDEN>)
 id 1kR0PS-0003eA-63
 for bug-guix@HIDDEN; Fri, 09 Oct 2020 17:58:43 -0400
Received: by mail-qv1-xf2e.google.com with SMTP id bl9so5496808qvb.10
 for <bug-guix@HIDDEN>; Fri, 09 Oct 2020 14:58:41 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:subject:date:message-id:mime-version;
 bh=uix3tNcfUP5UfZHfYPk8JFZUWSw/JWtK4RgE3O68JFQ=;
 b=CMqWXWWYab6YRR4Cb8lVcB4GU0nj8izgBYBtX5M9+VFskYMlLGGzE3c8q+xFz5ZNmK
 yI/+ppzzt9QqFdDYikWCzD43HGJ3griWlvmWmQRrpy4FMuUMLyoNhww7b3qhJB3Y4ixz
 OCxSHXHiGcpMisthZChmCPydfsTgp4/m2V5yEXlRKJ3XtennDG5UKfBLziAcIVi4IxK/
 uN4ruRZEZ5FUCafSMCaoL/AYw41ztD8uOrtKeeTjwZrCkFsypDjUenTQPLsWUBdCG0FG
 GH04HqqPYIsAuisNeD/lpj9omisnxXQHLYCOUV+SkzmRm6o3/Rf8rvCe+OD2uUCw4SRd
 YjHQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:subject:date:message-id:mime-version;
 bh=uix3tNcfUP5UfZHfYPk8JFZUWSw/JWtK4RgE3O68JFQ=;
 b=qctCl/jg9llROx6mBcK5kX/jSr66mKrhS0GPyEO+4FJ0rOkuAc++NP6G6uuunaQ768
 orciQ4DSEBr1hDmyG6ApNk36jZdPPQRGSfp53vxBv/HPP0UU5Z4g/hyD2kOMvDzEE0eo
 /DE6SFuvoHFtpWXd6vsZINLik70IFw79+IvChIg6TBGTSixDaF3YrPunFGZRy8hNRmIP
 eqCKTK3+TBQ4pmptuoHZnq1aeTYTmeQJANwI419By2hybdohwItg48wPU34a/+Ex9Zay
 mGDgnLmKBWPMwTFVPk5LroS69t8SqBX2tYerf2+X01mOVsN6Ad7EtXGNjUOXArnqxDKl
 9IXQ==
X-Gm-Message-State: AOAM531inLGOdAbAR37ozCgzy/zUpTRXwG2798FnxoMQtZPWZnfBKfhA
 Ror2gpuHkukGKP/2MwhcUj5nz/Ovwric7w==
X-Google-Smtp-Source: ABdhPJy4FberuyKYYp0ycXdNbQVA/7PtlQM0VWk/nf8wRRUbzBze1F7QsKSqWRsFmgOw5UntxwKDoA==
X-Received: by 2002:a05:6214:943:: with SMTP id
 dn3mr14408589qvb.45.1602280721187; 
 Fri, 09 Oct 2020 14:58:41 -0700 (PDT)
Received: from hurd (dsl-10-148-10.b2b2c.ca. [72.10.148.10])
 by smtp.gmail.com with ESMTPSA id i20sm1464439qkl.65.2020.10.09.14.58.40
 for <bug-guix@HIDDEN>
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Fri, 09 Oct 2020 14:58:40 -0700 (PDT)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: bug-guix <bug-guix@HIDDEN>
Subject: make update-guix-package produced an incorrect hash
Date: Fri, 09 Oct 2020 17:58:39 -0400
Message-ID: <87eem7qcxc.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Received-SPF: pass client-ip=2607:f8b0:4864:20::f2e;
 envelope-from=maxim.cournoyer@HIDDEN; helo=mail-qv1-xf2e.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
 That's all we know.
X-Spam_score_int: 0
X-Spam_score: -0.1
X-Spam_bar: /
X-Spam_report: (-0.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 PDS_TONAME_EQ_TOLOCAL_SHORT=1.999, RCVD_IN_DNSWL_NONE=-0.0001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 2.7 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hello, Today, I used 'make update-guix-package' and pushed
 the result (commit f08587682a). Users later reported being unable to pull
 because of a hash mismatch on the newly updated Guix package :-/. I haven't
 investigated why yet, and simply reverted the faulty update for now in commit
 a279f7c61c. 
 Content analysis details:   (2.7 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.0 FREEMAIL_FROM          Sender email is commonly abused enduser mail
 provider (maxim.cournoyer[at]gmail.com)
 1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 0.0 RCVD_IN_MSPIKE_H4      RBL: Very Good reputation (+4)
 [209.51.188.17 listed in wl.mailspike.net]
 -2.3 RCVD_IN_DNSWL_MED      RBL: Sender listed at https://www.dnswl.org/,
 medium trust [209.51.188.17 listed in list.dnswl.org]
 2.0 PDS_TONAME_EQ_TOLOCAL_SHORT Short body with To: name matches
 everything in local email
 0.0 RCVD_IN_MSPIKE_WL      Mailspike good senders
 2.0 SPOOFED_FREEMAIL       No description available.
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -0.3 (/)

Hello,

Today, I used 'make update-guix-package' and pushed the result (commit
f08587682a).  Users later reported being unable to pull because of a
hash mismatch on the newly updated Guix package :-/.

I haven't investigated why yet, and simply reverted the faulty update
for now in commit a279f7c61c.

Thank you,

Maxim




Acknowledgement sent to Maxim Cournoyer <maxim.cournoyer@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#43893; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Wed, 14 Oct 2020 04:15:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.