GNU bug report logs - #50274
[PATCH] guix: git: Adds feature to download git repository to the store.

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix-patches; Reported by: jgart <jgart@HIDDEN>; Keywords: patch; dated Mon, 30 Aug 2021 16:41:02 UTC; Maintainer for guix-patches is guix-patches@HIDDEN.

Message received at 50274 <at> debbugs.gnu.org:


Received: (at 50274) by debbugs.gnu.org; 31 Aug 2021 19:30:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Aug 31 15:30:42 2021
Received: from localhost ([127.0.0.1]:35446 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mL9Sz-00043H-MX
	for submit <at> debbugs.gnu.org; Tue, 31 Aug 2021 15:30:41 -0400
Received: from mx1.dismail.de ([78.46.223.134]:9089)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jgart@HIDDEN>) id 1mL9Sx-0003vj-8O
 for 50274 <at> debbugs.gnu.org; Tue, 31 Aug 2021 15:30:40 -0400
Received: from mx1.dismail.de (localhost [127.0.0.1])
 by mx1.dismail.de (OpenSMTPD) with ESMTP id de27769c;
 Tue, 31 Aug 2021 21:30:32 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=dismail.de; h=date
 :message-id:from:to:cc:subject:in-reply-to:references
 :mime-version:content-type:content-transfer-encoding; s=
 20190914; bh=ExAZfSyEnf39aRNf5VgAp6vjBJABviOMEvAacLZE5n8=; b=Y6m
 JH8JGePQpO50OMMNV5gsArC5YtRc9Q6nQWcVO6gykxTokJdJNcpvRxFshZD4US7x
 cnJ3Ea3OGuHDupgXpK6ESuWlNbBP/A6Nu+G9xrzBX2Z9RnNEKAjYWhVwQop3/vBj
 t3QbcXqKuGPA0ohj1PP5QiPa5RlpZ4J9SgS8yuRB6jS7VZxrVq26e1dJDRpYknnx
 856oiycRXOc6eCm5mK4vJHpVcvjcUULLBbsEF1buplihgpMfA1HVRE93UVsuq4Kk
 cbrkUNgIHTJF3r1DRdKpKUZOWV9MTNy+tqQYFSosYDB5J9a1xb7AWp7EFyYEGqHX
 CQQmQ9b7/1zMfKm158w==
Received: from smtp1.dismail.de (<unknown> [10.240.26.11])
 by mx1.dismail.de (OpenSMTPD) with ESMTP id 4e2619f5;
 Tue, 31 Aug 2021 21:30:32 +0200 (CEST)
Received: from smtp1.dismail.de (localhost [127.0.0.1])
 by smtp1.dismail.de (OpenSMTPD) with ESMTP id 913e0d21;
 Tue, 31 Aug 2021 21:30:32 +0200 (CEST)
Received: by dismail.de (OpenSMTPD) with ESMTPSA id 467adfcc
 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); 
 Tue, 31 Aug 2021 21:30:31 +0200 (CEST)
Date: Tue, 31 Aug 2021 15:30:29 -0400
Message-ID: <20210831153029.GC15239@HIDDEN>
From: jgart <jgart@HIDDEN>
To: Sarah Morgensen <iskarian@HIDDEN>
Subject: Re: [bug#50274] [PATCH] guix: git: Adds feature to download git
 repository to the store.
In-Reply-To: <865yvl1lg6.fsf@HIDDEN>
References: <865yvl1lg6.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Disposition: inline
Content-Transfer-Encoding: 8bit
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 50274
Cc: Julien Lepiller <julien@HIDDEN>, 50274 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

> Please disregard this comment.  I read too fast and didn't catch that
> git-fetch took the value of 'git-download-proc :)

No worries! :)
 _________________________________________ 
/ 3B1D 7F19 E36B B60C 0F5B 2CA9 A52A A2B4 \
\ 77B6 DD35                               /
 ----------------------------------------- 
        \   ^__^
         \  (oo)\_______
            (__)\       )\/\
                ||----w |
                ||     ||




Information forwarded to guix-patches@HIDDEN:
bug#50274; Package guix-patches. Full text available.

Message received at 50274 <at> debbugs.gnu.org:


Received: (at 50274) by debbugs.gnu.org; 31 Aug 2021 19:08:31 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Aug 31 15:08:31 2021
Received: from localhost ([127.0.0.1]:35408 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mL97X-0002ov-Ls
	for submit <at> debbugs.gnu.org; Tue, 31 Aug 2021 15:08:31 -0400
Received: from out0.migadu.com ([94.23.1.103]:31558)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mL97V-0002om-4l
 for 50274 <at> debbugs.gnu.org; Tue, 31 Aug 2021 15:08:30 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630436907;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to; bh=d32OmA4UkyArP7IF3Hd2lFyszFUKUTG0Yw+OkPMxNsM=;
 b=MF2/1WhAcvEjxm1c0fRLs6bQP+iQCPfrAILeoRg8YJdERHf/7ldhX/sBPJI/0cBtS4U75b
 fEiHrxqi/1bsNjVq57OBxj67XtnOsog5D5lTKzXtwG4mq6/6gUGCwvJFGQqMAta4ztk6TN
 AjRPH0zSiKnQnZp6xruY0Gl9bxSL7fg=
From: Sarah Morgensen <iskarian@HIDDEN>
To: jgart <jgart@HIDDEN>
Subject: Re: [bug#50274] [PATCH] guix: git: Adds feature to download git
 repository to the store.
Date: Tue, 31 Aug 2021 12:08:25 -0700
In-Reply-To: Sarah Morgensen's message of "Tue, 31 Aug 2021 11:50:48 -0700 (15
 minutes, 42 seconds ago)"
Message-ID: <865yvl1lg6.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 50274
Cc: Julien Lepiller <julien@HIDDEN>, 50274 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Hi again,

Sarah Morgensen <iskarian@HIDDEN> writes:

>>             (fetch (assq-ref opts 'download-proc))
>> +           (git-fetch (assq-ref opts 'git-download-proc))
>> +	   (commit (assq-ref opts 'commit))
>>             (path  (parameterize ((current-terminal-columns
>>                                    (terminal-columns)))
>> -                    (fetch (uri->string uri)
>> -                           #:verify-certificate?
>> -                           (assq-ref opts 'verify-certificate?))))
>> -           (hash  (call-with-input-file
>> -                      (or path
>> -                          (leave (G_ "~a: download failed~%")
>> -                                 arg))
>> -                    (cute port-hash (assoc-ref opts 'hash-algorithm) <>)))
>> +		    (if commit
>> +		        (git-fetch (uri->string uri) commit)
>
> You don't actually seem to use the download-git-to-store procedure you
> wrote above.  An oversight?

Please disregard this comment.  I read too fast and didn't catch that
git-fetch took the value of 'git-download-proc :)

--
Sarah




Information forwarded to guix-patches@HIDDEN:
bug#50274; Package guix-patches. Full text available.

Message received at 50274 <at> debbugs.gnu.org:


Received: (at 50274) by debbugs.gnu.org; 31 Aug 2021 18:50:57 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Tue Aug 31 14:50:57 2021
Received: from localhost ([127.0.0.1]:35386 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mL8qW-0002L0-V4
	for submit <at> debbugs.gnu.org; Tue, 31 Aug 2021 14:50:57 -0400
Received: from out0.migadu.com ([94.23.1.103]:27320)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mL8qT-0002Kp-Mf
 for 50274 <at> debbugs.gnu.org; Tue, 31 Aug 2021 14:50:55 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630435851;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=Ib47Ra2325KK7553QF+khNFLNaHPJ4+coN+eofxN5Yo=;
 b=G8QAVglXSJtVHJ+m2mvThR74OHFYA5c0YUO1B+lVTwwceij5g2X0UYEpK2fgxxav50DjhH
 kGMFMZhFgvIgXrDoUh2xn2L/QKeKx2bgVMBeICgxdZamp6XlLX+iURFj1RiudB3qW0zIk0
 NAswGw8qM0xf4GgWFNwm4UyrdLfhC5E=
From: Sarah Morgensen <iskarian@HIDDEN>
To: jgart <jgart@HIDDEN>
Subject: Re: [bug#50274] [PATCH] guix: git: Adds feature to download git
 repository to the store.
References: <20210830163918.19419-1-jgart@HIDDEN>
Date: Tue, 31 Aug 2021 11:50:48 -0700
In-Reply-To: <20210830163918.19419-1-jgart@HIDDEN> (jgart@HIDDEN's
 message of "Mon, 30 Aug 2021 12:39:19 -0400 (1 day, 1 hour, 24 minutes
 ago)")
Message-ID: <86a6kx1m9j.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: -0.7 (/)
X-Debbugs-Envelope-To: 50274
Cc: Julien Lepiller <julien@HIDDEN>, 50274 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)

Hello,

Thanks for the patch!  I've been wanting a feature like this.  I haven't
tested your patch, but just reading it, I have a few comments...

jgart <jgart@HIDDEN> writes:

> From: Julien Lepiller <julien@HIDDEN>
>
> * guix/git.scm (download-git-to-store): Download Git repository from
> URL at COMMIT to STORE, either under NAME or URL's basename if omitted.
> Write progress reports to LOG.  RECURSIVE? has the same effect as the
> same-named parameter of 'git-fetch'.
>
> * guix/scripts/download.scm (download-git-to-store*): Adds cli option.
> Examples:
> guix download --git-commit=v0.1.1 github.com/anaseto/gruid-tcell
> guix download -c v0.1.1 https://github.com/anaseto/gruid-tcell

These examples should probably be in the main commit message, not the
ChangeLog entry.

> ---
>  guix/git.scm              | 24 +++++++++++++++++-
>  guix/scripts/download.scm | 51 ++++++++++++++++++++++++++++++++-------
>  2 files changed, 65 insertions(+), 10 deletions(-)
>
> diff --git a/guix/git.scm b/guix/git.scm
> index 9c6f326c36..4c70782b97 100644
> --- a/guix/git.scm
> +++ b/guix/git.scm
> @@ -28,6 +28,7 @@
>    #:use-module (gcrypt hash)
>    #:use-module ((guix build utils)
>                  #:select (mkdir-p delete-file-recursively))
> +  #:use-module ((guix build git) #:select (git-fetch))
>    #:use-module (guix store)
>    #:use-module (guix utils)
>    #:use-module (guix records)
> @@ -43,6 +44,7 @@
>    #:use-module (srfi srfi-11)
>    #:use-module (srfi srfi-34)
>    #:use-module (srfi srfi-35)
> +  #:use-module (web uri)
>    #:export (%repository-cache-directory
>              honor-system-x509-certificates!
>  
> @@ -61,7 +63,9 @@
>              git-checkout-url
>              git-checkout-branch
>              git-checkout-commit
> -            git-checkout-recursive?))
> +            git-checkout-recursive?
> +
> +	    download-git-to-store))

Your patch has a number of tabs in it; they should be converted to spaces.

>  
>  (define %repository-cache-directory
>    (make-parameter (string-append (cache-directory #:ensure? #f)
> @@ -614,6 +618,24 @@ objects: 'ancestor (meaning that OLD is an ancestor of NEW), 'descendant, or
>                                  #:recursive? recursive?
>                                  #:log-port (current-error-port)))))
>  
> +(define* (download-git-to-store store url commit
> +				#:optional (name (basename url))
> +                                #:key (log (current-error-port)) recursive?)
> +  "Download Git repository from URL at COMMIT to STORE, either under NAME or
> +URL's basename if omitted.  Write progress reports to LOG.  RECURSIVE? has the
> +same effect as the same-named parameter of 'git-fetch'."
> +  (define uri
> +    (string->uri url))

You've defined uri but not used it anywhere.

> +
> +    (call-with-temporary-directory
> +     (lambda (temp)
> +       (let ((result
> +              (parameterize ((current-output-port log))
> +                (git-fetch url commit temp
> +        		   #:recursive? recursive?))))
> +         (and result
> +              (add-to-store store name #t "sha256" temp))))))
> +

Is there a reason for not using latest-repository-commit instead of
this?  That already takes care of temporary directories, ignores .git,
and reduces duplication if you download different tags from the same
repository:

  guix download -c v0.1.3 example.com/repo

  guix download -c v2.1.0 example.com/repo

>  ;; Local Variables:
>  ;; eval: (put 'with-repository 'scheme-indent-function 2)
>  ;; End:
> diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm
> index 5a91390358..6253ecaa5c 100644
> --- a/guix/scripts/download.scm
> +++ b/guix/scripts/download.scm
> @@ -26,15 +26,19 @@
>    #:use-module (guix base32)
>    #:autoload   (guix base64) (base64-encode)
>    #:use-module ((guix download) #:hide (url-fetch))
> +  #:use-module ((guix git) #:select (download-git-to-store))
>    #:use-module ((guix build download)
>                  #:select (url-fetch))
>    #:use-module ((guix progress)
>                  #:select (current-terminal-columns))
> +  #:use-module ((guix serialization)
> +		#:select (write-file))
>    #:use-module ((guix build syscalls)
>                  #:select (terminal-columns))
>    #:use-module (web uri)
>    #:use-module (ice-9 match)
>    #:use-module (srfi srfi-1)
> +  #:use-module (srfi srfi-11)
>    #:use-module (srfi srfi-14)
>    #:use-module (srfi srfi-26)
>    #:use-module (srfi srfi-37)
> @@ -76,12 +80,20 @@
>                         (ensure-valid-store-file-name (basename url))
>                         #:verify-certificate? verify-certificate?)))
>  
> +(define* (download-git-to-store* url commit #:key recursive?)
> +  (with-store store
> +    (download-git-to-store store url commit
> +			   (ensure-valid-store-file-name (basename url))
> +			   #:recursive? recursive?)))
> +
>  (define %default-options
>    ;; Alist of default option values.
>    `((format . ,bytevector->nix-base32-string)
>      (hash-algorithm . ,(hash-algorithm sha256))
>      (verify-certificate? . #t)
> -    (download-proc . ,download-to-store*)))
> +    (download-proc . ,download-to-store*)
> +    (git-download-proc . ,download-git-to-store*)
> +    (commit . #f)))
>  
>  (define (show-help)
>    (display (G_ "Usage: guix download [OPTION] URL
> @@ -100,6 +112,9 @@ and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
>                           do not validate the certificate of HTTPS servers "))
>    (format #t (G_ "
>    -o, --output=FILE      download to FILE"))

This option exists, so it would make sense for

  guix download -c v0.1 example.com/repo -o ~/src/repo

to work as well.

> +  (format #t (G_ "
> +  -c, --git-commit=COMMIT
> +                         download a Git repository"))

Would it be possible to find a way to express this option so it wouldn't
conflict with potential other repositories (SVN, hg, etc)?

Maybe something like

  guix download --git -c v0.1 example.com/repo
  guix download --svn -c v0.1 example.com/repo

But in SVN, "revisions" are used instead of "commits," so perhaps
removing that terminology would be better.  We could do

  guix download --git example.com/repo v0.1
  guix download --svn example.com/repo v0.1

Or! We could use (fake) sub-commands like so:

  guix download git example.com/repo -c v0.1

This feels nice, since it matches e.g. 'guix import go'.  See
guix/import.scm for how this works.

>    (newline)
>    (display (G_ "
>    -h, --help             display this help and exit"))
> @@ -143,6 +158,9 @@ and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
>                                (lambda* (url #:key verify-certificate?)
>                                  (download-to-file url arg))
>                                (alist-delete 'download result))))
> +	(option '(#\c "git-commit") #t #f
> +		(lambda (opt name arg result)
> +		  (alist-cons 'commit arg result)))
>  
>          (option '(#\h "help") #f #f
>                  (lambda args
> @@ -182,16 +200,31 @@ and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
>                        (leave (G_ "~a: failed to parse URI~%")
>                               arg)))
>             (fetch (assq-ref opts 'download-proc))
> +           (git-fetch (assq-ref opts 'git-download-proc))
> +	   (commit (assq-ref opts 'commit))
>             (path  (parameterize ((current-terminal-columns
>                                    (terminal-columns)))
> -                    (fetch (uri->string uri)
> -                           #:verify-certificate?
> -                           (assq-ref opts 'verify-certificate?))))
> -           (hash  (call-with-input-file
> -                      (or path
> -                          (leave (G_ "~a: download failed~%")
> -                                 arg))
> -                    (cute port-hash (assoc-ref opts 'hash-algorithm) <>)))
> +		    (if commit
> +		        (git-fetch (uri->string uri) commit)

You don't actually seem to use the download-git-to-store procedure you
wrote above.  An oversight?

> +                        (fetch (uri->string uri)
> +                               #:verify-certificate?
> +                               (assq-ref opts 'verify-certificate?)))))
> +           (hash  (if (or (assq-ref opts 'recursive) commit)

The 'recursive' option is used here, but not specified in options or the
help message.  I'm also not sure what it's supposed to mean here.
'Recursive' would have no meaning outside of fetching a repo, and for
fetching a repo, 'recursive' should mean that the fetch is recursive,
and would be applied above.

> +                      (let-values (((port get-hash)
> +                                    (open-hash-port
> +				      (assoc-ref opts 'hash-algorithm))))
> +                        (write-file path port
> +				    #:select?
> +				    (if commit
> +				        (lambda (file stat) (not (equal? (basename file) ".git")))
> +					(const #t)))
> +                        (force-output port)
> +                        (get-hash))
> +		      (call-with-input-file
> +                        (or path
> +                            (leave (G_ "~a: download failed~%")
> +                                   arg))
> +                        (cute port-hash (assoc-ref opts 'hash-algorithm) <>))))

Rather than special-casing repositories, perhaps you'd consider
incorporating the first 1-3 patches from [0]?  This would make it easier
to, say, add other repository formats later.  It would make all this
roughly

  (hash (and path
             (file-hash* path
                         #:algorithm
                         (assq-ref opts 'hash-algorithm))))

modulo error reporting, of course.

[0] https://issues.guix.gnu.org/50072

--
Sarah




Information forwarded to guix-patches@HIDDEN:
bug#50274; Package guix-patches. Full text available.

Message received at submit <at> debbugs.gnu.org:


Received: (at submit) by debbugs.gnu.org; 30 Aug 2021 16:40:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 30 12:40:39 2021
Received: from localhost ([127.0.0.1]:60082 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mKkKo-0001qj-TR
	for submit <at> debbugs.gnu.org; Mon, 30 Aug 2021 12:40:39 -0400
Received: from lists.gnu.org ([209.51.188.17]:59272)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <jgart@HIDDEN>) id 1mKkKj-0001qS-O7
 for submit <at> debbugs.gnu.org; Mon, 30 Aug 2021 12:40:33 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:49956)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <jgart@HIDDEN>) id 1mKkKj-0006YC-H0
 for guix-patches@HIDDEN; Mon, 30 Aug 2021 12:40:29 -0400
Received: from mx1.dismail.de ([78.46.223.134]:14469)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <jgart@HIDDEN>) id 1mKkKg-0001QC-RR
 for guix-patches@HIDDEN; Mon, 30 Aug 2021 12:40:29 -0400
Received: from mx1.dismail.de (localhost [127.0.0.1])
 by mx1.dismail.de (OpenSMTPD) with ESMTP id 70d5e8be;
 Mon, 30 Aug 2021 18:40:21 +0200 (CEST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed; d=dismail.de; h=from:to:cc
 :subject:date:message-id:mime-version:content-transfer-encoding;
 s=20190914; bh=uXbsGUGimnq1e+dyokYSl2B2iVQ1xmogC+gY5WbfHC4=; b=
 vqNMeW5tcpvw/Lqp7O6Nk5gBJdqela8Bzg+JR6kD4o4WgtjNuXJZ8WjGBjYtO/Ce
 aSbT2aq+XVdImfj2IkDZeW9ggR03xdGYdKmkk9W9UpXMO6j8Ay0LrEDmRuoIunLN
 TfwoOpAGXuJBCaYorJmkL9ohLzJl8s283ooHcWRtQGkmg4J3DhsH8X20MQ6G98hY
 zZFnpRNUtb+qBGnP0RzvsDV51wDNTJXNAMpSXBa9ZzN4p+mlpRsfwnO4L98zP0l6
 3zvIBGZQK2uHDgaSmbo+XdepImvYXnqvVOV4TvIrYnrgX9OaLiN9bX0kZvtlg1if
 7YemQuttJIAH8p2MOiJrBQ==
Received: from smtp2.dismail.de (<unknown> [10.240.26.12])
 by mx1.dismail.de (OpenSMTPD) with ESMTP id d6fcb696;
 Mon, 30 Aug 2021 18:40:21 +0200 (CEST)
Received: from smtp2.dismail.de (localhost [127.0.0.1])
 by smtp2.dismail.de (OpenSMTPD) with ESMTP id c06078a3;
 Mon, 30 Aug 2021 18:40:21 +0200 (CEST)
Received: by dismail.de (OpenSMTPD) with ESMTPSA id 8335a0ed
 (TLSv1.3:AEAD-AES256-GCM-SHA384:256:NO); 
 Mon, 30 Aug 2021 18:40:20 +0200 (CEST)
From: jgart <jgart@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] guix: git: Adds feature to download git repository to the
 store.
Date: Mon, 30 Aug 2021 12:39:19 -0400
Message-Id: <20210830163918.19419-1-jgart@HIDDEN>
X-Mailer: git-send-email 2.33.0
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=78.46.223.134; envelope-from=jgart@HIDDEN;
 helo=mx1.dismail.de
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
Cc: Julien Lepiller <julien@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

From: Julien Lepiller <julien@HIDDEN>

* guix/git.scm (download-git-to-store): Download Git repository from
URL at COMMIT to STORE, either under NAME or URL's basename if omitted.
Write progress reports to LOG.  RECURSIVE? has the same effect as the
same-named parameter of 'git-fetch'.

* guix/scripts/download.scm (download-git-to-store*): Adds cli option.
Examples:
guix download --git-commit=v0.1.1 github.com/anaseto/gruid-tcell
guix download -c v0.1.1 https://github.com/anaseto/gruid-tcell
---
 guix/git.scm              | 24 +++++++++++++++++-
 guix/scripts/download.scm | 51 ++++++++++++++++++++++++++++++++-------
 2 files changed, 65 insertions(+), 10 deletions(-)

diff --git a/guix/git.scm b/guix/git.scm
index 9c6f326c36..4c70782b97 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -28,6 +28,7 @@
   #:use-module (gcrypt hash)
   #:use-module ((guix build utils)
                 #:select (mkdir-p delete-file-recursively))
+  #:use-module ((guix build git) #:select (git-fetch))
   #:use-module (guix store)
   #:use-module (guix utils)
   #:use-module (guix records)
@@ -43,6 +44,7 @@
   #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-34)
   #:use-module (srfi srfi-35)
+  #:use-module (web uri)
   #:export (%repository-cache-directory
             honor-system-x509-certificates!
 
@@ -61,7 +63,9 @@
             git-checkout-url
             git-checkout-branch
             git-checkout-commit
-            git-checkout-recursive?))
+            git-checkout-recursive?
+
+	    download-git-to-store))
 
 (define %repository-cache-directory
   (make-parameter (string-append (cache-directory #:ensure? #f)
@@ -614,6 +618,24 @@ objects: 'ancestor (meaning that OLD is an ancestor of NEW), 'descendant, or
                                 #:recursive? recursive?
                                 #:log-port (current-error-port)))))
 
+(define* (download-git-to-store store url commit
+				#:optional (name (basename url))
+                                #:key (log (current-error-port)) recursive?)
+  "Download Git repository from URL at COMMIT to STORE, either under NAME or
+URL's basename if omitted.  Write progress reports to LOG.  RECURSIVE? has the
+same effect as the same-named parameter of 'git-fetch'."
+  (define uri
+    (string->uri url))
+
+    (call-with-temporary-directory
+     (lambda (temp)
+       (let ((result
+              (parameterize ((current-output-port log))
+                (git-fetch url commit temp
+        		   #:recursive? recursive?))))
+         (and result
+              (add-to-store store name #t "sha256" temp))))))
+
 ;; Local Variables:
 ;; eval: (put 'with-repository 'scheme-indent-function 2)
 ;; End:
diff --git a/guix/scripts/download.scm b/guix/scripts/download.scm
index 5a91390358..6253ecaa5c 100644
--- a/guix/scripts/download.scm
+++ b/guix/scripts/download.scm
@@ -26,15 +26,19 @@
   #:use-module (guix base32)
   #:autoload   (guix base64) (base64-encode)
   #:use-module ((guix download) #:hide (url-fetch))
+  #:use-module ((guix git) #:select (download-git-to-store))
   #:use-module ((guix build download)
                 #:select (url-fetch))
   #:use-module ((guix progress)
                 #:select (current-terminal-columns))
+  #:use-module ((guix serialization)
+		#:select (write-file))
   #:use-module ((guix build syscalls)
                 #:select (terminal-columns))
   #:use-module (web uri)
   #:use-module (ice-9 match)
   #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-11)
   #:use-module (srfi srfi-14)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-37)
@@ -76,12 +80,20 @@
                        (ensure-valid-store-file-name (basename url))
                        #:verify-certificate? verify-certificate?)))
 
+(define* (download-git-to-store* url commit #:key recursive?)
+  (with-store store
+    (download-git-to-store store url commit
+			   (ensure-valid-store-file-name (basename url))
+			   #:recursive? recursive?)))
+
 (define %default-options
   ;; Alist of default option values.
   `((format . ,bytevector->nix-base32-string)
     (hash-algorithm . ,(hash-algorithm sha256))
     (verify-certificate? . #t)
-    (download-proc . ,download-to-store*)))
+    (download-proc . ,download-to-store*)
+    (git-download-proc . ,download-git-to-store*)
+    (commit . #f)))
 
 (define (show-help)
   (display (G_ "Usage: guix download [OPTION] URL
@@ -100,6 +112,9 @@ and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
                          do not validate the certificate of HTTPS servers "))
   (format #t (G_ "
   -o, --output=FILE      download to FILE"))
+  (format #t (G_ "
+  -c, --git-commit=COMMIT
+                         download a Git repository"))
   (newline)
   (display (G_ "
   -h, --help             display this help and exit"))
@@ -143,6 +158,9 @@ and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
                               (lambda* (url #:key verify-certificate?)
                                 (download-to-file url arg))
                               (alist-delete 'download result))))
+	(option '(#\c "git-commit") #t #f
+		(lambda (opt name arg result)
+		  (alist-cons 'commit arg result)))
 
         (option '(#\h "help") #f #f
                 (lambda args
@@ -182,16 +200,31 @@ and 'base16' ('hex' and 'hexadecimal' can be used as well).\n"))
                       (leave (G_ "~a: failed to parse URI~%")
                              arg)))
            (fetch (assq-ref opts 'download-proc))
+           (git-fetch (assq-ref opts 'git-download-proc))
+	   (commit (assq-ref opts 'commit))
            (path  (parameterize ((current-terminal-columns
                                   (terminal-columns)))
-                    (fetch (uri->string uri)
-                           #:verify-certificate?
-                           (assq-ref opts 'verify-certificate?))))
-           (hash  (call-with-input-file
-                      (or path
-                          (leave (G_ "~a: download failed~%")
-                                 arg))
-                    (cute port-hash (assoc-ref opts 'hash-algorithm) <>)))
+		    (if commit
+		        (git-fetch (uri->string uri) commit)
+                        (fetch (uri->string uri)
+                               #:verify-certificate?
+                               (assq-ref opts 'verify-certificate?)))))
+           (hash  (if (or (assq-ref opts 'recursive) commit)
+                      (let-values (((port get-hash)
+                                    (open-hash-port
+				      (assoc-ref opts 'hash-algorithm))))
+                        (write-file path port
+				    #:select?
+				    (if commit
+				        (lambda (file stat) (not (equal? (basename file) ".git")))
+					(const #t)))
+                        (force-output port)
+                        (get-hash))
+		      (call-with-input-file
+                        (or path
+                            (leave (G_ "~a: download failed~%")
+                                   arg))
+                        (cute port-hash (assoc-ref opts 'hash-algorithm) <>))))
            (fmt   (assq-ref opts 'format)))
       (format #t "~a~%~a~%" path (fmt hash))
       #t)))
-- 
2.33.0





Acknowledgement sent to jgart <jgart@HIDDEN>:
New bug report received and forwarded. Copy sent to guix-patches@HIDDEN. Full text available.
Report forwarded to guix-patches@HIDDEN:
bug#50274; Package guix-patches. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Tue, 31 Aug 2021 19:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.