GNU logs - #50359, boring messages


Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 03 Sep 2021 15:52:02 +0000
Resent-Message-ID: <handler.50359.B.163068428430581 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 50359 <at> debbugs.gnu.org
X-Debbugs-Original-To: guix-patches@HIDDEN
Received: via spool by submit <at> debbugs.gnu.org id=B.163068428430581
          (code B ref -1); Fri, 03 Sep 2021 15:52:02 +0000
Received: (at submit) by debbugs.gnu.org; 3 Sep 2021 15:51:24 +0000
Received: from localhost ([127.0.0.1]:45788 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mMBTP-0007xA-Ar
	for submit <at> debbugs.gnu.org; Fri, 03 Sep 2021 11:51:23 -0400
Received: from lists.gnu.org ([209.51.188.17]:53982)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mMBTN-0007x3-TK
 for submit <at> debbugs.gnu.org; Fri, 03 Sep 2021 11:51:22 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:56522)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <public@HIDDEN>)
 id 1mMBTM-0006Q2-VL
 for guix-patches@HIDDEN; Fri, 03 Sep 2021 11:51:21 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:58080
 helo=mail.yoctocell.xyz)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <public@HIDDEN>)
 id 1mMBTJ-0004iq-L0
 for guix-patches@HIDDEN; Fri, 03 Sep 2021 11:51:20 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1630684266;
 bh=cdtrGN9Y8W0hGMyXnMzGssKIL6kIknZ9jKKMhgeCheg=;
 h=From:To:Subject:Date;
 b=FwV1KD+rP9LxCUJ7+HnDaOh1f8ePrHqNIdQtltOERgyZRm2C2o9vH9D84sO4ClQ6H
 J4NWCChfgbAgLHQCFKrBBYNQ1AjtlVgSMIUeUioc93ezzwQ77P5S7U5AtaX0QvpviM
 QBEz3uekTyR2cezX71GwZRBGeVDi2K7ubxuOgayw=
Message-Id: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Date: Fri, 03 Sep 2021 17:50:56 +0200
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=87.96.130.155; envelope-from=public@HIDDEN;
 helo=mail.yoctocell.xyz
X-Spam_score_int: 30
X-Spam_score: 3.0
X-Spam_bar: +++
X-Spam_report: (3.0 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1,
 DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FROM_SUSPICIOUS_NTLD=0.5,
 FROM_SUSPICIOUS_NTLD_FP=1.598, PDS_OTHER_BAD_TLD=1.999, RDNS_DYNAMIC=0.982,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 1.7 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  * guix/import/git.scm: New file. * doc/guix.texi (Invoking
    guix refresh): Document it. * Makefile.am (MODULES): Register it. --- This
    patch adds a new =?UTF-8?Q?=E2=80=98generic-git=E2=80=99?= updater which can check for new [...] 
 
 Content analysis details:   (1.7 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  1.0 SPF_SOFTFAIL           SPF: sender does not match SPF record (softfail)
 -0.0 SPF_HELO_PASS          SPF: HELO matches SPF record
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
                             [209.51.188.17 listed in wl.mailspike.net]
 -2.3 RCVD_IN_DNSWL_MED      RBL: Sender listed at https://www.dnswl.org/,
                             medium trust
                             [209.51.188.17 listed in list.dnswl.org]
  0.5 FROM_SUSPICIOUS_NTLD_FP From abused NTLD
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.2 (/)

* guix/import/git.scm: New file.
* doc/guix.texi (Invoking guix refresh): Document it.
* Makefile.am (MODULES): Register it.
---
This patch adds a new =E2=80=98generic-git=E2=80=99 updater which can check=
 for new tags
for package hosted on Git repos.  However, it cannot download Git repos
and update the package definitions, i.e. =E2=80=98guix refresh -u=E2=80=99.=
  There is a
pending patch that would add this feature though[1].

=E2=80=98guix refresh -L=E2=80=99 now reports

  Available updaters:
  [=E2=80=A6]
  94.5% of the packages are covered by these updaters.

We are getting close to 100% :-)

See it in action!

--8<---------------cut here---------------start------------->8---
$ ./pre-inst-env guix refresh harmonist scdoc gmnisrv
gnu/packages/web.scm:7931:4: warning: no tags were found for package `gmnis=
rv'
gnu/packages/web.scm:7931:4: warning: 'generic-git' updater failed to deter=
mine available releases for gmnisrv
gnu/packages/man.scm:339:12: scdoc would be upgraded from 1.10.1 to 1.11.1
gnu/packages/games.scm:9433:2: warning: failed to fetch Git repository for =
package `harmonist'
gnu/packages/games.scm:9433:2: warning: 'generic-git' updater failed to det=
ermine available releases for harmonist
--8<---------------cut here---------------end--------------->8---
=20=20
[1]: <https://issues.guix.gnu.org/50072>

 Makefile.am         |   1 +
 doc/guix.texi       |  27 ++++++
 guix/import/git.scm | 223 ++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 251 insertions(+)
 create mode 100644 guix/import/git.scm

diff --git a/Makefile.am b/Makefile.am
index 3c79760734..c4d3a456b1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -254,6 +254,7 @@ MODULES =3D					\
   guix/import/egg.scm   			\
   guix/import/elpa.scm   			\
   guix/import/gem.scm				\
+  guix/import/git.scm                           \
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/gnu.scm				\
diff --git a/doc/guix.texi b/doc/guix.texi
index 36a0c7f5ec..26afb1607a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, Launc=
hpad} packages.
 @item generic-html
 a generic updater that crawls the HTML page where the source tarball of
 the package is hosted, when applicable.
+@item generic-git
+a generic updater for packages hosted on Git repositories.  It tries to
+be smart about parsing Git tag names, but if it is not able to parse the
+tag name and compare tags correctly, users can define the following
+properties for a package.
+
+@itemize
+@item @code{tag-prefix}: a regular expression for matching a prefix of
+the tag name.
+
+@item @code{tag-suffix}: a regular expression for matching a suffix of
+the tag name.
+
+@item @code{tag-version-delimiter}: a string used as the delimiter in
+the tag name for separating the numbers of the version.
+@end itemize
+
+@lisp
+(package
+  (name "foo")
+  ;; ...
+  (properties
+    '((tag-prefix . "^release0-")
+      (tag-suffix . "[a-z]?$")
+      (tag-version-delimiter . ":"))))
+@end lisp=20=20=20=20=20=20
+
 @end table
=20
 For instance, the following command only checks for updates of Emacs
diff --git a/guix/import/git.scm b/guix/import/git.scm
new file mode 100644
index 0000000000..9a654c1972
--- /dev/null
+++ b/guix/import/git.scm
@@ -0,0 +1,223 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import git)
+  #:use-module (git)
+  #:use-module (guix build utils)
+  #:use-module (guix diagnostics)
+  #:use-module (guix git)
+  #:use-module (guix git-download)
+  #:use-module (guix i18n)
+  #:use-module (guix packages)
+  #:use-module (guix upstream)
+  #:use-module (guix utils)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 rdelim)
+  #:use-module (ice-9 regex)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-28)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
+  #:use-module (srfi srfi-71)
+  #:export (%generic-git-updater))
+
+;;; Commentary:
+;;;
+;;; This module provides a generic package updater for packages hosted on =
Git
+;;; repositories.
+;;;
+;;; It tries to be smart about tag names, but if it is not automatically a=
ble
+;;; to parse the tag names correctly, users can set the `tag-prefix',
+;;; `tag-suffix' and `tag-version-delimiter' properties of the package to =
make
+;;; the updater parse the Git tag name correctly.
+;;;
+;;; Code:
+
+;;; Errors & warnings
+
+(define-condition-type &git-tag-error &error
+  git-tag-error?
+  (kind git-tag-error-kind))
+
+(define (git-tag-error kind)
+  (raise (condition (&message (message (format "bad `~a' property")))
+                    (&git-tag-error
+                     (kind kind)))))
+
+(define (git-tag-warning package c)
+  (warning (package-location package)
+           (G_ "~a for package `~a'~%")
+           (condition-message c)
+           (package-name package)))
+
+(define-condition-type &git-no-tags-error &error
+  git-no-tags-error?)
+
+(define (git-no-tags-error)
+  (raise (condition (&message (message "no tags were found"))
+                    (&git-no-tags-error))))
+
+(define (git-no-tags-warning package c)
+  (warning (package-location package)
+           (G_ "~a for package `~a'~%")
+           (condition-message c)
+           (package-name package)))
+
+(define (git-fetch-warning package)
+  (warning (package-location package)
+           (G_ "failed to fetch Git repository for package `~a'~%")
+           (package-name package)))
+
+
+;;; Helper functions
+
+(define (string-split* str delim)
+  "Like `string-split', but DELIM is a string instead of a
+char-set."
+  (filter (lambda (str) (not (equal? str "")))
+          (string-split str (string->char-set delim))))
+
+(define* (get-version package tag #:key prefix suffix delim)
+  (define delim* (if delim delim "."))
+  (define prefix-regexp "^[^0-9]*")
+  (define suffix-regexp (string-append "[^0-9" (regexp-quote delim*) "]*$"=
))
+  (define delim-regexp (string-append "^[0-9]+" (regexp-quote delim*) "[0-=
9]+"))
+
+  (define no-prefix
+    (let ((match (string-match (or prefix prefix-regexp) tag)))
+      (if match
+          (regexp-substitute #f match 'post)
+          (git-tag-error 'tag-prefix))))
+
+  (define no-suffix
+    (let ((match (string-match (or suffix suffix-regexp) no-prefix)))
+      (if match
+          (regexp-substitute #f match 'pre)
+          (git-tag-error 'tag-suffix))))
+
+  (define no-delims
+    (if (string-match delim-regexp no-suffix)
+        (string-split* no-suffix delim*)
+        (git-tag-error 'tag-version-delimiter)))
+
+  (string-join no-delims "."))
+
+(define (sort-tags tags)
+  "Sort TAGS, a list if Git tags, such that the latest tag is the last ele=
ment."
+  (sort tags (lambda (a b)
+               (eq? (version-compare a b) '<))))
+
+
+;;; Updater
+
+(define (get-remote url git-uri)
+  "Given a URL and GIT-URI, a <git-reference> record, return the ``origin'=
' remote."
+  (let* ((checkout (update-cached-checkout url
+                                           #:recursive?
+                                           (git-reference-recursive? git-u=
ri)))
+         (repository (repository-open checkout)))
+    (remote-lookup repository "origin")))
+
+(define (get-latest-tag remote)
+  "Given a Git REMOTE, return that latest tag available."
+  (remote-connect remote)
+
+  (define tags
+    (sort-tags
+     (map (lambda (tag)
+            (string-drop tag (string-length "refs/tags/")))
+          (filter (lambda (ref)
+                    ;; Every tag has two refs:
+                    ;;
+                    ;; * refs/tags/1.2.3^{}
+                    ;; * refs/tags/1.2.3
+                    ;;
+                    ;; remove the one with the trailing ^{}
+                    (and (not (string-suffix? "^{}" ref))
+                         (string-prefix? "refs/tags/" ref)))
+                  (map (lambda (remote-head)
+                         (remote-head-name remote-head))
+                       (remote-ls remote))))))
+
+  (remote-disconnect remote)
+
+  (if (null? tags)
+      (git-no-tags-error)
+      (last tags)))
+
+(define (latest-git-tag-version package tag-prefix tag-suffix
+                                tag-version-delimiter)
+  "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, and TAG-VERSION-DELIMITER
+properties of PACKAGE, returns the latest version of PACKAGE."
+  (guard (c ((eq? (exception-kind c) 'git-error)
+             (git-fetch-warning package)
+             #f)
+            ((git-tag-error? c)
+             (git-tag-warning package c)
+             #f)
+            ((git-no-tags-error? c)
+             (git-no-tags-warning package c)
+             #f))
+    (let* ((source (package-source package))
+           (git-uri (origin-uri source))
+           (url (git-reference-url (origin-uri source)))
+           (remote (get-remote url git-uri))
+           (latest-tag (get-latest-tag remote)))
+      (get-version package
+                   latest-tag
+                   #:prefix tag-prefix
+                   #:suffix tag-suffix
+                   #:delim tag-version-delimiter))))
+
+(define (git-package? package)
+  "Whether the origin of PACKAGE is a Git repostiory."
+  (match (package-source package)
+    ((? origin? origin)
+     (and (eq? (origin-method origin) git-fetch)
+          (git-reference? (origin-uri origin))))
+    (_ #f)))
+
+(define (latest-git-release package)
+  "Return the latest release of PACKAGE."
+  (let* ((name (package-name package))
+         (properties (package-properties package))
+         (tag-prefix (assq-ref properties 'tag-prefix))
+         (tag-suffix (assq-ref properties 'tag-suffix))
+         (tag-version-delimiter (assq-ref properties 'tag-version-delimite=
r))
+         (old-version (package-version package))
+         (url (git-reference-url (origin-uri (package-source package))))
+         (new-version (latest-git-tag-version package
+                                              tag-prefix
+                                              tag-suffix
+                                              tag-version-delimiter)))
+
+    (if new-version
+        (upstream-source
+         (package name)
+         (version new-version)
+         (urls (list url)))
+        ;; No new release or no tags available.
+        #f)))
+
+(define %generic-git-updater
+  (upstream-updater
+   (name 'generic-git)
+   (description "Updater for packages hosted on Git repositories")
+   (pred git-package?)
+   (latest latest-git-release)))

base-commit: 9540323458de87b0b8aa421e449a4fe27af7c393
--=20
2.33.0







Message sent:


Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Xinglu Chen <public@HIDDEN>
Subject: bug#50359: Acknowledgement ([PATCH] import: Add 'generic-git'
 updater.)
Message-ID: <handler.50359.B.163068428430581.ack <at> debbugs.gnu.org>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
X-Gnu-PR-Message: ack 50359
X-Gnu-PR-Package: guix-patches
X-Gnu-PR-Keywords: patch
Reply-To: 50359 <at> debbugs.gnu.org
Date: Fri, 03 Sep 2021 15:52:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 guix-patches@HIDDEN

If you wish to submit further information on this problem, please
send it to 50359 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
50359: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D50359
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems


Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
In-Reply-To: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sun, 05 Sep 2021 00:20:01 +0000
Resent-Message-ID: <handler.50359.B50359.163080119032190 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163080119032190
          (code B ref 50359); Sun, 05 Sep 2021 00:20:01 +0000
Received: (at 50359) by debbugs.gnu.org; 5 Sep 2021 00:19:50 +0000
Received: from localhost ([127.0.0.1]:48868 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mMfsz-0008N7-MW
	for submit <at> debbugs.gnu.org; Sat, 04 Sep 2021 20:19:50 -0400
Received: from out2.migadu.com ([188.165.223.204]:31959)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mMfsw-0008Mx-8W
 for 50359 <at> debbugs.gnu.org; Sat, 04 Sep 2021 20:19:48 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630801184;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 references:references; bh=MKFB+NpBec8yWeoNs7VQUyviuqZy0NVyZz//X2MEXDk=;
 b=N7dMN8FCGRaqZ4MVYnaa0ZkhEz6dwSP6219YhbpdiasHZj7T2amr2GrNyTHWnL6nvYVECP
 o2kMA4ywPhE8kgITkLTq7R0ZKTqQjWS/bj3SvMoPk08s/icXgtpPyG1VHg4VFN0LlDiRxW
 0sGXZAvfPDH35SoIFg1PQIsooxcs+eg=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Date: Sat, 04 Sep 2021 17:19:40 -0700
Message-ID: <86k0jvkh5v.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hello, Thanks for the patch! Glad to see this idea becoming
 more polished. Xinglu Chen <public@HIDDEN> writes: 
 Content analysis details:   (2.0 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [188.165.223.204 listed in wl.mailspike.net]
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hello,

Thanks for the patch!  Glad to see this idea becoming more polished.

Xinglu Chen <public@HIDDEN> writes:

> * guix/import/git.scm: New file.
> * doc/guix.texi (Invoking guix refresh): Document it.
> * Makefile.am (MODULES): Register it.
> ---
> This patch adds a new =E2=80=98generic-git=E2=80=99 updater which can che=
ck for new tags
> for package hosted on Git repos.  However, it cannot download Git repos
> and update the package definitions, i.e. =E2=80=98guix refresh -u=E2=80=
=99.  There is a
> pending patch that would add this feature though[1].
>
> =E2=80=98guix refresh -L=E2=80=99 now reports
>
>   Available updaters:
>   [=E2=80=A6]
>   94.5% of the packages are covered by these updaters.
>
> We are getting close to 100% :-)

Wow, that is close!

>
> See it in action!
>
> $ ./pre-inst-env guix refresh harmonist scdoc gmnisrv
> gnu/packages/web.scm:7931:4: warning: no tags were found for package `gmn=
isrv'
> gnu/packages/web.scm:7931:4: warning: 'generic-git' updater failed to det=
ermine available releases for gmnisrv
> gnu/packages/man.scm:339:12: scdoc would be upgraded from 1.10.1 to 1.11.1
> gnu/packages/games.scm:9433:2: warning: failed to fetch Git repository fo=
r package `harmonist'
> gnu/packages/games.scm:9433:2: warning: 'generic-git' updater failed to d=
etermine available releases for harmonist

FWIW, harmonist and a few other packages fail to work because they use
an old git protocol which is not supported by libgit2.

[...]
> +
> +@itemize
> +@item @code{tag-prefix}: a regular expression for matching a prefix of
> +the tag name.
> +
> +@item @code{tag-suffix}: a regular expression for matching a suffix of
> +the tag name.
> +
> +@item @code{tag-version-delimiter}: a string used as the delimiter in
> +the tag name for separating the numbers of the version.
> +@end itemize
> +
> +@lisp
> +(package
> +  (name "foo")
> +  ;; ...
> +  (properties
> +    '((tag-prefix . "^release0-")
> +      (tag-suffix . "[a-z]?$")
> +      (tag-version-delimiter . ":"))))
> +@end lisp
             ^ extra whitespace

I do like the selection of (prefix, suffix, delimiter), though I think
there are only one or two packages which use a different delimiter.

[...]
> +;;; Errors & warnings
> +
> +(define-condition-type &git-tag-error &error
> +  git-tag-error?
> +  (kind git-tag-error-kind))
> +
> +(define (git-tag-error kind)
> +  (raise (condition (&message (message (format "bad `~a' property")))
> +                    (&git-tag-error
> +                     (kind kind)))))

When I trigger this error, I get:
--8<---------------cut here---------------start------------->8---
In ice-9/exceptions.scm:
   406:15  6 (latest-git-release _)
In ice-9/boot-9.scm:
  1752:10  5 (with-exception-handler _ _ #:unwind? _ # _)
In guix/import/git.scm:
    59:39  4 (get-version _ _ #:prefix _ #:suffix _ #:delim _)
In unknown file:
           3 (simple-format #f "bad `~a' property")
In ice-9/boot-9.scm:
  1685:16  2 (raise-exception _ #:continuable? _)
  1683:16  1 (raise-exception _ #:continuable? _)
  1685:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1685:16: In procedure raise-exception:
In procedure simple-format: FORMAT: Missing argument for ~a
--8<---------------cut here---------------end--------------->8---

> +
> +(define (git-tag-warning package c)
> +  (warning (package-location package)
> +           (G_ "~a for package `~a'~%")
> +           (condition-message c)
> +           (package-name package)))
> +
> +(define-condition-type &git-no-tags-error &error
> +  git-no-tags-error?)
> +
> +(define (git-no-tags-error)
> +  (raise (condition (&message (message "no tags were found"))
> +                    (&git-no-tags-error))))
> +
> +(define (git-no-tags-warning package c)
> +  (warning (package-location package)
> +           (G_ "~a for package `~a'~%")
> +           (condition-message c)
> +           (package-name package)))
> +
> +(define (git-fetch-warning package)
> +  (warning (package-location package)
> +           (G_ "failed to fetch Git repository for package `~a'~%")
> +           (package-name package)))
> +
> +
> +;;; Helper functions
> +
> +(define (string-split* str delim)
> +  "Like `string-split', but DELIM is a string instead of a
> +char-set."
> +  (filter (lambda (str) (not (equal? str "")))
> +          (string-split str (string->char-set delim))))

  (string-split* "1:2.3" ":.") -> ("1" "2" "3")
  (string-split* "1a2b3" "ab") -> ("1" "2" "3")

Is this what you intended?  The documentation above makes it sound like
the whole string serves as the delimiter.

> +
> +(define* (get-version package tag #:key prefix suffix delim)

PACKAGE is not used by this procedure.

> +  (define delim* (if delim delim "."))
> +  (define prefix-regexp "^[^0-9]*")
> +  (define suffix-regexp (string-append "[^0-9" (regexp-quote delim*) "]*=
$"))

With a delimiter of '.', this would say the suffix of '1.2.3.prerelease'
is 'prerelease', not '.prerelease'.  Is this correct?  (I would be
tempted to just remove delim* from this.)

> +  (define delim-regexp (string-append "^[0-9]+" (regexp-quote delim*) "[=
0-9]+"))

This fails to account for versions which use non-numerics, such as (all
taken from the package-version field of packages using git-fetch and
which use this version as the tag):

1.0.0-beta.0
0.0.9.4f
4.4-git.1
5.2.0-alpha
0.2.0-alpha-199-g3e7a475
20200701.154658.b0d6223
12-068oasis4
4.0.0.dev8
0.32-14-gcdfe14e
2.8-fix-2

There are about 50-60 packages like this.

I'm not sure how much effort should be spent including them, and for
some of them I'm not sure what our ideal behavior *is*.  Even if we
could reliably detect them, should "alpha" or "dev" packages be returned
by the updater?

Upon investigation, there is a deeper problem: version-compare thinks
"5.2.0" is a lower version than "5.2.0-alpha", and that "4.0.0" is lower
than "4.0.0.dev8".

scheme@(guile-user)> (version-compare "5.1.9" "5.2.0")
$5 =3D <
scheme@(guile-user)> (version-compare "5.2.0" "5.2.0-alpha")
$6 =3D <
scheme@(guile-user)> (version-compare "4.0.0" "4.0.0.dev8")
$7 =3D <

> +
> +  (define no-prefix
> +    (let ((match (string-match (or prefix prefix-regexp) tag)))
> +      (if match
> +          (regexp-substitute #f match 'post)
> +          (git-tag-error 'tag-prefix))))
> +
> +  (define no-suffix
> +    (let ((match (string-match (or suffix suffix-regexp) no-prefix)))
> +      (if match
> +          (regexp-substitute #f match 'pre)
> +          (git-tag-error 'tag-suffix))))
> +
> +  (define no-delims
> +    (if (string-match delim-regexp no-suffix)
> +        (string-split* no-suffix delim*)
> +        (git-tag-error 'tag-version-delimiter)))

This throws an error if the version doesn't have any delimiter.

Actually, it throws an error in a lot of other cases too, often saying
the 'tag-version-delimiter is wrong when it's something else.  Consider
the tags from the "openjpeg" package, sorted by 'sort-tags':

arelease
opj0-97
start
v2.1.1
v2.1.2
v2.2.0
v2.3.0
v2.3.1
v2.4.0
version.1.0
version.1.1
version.1.2
version.1.3
version.1.4
version.1.5
version.1.5.1
version.1.5.2
version.2.0
version.2.0.1
version.2.1
wg1n6848

At first, 'get-version' throws an error because "wg1n6848" doesn't have
a delimiter. But even disregarding that, it would return "version.2.1"
-> "2.1" as the latest version.

Probably we should process all tags with 'get-version' (simply skipping
any that don't parse) and use that to sort the tags.  If none parse with
'get-version' we could use the "no tags" error or have a separate error
for "there were tags but we couldn't process them".

And this lets us just do something like (untested):

(define* (get-version tag #:key prefix suffix delim)
  (define delim-rx (regexp-quote (or delim ".")))
  (define prefix-rx (or prefix "[^[:digit:]]*"))
  (define suffix-rx (or suffix ".*"))
  (define version-char-rx
   (string-append "[^" delim-rx "[:punct:]]"))

  (define tag-rx
   (string-append "^" prefix "(" version-char-rx "+("
                  delim-rx version-char-rx ")*)" suffix-rx "$"))

  (and=3D> (string-match tag-rx tag)
         (cut match-substring <> 1)))

Though at this point, 'tag-rx' should probably be constructed and
compiled outside the loop.

> +
> +  (string-join no-delims "."))
> +
> +(define (sort-tags tags)
> +  "Sort TAGS, a list if Git tags, such that the latest tag is the last e=
lement."
> +  (sort tags (lambda (a b)
> +               (eq? (version-compare a b) '<))))
> +
> +
> +;;; Updater
> +
> +(define (get-remote url git-uri)
> +  "Given a URL and GIT-URI, a <git-reference> record, return the ``origi=
n'' remote."
> +  (let* ((checkout (update-cached-checkout url
> +                                           #:recursive?
> +                                           (git-reference-recursive? git=
-uri)))
> +         (repository (repository-open checkout)))
> +    (remote-lookup repository "origin")))

We surely don't want 'update-cached-checkout' since that fetches the
whole repo history!  I've attached a patch below (based on top of this
one) which brings the total time-per-package to under 1s.  I moved it to
(guix git) to make use of 'with-libgit2' which ensures we use system
certificates.

Apologies for such a long reply. I hope it was helpful :)

--
Sarah


--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline; filename=0001-git-Add-ls-remote-refs.patch
Content-Description: git: Add 'ls-remote-refs'.

From 0b0973034711e15b52702c0aec0c653dfd41928c Mon Sep 17 00:00:00 2001
Message-Id: <0b0973034711e15b52702c0aec0c653dfd41928c.1630800771.git.iskarian@HIDDEN>
From: Sarah Morgensen <iskarian@HIDDEN>
Date: Fri, 3 Sep 2021 22:40:02 -0700
Subject: [PATCH] git: Add 'ls-remote-refs'.

---
 guix/git.scm        | 33 +++++++++++++++++++++++++++++++
 guix/import/git.scm | 47 ++++++++++-----------------------------------
 2 files changed, 43 insertions(+), 37 deletions(-)

diff --git a/guix/git.scm b/guix/git.scm
index 9c6f326c36..b784fd6d20 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -56,6 +56,8 @@
             commit-difference
             commit-relation
 
+            ls-remote-refs
+
             git-checkout
             git-checkout?
             git-checkout-url
@@ -556,6 +558,37 @@ objects: 'ancestor (meaning that OLD is an ancestor of NEW), 'descendant, or
               (if (set-contains? oldest new)
                   'descendant
                   'unrelated))))))
+
+;;
+;;; Remote operations.
+;;;
+
+(define* (ls-remote-refs url #:key tags?)
+  "Return the list of references advertised at Git repository URL.  If TAGS?
+is true, limit to only refs/tags."
+  (define (ref? ref)
+    ;; Like `git ls-remote --refs', only show actual references.
+    (and (string-prefix? "refs/" ref)
+         (not (string-suffix? "^{}" ref))))
+
+  (define (tag? ref)
+    (string-prefix? "refs/tags/" ref))
+
+  (define (include? ref)
+    (and ref?
+         (or (not tags?) (tag? ref))))
+
+  (with-libgit2
+   (with-temporary-directory
+    (lambda (cache-directory)
+      (let* ((repository (repository-init cache-directory))
+             ;; Create an in-memory remote so we don't touch disk.
+             (remote (remote-create-anonymous repository url)))
+        (remote-connect remote)
+        (remote-disconnect remote)
+        (repository-close! repository)
+
+        (filter include? (map remote-head-name (remote-ls remote))))))))
 
 
 ;;;
diff --git a/guix/import/git.scm b/guix/import/git.scm
index 9a654c1972..097a2f70bc 100644
--- a/guix/import/git.scm
+++ b/guix/import/git.scm
@@ -17,7 +17,6 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (guix import git)
-  #:use-module (git)
   #:use-module (guix build utils)
   #:use-module (guix diagnostics)
   #:use-module (guix git)
@@ -126,40 +125,15 @@ char-set."
 
 ;;; Updater
 
-(define (get-remote url git-uri)
-  "Given a URL and GIT-URI, a <git-reference> record, return the ``origin'' remote."
-  (let* ((checkout (update-cached-checkout url
-                                           #:recursive?
-                                           (git-reference-recursive? git-uri)))
-         (repository (repository-open checkout)))
-    (remote-lookup repository "origin")))
-
-(define (get-latest-tag remote)
-  "Given a Git REMOTE, return that latest tag available."
-  (remote-connect remote)
-
-  (define tags
-    (sort-tags
-     (map (lambda (tag)
-            (string-drop tag (string-length "refs/tags/")))
-          (filter (lambda (ref)
-                    ;; Every tag has two refs:
-                    ;;
-                    ;; * refs/tags/1.2.3^{}
-                    ;; * refs/tags/1.2.3
-                    ;;
-                    ;; remove the one with the trailing ^{}
-                    (and (not (string-suffix? "^{}" ref))
-                         (string-prefix? "refs/tags/" ref)))
-                  (map (lambda (remote-head)
-                         (remote-head-name remote-head))
-                       (remote-ls remote))))))
-
-  (remote-disconnect remote)
-
-  (if (null? tags)
-      (git-no-tags-error)
-      (last tags)))
+(define (get-latest-tag url)
+  "Return the latest tag available from the Git repository at URL."
+  (let ((tags (map (cut string-drop <> (string-length "refs/tags/"))
+                   (ls-remote-refs url #:tags? #t))))
+
+    (if (null? tags)
+        (git-no-tags-error)
+        (last (sort-tags tags)))))
+
 
 (define (latest-git-tag-version package tag-prefix tag-suffix
                                 tag-version-delimiter)
@@ -177,8 +151,7 @@ properties of PACKAGE, returns the latest version of PACKAGE."
     (let* ((source (package-source package))
            (git-uri (origin-uri source))
            (url (git-reference-url (origin-uri source)))
-           (remote (get-remote url git-uri))
-           (latest-tag (get-latest-tag remote)))
+           (latest-tag (get-latest-tag url)))
       (get-version package
                    latest-tag
                    #:prefix tag-prefix

base-commit: 522a3bf99cbc21a9093f63280b9508cd69b94ff0
prerequisite-patch-id: c60e771d96884a78a014e145723562a619c1a0e0
-- 
2.32.0


--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sun, 05 Sep 2021 01:04:02 +0000
Resent-Message-ID: <handler.50359.B50359.16308038053592 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.16308038053592
          (code B ref 50359); Sun, 05 Sep 2021 01:04:02 +0000
Received: (at 50359) by debbugs.gnu.org; 5 Sep 2021 01:03:25 +0000
Received: from localhost ([127.0.0.1]:48883 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mMgZA-0000vs-PL
	for submit <at> debbugs.gnu.org; Sat, 04 Sep 2021 21:03:24 -0400
Received: from out0.migadu.com ([94.23.1.103]:15773)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mMgZ8-0000vi-K4
 for 50359 <at> debbugs.gnu.org; Sat, 04 Sep 2021 21:03:23 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630803800;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=OrhwAzr3CgSgU7rh4QKxREoUhoPYi+iq4m9BIe0do3c=;
 b=hza3sadN31Zz00TLXUwkLSPglhxSDEHcFm7pAfs1P83bhKq+dnktAeoZyaectVAkGLCiUb
 arcl8uWSsuCuK944DvPOW7MmSlLIyKL6zxL911NocTj5e70+l9/ayq5yBg7QCPn09PGSYd
 XX/Z2hejPFiEL+493BWbebsqbvtXavE=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN>
Date: Sat, 04 Sep 2021 18:03:18 -0700
In-Reply-To: <86k0jvkh5v.fsf@HIDDEN> (Sarah Morgensen's message of "Sat, 04
 Sep 2021 17:19:40 -0700 (40 minutes, 11 seconds ago)")
Message-ID: <86czpnkf55.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)


Apologies, in my patch, 'with-temporary-directory' should be
'call-with-temporary-directory'...

Sarah Morgensen <iskarian@HIDDEN> writes:

> +(define* (ls-remote-refs url #:key tags?)
> +  "Return the list of references advertised at Git repository URL.  If TAGS?
> +is true, limit to only refs/tags."
> +  (define (ref? ref)
> +    ;; Like `git ls-remote --refs', only show actual references.
> +    (and (string-prefix? "refs/" ref)
> +         (not (string-suffix? "^{}" ref))))
> +
> +  (define (tag? ref)
> +    (string-prefix? "refs/tags/" ref))
> +
> +  (define (include? ref)
> +    (and ref?
> +         (or (not tags?) (tag? ref))))
> +
> +  (with-libgit2
> +   (with-temporary-directory

...right here.

--
Sarah




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sun, 05 Sep 2021 10:37:01 +0000
Resent-Message-ID: <handler.50359.B50359.163083819912595 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Sarah Morgensen <iskarian@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163083819912595
          (code B ref 50359); Sun, 05 Sep 2021 10:37:01 +0000
Received: (at 50359) by debbugs.gnu.org; 5 Sep 2021 10:36:39 +0000
Received: from localhost ([127.0.0.1]:49368 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mMpVu-0003H3-BC
	for submit <at> debbugs.gnu.org; Sun, 05 Sep 2021 06:36:38 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:34286
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mMpVs-0003Gj-Lc
 for 50359 <at> debbugs.gnu.org; Sun, 05 Sep 2021 06:36:37 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1630838188;
 bh=eXYEC//qZ2lxCKHOpD/Aij4T6gIZE8/Mcyd/QwTDA68=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=uT2UZp/QQBJ+dxk7Tj9GLEfHRF0msM/uZuZkEYo+fpvPRkSJ169xnebdhf9b529qP
 5+UXfYRRSEcGFNtPYYH1R9NkTqK5x9b8ecuiRqgB6MFA51O9VkZjv9/KRXRHzIovr4
 rpdXDdtdrDsuGO56O5fH9sv8J8k/GNWr4a0Bv1II=
In-Reply-To: <86k0jvkh5v.fsf@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN>
Date: Sun, 05 Sep 2021 12:36:24 +0200
Message-ID: <87h7ez48d3.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  On Sat, Sep 04 2021, Sarah Morgensen wrote: > Hello,
 > > Thanks
 for the patch! Glad to see this idea becoming more polished. > > Xinglu Chen
 <public@HIDDEN> writes: > >> * guix/import/git.scm: New file. >> *
 doc/guix.texi (Invoking guix [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Sat, Sep 04 2021, Sarah Morgensen wrote: > Hello, > > Thanks
    for the patch! Glad to see this idea becoming more polished. > > Xinglu Chen
    <public@HIDDEN> writes: > >> * guix/import/git.scm: New file. >> *
    doc/guix.texi (Invoking guix [...] 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Sat, Sep 04 2021, Sarah Morgensen wrote:

> Hello,
>
> Thanks for the patch!  Glad to see this idea becoming more polished.
>
> Xinglu Chen <public@HIDDEN> writes:
>
>> * guix/import/git.scm: New file.
>> * doc/guix.texi (Invoking guix refresh): Document it.
>> * Makefile.am (MODULES): Register it.
>> ---
>> This patch adds a new =E2=80=98generic-git=E2=80=99 updater which can ch=
eck for new tags
>> for package hosted on Git repos.  However, it cannot download Git repos
>> and update the package definitions, i.e. =E2=80=98guix refresh -u=E2=80=
=99.  There is a
>> pending patch that would add this feature though[1].
>>
>> =E2=80=98guix refresh -L=E2=80=99 now reports
>>
>>   Available updaters:
>>   [=E2=80=A6]
>>   94.5% of the packages are covered by these updaters.
>>
>> We are getting close to 100% :-)
>
> Wow, that is close!
>
>>
>> See it in action!
>>
>> $ ./pre-inst-env guix refresh harmonist scdoc gmnisrv
>> gnu/packages/web.scm:7931:4: warning: no tags were found for package `gm=
nisrv'
>> gnu/packages/web.scm:7931:4: warning: 'generic-git' updater failed to de=
termine available releases for gmnisrv
>> gnu/packages/man.scm:339:12: scdoc would be upgraded from 1.10.1 to 1.11=
.1
>> gnu/packages/games.scm:9433:2: warning: failed to fetch Git repository f=
or package `harmonist'
>> gnu/packages/games.scm:9433:2: warning: 'generic-git' updater failed to =
determine available releases for harmonist
>
> FWIW, harmonist and a few other packages fail to work because they use
> an old git protocol which is not supported by libgit2.
>
> [...]
>> +
>> +@itemize
>> +@item @code{tag-prefix}: a regular expression for matching a prefix of
>> +the tag name.
>> +
>> +@item @code{tag-suffix}: a regular expression for matching a suffix of
>> +the tag name.
>> +
>> +@item @code{tag-version-delimiter}: a string used as the delimiter in
>> +the tag name for separating the numbers of the version.
>> +@end itemize
>> +
>> +@lisp
>> +(package
>> +  (name "foo")
>> +  ;; ...
>> +  (properties
>> +    '((tag-prefix . "^release0-")
>> +      (tag-suffix . "[a-z]?$")
>> +      (tag-version-delimiter . ":"))))
>> +@end lisp
>              ^ extra whitespace
>
> I do like the selection of (prefix, suffix, delimiter), though I think
> there are only one or two packages which use a different delimiter.
>
> [...]
>> +;;; Errors & warnings
>> +
>> +(define-condition-type &git-tag-error &error
>> +  git-tag-error?
>> +  (kind git-tag-error-kind))
>> +
>> +(define (git-tag-error kind)
>> +  (raise (condition (&message (message (format "bad `~a' property")))
>> +                    (&git-tag-error
>> +                     (kind kind)))))
>
> When I trigger this error, I get:
> --8<---------------cut here---------------start------------->8---
> In ice-9/exceptions.scm:
>    406:15  6 (latest-git-release _)
> In ice-9/boot-9.scm:
>   1752:10  5 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/import/git.scm:
>     59:39  4 (get-version _ _ #:prefix _ #:suffix _ #:delim _)
> In unknown file:
>            3 (simple-format #f "bad `~a' property")
> In ice-9/boot-9.scm:
>   1685:16  2 (raise-exception _ #:continuable? _)
>   1683:16  1 (raise-exception _ #:continuable? _)
>   1685:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> In procedure simple-format: FORMAT: Missing argument for ~a
> --8<---------------cut here---------------end--------------->8---

Oops, it should be

  (format "bad `~a' property" kind)
=20=20
>> +
>> +(define (git-tag-warning package c)
>> +  (warning (package-location package)
>> +           (G_ "~a for package `~a'~%")
>> +           (condition-message c)
>> +           (package-name package)))
>> +
>> +(define-condition-type &git-no-tags-error &error
>> +  git-no-tags-error?)
>> +
>> +(define (git-no-tags-error)
>> +  (raise (condition (&message (message "no tags were found"))
>> +                    (&git-no-tags-error))))
>> +
>> +(define (git-no-tags-warning package c)
>> +  (warning (package-location package)
>> +           (G_ "~a for package `~a'~%")
>> +           (condition-message c)
>> +           (package-name package)))
>> +
>> +(define (git-fetch-warning package)
>> +  (warning (package-location package)
>> +           (G_ "failed to fetch Git repository for package `~a'~%")
>> +           (package-name package)))
>> +
>> +
>> +;;; Helper functions
>> +
>> +(define (string-split* str delim)
>> +  "Like `string-split', but DELIM is a string instead of a
>> +char-set."
>> +  (filter (lambda (str) (not (equal? str "")))
>> +          (string-split str (string->char-set delim))))
>
>   (string-split* "1:2.3" ":.") -> ("1" "2" "3")
>   (string-split* "1a2b3" "ab") -> ("1" "2" "3")
>
> Is this what you intended?  The documentation above makes it sound like
> the whole string serves as the delimiter.

It=E2=80=99s not what I wanted, indeed.  I will try to fix it.

>> +
>> +(define* (get-version package tag #:key prefix suffix delim)
>
> PACKAGE is not used by this procedure.

Good catch, it was some leftover I forgot to remove.

>> +  (define delim* (if delim delim "."))
>> +  (define prefix-regexp "^[^0-9]*")
>> +  (define suffix-regexp (string-append "[^0-9" (regexp-quote delim*) "]=
*$"))
>
> With a delimiter of '.', this would say the suffix of '1.2.3.prerelease'
> is 'prerelease', not '.prerelease'.  Is this correct?  (I would be
> tempted to just remove delim* from this.)

Good point, I think removing =E2=80=98delim*=E2=80=99 would be a good idea.

>> +  (define delim-regexp (string-append "^[0-9]+" (regexp-quote delim*) "=
[0-9]+"))
>
> This fails to account for versions which use non-numerics, such as (all
> taken from the package-version field of packages using git-fetch and
> which use this version as the tag):
>
> 1.0.0-beta.0
> 0.0.9.4f
> 4.4-git.1
> 5.2.0-alpha
> 0.2.0-alpha-199-g3e7a475
> 20200701.154658.b0d6223
> 12-068oasis4
> 4.0.0.dev8
> 0.32-14-gcdfe14e
> 2.8-fix-2
>
> There are about 50-60 packages like this.
>
> I'm not sure how much effort should be spent including them, and for
> some of them I'm not sure what our ideal behavior *is*.  Even if we
> could reliably detect them, should "alpha" or "dev" packages be returned
> by the updater?

I don=E2=80=99t think we usually include alpha or rc releases, so updater
probably shouldn=E2=80=99t return them either.  Not sure how we would try to
detect alpha/beta/rc releases, though, besides running something like

  (string-match? "(alpha|beta|rc|dev)" TAG)

On each tag.

> Upon investigation, there is a deeper problem: version-compare thinks
> "5.2.0" is a lower version than "5.2.0-alpha", and that "4.0.0" is lower
> than "4.0.0.dev8".
>
> scheme@(guile-user)> (version-compare "5.1.9" "5.2.0")
> $5 =3D <
> scheme@(guile-user)> (version-compare "5.2.0" "5.2.0-alpha")
> $6 =3D <
> scheme@(guile-user)> (version-compare "4.0.0" "4.0.0.dev8")
> $7 =3D <

Maybe we should filter the tags before comparing them; that should
get rid of these pre-release tags.

>> +
>> +  (define no-prefix
>> +    (let ((match (string-match (or prefix prefix-regexp) tag)))
>> +      (if match
>> +          (regexp-substitute #f match 'post)
>> +          (git-tag-error 'tag-prefix))))
>> +
>> +  (define no-suffix
>> +    (let ((match (string-match (or suffix suffix-regexp) no-prefix)))
>> +      (if match
>> +          (regexp-substitute #f match 'pre)
>> +          (git-tag-error 'tag-suffix))))
>> +
>> +  (define no-delims
>> +    (if (string-match delim-regexp no-suffix)
>> +        (string-split* no-suffix delim*)
>> +        (git-tag-error 'tag-version-delimiter)))
>
> This throws an error if the version doesn't have any delimiter.

Setting the =E2=80=98tag-version-delimiter=E2=80=99 prefix to an empty stri=
ng would
solve this, right?  Or, maybe we should just get rid of the delimiter
thing since only a few packages use a different delimiter.

> Actually, it throws an error in a lot of other cases too, often saying
> the 'tag-version-delimiter is wrong when it's something else.  Consider
> the tags from the "openjpeg" package, sorted by 'sort-tags':
>
> arelease
> opj0-97
> start
> v2.1.1
> v2.1.2
> v2.2.0
> v2.3.0
> v2.3.1
> v2.4.0
> version.1.0
> version.1.1
> version.1.2
> version.1.3
> version.1.4
> version.1.5
> version.1.5.1
> version.1.5.2
> version.2.0
> version.2.0.1
> version.2.1
> wg1n6848
>
> At first, 'get-version' throws an error because "wg1n6848" doesn't have
> a delimiter. But even disregarding that, it would return "version.2.1"
> -> "2.1" as the latest version.
>
> Probably we should process all tags with 'get-version' (simply skipping
> any that don't parse) and use that to sort the tags.  If none parse with
> 'get-version' we could use the "no tags" error or have a separate error
> for "there were tags but we couldn't process them".

Ah, yes, that would be a good idea.

> And this lets us just do something like (untested):
>
> (define* (get-version tag #:key prefix suffix delim)
>   (define delim-rx (regexp-quote (or delim ".")))
>   (define prefix-rx (or prefix "[^[:digit:]]*"))
>   (define suffix-rx (or suffix ".*"))
>   (define version-char-rx
>    (string-append "[^" delim-rx "[:punct:]]"))
>
>   (define tag-rx
>    (string-append "^" prefix "(" version-char-rx "+("
>                   delim-rx version-char-rx ")*)" suffix-rx "$"))
>
>   (and=3D> (string-match tag-rx tag)
>          (cut match-substring <> 1)))
>
> Though at this point, 'tag-rx' should probably be constructed and
> compiled outside the loop.
>
>> +
>> +  (string-join no-delims "."))
>> +
>> +(define (sort-tags tags)
>> +  "Sort TAGS, a list if Git tags, such that the latest tag is the last =
element."
>> +  (sort tags (lambda (a b)
>> +               (eq? (version-compare a b) '<))))
>> +
>> +
>> +;;; Updater
>> +
>> +(define (get-remote url git-uri)
>> +  "Given a URL and GIT-URI, a <git-reference> record, return the ``orig=
in'' remote."
>> +  (let* ((checkout (update-cached-checkout url
>> +                                           #:recursive?
>> +                                           (git-reference-recursive? gi=
t-uri)))
>> +         (repository (repository-open checkout)))
>> +    (remote-lookup repository "origin")))
>
> We surely don't want 'update-cached-checkout' since that fetches the
> whole repo history!  I've attached a patch below (based on top of this
> one) which brings the total time-per-package to under 1s.  I moved it to
> (guix git) to make use of 'with-libgit2' which ensures we use system
> certificates.
>
> Apologies for such a long reply. I hope it was helpful :)

No worries, it definitely helped a lot, thank you!

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE0nagVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5vCAQAI0BtYCXoZojo9O8Y4O128gOfCvo
ETQvtPRNMILtZ8mkv+6aY0gnB1foGIEUDWb7pdZs3IBY6Z7H9yAM/pIaWnlfldum
6x/3m6ESO1GVyEFsSZxZ4pPa1xeQ3+qIRrKqIPN8fpmyttxzEDR54djq8yBARlCR
RhSjEf+r65nsUZqlOBRfWCDp/Gx7oreb6+RiFo7ZYeBsLESlcLt8SmUbJSXzw0b5
gvigA232i3povwdbYdfGiX3np9K6xhaXeXcGO5pPike5jlJlpLoSYmBP0aqhHOqb
TgqUvmCoxHcnMc8ytEVR02CdSix3RBhP8z7+fbyb1emTGVDiLCGfHCUn8L1jg2pF
6zsxKEGbLshNHkkolSIlm1YuHELiPqo8s9aNCtPFYyfcpn5iNjjx/P6B0Ywb1Gm6
uFiZ4f4PT3+slS07WWwZz+AbJnts+NFuhP26TOSjDym/WzM2XNOYAxlmcjoEDriz
+H8RFOmt6p8nnE2oBxLZkRq1+6jQBJHrCJc6+ZrJm3N3ZlFa0qdbRMwEeSkHrj7Y
nJgp+ZdIBqm2zbvw/xyquzytysnCuu2mBlQBPuWlRzoa9HYTa4FW6YEa7lXX8FDf
75tgRA+A2o/Cjv19zn6XpJjtIYW1Rdg7NhJqWmqUoPpMMf9BqNw5vcXztXSM2/f6
A05zw5m5IumjEdiy
=ZAEz
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sun, 05 Sep 2021 13:12:02 +0000
Resent-Message-ID: <handler.50359.B50359.16308474922751 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Sarah Morgensen <iskarian@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.16308474922751
          (code B ref 50359); Sun, 05 Sep 2021 13:12:02 +0000
Received: (at 50359) by debbugs.gnu.org; 5 Sep 2021 13:11:32 +0000
Received: from localhost ([127.0.0.1]:49429 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mMrvm-0000hQ-VS
	for submit <at> debbugs.gnu.org; Sun, 05 Sep 2021 09:11:32 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:37500
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mMrvk-0000h8-4q
 for 50359 <at> debbugs.gnu.org; Sun, 05 Sep 2021 09:11:30 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1630847480;
 bh=symRydbRMAn9xIMs0ijSQ604qu2+5ZzJH8i12xOxy80=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=OExsn8Dfdto3EQ8A6pj+ET1SXwFNisN2wE0oHoxKzKMS3bzhC2e4Llu6C6J7+S7aY
 HIIQUuQsIF0r1RhfdGVP99bwJBdDUo7TCRHSRCHD69qaNrXkhGY9vpvi8UREGTU5f3
 3Q3ogLDRRVn8stXY5FTbeBVckGsQ6PE7trg9My3k=
In-Reply-To: <86k0jvkh5v.fsf@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN>
Date: Sun, 05 Sep 2021 15:11:18 +0200
Message-ID: <87eea3416x.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Some more comments after some testing On Sat, Sep 04 2021,
 Sarah Morgensen wrote: > Hello, > > Thanks for the patch! Glad to see this
 idea becoming more polished. > > Xinglu Chen <public@HIDDEN> writes:
 > >> * guix/import/git.scm: New file. >> * doc/guix.texi (Invoking guix [...]
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  Some more comments after some testing On Sat, Sep 04 2021,
    Sarah Morgensen wrote: > Hello, > > Thanks for the patch! Glad to see this
    idea becoming more polished. > > Xinglu Chen <public@HIDDEN> writes:
    > >> * guix/import/git.scm: New file. >> * doc/guix.texi (Invoking guix [...]
    
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Some more comments after some testing

On Sat, Sep 04 2021, Sarah Morgensen wrote:

> Hello,
>
> Thanks for the patch!  Glad to see this idea becoming more polished.
>
> Xinglu Chen <public@HIDDEN> writes:
>
>> * guix/import/git.scm: New file.
>> * doc/guix.texi (Invoking guix refresh): Document it.
>> * Makefile.am (MODULES): Register it.
>> ---
>> This patch adds a new =E2=80=98generic-git=E2=80=99 updater which can ch=
eck for new tags
>> for package hosted on Git repos.  However, it cannot download Git repos
>> and update the package definitions, i.e. =E2=80=98guix refresh -u=E2=80=
=99.  There is a
>> pending patch that would add this feature though[1].
>>
>> =E2=80=98guix refresh -L=E2=80=99 now reports
>>
>>   Available updaters:
>>   [=E2=80=A6]
>>   94.5% of the packages are covered by these updaters.
>>
>> We are getting close to 100% :-)
>
> Wow, that is close!
>
>>
>> See it in action!
>>
>> $ ./pre-inst-env guix refresh harmonist scdoc gmnisrv
>> gnu/packages/web.scm:7931:4: warning: no tags were found for package `gm=
nisrv'
>> gnu/packages/web.scm:7931:4: warning: 'generic-git' updater failed to de=
termine available releases for gmnisrv
>> gnu/packages/man.scm:339:12: scdoc would be upgraded from 1.10.1 to 1.11=
.1
>> gnu/packages/games.scm:9433:2: warning: failed to fetch Git repository f=
or package `harmonist'
>> gnu/packages/games.scm:9433:2: warning: 'generic-git' updater failed to =
determine available releases for harmonist
>
> FWIW, harmonist and a few other packages fail to work because they use
> an old git protocol which is not supported by libgit2.
>
> [...]
>> +
>> +@itemize
>> +@item @code{tag-prefix}: a regular expression for matching a prefix of
>> +the tag name.
>> +
>> +@item @code{tag-suffix}: a regular expression for matching a suffix of
>> +the tag name.
>> +
>> +@item @code{tag-version-delimiter}: a string used as the delimiter in
>> +the tag name for separating the numbers of the version.
>> +@end itemize
>> +
>> +@lisp
>> +(package
>> +  (name "foo")
>> +  ;; ...
>> +  (properties
>> +    '((tag-prefix . "^release0-")
>> +      (tag-suffix . "[a-z]?$")
>> +      (tag-version-delimiter . ":"))))
>> +@end lisp
>              ^ extra whitespace
>
> I do like the selection of (prefix, suffix, delimiter), though I think
> there are only one or two packages which use a different delimiter.
>
> [...]
>> +;;; Errors & warnings
>> +
>> +(define-condition-type &git-tag-error &error
>> +  git-tag-error?
>> +  (kind git-tag-error-kind))
>> +
>> +(define (git-tag-error kind)
>> +  (raise (condition (&message (message (format "bad `~a' property")))
>> +                    (&git-tag-error
>> +                     (kind kind)))))
>
> When I trigger this error, I get:
> --8<---------------cut here---------------start------------->8---
> In ice-9/exceptions.scm:
>    406:15  6 (latest-git-release _)
> In ice-9/boot-9.scm:
>   1752:10  5 (with-exception-handler _ _ #:unwind? _ # _)
> In guix/import/git.scm:
>     59:39  4 (get-version _ _ #:prefix _ #:suffix _ #:delim _)
> In unknown file:
>            3 (simple-format #f "bad `~a' property")
> In ice-9/boot-9.scm:
>   1685:16  2 (raise-exception _ #:continuable? _)
>   1683:16  1 (raise-exception _ #:continuable? _)
>   1685:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1685:16: In procedure raise-exception:
> In procedure simple-format: FORMAT: Missing argument for ~a
> --8<---------------cut here---------------end--------------->8---
>
>> +
>> +(define (git-tag-warning package c)
>> +  (warning (package-location package)
>> +           (G_ "~a for package `~a'~%")
>> +           (condition-message c)
>> +           (package-name package)))
>> +
>> +(define-condition-type &git-no-tags-error &error
>> +  git-no-tags-error?)
>> +
>> +(define (git-no-tags-error)
>> +  (raise (condition (&message (message "no tags were found"))
>> +                    (&git-no-tags-error))))
>> +
>> +(define (git-no-tags-warning package c)
>> +  (warning (package-location package)
>> +           (G_ "~a for package `~a'~%")
>> +           (condition-message c)
>> +           (package-name package)))
>> +
>> +(define (git-fetch-warning package)
>> +  (warning (package-location package)
>> +           (G_ "failed to fetch Git repository for package `~a'~%")
>> +           (package-name package)))
>> +
>> +
>> +;;; Helper functions
>> +
>> +(define (string-split* str delim)
>> +  "Like `string-split', but DELIM is a string instead of a
>> +char-set."
>> +  (filter (lambda (str) (not (equal? str "")))
>> +          (string-split str (string->char-set delim))))
>
>   (string-split* "1:2.3" ":.") -> ("1" "2" "3")
>   (string-split* "1a2b3" "ab") -> ("1" "2" "3")
>
> Is this what you intended?  The documentation above makes it sound like
> the whole string serves as the delimiter.
>
>> +
>> +(define* (get-version package tag #:key prefix suffix delim)
>
> PACKAGE is not used by this procedure.
>
>> +  (define delim* (if delim delim "."))
>> +  (define prefix-regexp "^[^0-9]*")
>> +  (define suffix-regexp (string-append "[^0-9" (regexp-quote delim*) "]=
*$"))
>
> With a delimiter of '.', this would say the suffix of '1.2.3.prerelease'
> is 'prerelease', not '.prerelease'.  Is this correct?  (I would be
> tempted to just remove delim* from this.)
>
>> +  (define delim-regexp (string-append "^[0-9]+" (regexp-quote delim*) "=
[0-9]+"))
>
> This fails to account for versions which use non-numerics, such as (all
> taken from the package-version field of packages using git-fetch and
> which use this version as the tag):
>
> 1.0.0-beta.0
> 0.0.9.4f
> 4.4-git.1
> 5.2.0-alpha
> 0.2.0-alpha-199-g3e7a475
> 20200701.154658.b0d6223
> 12-068oasis4
> 4.0.0.dev8
> 0.32-14-gcdfe14e
> 2.8-fix-2
>
> There are about 50-60 packages like this.
>
> I'm not sure how much effort should be spent including them, and for
> some of them I'm not sure what our ideal behavior *is*.  Even if we
> could reliably detect them, should "alpha" or "dev" packages be returned
> by the updater?
>
> Upon investigation, there is a deeper problem: version-compare thinks
> "5.2.0" is a lower version than "5.2.0-alpha", and that "4.0.0" is lower
> than "4.0.0.dev8".
>
> scheme@(guile-user)> (version-compare "5.1.9" "5.2.0")
> $5 =3D <
> scheme@(guile-user)> (version-compare "5.2.0" "5.2.0-alpha")
> $6 =3D <
> scheme@(guile-user)> (version-compare "4.0.0" "4.0.0.dev8")
> $7 =3D <
>
>> +
>> +  (define no-prefix
>> +    (let ((match (string-match (or prefix prefix-regexp) tag)))
>> +      (if match
>> +          (regexp-substitute #f match 'post)
>> +          (git-tag-error 'tag-prefix))))
>> +
>> +  (define no-suffix
>> +    (let ((match (string-match (or suffix suffix-regexp) no-prefix)))
>> +      (if match
>> +          (regexp-substitute #f match 'pre)
>> +          (git-tag-error 'tag-suffix))))
>> +
>> +  (define no-delims
>> +    (if (string-match delim-regexp no-suffix)
>> +        (string-split* no-suffix delim*)
>> +        (git-tag-error 'tag-version-delimiter)))
>
> This throws an error if the version doesn't have any delimiter.
>
> Actually, it throws an error in a lot of other cases too, often saying
> the 'tag-version-delimiter is wrong when it's something else.  Consider
> the tags from the "openjpeg" package, sorted by 'sort-tags':
>
> arelease
> opj0-97
> start
> v2.1.1
> v2.1.2
> v2.2.0
> v2.3.0
> v2.3.1
> v2.4.0
> version.1.0
> version.1.1
> version.1.2
> version.1.3
> version.1.4
> version.1.5
> version.1.5.1
> version.1.5.2
> version.2.0
> version.2.0.1
> version.2.1
> wg1n6848
>
> At first, 'get-version' throws an error because "wg1n6848" doesn't have
> a delimiter. But even disregarding that, it would return "version.2.1"
> -> "2.1" as the latest version.
>
> Probably we should process all tags with 'get-version' (simply skipping
> any that don't parse) and use that to sort the tags.  If none parse with
> 'get-version' we could use the "no tags" error or have a separate error
> for "there were tags but we couldn't process them".
>
> And this lets us just do something like (untested):
>
> (define* (get-version tag #:key prefix suffix delim)
>   (define delim-rx (regexp-quote (or delim ".")))
>   (define prefix-rx (or prefix "[^[:digit:]]*"))
>   (define suffix-rx (or suffix ".*"))
>   (define version-char-rx
>    (string-append "[^" delim-rx "[:punct:]]"))
>
>   (define tag-rx
>    (string-append "^" prefix "(" version-char-rx "+("
>                   delim-rx version-char-rx ")*)" suffix-rx "$"))

This wouldn=E2=80=99t match anything if the version is just a plain number,
e.g., 1 or 09.

>
>   (and=3D> (string-match tag-rx tag)
>          (cut match-substring <> 1)))

With this, something like =E2=80=9C1.4.0rc1-450-g2725ef99d=E2=80=9D will re=
sult in
=E2=80=9C1.4.0=E2=80=9D being returned, which is incorrect.  Changing (cut
match:substring <> 1) to just =E2=80=98match:substring=E2=80=99 would solve=
 the issue,
but then pre-release tags, which we usually don=E2=80=99t want,  would also=
 get
matched.  Not sure what the best option would be in this case.

> Though at this point, 'tag-rx' should probably be constructed and
> compiled outside the loop.
>
>> +
>> +  (string-join no-delims "."))
>> +
>> +(define (sort-tags tags)
>> +  "Sort TAGS, a list if Git tags, such that the latest tag is the last =
element."
>> +  (sort tags (lambda (a b)
>> +               (eq? (version-compare a b) '<))))
>> +
>> +
>> +;;; Updater
>> +
>> +(define (get-remote url git-uri)
>> +  "Given a URL and GIT-URI, a <git-reference> record, return the ``orig=
in'' remote."
>> +  (let* ((checkout (update-cached-checkout url
>> +                                           #:recursive?
>> +                                           (git-reference-recursive? gi=
t-uri)))
>> +         (repository (repository-open checkout)))
>> +    (remote-lookup repository "origin")))
>
> We surely don't want 'update-cached-checkout' since that fetches the
> whole repo history!  I've attached a patch below (based on top of this
> one) which brings the total time-per-package to under 1s.  I moved it to
> (guix git) to make use of 'with-libgit2' which ensures we use system
> certificates.
>
> Apologies for such a long reply. I hope it was helpful :)
>
> --
> Sarah
>
> From 0b0973034711e15b52702c0aec0c653dfd41928c Mon Sep 17 00:00:00 2001
> Message-Id: <0b0973034711e15b52702c0aec0c653dfd41928c.1630800771.git.iska=
rian@HIDDEN>
> From: Sarah Morgensen <iskarian@HIDDEN>
> Date: Fri, 3 Sep 2021 22:40:02 -0700
> Subject: [PATCH] git: Add 'ls-remote-refs'.
>
> ---
>  guix/git.scm        | 33 +++++++++++++++++++++++++++++++
>  guix/import/git.scm | 47 ++++++++++-----------------------------------
>  2 files changed, 43 insertions(+), 37 deletions(-)
>
> diff --git a/guix/git.scm b/guix/git.scm
> index 9c6f326c36..b784fd6d20 100644
> --- a/guix/git.scm
> +++ b/guix/git.scm
> @@ -56,6 +56,8 @@
>              commit-difference
>              commit-relation
>=20=20
> +            ls-remote-refs
> +
>              git-checkout
>              git-checkout?
>              git-checkout-url
> @@ -556,6 +558,37 @@ objects: 'ancestor (meaning that OLD is an ancestor =
of NEW), 'descendant, or
>                (if (set-contains? oldest new)
>                    'descendant
>                    'unrelated))))))
> +
> +;;
> +;;; Remote operations.
> +;;;
> +
> +(define* (ls-remote-refs url #:key tags?)
> +  "Return the list of references advertised at Git repository URL.  If T=
AGS?
> +is true, limit to only refs/tags."
> +  (define (ref? ref)
> +    ;; Like `git ls-remote --refs', only show actual references.
> +    (and (string-prefix? "refs/" ref)
> +         (not (string-suffix? "^{}" ref))))
> +
> +  (define (tag? ref)
> +    (string-prefix? "refs/tags/" ref))
> +
> +  (define (include? ref)
> +    (and ref?
> +         (or (not tags?) (tag? ref))))
> +
> +  (with-libgit2
> +   (with-temporary-directory
> +    (lambda (cache-directory)
> +      (let* ((repository (repository-init cache-directory))
> +             ;; Create an in-memory remote so we don't touch disk.
> +             (remote (remote-create-anonymous repository url)))
> +        (remote-connect remote)
> +        (remote-disconnect remote)
> +        (repository-close! repository)
> +
> +        (filter include? (map remote-head-name (remote-ls remote))))))))
>

For some reason it seems to include refs that do and don=E2=80=99t end with
=E2=80=9C^{}=E2=80=9D

=2D-8<---------------cut here---------------start------------->8---
scheme@(guile-user)> (ls-remote-refs "https://github.com/clementine-player/=
Clementine" #:tags? #t)
$6 =3D ("refs/tags/0.1" "refs/tags/0.1^{}" "refs/tags/0.2" "refs/tags/0.2^{=
}" "refs/tags/0.3" "refs/tags/0.3^{}" "refs/tags/0.3.1" "refs/tags/0.3.1^{}=
" "refs/tags/0.3.2" "refs/tags/0.3.2^{}" "refs/tags/0.3rc1" "refs/tags/0.3r=
c1^{}" "refs/tags/0.4" "refs/tags/0.4^{}" "refs/tags/0.4.1" "refs/tags/0.4.=
1^{}" "refs/tags/0.4.2" "refs/tags/0.4.2^{}" "refs/tags/0.4rc1" "refs/tags/=
0.4rc1^{}" "refs/tags/0.5" "refs/tags/0.5^{}" "refs/tags/0.5.1" "refs/tags/=
0.5.1^{}" "refs/tags/0.5.2" "refs/tags/0.5.2^{}" "refs/tags/0.5.3" "refs/ta=
gs/0.5.3^{}" "refs/tags/0.5rc1" "refs/tags/0.5rc1^{}" "refs/tags/0.6" "refs=
/tags/0.6^{}" "refs/tags/0.6rc1" "refs/tags/0.6rc1^{}" "refs/tags/0.7" "ref=
s/tags/0.7^{}" "refs/tags/0.7.1" "refs/tags/0.7.1^{}" "refs/tags/0.7.2" "re=
fs/tags/0.7.2^{}" "refs/tags/0.7.3" "refs/tags/0.7.3^{}" "refs/tags/0.7rc1"=
 "refs/tags/0.7rc1^{}" "refs/tags/1.0" "refs/tags/1.0^{}" "refs/tags/1.0.1"=
 "refs/tags/1.0.1^{}" "refs/tags/1.0rc1" "refs/tags/1.0rc1^{}" "refs/tags/1=
.1" "refs/tags/1.1^{}" "refs/tags/1.1.1" "refs/tags/1.1.1^{}" "refs/tags/1.=
2" "refs/tags/1.2^{}" "refs/tags/1.2.1" "refs/tags/1.2.1^{}" "refs/tags/1.2=
.2" "refs/tags/1.2.2^{}" "refs/tags/1.2.3" "refs/tags/1.2.3^{}" "refs/tags/=
1.3" "refs/tags/1.3.1" "refs/tags/1.3rc1" "refs/tags/1.4.0rc1" "refs/tags/1=
.4.0rc1^{}" "refs/tags/1.4.0rc1-153-g06ba55549" "refs/tags/1.4.0rc1-156-gca=
6f42fae" "refs/tags/1.4.0rc1-157-g176b1d6c7" "refs/tags/1.4.0rc1-163-gef302=
1dff" "refs/tags/1.4.0rc1-167-gb0c92ae78" "refs/tags/1.4.0rc1-168-g6285c11b=
c" "refs/tags/1.4.0rc1-169-g934fd336d" "refs/tags/1.4.0rc1-170-g509c65ced" =
"refs/tags/1.4.0rc1-171-g0ecb77335" "refs/tags/1.4.0rc1-172-gb007e54b3" "re=
fs/tags/1.4.0rc1-174-gcb64d9705" "refs/tags/1.4.0rc1-176-g7e7d271b3" "refs/=
tags/1.4.0rc1-177-g096203ac8" "refs/tags/1.4.0rc1-188-g83fc376b0" "refs/tag=
s/1.4.0rc1-189-g58569d9d0" "refs/tags/1.4.0rc1-194-gbaea2d488" "refs/tags/1=
.4.0rc1-198-g6a5cb0712" "refs/tags/1.4.0rc1-200-g18497dcb6" "refs/tags/1.4.=
0rc1-201-gf46241e75" "refs/tags/1.4.0rc1-202-g833f8256c" "refs/tags/1.4.0rc=
1-203-gbc1674700" "refs/tags/1.4.0rc1-204-g912589439" "refs/tags/1.4.0rc1-2=
06-g8f56fbb83" "refs/tags/1.4.0rc1-207-g879dfa3d7" "refs/tags/1.4.0rc1-211-=
g949c20abd" "refs/tags/1.4.0rc1-230-gc934fef63" "refs/tags/1.4.0rc1-231-g60=
a46d193" "refs/tags/1.4.0rc1-234-g0271f43cc" "refs/tags/1.4.0rc1-235-g92b16=
0d2a" "refs/tags/1.4.0rc1-236-g13ee11f81" "refs/tags/1.4.0rc1-237-g54f200d9=
b" "refs/tags/1.4.0rc1-239-gfa067bf5c" "refs/tags/1.4.0rc1-241-ge7c5c76ea" =
"refs/tags/1.4.0rc1-242-gcf1067e74" "refs/tags/1.4.0rc1-243-g5612c9cb5" "re=
fs/tags/1.4.0rc1-244-g84099f249" "refs/tags/1.4.0rc1-245-g0555cf5a3" "refs/=
tags/1.4.0rc1-246-gf90babefa" "refs/tags/1.4.0rc1-247-g1a73918f9" "refs/tag=
s/1.4.0rc1-248-ged0078b8d" "refs/tags/1.4.0rc1-250-ga63a37a7a" "refs/tags/1=
.4.0rc1-251-g6f5fe724b" "refs/tags/1.4.0rc1-252-gc8d56776a" "refs/tags/1.4.=
0rc1-253-g00f9597d3" "refs/tags/1.4.0rc1-254-gbf3d3db23" "refs/tags/1.4.0rc=
1-257-g236cfa7ad" "refs/tags/1.4.0rc1-258-g84fc00d55" "refs/tags/1.4.0rc1-2=
61-g48ae27b4a" "refs/tags/1.4.0rc1-262-g536f34526" "refs/tags/1.4.0rc1-263-=
g4c9241db1" "refs/tags/1.4.0rc1-264-g22537a450" "refs/tags/1.4.0rc1-265-g22=
cfade4a" "refs/tags/1.4.0rc1-268-gc299c198d" "refs/tags/1.4.0rc1-269-gcf8d2=
004b" "refs/tags/1.4.0rc1-270-g6900197a8" "refs/tags/1.4.0rc1-271-g56ed6d4f=
7" "refs/tags/1.4.0rc1-272-gedb8c3b4e" "refs/tags/1.4.0rc1-274-g1ef5ec259" =
"refs/tags/1.4.0rc1-275-g0d25a1b39" "refs/tags/1.4.0rc1-276-g8c25c443c" "re=
fs/tags/1.4.0rc1-279-g76a24a0a2" "refs/tags/1.4.0rc1-280-gcf279e6f4" "refs/=
tags/1.4.0rc1-282-gad882cc99" "refs/tags/1.4.0rc1-283-g0fcb1df20" "refs/tag=
s/1.4.0rc1-284-g7d28e8700" "refs/tags/1.4.0rc1-285-gebf9ebf08" "refs/tags/1=
.4.0rc1-289-g834b1d451" "refs/tags/1.4.0rc1-290-g3bfaf3ff3" "refs/tags/1.4.=
0rc1-291-gc75fa0077" "refs/tags/1.4.0rc1-292-gdd9ed2334" "refs/tags/1.4.0rc=
1-293-g1f7607b1d" "refs/tags/1.4.0rc1-294-g987fe047c" "refs/tags/1.4.0rc1-2=
95-gee72793b5" "refs/tags/1.4.0rc1-296-g68d375c43" "refs/tags/1.4.0rc1-310-=
gd131c66f0" "refs/tags/1.4.0rc1-315-g16843da41" "refs/tags/1.4.0rc1-318-g44=
af6f9d5" "refs/tags/1.4.0rc1-319-gd3e327022" "refs/tags/1.4.0rc1-320-g3a4d7=
f3a3" "refs/tags/1.4.0rc1-321-g2d280734a" "refs/tags/1.4.0rc1-322-g6821f6d7=
b" "refs/tags/1.4.0rc1-323-g29aad2ae3" "refs/tags/1.4.0rc1-324-g06855ea6c" =
"refs/tags/1.4.0rc1-325-g598f84007" "refs/tags/1.4.0rc1-326-gd0bf92f06" "re=
fs/tags/1.4.0rc1-327-g7b3a0f397" "refs/tags/1.4.0rc1-328-ge9b62fa34" "refs/=
tags/1.4.0rc1-329-gf7bece3b8" "refs/tags/1.4.0rc1-332-g62d2f0de9" "refs/tag=
s/1.4.0rc1-340-g2172732b1" "refs/tags/1.4.0rc1-341-g54f7637ad" "refs/tags/1=
.4.0rc1-342-g2bac3626c" "refs/tags/1.4.0rc1-343-gb49afcc5b" "refs/tags/1.4.=
0rc1-344-gad354276b" "refs/tags/1.4.0rc1-345-g9e8d4434a" "refs/tags/1.4.0rc=
1-346-g4e3e9c8d1" "refs/tags/1.4.0rc1-347-gfc4cb6fc7" "refs/tags/1.4.0rc1-3=
48-gcac606186" "refs/tags/1.4.0rc1-349-g16d09ace0" "refs/tags/1.4.0rc1-350-=
geefb96bdc" "refs/tags/1.4.0rc1-351-g1daf43f91" "refs/tags/1.4.0rc1-352-gaa=
ee0b701" "refs/tags/1.4.0rc1-353-gae4948ce3" "refs/tags/1.4.0rc1-354-gd970b=
7400" "refs/tags/1.4.0rc1-355-gc856a6617" "refs/tags/1.4.0rc1-356-gd417aed2=
9" "refs/tags/1.4.0rc1-357-geec7641ef" "refs/tags/1.4.0rc1-358-gc536dc88e" =
"refs/tags/1.4.0rc1-360-gb2044a5be" "refs/tags/1.4.0rc1-361-gf17e29f41" "re=
fs/tags/1.4.0rc1-362-g7b3e2dfd8" "refs/tags/1.4.0rc1-363-gf60c42224" "refs/=
tags/1.4.0rc1-364-gc4d22d441" "refs/tags/1.4.0rc1-365-g41b1ba8ff" "refs/tag=
s/1.4.0rc1-366-g20f49c445" "refs/tags/1.4.0rc1-368-g1a0b288a8" "refs/tags/1=
.4.0rc1-369-gf5c904b26" "refs/tags/1.4.0rc1-370-gcca48b1eb" "refs/tags/1.4.=
0rc1-371-gdf262c5c7" "refs/tags/1.4.0rc1-372-g01f072764" "refs/tags/1.4.0rc=
1-373-gba8fc09a6" "refs/tags/1.4.0rc1-374-g91bad31f6" "refs/tags/1.4.0rc1-3=
77-gccba649f6" "refs/tags/1.4.0rc1-378-ga3a51ae11" "refs/tags/1.4.0rc1-379-=
gcfcd0a956" "refs/tags/1.4.0rc1-380-gd7966c828" "refs/tags/1.4.0rc1-384-g41=
513527c" "refs/tags/1.4.0rc1-386-gbbb6a773f" "refs/tags/1.4.0rc1-387-g627dd=
c398" "refs/tags/1.4.0rc1-388-g6a6ef729e" "refs/tags/1.4.0rc1-389-g51c600a5=
3" "refs/tags/1.4.0rc1-390-gaf4810a58" "refs/tags/1.4.0rc1-391-g863a66824" =
"refs/tags/1.4.0rc1-392-g9f8093a22" "refs/tags/1.4.0rc1-393-gc999fc70e" "re=
fs/tags/1.4.0rc1-394-g870969ef4" "refs/tags/1.4.0rc1-395-gfaab7fa6c" "refs/=
tags/1.4.0rc1-396-g549544517" "refs/tags/1.4.0rc1-397-g616ccc6fd" "refs/tag=
s/1.4.0rc1-398-g0393d865c" "refs/tags/1.4.0rc1-399-ga012e7e27" "refs/tags/1=
.4.0rc1-400-g87cd3d2ab" "refs/tags/1.4.0rc1-401-gdc2c1e111" "refs/tags/1.4.=
0rc1-402-g63a73a4a5" "refs/tags/1.4.0rc1-403-g2b99d32be" "refs/tags/1.4.0rc=
1-406-g409c6b89d" "refs/tags/1.4.0rc1-407-g3efa68f07" "refs/tags/1.4.0rc1-4=
08-g8f863bc96" "refs/tags/1.4.0rc1-409-g8201c1035" "refs/tags/1.4.0rc1-410-=
g479f1d4de" "refs/tags/1.4.0rc1-413-g25d3fca07" "refs/tags/1.4.0rc1-414-g8c=
774e388" "refs/tags/1.4.0rc1-416-g7b9430982" "refs/tags/1.4.0rc1-417-gf7796=
52aa" "refs/tags/1.4.0rc1-418-gb3aed042e" "refs/tags/1.4.0rc1-420-g596cd9b0=
a" "refs/tags/1.4.0rc1-421-ge1e559732" "refs/tags/1.4.0rc1-422-gace5234e6" =
"refs/tags/1.4.0rc1-423-g2dd424a19" "refs/tags/1.4.0rc1-425-g4f5bf1cc6" "re=
fs/tags/1.4.0rc1-426-g72e2e62eb" "refs/tags/1.4.0rc1-427-gcf842a8c5" "refs/=
tags/1.4.0rc1-428-g81a3c0f83" "refs/tags/1.4.0rc1-429-gf1678fd33" "refs/tag=
s/1.4.0rc1-430-g7854aefdd" "refs/tags/1.4.0rc1-431-ga9e193234" "refs/tags/1=
.4.0rc1-432-g447e91a68" "refs/tags/1.4.0rc1-433-g76c87146d" "refs/tags/1.4.=
0rc1-434-ga7a32b08b" "refs/tags/1.4.0rc1-436-g8c2ab8fa0" "refs/tags/1.4.0rc=
1-438-gcb88954a3" "refs/tags/1.4.0rc1-439-g79ca9147e" "refs/tags/1.4.0rc1-4=
40-g7ba322b10" "refs/tags/1.4.0rc1-441-gb9a844263" "refs/tags/1.4.0rc1-442-=
g78d4c4f3f" "refs/tags/1.4.0rc1-444-g8d11e9ffa" "refs/tags/1.4.0rc1-446-g18=
eef830a" "refs/tags/1.4.0rc1-447-g8db8b1e78" "refs/tags/1.4.0rc1-448-g816fd=
88d4" "refs/tags/1.4.0rc1-449-g50ee78613" "refs/tags/1.4.0rc1-450-g2725ef99=
d" "refs/tags/1.4.0rc1-451-g66ea25bca" "refs/tags/1.4.0rc1-453-g281da0532" =
"refs/tags/1.4.0rc1-454-g57a6fe4f2" "refs/tags/1.4.0rc1-456-geb7a9bfa4" "re=
fs/tags/1.4.0rc1-457-g8f3772b59" "refs/tags/1.4.0rc1-459-ge84f87f62" "refs/=
tags/1.4.0rc1-461-gf7b6708e4" "refs/tags/1.4.0rc1-462-gfffc50c79" "refs/tag=
s/1.4.0rc1-463-gf7ed4a309" "refs/tags/1.4.0rc1-464-gcde0343a6" "refs/tags/1=
.4.0rc1-465-gb69dd2d90" "refs/tags/1.4.0rc1-466-gd9a48b90b" "refs/tags/1.4.=
0rc1-467-gd93bd9ca2" "refs/tags/1.4.0rc1-468-g1a3828e2c" "refs/tags/1.4.0rc=
1-469-gb40d9ed44" "refs/tags/1.4.0rc1-471-gb989a674a" "refs/tags/1.4.0rc1-4=
72-g4e8a12f37" "refs/tags/1.4.0rc1-473-gbce55d0ef" "refs/tags/1.4.0rc1-477-=
g576731767" "refs/tags/1.4.0rc1-480-g05f513ab6" "refs/tags/1.4.0rc1-481-g2b=
988ed7b" "refs/tags/1.4.0rc1-482-g0c099ab6f" "refs/tags/1.4.0rc1-483-gc7f5c=
0f40" "refs/tags/1.4.0rc1-484-g2d8a56b7c" "refs/tags/1.4.0rc1-486-gf92690c1=
4" "refs/tags/1.4.0rc1-487-g15474ada3" "refs/tags/1.4.0rc1-488-g7bb0c59f2" =
"refs/tags/1.4.0rc1-489-g6314c8cb2" "refs/tags/1.4.0rc1-491-g651eee13e" "re=
fs/tags/1.4.0rc1-494-gdfb953a78" "refs/tags/1.4.0rc1-495-g10bf5dc17" "refs/=
tags/1.4.0rc1-496-gcef1d7e74" "refs/tags/1.4.0rc1-497-g3bd15aea0" "refs/tag=
s/1.4.0rc1-498-g681e7bea5" "refs/tags/1.4.0rc1-502-gaf75ebbd6" "refs/tags/1=
.4.0rc1-509-g89e9b20df" "refs/tags/1.4.0rc1-510-g3f34b332c" "refs/tags/1.4.=
0rc1-512-g8b2f7f08a" "refs/tags/1.4.0rc1-514-g05e450c3c" "refs/tags/1.4.0rc=
1-515-g1154c0f54" "refs/tags/1.4.0rc1-516-g8b566b2a7" "refs/tags/1.4.0rc1-5=
18-g3244cf083" "refs/tags/1.4.0rc1-519-gd1e9ee9f9" "refs/tags/1.4.0rc1-520-=
gc394d7d2d" "refs/tags/1.4.0rc1-521-gb68b12010" "refs/tags/1.4.0rc1-522-gfd=
b3f7ac3" "refs/tags/1.4.0rc1-525-gc12294c5e" "refs/tags/1.4.0rc1-526-g88189=
8f84" "refs/tags/1.4.0rc1-527-g438e8ca61" "refs/tags/1.4.0rc1-528-g86d782cb=
6" "refs/tags/1.4.0rc1-531-g641279072" "refs/tags/1.4.0rc1-533-gf4e70face" =
"refs/tags/1.4.0rc1-534-gd13410c91" "refs/tags/1.4.0rc1-536-g4edf77082" "re=
fs/tags/1.4.0rc1-537-gada6752ea" "refs/tags/1.4.0rc1-538-g15fdad3d5" "refs/=
tags/1.4.0rc1-540-g4f86e0b2b" "refs/tags/1.4.0rc1-541-ge077df22d" "refs/tag=
s/1.4.0rc1-542-g8a7120e1e" "refs/tags/1.4.0rc1-544-g3b8519fda" "refs/tags/1=
.4.0rc1-545-g2d6bb4abd" "refs/tags/1.4.0rc1-548-g354f6a23e" "refs/tags/1.4.=
0rc1-549-ge8875faf8" "refs/tags/1.4.0rc1-550-g72c1f91c0" "refs/tags/1.4.0rc=
1-551-g144bdc249" "refs/tags/1.4.0rc1-552-gdb55c541b" "refs/tags/1.4.0rc1-5=
53-ga86558f9a" "refs/tags/1.4.0rc1-554-g2d34588b8" "refs/tags/1.4.0rc1-555-=
g32944a15d" "refs/tags/1.4.0rc1-556-g3440f90a6" "refs/tags/1.4.0rc1-557-g00=
9642d12" "refs/tags/1.4.0rc1-558-g47f7b307f" "refs/tags/1.4.0rc1-559-ge7364=
263b" "refs/tags/1.4.0rc1-560-g7303f72ee" "refs/tags/1.4.0rc1-562-g99ee1394=
a" "refs/tags/1.4.0rc1-563-g163ebe71d" "refs/tags/1.4.0rc1-564-g429d8ee0f" =
"refs/tags/1.4.0rc1-565-g6b21079fd" "refs/tags/1.4.0rc1-566-gf04657e7e" "re=
fs/tags/1.4.0rc1-567-g280a514eb" "refs/tags/1.4.0rc1-568-gc51d2f954" "refs/=
tags/1.4.0rc1-569-gf17b79a10" "refs/tags/1.4.0rc1-570-g73c0af197" "refs/tag=
s/1.4.0rc1-571-g5f75bde39" "refs/tags/1.4.0rc1-572-g59f6d95b8" "refs/tags/1=
.4.0rc1-573-g8258c78c0" "refs/tags/1.4.0rc1-574-gb2ed9499f" "refs/tags/1.4.=
0rc1-575-g94f4f65a6" "refs/tags/1.4.0rc1-576-g7e48b78c1" "refs/tags/1.4.0rc=
1-577-gfc83e4127" "refs/tags/1.4.0rc1-578-gd59ed1e70" "refs/tags/1.4.0rc1-5=
79-g8fddc816a" "refs/tags/1.4.0rc1-585-g8c1bdc1a4" "refs/tags/1.4.0rc1-586-=
g20647e8a9" "refs/tags/1.4.0rc1-587-g708385c71" "refs/tags/1.4.0rc1-588-g9a=
337a9ef" "refs/tags/1.4.0rc1-589-gf48888a43" "refs/tags/1.4.0rc1-591-g579d8=
6904" "refs/tags/1.4.0rc1-593-g783213f9c" "refs/tags/1.4.0rc1-594-gf5d3079d=
b" "refs/tags/1.4.0rc1-596-g590bcf1c7" "refs/tags/1.4.0rc1-597-g83157100c" =
"refs/tags/1.4.0rc1-598-gd16d9ba28" "refs/tags/1.4.0rc1-600-g3f614464e" "re=
fs/tags/1.4.0rc1-601-ga7468dcd4" "refs/tags/1.4.0rc1-602-g89155ace7" "refs/=
tags/1.4.0rc1-603-g75de59703" "refs/tags/1.4.0rc1-604-g1309c76be" "refs/tag=
s/1.4.0rc1-613-ge756f2d68" "refs/tags/1.4.0rc1-614-g89831f8dc" "refs/tags/1=
.4.0rc1-617-g776bd3b02" "refs/tags/1.4.0rc1-618-gf071075e8" "refs/tags/1.4.=
0rc1-619-gd71eba97f" "refs/tags/1.4.0rc1-620-g684c9d232" "refs/tags/1.4.0rc=
1-621-g2132e99fb" "refs/tags/1.4.0rc1-622-gf7369d2c4" "refs/tags/1.4.0rc1-6=
23-gf67475375" "refs/tags/1.4.0rc1-624-g72cfdf25a" "refs/tags/1.4.0rc1-626-=
g058fe6f4b" "refs/tags/1.4.0rc1-627-g0dbefa306" "refs/tags/1.4.0rc1-628-gb0=
9ab3ff3" "refs/tags/1.4.0rc1-629-g612767c87" "refs/tags/1.4.0rc1-631-g4e4fc=
cc07" "refs/tags/1.4.0rc1-633-g3a00403ad" "refs/tags/1.4.0rc1-634-g4aa4f4fc=
e" "refs/tags/1.4.0rc1-635-g418a36693" "refs/tags/1.4.0rc1-636-g2bf8f1388" =
"refs/tags/1.4.0rc1-637-gffdaeba09" "refs/tags/1.4.0rc1-638-gc3c77aef1" "re=
fs/tags/1.4.0rc1-639-g11bd0db03" "refs/tags/1.4.0rc1-657-g57b5911f1" "refs/=
tags/1.4.0rc1-658-g6240fd3d0" "refs/tags/1.4.0rc1-659-g54be35f52" "refs/tag=
s/1.4.0rc1-660-ge46503d0c" "refs/tags/1.4.0rc1-661-g62cb889a3" "refs/tags/1=
.4.0rc1-662-g5ab81fd8b" "refs/tags/1.4.0rc1-663-gf9854e564" "refs/tags/1.4.=
0rc1-664-g1db1e3231" "refs/tags/1.4.0rc1-665-g67a947f11" "refs/tags/1.4.0rc=
1-666-g4a83f8c81" "refs/tags/1.4.0rc1-668-gf35a640ce" "refs/tags/1.4.0rc1-6=
69-g67aa15418" "refs/tags/1.4.0rc1-670-g8c660e278" "refs/tags/1.4.0rc1-671-=
g25b537cf2" "refs/tags/1.4.0rc1-672-ga5fd484a6" "refs/tags/1.4.0rc1-673-gdb=
8de64ab" "refs/tags/1.4.0rc1-674-g7cb5f5c80" "refs/tags/1.4.0rc1-675-ga5e84=
bbe9" "refs/tags/1.4.0rc1-676-g6b2918ee9" "refs/tags/1.4.0rc1-677-g4acfdae7=
4" "refs/tags/1.4.0rc1-678-g2902a8786" "refs/tags/1.4.0rc1-679-gb3b769f0e" =
"refs/tags/1.4.0rc1-680-g4d3474840" "refs/tags/1.4.0rc1-681-g598e660ae" "re=
fs/tags/1.4.0rc1-682-g0c1b6a2a4" "refs/tags/1.4.0rc1-683-g320a1b81c" "refs/=
tags/1.4.0rc1-684-g1d1d3b157" "refs/tags/1.4.0rc1-685-gf379ad84d" "refs/tag=
s/1.4.0rc1-686-gdaa2f25e3" "refs/tags/1.4.0rc1-687-g1e39ce29a" "refs/tags/1=
.4.0rc1-688-g98dd3e48a" "refs/tags/1.4.0rc1-689-g6982b4781" "refs/tags/1.4.=
0rc1-690-gc0c903767" "refs/tags/1.4.0rc1-691-gdbe15e5e9" "refs/tags/1.4.0rc=
1-692-g224c475b5" "refs/tags/1.4.0rc1-693-gac3a0d33f" "refs/tags/1.4.0rc1-6=
94-g102317e5c" "refs/tags/1.4.0rc1-695-ge2d6759d5" "refs/tags/1.4.0rc1-696-=
gbf424ce98" "refs/tags/1.4.0rc1-697-gcddc08e14" "refs/tags/1.4.0rc1-698-gb5=
5e54388" "refs/tags/1.4.0rc1-699-g327d5fdac" "refs/tags/1.4.0rc1-700-g03e13=
c69e" "refs/tags/1.4.0rc1-701-g8682d4de4" "refs/tags/1.4.0rc1-702-g922afe50=
6" "refs/tags/1.4.0rc1-708-gc8c110efa" "refs/tags/1.4.0rc1-709-g628ff6582" =
"refs/tags/1.4.0rc1-710-g7eb62b626" "refs/tags/1.4.0rc1-711-g3b7d5880f" "re=
fs/tags/1.4.0rc1-712-g769d8bbe6" "refs/tags/1.4.0rc1-713-gc58335c6c" "refs/=
tags/1.4.0rc1-715-ge556a59ae" "refs/tags/1.4.0rc1-716-g2cca75d93")
scheme@(guile-user)>
=2D-8<---------------cut here---------------end--------------->8---

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE0wfcVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5v8QP/1CC/R9sAsolZ2UOQ6I8R4RXOTqx
2eGD9CyQGvD+xxliaP+cWzom/ti8vjGf8EEQfSIZ4F5db3z1VdbfU8XmMg1lwhXx
/pdH5tjCjASrWioMKMZh7ZI4VMjZmSTCtvkGqsIiYdZ1xP11bJyIqDCxzib1Ue61
irN2Yh9UEQzyrW5a+TQrUkEWAY35ayNusiJHA0ANml6k7SyM6Y3CSTCfr0zk9EK+
UEiGuTpdUnezR/1czsxl/CvCepVmS9Xm6Ys1BO8v1DzXAtKGzxne4n5QuTUEL2Lo
5yFkIEtiFo1ccu6kbh59KsmOsljVtBZOuWNsEnM5d2ybKVyD350n2XoEOztv+VUQ
vZjpqbuDXt17MovXQGmNU6mS0r176X9KlXPN227uR7rZc9CyiTfcAvttrTMJ8pFG
FNJwhbuAzX271hrOHkLgAbj03Q6B1J5kyTMnfsD0jct4NDypsKZnw5CGltRyGCSZ
d9R52wyeiNIJv4nwxrfe3a4Oi1tIkJI81nqOM674Z1v0lGqUIk0yC63d/A+whngs
0cOs/qCiZDF+xxFF/3ptXKSsJyKSBu2cLZbp7aUhOj1Oqj40zE2Itf2qz6wgpXc0
wovuyEPjLb6SF3z9Fz1YwPusgm99zWyrM4/XMMZ6UWMQAd3jkCL8CBYm7pavbFsr
QnYIhfSYyoo7stod
=mfA0
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 06 Sep 2021 03:15:01 +0000
Resent-Message-ID: <handler.50359.B50359.16308980961992 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.16308980961992
          (code B ref 50359); Mon, 06 Sep 2021 03:15:01 +0000
Received: (at 50359) by debbugs.gnu.org; 6 Sep 2021 03:14:56 +0000
Received: from localhost ([127.0.0.1]:51288 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mN560-0000W4-IS
	for submit <at> debbugs.gnu.org; Sun, 05 Sep 2021 23:14:56 -0400
Received: from out2.migadu.com ([188.165.223.204]:13490)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mN55x-0000Vt-3z
 for 50359 <at> debbugs.gnu.org; Sun, 05 Sep 2021 23:14:54 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630898091;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=xUrn6lZE3xVKauxh73rs8SYlgQuSTAK1Bn2VQe/8pXk=;
 b=LvTReMGbQawATevtl9is1eatz2cFz9S72jIAZrnTo53GLiKakkU00gpLyCCgfiF4ZzztXB
 V3tj8yyF1RJAOgbsflLR1wmTAlP+VnIqwreOMyWAM4vW/4BuVDdIwcUxvIm2DT7cJpw1xx
 GuIJNX8nVJ7ReXZiYL5EO3R8kjVd72o=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87eea3416x.fsf@HIDDEN>
Date: Sun, 05 Sep 2021 20:14:49 -0700
In-Reply-To: <87eea3416x.fsf@HIDDEN> (Xinglu Chen's message of "Sun, 05
 Sep 2021 15:11:18 +0200 (5 hours, 24 minutes, 26 seconds ago)")
Message-ID: <861r62jsye.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hello, Xinglu Chen <public@HIDDEN> writes: >> And this
 lets us just do something like (untested): >> >> (define* (get-version tag
 #:key prefix suffix delim) >> (define delim-rx (regexp-quote (or delim ".")))
 >> (define prefix-rx (or prefix "[^ [...] 
 Content analysis details:   (2.0 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [188.165.223.204 listed in wl.mailspike.net]
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

Hello,

Xinglu Chen <public@HIDDEN> writes:

>> And this lets us just do something like (untested):
>>
>> (define* (get-version tag #:key prefix suffix delim)
>>   (define delim-rx (regexp-quote (or delim ".")))
>>   (define prefix-rx (or prefix "[^[:digit:]]*"))
>>   (define suffix-rx (or suffix ".*"))
>>   (define version-char-rx
>>    (string-append "[^" delim-rx "[:punct:]]"))
>>
>>   (define tag-rx
>>    (string-append "^" prefix "(" version-char-rx "+("
>>                   delim-rx version-char-rx ")*)" suffix-rx "$"))
>
> This wouldn=E2=80=99t match anything if the version is just a plain numbe=
r,
> e.g., 1 or 09.

It does, but I had many errors in the definition.  Again, apologies.  I
shouldn't send emails that late, haha.  This method should read:

 --8<---------------cut here---------------start------------->8---
 (define* (get-version tag #:key prefix suffix delim)
   (define delim-rx (regexp-quote (or delim ".")))
   (define prefix-rx (or prefix "[^[:digit:]]*"))
   (define suffix-rx (or suffix ".*"))
   (define version-char-rx
    (string-append "[^" delim-rx "[:punct:]]"))

   (define tag-rx
    (string-append "^" prefix-rx "(" version-char-rx "+("
                   delim-rx version-char-rx "+)*)" suffix-rx "$"))
   (and=3D> (string-match tag-rx tag)
          (cut match:substring <> 1)))
--8<---------------cut here---------------end--------------->8---

>
> With this, something like =E2=80=9C1.4.0rc1-450-g2725ef99d=E2=80=9D will =
result in
> =E2=80=9C1.4.0=E2=80=9D being returned, which is incorrect.  Changing (cut
> match:substring <> 1) to just =E2=80=98match:substring=E2=80=99 would sol=
ve the issue,
> but then pre-release tags, which we usually don=E2=80=99t want,  would al=
so get
> matched.  Not sure what the best option would be in this case.
>

With the fixed method above:

scheme@(emacs-guix)> (get-version "8")
$16 =3D "8"
scheme@(emacs-guix)> (get-version "1.4.0rc1-450-g2725ef99d")
$17 =3D "1.4.0rc1"

But, we still get:

scheme@(emacs-guix)> (get-version "1.4.0-rc1")
$18 =3D "1.4.0"

which leads us to what you talked about in your other message.

[...]
>> +(define* (ls-remote-refs url #:key tags?)
>> +  "Return the list of references advertised at Git repository URL.  If =
TAGS?
>> +is true, limit to only refs/tags."
>> +  (define (ref? ref)
>> +    ;; Like `git ls-remote --refs', only show actual references.
>> +    (and (string-prefix? "refs/" ref)
>> +         (not (string-suffix? "^{}" ref))))
>> +
>> +  (define (tag? ref)
>> +    (string-prefix? "refs/tags/" ref))
>> +
>> +  (define (include? ref)
>> +    (and ref?

This should be:
        (and (ref? ref)

>> +         (or (not tags?) (tag? ref))))
>> +
>> +  (with-libgit2
>> +   (with-temporary-directory
>> +    (lambda (cache-directory)
>> +      (let* ((repository (repository-init cache-directory))
>> +             ;; Create an in-memory remote so we don't touch disk.
>> +             (remote (remote-create-anonymous repository url)))
>> +        (remote-connect remote)
>> +        (remote-disconnect remote)
>> +        (repository-close! repository)
>> +
>> +        (filter include? (map remote-head-name (remote-ls remote))))))))
>>
>
> For some reason it seems to include refs that do and don=E2=80=99t end wi=
th
> =E2=80=9C^{}=E2=80=9D

Sorry, another typo I missed.  See above.

--
Sarah




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 06 Sep 2021 05:41:01 +0000
Resent-Message-ID: <handler.50359.B50359.163090681524932 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163090681524932
          (code B ref 50359); Mon, 06 Sep 2021 05:41:01 +0000
Received: (at 50359) by debbugs.gnu.org; 6 Sep 2021 05:40:15 +0000
Received: from localhost ([127.0.0.1]:51401 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mN7Mc-0006U2-Hv
	for submit <at> debbugs.gnu.org; Mon, 06 Sep 2021 01:40:15 -0400
Received: from out0.migadu.com ([94.23.1.103]:32941)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mN7MZ-0006To-NK
 for 50359 <at> debbugs.gnu.org; Mon, 06 Sep 2021 01:40:13 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630906809;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=Vc21lp4iSw+Pfq9KNBd4IwNHCdIxnnMcXqV5hIbaYqw=;
 b=AR5kqWPz/Bu+HBa80vmM5kB+w3L4XO6NyGAGNSAH//wg8932gj+hrvp8527DMR99ksy/sT
 gw1kw9bYwqcnKTFko1INrvO3ZqVYv3jm4MfqAlYDdzcz9ug3Lflc9YkP8tVNFtxPKsWuSo
 8DOgHXc9++Cw4jhGuZaJ5BCNPm4U7eQ=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
Date: Sun, 05 Sep 2021 22:40:07 -0700
In-Reply-To: <87h7ez48d3.fsf@HIDDEN> (Xinglu Chen's message of "Sun, 05
 Sep 2021 12:36:24 +0200 (16 hours, 38 minutes, 30 seconds ago)")
Message-ID: <86y28ai7ns.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 1.3 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hi, Xinglu Chen <public@HIDDEN> writes: >> There are
 about 50-60 packages like this. >> >> I'm not sure how much effort should
 be spent including them, and for >> some of them I'm not sure what our ideal
 behavior *is*. Even if we >> could r [...] 
 Content analysis details:   (1.3 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at https://www.dnswl.org/,
 low trust [94.23.1.103 listed in list.dnswl.org]
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [94.23.1.103 listed in wl.mailspike.net]
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.3 (/)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi,

Xinglu Chen <public@HIDDEN> writes:

>> There are about 50-60 packages like this.
>>
>> I'm not sure how much effort should be spent including them, and for
>> some of them I'm not sure what our ideal behavior *is*.  Even if we
>> could reliably detect them, should "alpha" or "dev" packages be returned
>> by the updater?
>
> I don=E2=80=99t think we usually include alpha or rc releases, so updater
> probably shouldn=E2=80=99t return them either.  Not sure how we would try=
 to
> detect alpha/beta/rc releases, though, besides running something like
>
>   (string-match? "(alpha|beta|rc|dev)" TAG)
>
> On each tag.

That heuristic is pretty good.  (It might miss a few, but I'd rather
accidentally include some alpha/beta/rc releases than risk excluding
real ones.)

We could then safely sort tags with just the prefix removed -- this takes c=
are
of "1.1f" coming after "1.1", and so on.

Actually, it looks like there's only a few packages with a suffix;
instead, we can probably just only use a suffix if they provide
'tag-suffix.  This is all of them (and the "dev" ones probably shouldn't
be suffixes, just part of the version):

(commit (string-append "v" version "-stable"))))
(commit (string-append version "-stable"))))
(commit (string-append version "-Leia"))))
(commit (string-append "haddock-" version "-release"))))
(commit (string-append "v" version "-8.13"))))
(commit (string-append "v" version "-oss"))))
(commit (string-append "v" version "-stable"))))
(commit (string-append "ddskk-" version "_" code-name))))
(commit (string-append version "-freebsdport"))))
(commit (string-append version "-dev"))))
(commit (string-append version "-release-20210531143054"))))
(commit (string-append version "-release-20210412001032"))))
(commit (string-append "v" version "-debian"))))
(commit (string-append version "dev"))))
(commit (string-append version "_Linux"))
(commit (string-append version "R"))))
(commit (string-append "jdk-" version "-ga"))))
(commit (string-append "jdk-" version "-ga"))))
(commit (string-append "jdk-" version "-ga"))))
(commit (string-append "jdk-" version "-ga"))))
(commit (string-append version "-opt"))))
(commit (string-append "1.1-" version "-RELEASE"))))

Additionally, these are all the weird version strings I could find that
are actually used as the commit:

--=-=-=
Content-Type: text/plain
Content-Disposition: inline; filename=versions.txt
Content-Description: versions

1.2.2.rc2
12-068oasis4
1.2-2
0.F-2
0.2.0-alpha
5.1.0-b2
1.5-11
1.9.14-20210407
0.9.3b
2.8-fix-2
2020-05-19
10-11.0.0
1.0.12-2
0.9.3+16.04.20160218-0ubuntu1
2.12.c
1.1+11
5.1+4.06.0
4.2-411
1.4.0rc1-450-g2725ef99d
2.0.0-alpha14
60.2.3-2
1.21c
1.0.0rc4
2.1.0b1
1.02r6
0.32-14-gcdfe14e
3.0.0a3
2.00a2.3
1.0beta.18
2.1b
2.7.8a
2.7.3a
1.1.alpha19
1.0.2-rc4
0.5.3+git20200502
1.0.3-rc3
4.0.0.dev8
3.0.0beta1-24-g024cc9fa2
0.16-2-ge145396
2.0b6
2.0M10
1.0.7+0
1.16.0+5
0.4.0+1
2.2.10+0
4.3.1+2
2.13.93+0
2.10.4+0
1.0.5+5
0.21.0+0
3.3.4+0
2.68.1+0
0.10.1+1
6.9.10-12+3
2.0.1+2
3.100.0+1
0.14.0+2
0.1.6+2
3.3.0+0
1.8.7+0
1.3.0+2
1.42.0+0
1.16.1+0
2.35.0+0
1.6.37+5
4.1.0+1
2.36.0+0
1.3.6+4
2.10.1+0
2.26.0+0
1.3.4+0
1.1.1+2
1.3.1+1
8.44.0+0
0.40.1+0
5.15.2+0
1.17.0+3
1.18.0+3
2020.7.14+0
3.0.0+1
0.9.1+3
2.9.12+0
0.1.0+2
1.6.9+2
1.0.9+3
1.13.0+2
1.2.0+3
1.1.3+3
1.3.4+2
5.0.3+3
1.7.10+3
1.1.4+3
1.1.0+3
1.5.2+3
0.9.10+3
0.4.0+1
0.4.0+1
0.4.0+1
0.3.9+1
0.4.1+1
1.4.2+3
2.27.0+3
1.4.0+2
1.1.34+0
1.2.12+1
1.5.0+0
2021-06-07
1.2.2-5-g20dc8ed
2.1-20201229
3.0-rc1
1.32.0-0
20200701.154658.b0d6223
3.9-0
R63-10032.B
0.58.2.a
0.0.0+git20200527
3.028R
3.001R
2.57b
1.0.0-20201130134442-10cb98267c6c
0.0.0-20161123171359-e6a2ba005892
0.0.0-20210615171337-6886f2dfbf5b
2020-11-10
5.2.0-alpha
2021-01-01
2021-02-28
2020-11-01
4.4-git.1
1.217-2
3.3.06-1
0.2.0-alpha-199-g3e7a475
1.0.0-beta.0
1.20190621-4
1.9.0-147-g61edec1ef
0.0.9.4f

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


Yep, it looks like the above would work for the majority of these.
That's probably Good Enough^tm.

>>> +  (define no-delims
>>> +    (if (string-match delim-regexp no-suffix)
>>> +        (string-split* no-suffix delim*)
>>> +        (git-tag-error 'tag-version-delimiter)))
>>
>> This throws an error if the version doesn't have any delimiter.
>
> Setting the =E2=80=98tag-version-delimiter=E2=80=99 prefix to an empty st=
ring would
> solve this, right?  Or, maybe we should just get rid of the delimiter
> thing since only a few packages use a different delimiter.

IMO, just get rid of the delimiter.  If we wanted to be *that* flexible,
we could make it so they provide a tag->version proc instead of (prefix,
suffix, delimiter).

--
Sarah

--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 06 Sep 2021 12:21:01 +0000
Resent-Message-ID: <handler.50359.B50359.163093085329818 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Sarah Morgensen <iskarian@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163093085329818
          (code B ref 50359); Mon, 06 Sep 2021 12:21:01 +0000
Received: (at 50359) by debbugs.gnu.org; 6 Sep 2021 12:20:53 +0000
Received: from localhost ([127.0.0.1]:52280 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mNDcK-0007kr-Kz
	for submit <at> debbugs.gnu.org; Mon, 06 Sep 2021 08:20:53 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:37942
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mNDcI-0007ka-Ag
 for 50359 <at> debbugs.gnu.org; Mon, 06 Sep 2021 08:20:51 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1630930837;
 bh=rTeAfqqkrZ8uCrLD42TmYnmw6Zfc0mWDeJW9zRplpEc=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=MYYVUOoe+DrZ96dkt00vEYMFEyxSINEtCD0m6tE9lhiRsm8VlNqbqiwK2gv9ooEVL
 OTFRKDlomoQ8fgVPqa4Aa/k+tTnu2tQFbO4GngHrLonWRvNWg20F1hGOi7bGdIBzS8
 hTGdJsdFudoC2F/mSQqDu+WCMDT9WzKcjQD9cP/g=
In-Reply-To: <86y28ai7ns.fsf@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
 <86y28ai7ns.fsf@HIDDEN>
Date: Mon, 06 Sep 2021 14:20:34 +0200
Message-ID: <87y28928vh.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  On Sun, Sep 05 2021, Sarah Morgensen wrote: > Hi, > > Xinglu
 Chen <public@HIDDEN> writes: > >>> There are about 50-60 packages
 like this. >>> >>> I'm not sure how much effort should be spent including
 them, and for >>> some of them I'm n [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Sun, Sep 05 2021, Sarah Morgensen wrote: > Hi, > > Xinglu
    Chen <public@HIDDEN> writes: > >>> There are about 50-60 packages
   like this. >>> >>> I'm not sure how much effort should be spent including
   them, and for >>> some of them I'm n [...] 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Sun, Sep 05 2021, Sarah Morgensen wrote:

> Hi,
>
> Xinglu Chen <public@HIDDEN> writes:
>
>>> There are about 50-60 packages like this.
>>>
>>> I'm not sure how much effort should be spent including them, and for
>>> some of them I'm not sure what our ideal behavior *is*.  Even if we
>>> could reliably detect them, should "alpha" or "dev" packages be returned
>>> by the updater?
>>
>> I don=E2=80=99t think we usually include alpha or rc releases, so updater
>> probably shouldn=E2=80=99t return them either.  Not sure how we would tr=
y to
>> detect alpha/beta/rc releases, though, besides running something like
>>
>>   (string-match? "(alpha|beta|rc|dev)" TAG)
>>
>> On each tag.
>
> That heuristic is pretty good.  (It might miss a few, but I'd rather
> accidentally include some alpha/beta/rc releases than risk excluding
> real ones.)
>
> We could then safely sort tags with just the prefix removed -- this takes=
 care
> of "1.1f" coming after "1.1", and so on.
>
> Actually, it looks like there's only a few packages with a suffix;
> instead, we can probably just only use a suffix if they provide
> 'tag-suffix.  This is all of them (and the "dev" ones probably shouldn't
> be suffixes, just part of the version):

Yeah, I think that sounds like a reasonable thing to do.

> (commit (string-append "v" version "-stable"))))
> (commit (string-append version "-stable"))))
> (commit (string-append version "-Leia"))))
> (commit (string-append "haddock-" version "-release"))))
> (commit (string-append "v" version "-8.13"))))
> (commit (string-append "v" version "-oss"))))
> (commit (string-append "v" version "-stable"))))
> (commit (string-append "ddskk-" version "_" code-name))))
> (commit (string-append version "-freebsdport"))))
> (commit (string-append version "-dev"))))
> (commit (string-append version "-release-20210531143054"))))
> (commit (string-append version "-release-20210412001032"))))
> (commit (string-append "v" version "-debian"))))
> (commit (string-append version "dev"))))
> (commit (string-append version "_Linux"))
> (commit (string-append version "R"))))
> (commit (string-append "jdk-" version "-ga"))))
> (commit (string-append "jdk-" version "-ga"))))
> (commit (string-append "jdk-" version "-ga"))))
> (commit (string-append "jdk-" version "-ga"))))
> (commit (string-append version "-opt"))))
> (commit (string-append "1.1-" version "-RELEASE"))))
>
> Additionally, these are all the weird version strings I could find that
> are actually used as the commit:
> 1.2.2.rc2
> 12-068oasis4
> 1.2-2
> 0.F-2
> 0.2.0-alpha
> 5.1.0-b2
> 1.5-11
> 1.9.14-20210407
> 0.9.3b
> 2.8-fix-2
> 2020-05-19
> 10-11.0.0
> 1.0.12-2
> 0.9.3+16.04.20160218-0ubuntu1
> 2.12.c
> 1.1+11
> 5.1+4.06.0
> 4.2-411
> 1.4.0rc1-450-g2725ef99d
> 2.0.0-alpha14
> 60.2.3-2
> 1.21c
> 1.0.0rc4
> 2.1.0b1
> 1.02r6
> 0.32-14-gcdfe14e
> 3.0.0a3
> 2.00a2.3
> 1.0beta.18
> 2.1b
> 2.7.8a
> 2.7.3a
> 1.1.alpha19
> 1.0.2-rc4
> 0.5.3+git20200502
> 1.0.3-rc3
> 4.0.0.dev8
> 3.0.0beta1-24-g024cc9fa2
> 0.16-2-ge145396
> 2.0b6
> 2.0M10
> 1.0.7+0
> 1.16.0+5
> 0.4.0+1
> 2.2.10+0
> 4.3.1+2
> 2.13.93+0
> 2.10.4+0
> 1.0.5+5
> 0.21.0+0
> 3.3.4+0
> 2.68.1+0
> 0.10.1+1
> 6.9.10-12+3
> 2.0.1+2
> 3.100.0+1
> 0.14.0+2
> 0.1.6+2
> 3.3.0+0
> 1.8.7+0
> 1.3.0+2
> 1.42.0+0
> 1.16.1+0
> 2.35.0+0
> 1.6.37+5
> 4.1.0+1
> 2.36.0+0
> 1.3.6+4
> 2.10.1+0
> 2.26.0+0
> 1.3.4+0
> 1.1.1+2
> 1.3.1+1
> 8.44.0+0
> 0.40.1+0
> 5.15.2+0
> 1.17.0+3
> 1.18.0+3
> 2020.7.14+0
> 3.0.0+1
> 0.9.1+3
> 2.9.12+0
> 0.1.0+2
> 1.6.9+2
> 1.0.9+3
> 1.13.0+2
> 1.2.0+3
> 1.1.3+3
> 1.3.4+2
> 5.0.3+3
> 1.7.10+3
> 1.1.4+3
> 1.1.0+3
> 1.5.2+3
> 0.9.10+3
> 0.4.0+1
> 0.4.0+1
> 0.4.0+1
> 0.3.9+1
> 0.4.1+1
> 1.4.2+3
> 2.27.0+3
> 1.4.0+2
> 1.1.34+0
> 1.2.12+1
> 1.5.0+0
> 2021-06-07
> 1.2.2-5-g20dc8ed
> 2.1-20201229
> 3.0-rc1
> 1.32.0-0
> 20200701.154658.b0d6223
> 3.9-0
> R63-10032.B
> 0.58.2.a
> 0.0.0+git20200527
> 3.028R
> 3.001R
> 2.57b
> 1.0.0-20201130134442-10cb98267c6c
> 0.0.0-20161123171359-e6a2ba005892
> 0.0.0-20210615171337-6886f2dfbf5b
> 2020-11-10
> 5.2.0-alpha
> 2021-01-01
> 2021-02-28
> 2020-11-01
> 4.4-git.1
> 1.217-2
> 3.3.06-1
> 0.2.0-alpha-199-g3e7a475
> 1.0.0-beta.0
> 1.20190621-4
> 1.9.0-147-g61edec1ef
> 0.0.9.4f
>
> Yep, it looks like the above would work for the majority of these.
> That's probably Good Enough^tm.

Any trick you used to find all of there weird version numbers?  :-)

>>>> +  (define no-delims
>>>> +    (if (string-match delim-regexp no-suffix)
>>>> +        (string-split* no-suffix delim*)
>>>> +        (git-tag-error 'tag-version-delimiter)))
>>>
>>> This throws an error if the version doesn't have any delimiter.
>>
>> Setting the =E2=80=98tag-version-delimiter=E2=80=99 prefix to an empty s=
tring would
>> solve this, right?  Or, maybe we should just get rid of the delimiter
>> thing since only a few packages use a different delimiter.
>
> IMO, just get rid of the delimiter.  If we wanted to be *that* flexible,
> we could make it so they provide a tag->version proc instead of (prefix,
> suffix, delimiter).

a =E2=80=98tag->version=E2=80=99 procedure would probably make things a bit=
 too
complicated for the people writing package definitions.  For example,
having a delimiter would make it easy to match a tag like
=E2=80=9C2021-01-01-release=E2=80=9D

Delimiter is =E2=80=9C.=E2=80=9D (sorry if this hurts your eyes ;-))

scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^\\.[:punct:=
]]+(\\.[^\\.[:punct:]]+)*).*$" "2021-01-01-release") 1)
$28 =3D "2021"

Delimiter is =E2=80=9C-=E2=80=9D

scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^-[:punct:]]=
+(-[^-[:punct:]]+)*).*$" "2021-01-01-release") 1)
$29 =3D "2021-01-01-release"

And then, setting the suffix to =E2=80=9C-release=E2=80=9D would match just=
 the version
part.


On Sun, Sep 05 2021, Sarah Morgensen wrote (again):

>>> And this lets us just do something like (untested):
>>>
>>> (define* (get-version tag #:key prefix suffix delim)
>>>   (define delim-rx (regexp-quote (or delim ".")))
>>>   (define prefix-rx (or prefix "[^[:digit:]]*"))
>>>   (define suffix-rx (or suffix ".*"))
>>>   (define version-char-rx
>>>    (string-append "[^" delim-rx "[:punct:]]"))
>>>
>>>   (define tag-rx
>>>    (string-append "^" prefix "(" version-char-rx "+("
>>>                   delim-rx version-char-rx ")*)" suffix-rx "$"))
>>
>> This wouldn=E2=80=99t match anything if the version is just a plain numb=
er,
>> e.g., 1 or 09.
>
> It does,

Oh, I missed the extra pair of parens, sorry about that.

> but I had many errors in the definition.  Again, apologies.  I
> shouldn't send emails that late, haha.  This method should read:
>
>  --8<---------------cut here---------------start------------->8---
>  (define* (get-version tag #:key prefix suffix delim)
>    (define delim-rx (regexp-quote (or delim ".")))
>    (define prefix-rx (or prefix "[^[:digit:]]*"))
>    (define suffix-rx (or suffix ".*"))
>    (define version-char-rx
>     (string-append "[^" delim-rx "[:punct:]]"))
>
>    (define tag-rx
>     (string-append "^" prefix-rx "(" version-char-rx "+("
>                    delim-rx version-char-rx "+)*)" suffix-rx "$"))
>    (and=3D> (string-match tag-rx tag)
>           (cut match:substring <> 1)))
> --8<---------------cut here---------------end--------------->8---
>
>>
>> With this, something like =E2=80=9C1.4.0rc1-450-g2725ef99d=E2=80=9D will=
 result in
>> =E2=80=9C1.4.0=E2=80=9D being returned, which is incorrect.  Changing (c=
ut
>> match:substring <> 1) to just =E2=80=98match:substring=E2=80=99 would so=
lve the issue,
>> but then pre-release tags, which we usually don=E2=80=99t want,  would a=
lso get
>> matched.  Not sure what the best option would be in this case.
>>
>
> With the fixed method above:
>
> scheme@(emacs-guix)> (get-version "8")
> $16 =3D "8"
> scheme@(emacs-guix)> (get-version "1.4.0rc1-450-g2725ef99d")
> $17 =3D "1.4.0rc1"
>
> But, we still get:
>
> scheme@(emacs-guix)> (get-version "1.4.0-rc1")
> $18 =3D "1.4.0"
>
> which leads us to what you talked about in your other message.

Hmm, maybe we could check that if (string-append "1.4.0" suffix) is a
suffix on of the tag, or is this too much overhead?  This would only
work if we set the suffix to the empty string by default.

Since (string-match "1.4.0$" "1.4.0-rc1") returns #f, this tag would get
filtered out.  But then a suffix would have to be specified to match
"1.4.0rc1-450-g2725ef99d".

Not sure what the best option is here.

>>> +(define* (ls-remote-refs url #:key tags?)
>>> +  "Return the list of references advertised at Git repository URL.  If=
 TAGS?
>>> +is true, limit to only refs/tags."
>>> +  (define (ref? ref)
>>> +    ;; Like `git ls-remote --refs', only show actual references.
>>> +    (and (string-prefix? "refs/" ref)
>>> +         (not (string-suffix? "^{}" ref))))
>>> +
>>> +  (define (tag? ref)
>>> +    (string-prefix? "refs/tags/" ref))
>>> +
>>> +  (define (include? ref)
>>> +    (and ref?
>
> This should be:
>         (and (ref? ref)

Ah, problem solved!  :-)

>>> +         (or (not tags?) (tag? ref))))
>>> +
>>> +  (with-libgit2
>>> +   (with-temporary-directory
>>> +    (lambda (cache-directory)
>>> +      (let* ((repository (repository-init cache-directory))
>>> +             ;; Create an in-memory remote so we don't touch disk.
>>> +             (remote (remote-create-anonymous repository url)))
>>> +        (remote-connect remote)
>>> +        (remote-disconnect remote)
>>> +        (repository-close! repository)
>>> +
>>> +        (filter include? (map remote-head-name (remote-ls remote))))))=
))
>>>
>>
>> For some reason it seems to include refs that do and don=E2=80=99t end w=
ith
>> =E2=80=9C^{}=E2=80=9D
>
> Sorry, another typo I missed.  See above.

No worries, thanks for taking the time to review this!

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE2B5IVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5BgIQAIZHtTauloSjYVC1T6cs8epK0r8B
gUw3Dk3ZdBbjd9TfpfjsUnaGN2o6KwBKTnpj5atLPiM2Vo5an8BLmSOTlJQ9+O8p
DBiKYY0kKdknJYhboSIKj/ejCRIkdLQy6Wj6ucWmYmxipqk+muBO0c5qoMsIYHFp
iLUH0rrT7R0PMkBOSGrGgLi5Y4RRBlM9O4QqMp4RZ5Vw+brn6o3Z/+ja9/wlPSl9
md9W0g3D8d7eTmu/ATW71GDaI3UndGOUeCW4X3uTV0SqPUkyFuLKFFsqiQMxZHe1
iQ+uu1vj6shpxa+tXvpzqT3CdPDRzU8UyCtPNCUc8TaoLLj+SRlsdGv/xe1DSRYa
lZhX1kxkUdJxpAjKuyU4i9N2b0Wftbmsfg4PATQJgP/XbY9RiSp6pXNY0uKNIhLF
CBEoC5cq3MDjmLMswaipdlcVG4a5gxzhGI37aMBpE/of9hCDlY+rxth02m4nr4Rf
6SMeXcwYlcLNTU8FRw90d6aE+8NJxH4XR5VOW/CvZw6aArzYE3lwUvyYmFD78xfu
9LfehCH0DR3q98GCabyswVxpMrNdv8Z+wVNlpF+kDnCDzcD9vIhjKU6RTkzjp1UN
zc5hQv4VjEqTs2hVSjf04X9L0s6gO2TRjpFJtquPq+GPqnozLJBuzIt+v0MRj/AH
DF2EcH0wmD9yKpdI
=l3fr
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Tue, 07 Sep 2021 01:01:02 +0000
Resent-Message-ID: <handler.50359.B50359.163097646117463 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163097646117463
          (code B ref 50359); Tue, 07 Sep 2021 01:01:02 +0000
Received: (at 50359) by debbugs.gnu.org; 7 Sep 2021 01:01:01 +0000
Received: from localhost ([127.0.0.1]:54822 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mNPTw-0004XY-33
	for submit <at> debbugs.gnu.org; Mon, 06 Sep 2021 21:01:00 -0400
Received: from out2.migadu.com ([188.165.223.204]:62202)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mNPTs-0004XL-RF
 for 50359 <at> debbugs.gnu.org; Mon, 06 Sep 2021 21:00:58 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1630976455;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=g9fElk3LvhkfJh78D55tZ1OXS453jmAETo9WcoQbHpk=;
 b=ktJw6RDJpOrKXdxxSvG+IYg6UFk76jZUmg2MEy1uc6S/6qqaZb/GRz0q6iA8IVlL/QyqsS
 nRDsxV5rBwjfHyFKordYG21MVSKF4TS56QrlSToI2xfLcKbTcZuffIIcyX5mwcjnMIzyiW
 xtjygFP2c0riO009Hvzg6Cj4hNHkglo=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
 <86y28ai7ns.fsf@HIDDEN> <87y28928vh.fsf@HIDDEN>
Date: Mon, 06 Sep 2021 18:00:52 -0700
In-Reply-To: <87y28928vh.fsf@HIDDEN> (Xinglu Chen's message of "Mon, 06
 Sep 2021 14:20:34 +0200 (5 hours, 48 minutes ago)")
Message-ID: <86pmtli4hn.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hi, Xinglu Chen <public@HIDDEN> writes: > Any trick
 you used to find all of there weird version numbers? :-) 
 Content analysis details:   (2.0 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [188.165.223.204 listed in wl.mailspike.net]
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi,

Xinglu Chen <public@HIDDEN> writes:

> Any trick you used to find all of there weird version numbers?  :-)

This monstrosity:

  rg -U -B4 --pcre2 '(?!\(let.*(\n.*){0,1})\(version "([^\n"]*[^0-9\.][^"\n=
]*)".*(\n.*){0,10}commit.*version' gnu/packages

and to show just the versions:

  rg -Uor '$2' --pcre2 --no-filename --no-line-number=20

>> IMO, just get rid of the delimiter.  If we wanted to be *that* flexible,
>> we could make it so they provide a tag->version proc instead of (prefix,
>> suffix, delimiter).
>
> a =E2=80=98tag->version=E2=80=99 procedure would probably make things a b=
it too
> complicated for the people writing package definitions.  For example,
> having a delimiter would make it easy to match a tag like
> =E2=80=9C2021-01-01-release=E2=80=9D
>
> Delimiter is =E2=80=9C.=E2=80=9D (sorry if this hurts your eyes ;-))
>
> scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^\\.[:punc=
t:]]+(\\.[^\\.[:punct:]]+)*).*$" "2021-01-01-release") 1)
> $28 =3D "2021"
>
> Delimiter is =E2=80=9C-=E2=80=9D
>
> scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^-[:punct:=
]]+(-[^-[:punct:]]+)*).*$" "2021-01-01-release") 1)
> $29 =3D "2021-01-01-release"
>
> And then, setting the suffix to =E2=80=9C-release=E2=80=9D would match ju=
st the version
> part.

Right.  I missed that.

In that vein, should we keep the dashes in "2021-01-01" or convert them
to periods?  What about when a tag has underscores?

What if a repo has tags in both formats?  Then "3.0.1" would be
considered older than "2011-01-01".

Maybe we should just add an extra bit to detect a date format and only
consider it when there's no "proper versions"?

Aaaand I fell down a rabbit hole after that :) I've attached a patch
with what I've done.  It still has lots of issues--it requires the tag
to contain at least one version delimiter, it requires the first
character of the version to be a number... it might not even be better
than before I touched it, and even so the added complexity might not be
worth it.  But if you'd like to take it for a spin, I've attached it (it
applies straight on master).

--
Sarah


--=-=-=
Content-Type: text/x-patch; charset=utf-8
Content-Disposition: inline;
 filename=0001-import-Add-generic-git-updater.patch
Content-Transfer-Encoding: quoted-printable
Content-Description: the rabbit hole

From 08bd59a7fa1aa9735a1794672ce8d1f683d3d6db Mon Sep 17 00:00:00 2001
Message-Id: <08bd59a7fa1aa9735a1794672ce8d1f683d3d6db.1630975873.git.iskari=
an@HIDDEN>
From: Xinglu Chen <public@HIDDEN>
Date: Fri, 3 Sep 2021 17:50:56 +0200
Subject: [PATCH] import: Add 'generic-git' updater.

* guix/import/git.scm: New file.
* doc/guix.texi (Invoking guix refresh): Document it.
* Makefile.am (MODULES): Register it.
---
 Makefile.am         |   1 +
 doc/guix.texi       |  27 +++++++
 guix/git.scm        |  33 ++++++++
 guix/import/git.scm | 191 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 252 insertions(+)
 create mode 100644 guix/import/git.scm

diff --git a/Makefile.am b/Makefile.am
index 3c79760734..c4d3a456b1 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -254,6 +254,7 @@ MODULES =3D					\
   guix/import/egg.scm   			\
   guix/import/elpa.scm   			\
   guix/import/gem.scm				\
+  guix/import/git.scm                           \
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/gnu.scm				\
diff --git a/doc/guix.texi b/doc/guix.texi
index 36a0c7f5ec..26afb1607a 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, Launc=
hpad} packages.
 @item generic-html
 a generic updater that crawls the HTML page where the source tarball of
 the package is hosted, when applicable.
+@item generic-git
+a generic updater for packages hosted on Git repositories.  It tries to
+be smart about parsing Git tag names, but if it is not able to parse the
+tag name and compare tags correctly, users can define the following
+properties for a package.
+
+@itemize
+@item @code{tag-prefix}: a regular expression for matching a prefix of
+the tag name.
+
+@item @code{tag-suffix}: a regular expression for matching a suffix of
+the tag name.
+
+@item @code{tag-version-delimiter}: a string used as the delimiter in
+the tag name for separating the numbers of the version.
+@end itemize
+
+@lisp
+(package
+  (name "foo")
+  ;; ...
+  (properties
+    '((tag-prefix . "^release0-")
+      (tag-suffix . "[a-z]?$")
+      (tag-version-delimiter . ":"))))
+@end lisp=20=20=20=20=20=20
+
 @end table
=20
 For instance, the following command only checks for updates of Emacs
diff --git a/guix/git.scm b/guix/git.scm
index 9c6f326c36..c5d0d2da8e 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -56,6 +56,8 @@
             commit-difference
             commit-relation
=20
+            ls-remote-refs
+
             git-checkout
             git-checkout?
             git-checkout-url
@@ -556,6 +558,37 @@ objects: 'ancestor (meaning that OLD is an ancestor of=
 NEW), 'descendant, or
               (if (set-contains? oldest new)
                   'descendant
                   'unrelated))))))
+
+;;
+;;; Remote operations.
+;;;
+
+(define* (ls-remote-refs url #:key tags?)
+  "Return the list of references advertised at Git repository URL.  If TAG=
S?
+is true, limit to only refs/tags."
+  (define (ref? ref)
+    ;; Like `git ls-remote --refs', only show actual references.
+    (and (string-prefix? "refs/" ref)
+         (not (string-suffix? "^{}" ref))))
+
+  (define (tag? ref)
+    (string-prefix? "refs/tags/" ref))
+
+  (define (include? ref)
+    (and (ref? ref)
+         (or (not tags?) (tag? ref))))
+
+  (with-libgit2
+   (call-with-temporary-directory
+    (lambda (cache-directory)
+      (let* ((repository (repository-init cache-directory))
+             ;; Create an in-memory remote so we don't touch disk.
+             (remote (remote-create-anonymous repository url)))
+        (remote-connect remote)
+        (remote-disconnect remote)
+        (repository-close! repository)
+
+        (filter include? (map remote-head-name (remote-ls remote))))))))
=20
 
 ;;;
diff --git a/guix/import/git.scm b/guix/import/git.scm
new file mode 100644
index 0000000000..8568981af2
--- /dev/null
+++ b/guix/import/git.scm
@@ -0,0 +1,191 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN>
+;;; Copyright =C2=A9 2021 Sarah Morgensen <iskarian@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import git)
+  #:use-module (guix build utils)
+  #:use-module (guix diagnostics)
+  #:use-module (guix git)
+  #:use-module (guix git-download)
+  #:use-module (guix i18n)
+  #:use-module (guix packages)
+  #:use-module (guix upstream)
+  #:use-module (guix utils)
+  #:use-module (ice-9 format)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 rdelim)
+  #:use-module (ice-9 regex)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
+  #:use-module (srfi srfi-71)
+  #:export (%generic-git-updater))
+
+;;; Commentary:
+;;;
+;;; This module provides a generic package updater for packages hosted on =
Git
+;;; repositories.
+;;;
+;;; It tries to be smart about tag names, but if it is not automatically a=
ble
+;;; to parse the tag names correctly, users can set the `tag-prefix',
+;;; `tag-suffix' and `tag-version-delimiter' properties of the package to =
make
+;;; the updater parse the Git tag name correctly.
+;;;
+;;; Code:
+
+;;; Errors & warnings
+
+(define-condition-type &git-no-valid-tags-error &error
+  git-no-valid-tags-error?)
+
+(define (git-no-valid-tags-error)
+  (raise (condition (&message (message "no valid tags found"))
+                    (&git-no-valid-tags-error))))
+
+(define-condition-type &git-no-tags-error &error
+  git-no-tags-error?)
+
+(define (git-no-tags-error)
+  (raise (condition (&message (message "no tags were found"))
+                    (&git-no-tags-error))))
+
+
+;;; Updater
+
+(define* (get-version-mapping tags #:key prefix suffix delim)
+  (define (guess-delim)
+    (let ((total (length tags))
+          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
+          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
+          (underscores (reduce + 0 (map (cut string-count <> #\_) tags))))
+      ;; (format #t "total: ~d, dots: ~d, dashes ~d, underscores ~d~%"
+      ;;         total dots dashes underscores)
+      (cond
+       ((> dots (* total 0.35)) ".")
+       ((> dashes (* total 0.8)) "-")
+       ((> underscores (* total 0.8)) "_")
+       (else "."))))
+
+  (define delim-rx (regexp-quote (or delim (guess-delim))))
+  (define suffix-rx (or suffix ""))
+
+  (define prefix-rx
+    (make-regexp (string-append "^" (or prefix ".*") "$")))
+  (define tag-rx
+    (make-regexp
+     (string-append "([[:digit:]][^" delim-rx "[:punct:]]*"
+                    "(" delim-rx "[^" delim-rx "]+)+"
+                    ")" suffix-rx "$")))
+
+  (define (get-version tag)
+    (let ((tag-match (regexp-exec tag-rx tag)))
+      (and tag-match
+           (regexp-exec prefix-rx (match:prefix tag-match))
+           (regexp-substitute/global
+            #f delim-rx (match:substring tag-match 1)
+            'pre "." 'post))))
+
+  (define (entry<? a b)
+    (eq? (version-compare (car a) (car b)) '<))
+
+  (let ((mapping (fold alist-cons '() (map get-version tags) tags)))
+    (stable-sort! (filter car mapping) entry<?)))
+
+(define* (get-latest-tag url #:key prefix suffix delim)
+  "Return the latest tag available from the Git repository at URL."
+  (define (pre-release? tag)
+    (any (cut string-contains tag <>)
+         '("alpha" "beta" "rc" "dev" "test")))
+
+  (let* ((tags (map (cut string-drop <> (string-length "refs/tags/"))
+                    (ls-remote-refs url #:tags? #t)))
+         (versions->tags
+          (get-version-mapping (filter (negate pre-release?) tags)
+                               #:prefix prefix
+                               #:suffix suffix
+                               #:delim delim)))
+    (cond
+     ((null? tags)
+      (git-no-tags-error))
+     ((null? versions->tags)
+      (git-no-valid-tags-error))
+     (else
+      (match (last versions->tags)
+        ((version . tag)
+         (values version tag)))))))
+
+(define (latest-git-tag-version package tag-prefix tag-suffix
+                                tag-version-delimiter)
+  "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, and TAG-VERSION-DELIMITER
+properties of PACKAGE, returns the latest version of PACKAGE."
+  (guard (c ((or (git-no-tags-error? c) (git-no-valid-tags-error? c))
+             (warning (or (package-field-location package 'source)
+                          (package-location package))
+                      (G_ "~a for ~a~%")
+                      (condition-message c)
+                      (package-name package))
+             #f)
+            ((eq? (exception-kind c) 'git-error)
+             (warning (or (package-field-location package 'source)
+                          (package-location package))
+                      (G_ "failed to fetch Git repository for ~a~%")
+                      (package-name package))
+             #f))
+    (let* ((source (package-source package))
+           (url (git-reference-url (origin-uri source))))
+      ;;(format #t "~a~%" (package-name package))
+      (get-latest-tag url #:prefix tag-prefix #:suffix tag-suffix
+                      #:delim tag-version-delimiter))))
+
+(define (git-package? package)
+  "Whether the origin of PACKAGE is a Git repostiory."
+  (match (package-source package)
+    ((? origin? origin)
+     (and (eq? (origin-method origin) git-fetch)
+          (git-reference? (origin-uri origin))))
+    (_ #f)))
+
+(define (latest-git-release package)
+  "Return the latest release of PACKAGE."
+  (let* ((name (package-name package))
+         (properties (package-properties package))
+         (tag-prefix (assq-ref properties 'tag-prefix))
+         (tag-suffix (assq-ref properties 'tag-suffix))
+         (tag-version-delimiter (assq-ref properties 'tag-version-delimite=
r))
+         (old-version (package-version package))
+         (url (git-reference-url (origin-uri (package-source package))))
+         (new-version (latest-git-tag-version package
+                                              tag-prefix
+                                              tag-suffix
+                                              tag-version-delimiter)))
+
+    (if new-version
+        (upstream-source
+         (package name)
+         (version new-version)
+         (urls (list url)))
+        ;; No new release or no tags available.
+        #f)))
+
+(define %generic-git-updater
+  (upstream-updater
+   (name 'generic-git)
+   (description "Updater for packages hosted on Git repositories")
+   (pred git-package?)
+   (latest latest-git-release)))

base-commit: 522a3bf99cbc21a9093f63280b9508cd69b94ff0
--=20
2.32.0


--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Tue, 07 Sep 2021 19:14:02 +0000
Resent-Message-ID: <handler.50359.B50359.163104201820539 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Sarah Morgensen <iskarian@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163104201820539
          (code B ref 50359); Tue, 07 Sep 2021 19:14:02 +0000
Received: (at 50359) by debbugs.gnu.org; 7 Sep 2021 19:13:38 +0000
Received: from localhost ([127.0.0.1]:57771 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mNgXJ-0005LB-6p
	for submit <at> debbugs.gnu.org; Tue, 07 Sep 2021 15:13:38 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:51562
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mNgXG-0005Kt-4E
 for 50359 <at> debbugs.gnu.org; Tue, 07 Sep 2021 15:13:35 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631042005;
 bh=DLb7xkBOXXKVJSPIRcO86Zhb4/KB0Iu8cDZSok1Pkhk=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=i9smTY8/QkOYWKejtO5vC8Fy24djHjLWQYhEKwznR3WHQZ3AwM/8KLYNMJErr2fr+
 Ky+4vIx/iRGF9oInQj9zfPgyH8LPq/DqUsIgPqYvEAuLgyiv+Lo+zRVmvSGy+bEVYB
 1RzSz3b7JCBn5Mf5yVAZtmUXXEXm9m+8k7qovVW8=
In-Reply-To: <86pmtli4hn.fsf@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
 <86y28ai7ns.fsf@HIDDEN> <87y28928vh.fsf@HIDDEN>
 <86pmtli4hn.fsf@HIDDEN>
Date: Tue, 07 Sep 2021 21:13:22 +0200
Message-ID: <87wnnsyzal.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  On Mon, Sep 06 2021, Sarah Morgensen wrote: > Hi, > > Xinglu
 Chen <public@HIDDEN> writes: > >> Any trick you used to find all of
 there weird version numbers? :-) > > This monstrosity: > > rg -U -B4 --pcre2
 '(?!\(let.*(\n.*){0,1})\(versio [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Mon, Sep 06 2021, Sarah Morgensen wrote: > Hi, > > Xinglu
    Chen <public@HIDDEN> writes: > >> Any trick you used to find all of
    there weird version numbers? :-) > > This monstrosity: > > rg -U -B4 --pcre2
    '(?!\(let.*(\n.*){0,1})\(versio [...] 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Mon, Sep 06 2021, Sarah Morgensen wrote:

> Hi,
>
> Xinglu Chen <public@HIDDEN> writes:
>
>> Any trick you used to find all of there weird version numbers?  :-)
>
> This monstrosity:
>
>   rg -U -B4 --pcre2 '(?!\(let.*(\n.*){0,1})\(version "([^\n"]*[^0-9\.][^"=
\n]*)".*(\n.*){0,10}commit.*version' gnu/packages
>
> and to show just the versions:
>
>   rg -Uor '$2' --pcre2 --no-filename --no-line-number=20

Wow!  I will try that and see for myself!  :-)

>>> IMO, just get rid of the delimiter.  If we wanted to be *that* flexible,
>>> we could make it so they provide a tag->version proc instead of (prefix,
>>> suffix, delimiter).
>>
>> a =E2=80=98tag->version=E2=80=99 procedure would probably make things a =
bit too
>> complicated for the people writing package definitions.  For example,
>> having a delimiter would make it easy to match a tag like
>> =E2=80=9C2021-01-01-release=E2=80=9D
>>
>> Delimiter is =E2=80=9C.=E2=80=9D (sorry if this hurts your eyes ;-))
>>
>> scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^\\.[:pun=
ct:]]+(\\.[^\\.[:punct:]]+)*).*$" "2021-01-01-release") 1)
>> $28 =3D "2021"
>>
>> Delimiter is =E2=80=9C-=E2=80=9D
>>
>> scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^-[:punct=
:]]+(-[^-[:punct:]]+)*).*$" "2021-01-01-release") 1)
>> $29 =3D "2021-01-01-release"
>>
>> And then, setting the suffix to =E2=80=9C-release=E2=80=9D would match j=
ust the version
>> part.
>
> Right.  I missed that.
>
> In that vein, should we keep the dashes in "2021-01-01" or convert them
> to periods?

Having periods would be more consistent, then could have a
=E2=80=98date->version=E2=80=99 procedure that replaces the hyphens with do=
ts and have

  (git-reference
    (url "https://git.example.org")
    (commit (date->version version)))

> What about when a tag has underscores?

Hmm, not sure about that, below is a list of packages I could find which had
underscores as delimiters

gnu/packages/graphics.scm
239:          (commit "DIRECTFB_1_7_7")))
gnu/packages/gstreamer.scm
326:          (commit "ESOUND_0_2_41")))
gnu/packages/java.scm
13925:                      (commit "jboss-transaction-api_1.2_spec-1.1.1.F=
inal")))

They all seem to use periods in the =E2=80=98version=E2=80=99 field, though=
, so I would
say that the underscroes, should also be converted to periods.

> What if a repo has tags in both formats?  Then "3.0.1" would be
> considered older than "2011-01-01".

That=E2=80=99s tricky, there isn=E2=80=99t really a way to know how old =E2=
=80=9C3.0.1=E2=80=9D is,
without looking at the metadata of the tag.  Maybe this is one of those
corner cases which can=E2=80=99t really automatically determine the latest
release.  Should we have a =E2=80=98no-refresh?=E2=80=99 property to tell t=
he refresh to
not try to update the package?

> Maybe we should just add an extra bit to detect a date format and only
> consider it when there's no "proper versions"?

That could be a good idea!

> Aaaand I fell down a rabbit hole after that :) I've attached a patch
> with what I've done.  It still has lots of issues--it requires the tag
> to contain at least one version delimiter, it requires the first
> character of the version to be a number... it might not even be better
> than before I touched it, and even so the added complexity might not be
> worth it.  But if you'd like to take it for a spin, I've attached it (it
> applies straight on master).

Great! I will try it out and see how it compares to my current WIP
version.

Not having characters in the first version number probably isn=E2=80=99t su=
ch a
big deal, most version that contain characters end with a character.
E.g., =E2=80=9C1.2.3a=E2=80=9D is not to uncommon, but =E2=80=9Ca1.2.3=E2=
=80=9D is rarely seen.

> --
> Sarah
>
> From 08bd59a7fa1aa9735a1794672ce8d1f683d3d6db Mon Sep 17 00:00:00 2001
> Message-Id: <08bd59a7fa1aa9735a1794672ce8d1f683d3d6db.1630975873.git.iska=
rian@HIDDEN>
> From: Xinglu Chen <public@HIDDEN>
> Date: Fri, 3 Sep 2021 17:50:56 +0200
> Subject: [PATCH] import: Add 'generic-git' updater.
>
> * guix/import/git.scm: New file.
> * doc/guix.texi (Invoking guix refresh): Document it.
> * Makefile.am (MODULES): Register it.
> ---
>  Makefile.am         |   1 +
>  doc/guix.texi       |  27 +++++++
>  guix/git.scm        |  33 ++++++++
>  guix/import/git.scm | 191 ++++++++++++++++++++++++++++++++++++++++++++
>  4 files changed, 252 insertions(+)
>  create mode 100644 guix/import/git.scm
>
> diff --git a/Makefile.am b/Makefile.am
> index 3c79760734..c4d3a456b1 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -254,6 +254,7 @@ MODULES =3D					\
>    guix/import/egg.scm   			\
>    guix/import/elpa.scm   			\
>    guix/import/gem.scm				\
> +  guix/import/git.scm                           \
>    guix/import/github.scm   			\
>    guix/import/gnome.scm				\
>    guix/import/gnu.scm				\
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 36a0c7f5ec..26afb1607a 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, Lau=
nchpad} packages.
>  @item generic-html
>  a generic updater that crawls the HTML page where the source tarball of
>  the package is hosted, when applicable.
> +@item generic-git
> +a generic updater for packages hosted on Git repositories.  It tries to
> +be smart about parsing Git tag names, but if it is not able to parse the
> +tag name and compare tags correctly, users can define the following
> +properties for a package.
> +
> +@itemize
> +@item @code{tag-prefix}: a regular expression for matching a prefix of
> +the tag name.
> +
> +@item @code{tag-suffix}: a regular expression for matching a suffix of
> +the tag name.
> +
> +@item @code{tag-version-delimiter}: a string used as the delimiter in
> +the tag name for separating the numbers of the version.
> +@end itemize
> +
> +@lisp
> +(package
> +  (name "foo")
> +  ;; ...
> +  (properties
> +    '((tag-prefix . "^release0-")
> +      (tag-suffix . "[a-z]?$")
> +      (tag-version-delimiter . ":"))))
> +@end lisp=20=20=20=20=20=20
> +
>  @end table
>=20=20
>  For instance, the following command only checks for updates of Emacs
> diff --git a/guix/git.scm b/guix/git.scm
> index 9c6f326c36..c5d0d2da8e 100644
> --- a/guix/git.scm
> +++ b/guix/git.scm
> @@ -56,6 +56,8 @@
>              commit-difference
>              commit-relation
>=20=20
> +            ls-remote-refs
> +
>              git-checkout
>              git-checkout?
>              git-checkout-url
> @@ -556,6 +558,37 @@ objects: 'ancestor (meaning that OLD is an ancestor =
of NEW), 'descendant, or
>                (if (set-contains? oldest new)
>                    'descendant
>                    'unrelated))))))
> +
> +;;
> +;;; Remote operations.
> +;;;
> +
> +(define* (ls-remote-refs url #:key tags?)
> +  "Return the list of references advertised at Git repository URL.  If T=
AGS?
> +is true, limit to only refs/tags."
> +  (define (ref? ref)
> +    ;; Like `git ls-remote --refs', only show actual references.
> +    (and (string-prefix? "refs/" ref)
> +         (not (string-suffix? "^{}" ref))))
> +
> +  (define (tag? ref)
> +    (string-prefix? "refs/tags/" ref))
> +
> +  (define (include? ref)
> +    (and (ref? ref)
> +         (or (not tags?) (tag? ref))))
> +
> +  (with-libgit2
> +   (call-with-temporary-directory
> +    (lambda (cache-directory)
> +      (let* ((repository (repository-init cache-directory))
> +             ;; Create an in-memory remote so we don't touch disk.
> +             (remote (remote-create-anonymous repository url)))
> +        (remote-connect remote)
> +        (remote-disconnect remote)
> +        (repository-close! repository)
> +
> +        (filter include? (map remote-head-name (remote-ls remote))))))))
>=20=20
>  
>  ;;;
> diff --git a/guix/import/git.scm b/guix/import/git.scm
> new file mode 100644
> index 0000000000..8568981af2
> --- /dev/null
> +++ b/guix/import/git.scm
> @@ -0,0 +1,191 @@
> +;;; GNU Guix --- Functional package management for GNU
> +;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN>
> +;;; Copyright =C2=A9 2021 Sarah Morgensen <iskarian@HIDDEN>
> +;;;
> +;;; This file is part of GNU Guix.
> +;;;
> +;;; GNU Guix is free software; you can redistribute it and/or modify it
> +;;; under the terms of the GNU General Public License as published by
> +;;; the Free Software Foundation; either version 3 of the License, or (at
> +;;; your option) any later version.
> +;;;
> +;;; GNU Guix is distributed in the hope that it will be useful, but
> +;;; WITHOUT ANY WARRANTY; without even the implied warranty of
> +;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
> +;;; GNU General Public License for more details.
> +;;;
> +;;; You should have received a copy of the GNU General Public License
> +;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
> +
> +(define-module (guix import git)
> +  #:use-module (guix build utils)
> +  #:use-module (guix diagnostics)
> +  #:use-module (guix git)
> +  #:use-module (guix git-download)
> +  #:use-module (guix i18n)
> +  #:use-module (guix packages)
> +  #:use-module (guix upstream)
> +  #:use-module (guix utils)
> +  #:use-module (ice-9 format)
> +  #:use-module (ice-9 match)
> +  #:use-module (ice-9 rdelim)
> +  #:use-module (ice-9 regex)
> +  #:use-module (srfi srfi-1)
> +  #:use-module (srfi srfi-26)
> +  #:use-module (srfi srfi-34)
> +  #:use-module (srfi srfi-35)
> +  #:use-module (srfi srfi-71)
> +  #:export (%generic-git-updater))
> +
> +;;; Commentary:
> +;;;
> +;;; This module provides a generic package updater for packages hosted o=
n Git
> +;;; repositories.
> +;;;
> +;;; It tries to be smart about tag names, but if it is not automatically=
 able
> +;;; to parse the tag names correctly, users can set the `tag-prefix',
> +;;; `tag-suffix' and `tag-version-delimiter' properties of the package t=
o make
> +;;; the updater parse the Git tag name correctly.
> +;;;
> +;;; Code:
> +
> +;;; Errors & warnings
> +
> +(define-condition-type &git-no-valid-tags-error &error
> +  git-no-valid-tags-error?)
> +
> +(define (git-no-valid-tags-error)
> +  (raise (condition (&message (message "no valid tags found"))
> +                    (&git-no-valid-tags-error))))
> +
> +(define-condition-type &git-no-tags-error &error
> +  git-no-tags-error?)
> +
> +(define (git-no-tags-error)
> +  (raise (condition (&message (message "no tags were found"))
> +                    (&git-no-tags-error))))
> +
> +
> +;;; Updater
> +
> +(define* (get-version-mapping tags #:key prefix suffix delim)
> +  (define (guess-delim)
> +    (let ((total (length tags))
> +          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
> +          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
> +          (underscores (reduce + 0 (map (cut string-count <> #\_) tags))=
))
> +      ;; (format #t "total: ~d, dots: ~d, dashes ~d, underscores ~d~%"
> +      ;;         total dots dashes underscores)
> +      (cond
> +       ((> dots (* total 0.35)) ".")
> +       ((> dashes (* total 0.8)) "-")
> +       ((> underscores (* total 0.8)) "_")
> +       (else "."))))

These numbers seem rather arbitrary, how did you arrive at them?

Also, AFAICS, versions without delimiters won=E2=80=99t be matched.

> +  (define delim-rx (regexp-quote (or delim (guess-delim))))
> +  (define suffix-rx (or suffix ""))
> +
> +  (define prefix-rx
> +    (make-regexp (string-append "^" (or prefix ".*") "$")))

Why is there a $ here?

> +  (define tag-rx
> +    (make-regexp
> +     (string-append "([[:digit:]][^" delim-rx "[:punct:]]*"
> +                    "(" delim-rx "[^" delim-rx "]+)+"
> +                    ")" suffix-rx "$")))
> +
> +  (define (get-version tag)
> +    (let ((tag-match (regexp-exec tag-rx tag)))
> +      (and tag-match
> +           (regexp-exec prefix-rx (match:prefix tag-match))
> +           (regexp-substitute/global
> +            #f delim-rx (match:substring tag-match 1)
> +            'pre "." 'post))))
> +
> +  (define (entry<? a b)
> +    (eq? (version-compare (car a) (car b)) '<))
> +
> +  (let ((mapping (fold alist-cons '() (map get-version tags) tags)))
> +    (stable-sort! (filter car mapping) entry<?)))
> +
> +(define* (get-latest-tag url #:key prefix suffix delim)
> +  "Return the latest tag available from the Git repository at URL."
> +  (define (pre-release? tag)
> +    (any (cut string-contains tag <>)
> +         '("alpha" "beta" "rc" "dev" "test")))

This would only match lower-case characters, but sometimes upper-case
characters are used.

  (define (pre-release? tag)
    (any (lambda (rx) (regexp-exec (make-regexp rx regexp/icase) tag))
              '("alpha" "beta" "rc" "dev" "test")))
=20=20=20=20=20=20=20=20=20=20=20=20=20=20
> +  (let* ((tags (map (cut string-drop <> (string-length "refs/tags/"))
> +                    (ls-remote-refs url #:tags? #t)))
> +         (versions->tags
> +          (get-version-mapping (filter (negate pre-release?) tags)
> +                               #:prefix prefix
> +                               #:suffix suffix
> +                               #:delim delim)))
> +    (cond
> +     ((null? tags)
> +      (git-no-tags-error))
> +     ((null? versions->tags)
> +      (git-no-valid-tags-error))
> +     (else
> +      (match (last versions->tags)
> +        ((version . tag)
> +         (values version tag)))))))
> +
> +(define (latest-git-tag-version package tag-prefix tag-suffix
> +                                tag-version-delimiter)
> +  "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, and TAG-VERSION-DELIMITER
> +properties of PACKAGE, returns the latest version of PACKAGE."
> +  (guard (c ((or (git-no-tags-error? c) (git-no-valid-tags-error? c))
> +             (warning (or (package-field-location package 'source)
> +                          (package-location package))
> +                      (G_ "~a for ~a~%")
> +                      (condition-message c)
> +                      (package-name package))
> +             #f)
> +            ((eq? (exception-kind c) 'git-error)
> +             (warning (or (package-field-location package 'source)
> +                          (package-location package))
> +                      (G_ "failed to fetch Git repository for ~a~%")
> +                      (package-name package))
> +             #f))
> +    (let* ((source (package-source package))
> +           (url (git-reference-url (origin-uri source))))
> +      ;;(format #t "~a~%" (package-name package))
> +      (get-latest-tag url #:prefix tag-prefix #:suffix tag-suffix
> +                      #:delim tag-version-delimiter))))
> +
> +(define (git-package? package)
> +  "Whether the origin of PACKAGE is a Git repostiory."
> +  (match (package-source package)
> +    ((? origin? origin)
> +     (and (eq? (origin-method origin) git-fetch)
> +          (git-reference? (origin-uri origin))))
> +    (_ #f)))
> +
> +(define (latest-git-release package)
> +  "Return the latest release of PACKAGE."
> +  (let* ((name (package-name package))
> +         (properties (package-properties package))
> +         (tag-prefix (assq-ref properties 'tag-prefix))
> +         (tag-suffix (assq-ref properties 'tag-suffix))
> +         (tag-version-delimiter (assq-ref properties 'tag-version-delimi=
ter))
> +         (old-version (package-version package))
> +         (url (git-reference-url (origin-uri (package-source package))))
> +         (new-version (latest-git-tag-version package
> +                                              tag-prefix
> +                                              tag-suffix
> +                                              tag-version-delimiter)))
> +
> +    (if new-version
> +        (upstream-source
> +         (package name)
> +         (version new-version)
> +         (urls (list url)))
> +        ;; No new release or no tags available.
> +        #f)))
> +
> +(define %generic-git-updater
> +  (upstream-updater
> +   (name 'generic-git)
> +   (description "Updater for packages hosted on Git repositories")
> +   (pred git-package?)
> +   (latest latest-git-release)))
>
> base-commit: 522a3bf99cbc21a9093f63280b9508cd69b94ff0
> --=20
> 2.32.0

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE3udIVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x59rgP/i9zonrR51YoZ4Dw68xfhW66m2op
K7Xf9B45E+pAYKxo2R31HjO2B9CNYUXb5LBLAMvkqaH0KmP7Yv3LpBUpOm6EOmMh
gtgTJTBlLetvjxrQmcOHBd6n/vIFp74wlukV17vPrxtlrQVCM50IBYvPcVuq8GtU
eXQZhL3HYNc8K5ddKSRGwHNogZg3casFP3O2/x9J3wImFhIqA57IHan4n+mNAOUk
ddUMhiypYNTM74gYQZ4tqspJNyWD7l7YNF2CA+mpsn+Vadz+tCALffKZTO6f1mQT
ibuOV75ZbtuAcEkR+43o3dtWFamaQOuUZWLMBXFYU8iy91Suyqh3I1QazoKE0gXU
3hJO2x0+TdQV/bI8U/Hoj7nimXDbicJovFH9KCzOp2fPhcFxLCLwIMjc++BlMLx8
0TfkIZfHti962MlnYFgo0I0N/F0SiI6fFQBJ7L7i0nm3/el1wB23IIbjrNCQ9841
fLNw/Vz8QsO1xBNEFhrLhoFerzHLqmy+TggKVCzdlnlwPlrK5Y9cZZ1FIJq5LKli
sirHcyfi22YL7B19sTRhqZYBY1Gttucykn1ttuRZg56JhJvPoDunoluZyTtYXUQt
QkSpbjD/9VkyODCKgAjvm+3+e68uNlVePPCN5W/AroVPxw8oMjCeLakUTrKavM6w
vl+l3fGzkXs14GR0
=d2MH
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 08 Sep 2021 18:30:01 +0000
Resent-Message-ID: <handler.50359.B50359.163112574622357 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Sarah Morgensen <iskarian@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163112574622357
          (code B ref 50359); Wed, 08 Sep 2021 18:30:01 +0000
Received: (at 50359) by debbugs.gnu.org; 8 Sep 2021 18:29:06 +0000
Received: from localhost ([127.0.0.1]:33546 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mO2Jl-0005oW-RI
	for submit <at> debbugs.gnu.org; Wed, 08 Sep 2021 14:29:06 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:55900
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mO2Ji-0005ny-Su
 for 50359 <at> debbugs.gnu.org; Wed, 08 Sep 2021 14:29:04 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631125733;
 bh=2HyG349cNWVnzUPqUcYxl3f8KOxWIJNaD9SmLCyD2nY=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=jIpMh18dY+n2zCAWKPgSQqXNpv8mVairlZv2VcMCXstWNpQ6LxAU5GIHsQI7zXH1v
 y4btlj+tlIa4DEW4xwVl+0bCpXVwuAUSGzfcp/g25KCzaaremz4tSWwLIrbARhDjul
 iKdQ5COm1M6x2xHdiiDF7rTSoQ5DPVUF5HDehpL0=
In-Reply-To: <87wnnsyzal.fsf@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
 <86y28ai7ns.fsf@HIDDEN> <87y28928vh.fsf@HIDDEN>
 <86pmtli4hn.fsf@HIDDEN> <87wnnsyzal.fsf@HIDDEN>
Date: Wed, 08 Sep 2021 20:28:38 +0200
Message-ID: <87mtomzzu1.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  On Tue, Sep 07 2021, Xinglu Chen wrote: > On Mon, Sep 06 2021,
 Sarah Morgensen wrote: > >> Hi, >> >> Xinglu Chen <public@HIDDEN>
 writes: >> >>> Any trick you used to find all of there weird version numbers?
 :-) >> >> This monstrosity [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Tue, Sep 07 2021, Xinglu Chen wrote: > On Mon, Sep 06 2021,
    Sarah Morgensen wrote: > >> Hi, >> >> Xinglu Chen <public@HIDDEN>
   writes: >> >>> Any trick you used to find all of there weird version numbers?
    :-) >> >> This monstrosity [...] 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Tue, Sep 07 2021, Xinglu Chen wrote:

> On Mon, Sep 06 2021, Sarah Morgensen wrote:
>
>> Hi,
>>
>> Xinglu Chen <public@HIDDEN> writes:
>>
>>> Any trick you used to find all of there weird version numbers?  :-)
>>
>> This monstrosity:
>>
>>   rg -U -B4 --pcre2 '(?!\(let.*(\n.*){0,1})\(version "([^\n"]*[^0-9\.][^=
"\n]*)".*(\n.*){0,10}commit.*version' gnu/packages
>>
>> and to show just the versions:
>>
>>   rg -Uor '$2' --pcre2 --no-filename --no-line-number=20
>
> Wow!  I will try that and see for myself!  :-)
>
>>>> IMO, just get rid of the delimiter.  If we wanted to be *that* flexibl=
e,
>>>> we could make it so they provide a tag->version proc instead of (prefi=
x,
>>>> suffix, delimiter).
>>>
>>> a =E2=80=98tag->version=E2=80=99 procedure would probably make things a=
 bit too
>>> complicated for the people writing package definitions.  For example,
>>> having a delimiter would make it easy to match a tag like
>>> =E2=80=9C2021-01-01-release=E2=80=9D
>>>
>>> Delimiter is =E2=80=9C.=E2=80=9D (sorry if this hurts your eyes ;-))
>>>
>>> scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^\\.[:pu=
nct:]]+(\\.[^\\.[:punct:]]+)*).*$" "2021-01-01-release") 1)
>>> $28 =3D "2021"
>>>
>>> Delimiter is =E2=80=9C-=E2=80=9D
>>>
>>> scheme@(guile-user)> (match:substring (string-match "^[^0-9]*([^-[:punc=
t:]]+(-[^-[:punct:]]+)*).*$" "2021-01-01-release") 1)
>>> $29 =3D "2021-01-01-release"
>>>
>>> And then, setting the suffix to =E2=80=9C-release=E2=80=9D would match =
just the version
>>> part.
>>
>> Right.  I missed that.
>>
>> In that vein, should we keep the dashes in "2021-01-01" or convert them
>> to periods?
>
> Having periods would be more consistent, then could have a
> =E2=80=98date->version=E2=80=99 procedure that replaces the hyphens with =
dots and have
>
>   (git-reference
>     (url "https://git.example.org")
>     (commit (date->version version)))
>
>> What about when a tag has underscores?
>
> Hmm, not sure about that, below is a list of packages I could find which =
had
> underscores as delimiters
>
> gnu/packages/graphics.scm
> 239:          (commit "DIRECTFB_1_7_7")))
> gnu/packages/gstreamer.scm
> 326:          (commit "ESOUND_0_2_41")))
> gnu/packages/java.scm
> 13925:                      (commit "jboss-transaction-api_1.2_spec-1.1.1=
.Final")))
>
> They all seem to use periods in the =E2=80=98version=E2=80=99 field, thou=
gh, so I would
> say that the underscroes, should also be converted to periods.
>
>> What if a repo has tags in both formats?  Then "3.0.1" would be
>> considered older than "2011-01-01".
>
> That=E2=80=99s tricky, there isn=E2=80=99t really a way to know how old =
=E2=80=9C3.0.1=E2=80=9D is,
> without looking at the metadata of the tag.  Maybe this is one of those
> corner cases which can=E2=80=99t really automatically determine the latest
> release.  Should we have a =E2=80=98no-refresh?=E2=80=99 property to tell=
 the refresh to
> not try to update the package?
>
>> Maybe we should just add an extra bit to detect a date format and only
>> consider it when there's no "proper versions"?
>
> That could be a good idea!
>
>> Aaaand I fell down a rabbit hole after that :) I've attached a patch
>> with what I've done.  It still has lots of issues--it requires the tag
>> to contain at least one version delimiter, it requires the first
>> character of the version to be a number... it might not even be better
>> than before I touched it, and even so the added complexity might not be
>> worth it.  But if you'd like to take it for a spin, I've attached it (it
>> applies straight on master).
>
> Great! I will try it out and see how it compares to my current WIP
> version.
>
> Not having characters in the first version number probably isn=E2=80=99t =
such a
> big deal, most version that contain characters end with a character.
> E.g., =E2=80=9C1.2.3a=E2=80=9D is not to uncommon, but =E2=80=9Ca1.2.3=E2=
=80=9D is rarely seen.

I made some changes, to the patch and ran =E2=80=98guix refresh -t generic-=
git |
grep "^gnu/packages.*no valid=E2=80=99; I also modified the =E2=80=98github=
=E2=80=99 updater so
that it wouldn=E2=80=99t be used.
=20=20
  (define %github-updater
    (upstream-updater
     (name 'github)
     (description "Updater for GitHub packages")
     (pred (const #f)) ;this right here
     (latest latest-release)))

Attached is a file with all the packages that didn=E2=80=99t have any valid
tags, and with a short description of perharps why no valid tags were
found.

Something I noticed was the a lot of Julia package use a version scheme
like this:

  (version "1.2.3+0")

The "+0" is included in the version field and acts like a =E2=80=9Crevision=
=E2=80=9D; I
am not familiar with the Julia ecosystem, and I am not sure how we
should handle this situation.

The updated patch is also attached.


--=-=-=
Content-Type: application/octet-stream
Content-Disposition: attachment; filename=no-valid-tags
Content-Transfer-Encoding: base64

Ozs7IEN1c3RvbSBwcmVmaXggb3Igc3VmZml4IChzaG91bGQgYmUgc29sdmFibGUgYnkgdXMpCjs7
IGttc2Nvbi0gcHJlZml4CmdudS9wYWNrYWdlcy90ZXJtaW5hbHMuc2NtOjI4MToxNDogd2Fybmlu
Zzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Iga21zY29uCgo7OyB2MS4yLTxudW0+CmdudS9wYWNr
YWdlcy9zdGF0aXN0aWNzLnNjbTo1NzAxOjU6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQg
Zm9yIHItY29sb3JvdXQKCjs7IDxDSEFSPjxudW0+CmdudS9wYWNrYWdlcy9zZGwuc2NtOjY2Mjox
NDogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igc2RsMi1jcwoKOzsgdGVzdC0gcHJl
Zml4CmdudS9wYWNrYWdlcy9xdC5zY206MTE4Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91
bmQgZm9yIHFpdGUKCjs7ICJ2IiBwcmVmaXgsICItc3RhYmxlIiBzdWZmaXgKZ251L3BhY2thZ2Vz
L29wZW5jb2cuc2NtOjc2OjE0OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBhdG9t
c3BhY2UKZ251L3BhY2thZ2VzL29wZW5jb2cuc2NtOjQ0OjE0OiB3YXJuaW5nOiBubyB2YWxpZCB0
YWdzIGZvdW5kIGZvciBjb2d1dGlsCgo7OyArPG51bT4gc3VmZml4CmdudS9wYWNrYWdlcy9vY2Ft
bC5zY206NjAzOjEyOiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBvY2FtbC1tY2Nz
Cgo7OyAidmVyc2lvbi0iIHByZWZpeApnbnUvcGFja2FnZXMvbWluZXRlc3Quc2NtOjYwMDo1OiB3
YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBtaW5ldGVzdC11bmlmaWVkLWludmVudG9y
eQoKOzsgIm1hdmVuLXJlc291cmNlcy1wbHVnaW4tIiBwcmVmaXgKZ251L3BhY2thZ2VzL21hdmVu
LnNjbToyOTgwOjEyOiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBtYXZlbi1yZXNv
dXJjZXMtcGx1Z2luCgo7OyAidHhyLSIgcHJlZml4CmdudS9wYWNrYWdlcy9saXNwLnNjbTo5MTI6
NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgdHhyCgo7OyByZWxlYXNlLSBwcmVm
aXgKZ251L3BhY2thZ2VzL2FsZ2VicmEuc2NtOjE0MjA6MTI6IHdhcm5pbmc6IG5vIHZhbGlkIHRh
Z3MgZm91bmQgZm9yIG00cmkKZ251L3BhY2thZ2VzL2FsZ2VicmEuc2NtOjE1MTA6MTI6IHdhcm5p
bmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIG00cmllCgo7OyBscmNhbGMtIHByZWZpeApnbnUv
cGFja2FnZXMvYWxnZWJyYS5zY206MTU2NjoxMjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3Vu
ZCBmb3IgbHJjYWxjCgo7OyAzLjMuMDYtMSwgLTxudW0+IHN1ZmZpeApnbnUvcGFja2FnZXMvYWRt
aW4uc2NtOjQwNDY6Mjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgaW54aQpnbnUv
cGFja2FnZXMvYWRtaW4uc2NtOjM5Nzc6Nzogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBm
b3IgaW54aS1taW5pbWFsCgo7OyAtcmVsZWFzZS1EQVRFK1NPTUUtUkFORE9NLU5VTUJFUlMsIHJl
ZnMvdGFncy8yLjAuMTgwLXJlbGVhc2UtMjAyMTAzMTUyMzE3MDcKZ251L3BhY2thZ2VzL2Jpb2lu
Zm9ybWF0aWNzLnNjbToxMTE4OToxMjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Ig
YmlvYmFtYmFtMgpnbnUvcGFja2FnZXMvYmlvaW5mb3JtYXRpY3Muc2NtOjExMTQ5OjEyOiB3YXJu
aW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBsaWJtYXVzMgoKOzsgIitnaXQ8ZGF0ZT4iIHN1
ZmZpeApnbnUvcGFja2FnZXMvZ25vbWUuc2NtOjExODc4OjEyOiB3YXJuaW5nOiBubyB2YWxpZCB0
YWdzIGZvdW5kIGZvciBmZWVkYmFja2QKCjs7ICJmZXN0LXRlc3QtIiBwcmVmaXgKZ251L3BhY2th
Z2VzL2phdmEuc2NtOjEwOTAwOjEyOiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBq
YXZhLWZlc3QtdGVzdAoKOzsgIi1zdGFibGUiIHN1ZmZpeApnbnUvcGFja2FnZXMvZ2FtZS1kZXZl
bG9wbWVudC5zY206MTcyMjoxMjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgZ29k
b3QKCjs7ICJ2IiBwcmVmaXgsICItZGViaWFuIiBzdWZmaXgKZ251L3BhY2thZ2VzL2dzdHJlYW1l
ci5zY206OTU6NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igb3Blbm5pMgoKCjs7
ICs8bnVtPiBzdWZmaXgsIGJ1dCB0aGlzIGlzIHBhcnQgb2YgdGhlIGB2ZXJzaW9uJyBmaWVsZCBh
cyB3ZWxsLCBub3QKOzsgc3VyZSB3aGF0IHRvIGRvIGFib3V0IHRoaXMKOzsKOzsgKHZlcnNpb24g
IjEuNS4wKzAiKQpnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbTozNjY6Njogd2FybmluZzogbm8g
dmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEtZnJpYmlkaS1qbGwKZ251L3BhY2thZ2VzL2p1bGlh
LWpsbC5zY206MjUyMDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14
b3JnLXh0cmFucy1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MTgxMjo2OiB3YXJuaW5n
OiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14b3JnLWxpYngxMS1qbGwKZ251L3BhY2th
Z2VzL2p1bGlhLWpsbC5zY206MTE0OTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBqdWxpYS1saWJ2b3JiaXMtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjE0OTE6Njog
d2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEtcXQ1YmFzZS1qbGwKZ251L3Bh
Y2thZ2VzL2p1bGlhLWpsbC5zY206MTQxNTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5k
IGZvciBqdWxpYS1wY3JlLWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxOTY0OjY6IHdh
cm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLXhvcmctbGlieGRtY3AtamxsCmdu
dS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjE1ODE6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBm
b3VuZCBmb3IganVsaWEtd2F5bGFuZC1wcm90b2NvbHMtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1q
bGwuc2NtOjY5NDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS1sYW1l
LWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToyMjYxOjY6IHdhcm5pbmc6IG5vIHZhbGlk
IHRhZ3MgZm91bmQgZm9yIGp1bGlhLXhvcmcteGNiLXV0aWwtamxsCmdudS9wYWNrYWdlcy9qdWxp
YS1qbGwuc2NtOjE2MTg6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEt
eDI2NC1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MTkyNjo2OiB3YXJuaW5nOiBubyB2
YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14b3JnLWxpYnhjdXJzb3ItamxsCmdudS9wYWNrYWdl
cy9qdWxpYS1qbGwuc2NtOjkyMDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBq
dWxpYS1saWJncGctZXJyb3ItamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjg4Mjo2OiB3
YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS1saWJnbHZuZC1qbGwKZ251L3Bh
Y2thZ2VzL2p1bGlhLWpsbC5zY206NDg2OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQg
Zm9yIGp1bGlhLWdsaWItamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjE4NTA6Njogd2Fy
bmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEteG9yZy1saWJ4YXUtamxsCmdudS9w
YWNrYWdlcy9qdWxpYS1qbGwuc2NtOjEwNjo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5k
IGZvciBqdWxpYS1jYWlyby1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MjU1Njo2OiB3
YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14c2x0LWpsbApnbnUvcGFja2Fn
ZXMvanVsaWEtamxsLnNjbToyMjk4OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9y
IGp1bGlhLXhvcmcteGNiLXV0aWwtaW1hZ2UtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2Nt
OjI0MDk6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEteG9yZy14Y2It
dXRpbC13bS1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206NjU3OjY6IHdhcm5pbmc6IG5v
IHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLWpwZWd0dXJiby1qbGwKZ251L3BhY2thZ2VzL2p1
bGlhLWpsbC5zY206MjA3NDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxp
YS14b3JnLWxpYnhpLWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxMTg3OjY6IHdhcm5p
bmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLWx6by1qbGwKZ251L3BhY2thZ2VzL2p1
bGlhLWpsbC5zY206MjAwMDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxp
YS14b3JnLWxpYnhleHQtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjg0Mzo2OiB3YXJu
aW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS1saWJnY3J5cHQtamxsCmdudS9wYWNr
YWdlcy9qdWxpYS1qbGwuc2NtOjczMDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBqdWxpYS1saWJhc3MtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjE1NDI6Njogd2Fy
bmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEtd2F5bGFuZC1qbGwKZ251L3BhY2th
Z2VzL2p1bGlhLWpsbC5zY206MTExMTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBqdWxpYS1saWJ1dWlkLWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxMDcyOjY6IHdh
cm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLWxpYnRpZmYtamxsCmdudS9wYWNr
YWdlcy9qdWxpYS1qbGwuc2NtOjIyMjQ6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBm
b3IganVsaWEteG9yZy1saWJ4cmVuZGVyLWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbTo5
NTg6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEtbGliaWNvbnYtamxs
CmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjk5Njo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdz
IGZvdW5kIGZvciBqdWxpYS1saWJtb3VudC1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206
MTAzNTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS1saWJwbmctamxs
CmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjc3MDo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdz
IGZvdW5kIGZvciBqdWxpYS1saWJmZGstYWFjLWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNj
bToxNzM1OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLXhtbDItamxs
CmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjgwNjo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdz
IGZvdW5kIGZvciBqdWxpYS1saWJmZmktamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjE4
ODY6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEteG9yZy1saWJ4Y2It
amxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjI0NDY6Njogd2FybmluZzogbm8gdmFsaWQg
dGFncyBmb3VuZCBmb3IganVsaWEteG9yZy14a2Jjb21wLWpsbApnbnUvcGFja2FnZXMvanVsaWEt
amxsLnNjbToxNjU5OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLXgy
NjUtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjI0ODM6Njogd2FybmluZzogbm8gdmFs
aWQgdGFncyBmb3VuZCBmb3IganVsaWEteG9yZy14a2V5Ym9hcmQtY29uZmlnLWpsbApnbnUvcGFj
a2FnZXMvanVsaWEtamxsLnNjbTo0NDM6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBm
b3IganVsaWEtZ2xmdy1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MjExMjo2OiB3YXJu
aW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14b3JnLWxpYnhpbmVyYW1hLWpsbApn
bnUvcGFja2FnZXMvanVsaWEtamxsLnNjbTo2Njo1OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZv
dW5kIGZvciBqdWxpYS1iemlwMi1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206NTY5OjY6
IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLWltYWdlbWFnaWNrLWpsbApn
bnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxMjYzOjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3Mg
Zm91bmQgZm9yIGp1bGlhLW9nZy1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MjMyOjY6
IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLWZmbXBlZy1qbGwKZ251L3Bh
Y2thZ2VzL2p1bGlhLWpsbC5zY206MTY5NTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5k
IGZvciBqdWxpYS14a2Jjb21tb24tamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjI1OTk6
Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEtemxpYi1qbGwKZ251L3Bh
Y2thZ2VzL2p1bGlhLWpsbC5zY206MjMzNTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5k
IGZvciBqdWxpYS14b3JnLXhjYi11dGlsLWtleXN5bXMtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1q
bGwuc2NtOjUzMDo1OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS1ndW1i
by1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MjE4Njo2OiB3YXJuaW5nOiBubyB2YWxp
ZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14b3JnLWxpYnhyYW5kci1qbGwKZ251L3BhY2thZ2VzL2p1
bGlhLWpsbC5zY206MTk0OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlh
LWV4cGF0LWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxMzAwOjc6IHdhcm5pbmc6IG5v
IHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLW9wZW5zcGVjZnVuLWpsbApnbnUvcGFja2FnZXMv
anVsaWEtamxsLnNjbTozMjQ6NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVs
aWEtZnJlZXR5cGUyLWpsbApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxMzQzOjY6IHdhcm5p
bmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLW9wZW5zc2wtamxsCmdudS9wYWNrYWdl
cy9qdWxpYS1qbGwuc2NtOjEyMjU6NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Ig
anVsaWEtbWJlZHRscy1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MTUyOjU6IHdhcm5p
bmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlhLWNvbXBpbGVyc3VwcG9ydGxpYnJhcmll
cy1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MjAzNzo2OiB3YXJuaW5nOiBubyB2YWxp
ZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14b3JnLWxpYnhmaXhlcy1qbGwKZ251L3BhY2thZ2VzL2p1
bGlhLWpsbC5zY206MjgxOjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGp1bGlh
LWZvbnRjb25maWctamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjE3NzU6Njogd2Fybmlu
Zzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEteG9yZy1saWJwdGhyZWFkLXN0dWJzLWps
bApnbnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxMzc5OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRh
Z3MgZm91bmQgZm9yIGp1bGlhLW9wdXMtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjQw
Mjo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS1nZXR0ZXh0LWpsbApn
bnUvcGFja2FnZXMvanVsaWEtamxsLnNjbToxNDUzOjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3Mg
Zm91bmQgZm9yIGp1bGlhLXBpeG1hbi1qbGwKZ251L3BhY2thZ2VzL2p1bGlhLWpsbC5zY206MjM3
Mjo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxpYS14b3JnLXhjYi11dGls
LXJlbmRlcnV0aWwtamxsCmdudS9wYWNrYWdlcy9qdWxpYS1qbGwuc2NtOjI2Mzc6Njogd2Fybmlu
Zzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IganVsaWEtenN0ZC1qbGwKZ251L3BhY2thZ2VzL2p1
bGlhLWpsbC5zY206MjE0OTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBqdWxp
YS14b3JnLWxpYnhrYmZpbGUtamxsCgoMCjs7OyBvbmx5IHByZS1yZWxlYXNlcwpnbnUvcGFja2Fn
ZXMvbGludXguc2NtOjEzMDY6Nzogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgcnRs
ODgxMmF1LWFpcmNyYWNrLW5nLWxpbnV4LW1vZHVsZQpnbnUvcGFja2FnZXMvdmltLnNjbTo1OTY6
Nzogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgdmltLXNvbGFyaXplZApnbnUvcGFj
a2FnZXMvbXVzaWMuc2NtOjIxMzM6MTI6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9y
IHBvd2VydGFiZWRpdG9yCmdudS9wYWNrYWdlcy9tdXNpYy5zY206NDQ2MzoxNDogd2FybmluZzog
bm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgcmtybHYyCmdudS9wYWNrYWdlcy9sdWEuc2NtOjI0ODox
Mjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgbHVhNS4xLXNvY2tldApnbnUvcGFj
a2FnZXMvbHVhLnNjbToyNDg6MTI6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGx1
YTUuMi1zb2NrZXQKZ251L3BhY2thZ2VzL2d1aWxlLXh5ei5zY206Mzg2MzoxMjogd2FybmluZzog
bm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgZ3VpbGUtYWMtZC1idXMKZ251L3BhY2thZ2VzL2dvbGFu
Zy5zY206NDEzNjo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBnby1naXRodWIt
Y29tLWJ0Y3N1aXRlLWJ0Y2QtYnRjZWMKZ251L3BhY2thZ2VzL2dhbWVzLnNjbTo3Mjc0OjU6IHdh
cm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNvbG9ib3QKCjs7IF9iZXRhPG51bT4gc3Vm
Zml4LCByZWZzL3RhZ3MvdjAuMV9iZXRhMgpnbnUvcGFja2FnZXMvY29tcHRvbi5zY206NTA6Nzog
d2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgY29tcHRvbgoKOzsgYWxtb3N0IGFsbCB0
YWdzIGFyZSBwcmUtcmVsZWFzZXMKOzsgcmVmcy90YWdzLzAuMC40LTE1Mi1hbHBoYQo7OyByZWZz
L3RhZ3MvMC4wLjQtMTUyLWFscGhhXnt9Cjs7IHJlZnMvdGFncy8wLjAuNC0xNzItYWxwaGEKOzsg
cmVmcy90YWdzLzAuMC40LTE3Mi1hbHBoYV57fQo7OyByZWZzL3RhZ3MvMC4xLjAtYWxwaGEKOzsg
cmVmcy90YWdzLzAuMS4wLWFscGhhXnt9Cjs7IHJlZnMvdGFncy8wLjEuMS1hbHBoYQo7OyByZWZz
L3RhZ3MvMC4xLjEtYWxwaGFee30KOzsgcmVmcy90YWdzLzAuMS4yLWFscGhhCjs7IHJlZnMvdGFn
cy8wLjEuMi1hbHBoYV57fQo7OyByZWZzL3RhZ3MvMC4yLjAtYWxwaGEKOzsgcmVmcy90YWdzLzAu
Mi4wLWFscGhhXnt9Cjs7IHJlZnMvdGFncy9wcmUtZXh0ZW5zaW9uCjs7IHJlZnMvdGFncy92MC4w
LjItYWxwaGEKOzsgcmVmcy90YWdzL3YwLjAuMy1hbHBoYQo7OyByZWZzL3RhZ3MvdjAuMC40LWFs
cGhhCjs7IHJlZnMvdGFncy92MC4wLjQtYWxwaGFee30KOzsgcmVmcy90YWdzL3YwLjEuMC1hbHBo
YQo7OyByZWZzL3RhZ3MvdjAuMS4wLWFscGhhXnt9Cjs7IHJlZnMvdGFncy92MC4xLjItY2FuZGlk
YXRlCjs7IHJlZnMvdGFncy92MC4xLjItY2FuZGlkYXRlXnt9CmdudS9wYWNrYWdlcy9ndWlsZS14
eXouc2NtOjM0ODg6MTI6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIG5vbWFkCgoM
Cjs7OyBubyBkZWxpbWl0ZXIgb3IganVzdCBvbmUgdmVyc2lvbiBudW1iZXIKZ251L3BhY2thZ2Vz
L21haWwuc2NtOjU4Nzo1OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBuZW9tdXR0
CmdudS9wYWNrYWdlcy9zeW5jLnNjbTo1MTg6MTI6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91
bmQgZm9yIGNhc3luYwpnbnUvcGFja2FnZXMvcHVsc2VhdWRpby5zY206MzE1OjEyOiB3YXJuaW5n
OiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBwb255bWl4CmdudS9wYWNrYWdlcy9uZXR3b3JraW5n
LnNjbToxMjY2OjEyOiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBpcHV0aWxzCmdu
dS9wYWNrYWdlcy9tdXNpYy5zY206Njg4NDoxMjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3Vu
ZCBmb3IgYTJqbWlkaWQKZ251L3BhY2thZ2VzL2VtdWxhdG9ycy5zY206NTY3OjU6IHdhcm5pbmc6
IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGhpZ2FuCmdudS9wYWNrYWdlcy9lbWFjcy14eXouc2Nt
OjExNDg5OjU6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVtYWNzLWx1YS1tb2Rl
CmdudS9wYWNrYWdlcy9kaWZmb3Njb3BlLnNjbTo3NjoxMjogd2FybmluZzogbm8gdmFsaWQgdGFn
cyBmb3VuZCBmb3IgZGlmZm9zY29wZQpnbnUvcGFja2FnZXMvY3Vwcy5zY206Njc6MTQ6IHdhcm5p
bmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGJybGFzZXIKZ251L3BhY2thZ2VzL2NvZGUuc2Nt
OjU3NToxMjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Iga2NvdgpnbnUvcGFja2Fn
ZXMvZ25vbWUteHl6LnNjbToxODU6NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Ig
cGFwaXJ1cy1pY29uLXRoZW1lCmdudS9wYWNrYWdlcy9nbm9tZS14eXouc2NtOjM2NzoxMjogd2Fy
bmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgZ25vbWUtc2hlbGwtZXh0ZW5zaW9uLWdzY29u
bmVjdApnbnUvcGFja2FnZXMvZ25vbWUteHl6LnNjbToyODk6NTogd2FybmluZzogbm8gdmFsaWQg
dGFncyBmb3VuZCBmb3IgZ25vbWUtc2hlbGwtZXh0ZW5zaW9uLXRvcGljb25zLXJlZHV4CmdudS9w
YWNrYWdlcy9nb2xhbmcuc2NtOjExMTY6NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBm
b3IgZ28tZ2l0aHViLWNvbS1vcC1nby1sb2dnaW5nCmdudS9wYWNrYWdlcy9qYXZhLnNjbToxMDU3
OToxMjogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgamF2YS1qYXZheC1pbmplY3QK
Z251L3BhY2thZ2VzL21haWwuc2NtOjE0MTU6MTI6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91
bmQgZm9yIG5vdG11Y2gtYWRkcmxvb2t1cC1jCgoKCgoKCgwKOzs7IChNb3N0IGxpa2VseSkgbm90
IHNvbHZhYmxlIGJ5IHVzCjs7IG9ubHkgb25lIHRhZyAiaW5pdGlhbC1jb21taXQiCmdudS9wYWNr
YWdlcy9idWlsZC10b29scy5zY206MjA4OjE0OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5k
IGZvciBnbgoKOzsgPz8/IHRoZSB3ZWlyZGVzdCB2ZXJzaW9uIHNjaGVtZT8/Pwo7OyByZWZzL3Rh
Z3MvYnpyLXJldm5vLTc4OTQKOzsgcmVmcy90YWdzL21vc2VzLTMuMi4xMy0xNS1Ob3YtMjAxMgo7
OyByZWZzL3RhZ3MvbW92ZWQtdG8tdXJlLWJhc2VkLXIybAo7OyByZWZzL3RhZ3Mvb2Jzb2xldGUt
QysrLVBMTgo7OyByZWZzL3RhZ3Mvb2Jzb2xldGUtUHl0aG9uLVBMTgo7OyByZWZzL3RhZ3Mvb2xk
LWVtYm9kaW1lbnRfOC1ub3YtMjAxNQo7OyByZWZzL3RhZ3Mvb2xkLW9wZW5wc2ktSmluaHVhLUNo
dWEtMjAxMQo7OyByZWZzL3RhZ3Mvb2xkLXBhdHRlcm4tbWluZXItMTEtSnVuLTIwMTkKOzsgcmVm
cy90YWdzL3YwLjEuNC1zdGFibGUKZ251L3BhY2thZ2VzL29wZW5jb2cuc2NtOjIwOToxNDogd2Fy
bmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igb3BlbmNvZwoKOzsgInF1aWNrbGlzcC0iIHBy
ZWZpeCwgZGF0ZSBpbnN0ZWFkIG9mIHZlcnNpb24gbnVtYmVyCmdudS9wYWNrYWdlcy9saXNwLXh5
ei5zY206MzcwMzo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBzYmNsLXBvcnRh
YmxlLXRocmVhZHMKCjs7IG5vIHZlcnNpb24gbnVtYmVyIGluIHRhZwpnbnUvcGFja2FnZXMvZ3Vp
bGUteHl6LnNjbTozMjgzOjc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGd1aWxl
LXNyZmktMTgwCmdudS9wYWNrYWdlcy9nb2xhbmcuc2NtOjYxOTM6MTQ6IHdhcm5pbmc6IG5vIHZh
bGlkIHRhZ3MgZm91bmQgZm9yIGdvLWdpdGh1Yi1jb20tZ29sYW5nLWZyZWV0eXBlCmdudS9wYWNr
YWdlcy9tZXNzYWdpbmcuc2NtOjE0OTU6MTQ6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQg
Zm9yIGxpYnRveGNvcmUKZ251L3BhY2thZ2VzL25ldHdvcmtpbmcuc2NtOjM4MTQ6NTogd2Fybmlu
Zzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgdmRlMgpnbnUvcGFja2FnZXMvbGlzcC14eXouc2Nt
OjE1ODkyOjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLWJvZGdlLWJsb2Jz
LXN1cHBvcnQKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTozMTIwOjc6IHdhcm5pbmc6IG5vIHZh
bGlkIHRhZ3MgZm91bmQgZm9yIGNsLWNmZmktYy1yZWYKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNj
bTo1OTc5Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIHNiY2wtaWVlZS1mbG9h
dHMKZ251L3BhY2thZ2VzL2VuZ2luZWVyaW5nLnNjbTo4MTQ6MTQ6IHdhcm5pbmc6IG5vIHZhbGlk
IHRhZ3MgZm91bmQgZm9yIGxpYmZpdmUKZ251L3BhY2thZ2VzL2NwcC5zY206MjIxOjE0OiB3YXJu
aW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciByY3QKCjs7ICJnby53ZWVrbHkuIiBwcmVmaXgs
IGRhdGUgaW5zdGVhZCBvZiB2ZXJzaW9uCmdudS9wYWNrYWdlcy9nb2xhbmcuc2NtOjM5MDU6Nzog
d2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgZ28tZ2l0aHViLWNvbS1taWNoaXdlbmQt
Z29sYW5nLXByZXR0eQoKOzsgd2VpcmQgdmVyc2lvbiBzY2hlbWUKOzsgcmVmcy90YWdzL1YzCjs7
IHJlZnMvdGFncy92ZXJzaW9uXzEKOzsgcmVmcy90YWdzL3d5aGFzaF9hbHBoYQo7OyByZWZzL3Rh
Z3Mvd3loYXNoX2ZpbmFsCjs7IHJlZnMvdGFncy93eWhhc2hfdjQKOzsgcmVmcy90YWdzL3d5aGFz
aF92NQpnbnUvcGFja2FnZXMvZGlnZXN0LnNjbTozNjoxMjogd2FybmluZzogbm8gdmFsaWQgdGFn
cyBmb3VuZCBmb3Igd3loYXNoCgoKCgo7OzsgY291bGRuJ3QgYmUgYm90aGVyZWQgdG8gY2hlY2su
Li4KZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo2NTA4OjI6IHdhcm5pbmc6IG5vIHZhbGlkIHRh
Z3MgZm91bmQgZm9yIGNsLXRyaXZpYQpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjEyMjE0Ojc6
IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLW1jY2xpbQpnbnUvcGFja2FnZXMv
bGlzcC14eXouc2NtOjEzNjMzOjc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNs
LWNsYXctdXRpbHMKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTozMTk1Ojc6IHdhcm5pbmc6IG5v
IHZhbGlkIHRhZ3MgZm91bmQgZm9yIHNiY2wtcGFyZW5zY3JpcHQKZ251L3BhY2thZ2VzL2xpc3At
eHl6LnNjbTo2NDY2Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLXRyaXZp
YS50cml2aWFsCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206NjUwODoyOiB3YXJuaW5nOiBubyB2
YWxpZCB0YWdzIGZvdW5kIGZvciBlY2wtdHJpdmlhCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206
MTM1NjY6NTogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgY2wtY2xhdy1zdXBwb3J0
CmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206OTMyNjo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdz
IGZvdW5kIGZvciBjbC1qcGVnCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206MTM1NjY6NTogd2Fy
bmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igc2JjbC1jbGF3LXN1cHBvcnQKZ251L3BhY2th
Z2VzL2xpc3AteHl6LnNjbToxODM5NDo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBjbC10YWlscmVjCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206MTM5OjY6IHdhcm5pbmc6IG5v
IHZhbGlkIHRhZ3MgZm91bmQgZm9yIHNiY2wtYm9kZ2UtdXRpbGl0aWVzCmdudS9wYWNrYWdlcy9s
aXNwLXh5ei5zY206MTU4NjA6Njogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgZWNs
LWJvZGdlLW1hdGgKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbToxMzU2Njo1OiB3YXJuaW5nOiBu
byB2YWxpZCB0YWdzIGZvdW5kIGZvciBlY2wtY2xhdy1zdXBwb3J0CmdudS9wYWNrYWdlcy9saXNw
LXh5ei5zY206MTc4Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLWJvZGdl
LXF1ZXVlCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206NjUwODoyOiB3YXJuaW5nOiBubyB2YWxp
ZCB0YWdzIGZvdW5kIGZvciBzYmNsLXRyaXZpYQpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjY4
MDc6Nzogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igc2JjbC1zLXN5c2RlcHMKZ251
L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo2NDY2Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91
bmQgZm9yIGVjbC10cml2aWEudHJpdmlhbApnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjUxOTo3
OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBjbC1maWFzY28KZ251L3BhY2thZ2Vz
L2xpc3AteHl6LnNjbTo2ODA3Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVj
bC1zLXN5c2RlcHMKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo4OTI5Ojc6IHdhcm5pbmc6IG5v
IHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVjbC1jbC1hc3luYwpnbnUvcGFja2FnZXMvbGlzcC14eXou
c2NtOjMxOTU6Nzogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3IgY2wtcGFyZW5zY3Jp
cHQKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo5MzI2Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRh
Z3MgZm91bmQgZm9yIGVjbC1jbC1qcGVnCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206NTE5Ojc6
IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVjbC1maWFzY28KZ251L3BhY2thZ2Vz
L2xpc3AteHl6LnNjbTozNzAzOjc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVj
bC1wb3J0YWJsZS10aHJlYWRzCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206MTIyMTQ6Nzogd2Fy
bmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igc2JjbC1tY2NsaW0KZ251L3BhY2thZ2VzL2xp
c3AteHl6LnNjbTo4NjI4Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLW1p
c2MtZXh0ZW5zaW9ucwpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjEzNjMzOjc6IHdhcm5pbmc6
IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVjbC1jbGF3LXV0aWxzCmdudS9wYWNrYWdlcy9saXNw
LXh5ei5zY206NTk3OTo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBjbC1pZWVl
LWZsb2F0cwpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjE1ODkyOjY6IHdhcm5pbmc6IG5vIHZh
bGlkIHRhZ3MgZm91bmQgZm9yIGVjbC1ib2RnZS1ibG9icy1zdXBwb3J0CmdudS9wYWNrYWdlcy9s
aXNwLXh5ei5zY206MzEyMDo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBlY2wt
Y2ZmaS1jLXJlZgpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjU5Nzk6Nzogd2FybmluZzogbm8g
dmFsaWQgdGFncyBmb3VuZCBmb3IgZWNsLWllZWUtZmxvYXRzCmdudS9wYWNrYWdlcy9saXNwLXh5
ei5zY206MTM5OjY6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLWJvZGdlLXV0
aWxpdGllcwpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjE3ODo3OiB3YXJuaW5nOiBubyB2YWxp
ZCB0YWdzIGZvdW5kIGZvciBlY2wtYm9kZ2UtcXVldWUKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNj
bTo5MzI2Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIHNiY2wtY2wtanBlZwpn
bnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjUxOTo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZv
dW5kIGZvciBzYmNsLWZpYXNjbwpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjE1ODkyOjY6IHdh
cm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIHNiY2wtYm9kZ2UtYmxvYnMtc3VwcG9ydApn
bnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjEzOTo2OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZv
dW5kIGZvciBlY2wtYm9kZ2UtdXRpbGl0aWVzCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206MzE5
NTo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBlY2wtcGFyZW5zY3JpcHQKZ251
L3BhY2thZ2VzL2xpc3AteHl6LnNjbTozMTIwOjc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91
bmQgZm9yIHNiY2wtY2ZmaS1jLXJlZgpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjE3ODo3OiB3
YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBzYmNsLWJvZGdlLXF1ZXVlCmdudS9wYWNr
YWdlcy9saXNwLXh5ei5zY206ODYyODo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBlY2wtbWlzYy1leHRlbnNpb25zCmdudS9wYWNrYWdlcy9saXNwLXh5ei5zY206MTU4NjA6Njog
d2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBmb3Igc2JjbC1ib2RnZS1tYXRoCmdudS9wYWNr
YWdlcy9saXNwLXh5ei5zY206ODYyODo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBzYmNsLW1pc2MtZXh0ZW5zaW9ucwpnbnUvcGFja2FnZXMvbGlzcC14eXouc2NtOjE1ODYwOjY6
IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLWJvZGdlLW1hdGgKZ251L3BhY2th
Z2VzL2xpc3AteHl6LnNjbToxMzYzMzo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBzYmNsLWNsYXctdXRpbHMKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo2NDY2Ojc6IHdhcm5p
bmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIHNiY2wtdHJpdmlhLnRyaXZpYWwKZ251L3BhY2th
Z2VzL2xpc3AteHl6LnNjbToxODM5NDo3OiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZvdW5kIGZv
ciBzYmNsLXRhaWxyZWMKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbToxODM5NDo3OiB3YXJuaW5n
OiBubyB2YWxpZCB0YWdzIGZvdW5kIGZvciBlY2wtdGFpbHJlYwpnbnUvcGFja2FnZXMvbGlzcC14
eXouc2NtOjEyMjE0Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGVjbC1tY2Ns
aW0KZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo4OTI5Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRh
Z3MgZm91bmQgZm9yIHNiY2wtY2wtYXN5bmMKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTo2ODA3
Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLXMtc3lzZGVwcwpnbnUvcGFj
a2FnZXMvbGlzcC14eXouc2NtOjg5Mjk6Nzogd2FybmluZzogbm8gdmFsaWQgdGFncyBmb3VuZCBm
b3IgY2wtYXN5bmMKZ251L3BhY2thZ2VzL2xpc3AteHl6LnNjbTozNzAzOjc6IHdhcm5pbmc6IG5v
IHZhbGlkIHRhZ3MgZm91bmQgZm9yIGNsLXBvcnRhYmxlLXRocmVhZHMKZ251L3BhY2thZ2VzL2xp
bnV4LnNjbToxMzA2Ojc6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIHJ0bDg4MTJh
dS1haXJjcmFjay1uZy1saW51eC1tb2R1bGUKZ251L3BhY2thZ2VzL2xpbnV4LnNjbTo2Mjg0OjU6
IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIG1jZWxvZwpnbnUvcGFja2FnZXMva29k
aS5zY206NDcwOjI6IHdhcm5pbmc6IG5vIHZhbGlkIHRhZ3MgZm91bmQgZm9yIGtvZGktd2F5bGFu
ZApnbnUvcGFja2FnZXMva29kaS5zY206Mjc1OjEyOiB3YXJuaW5nOiBubyB2YWxpZCB0YWdzIGZv
dW5kIGZvciBrb2RpCg==
--=-=-=
Content-Type: text/x-patch; charset=utf-8
Content-Disposition: attachment;
 filename=0001-import-Add-generic-git-updater.patch
Content-Transfer-Encoding: quoted-printable

From=20f924dbb835425f6b9a5796918125592870391405 Mon Sep 17 00:00:00 2001
Message-Id: <f924dbb835425f6b9a5796918125592870391405.1631125652.git.public=
@yoctocell.xyz>
From: Xinglu Chen <public@HIDDEN>
Date: Fri, 3 Sep 2021 17:50:56 +0200
Subject: [PATCH] import: Add 'generic-git' updater.

* guix/import/git.scm: New file.
* doc/guix.texi (Invoking guix refresh): Document it.
* Makefile.am (MODULES): Register it.
* guix/git.scm (ls-remote-refs): New procedure.

Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>
=2D--
 Makefile.am         |   1 +
 doc/guix.texi       |  27 ++++++
 guix/git.scm        |  33 +++++++
 guix/import/git.scm | 217 ++++++++++++++++++++++++++++++++++++++++++++
 4 files changed, 278 insertions(+)
 create mode 100644 guix/import/git.scm

diff --git a/Makefile.am b/Makefile.am
index 3c79760734..c4d3a456b1 100644
=2D-- a/Makefile.am
+++ b/Makefile.am
@@ -254,6 +254,7 @@ MODULES =3D					\
   guix/import/egg.scm   			\
   guix/import/elpa.scm   			\
   guix/import/gem.scm				\
+  guix/import/git.scm                           \
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/gnu.scm				\
diff --git a/doc/guix.texi b/doc/guix.texi
index 36a0c7f5ec..26afb1607a 100644
=2D-- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, Launc=
hpad} packages.
 @item generic-html
 a generic updater that crawls the HTML page where the source tarball of
 the package is hosted, when applicable.
+@item generic-git
+a generic updater for packages hosted on Git repositories.  It tries to
+be smart about parsing Git tag names, but if it is not able to parse the
+tag name and compare tags correctly, users can define the following
+properties for a package.
+
+@itemize
+@item @code{tag-prefix}: a regular expression for matching a prefix of
+the tag name.
+
+@item @code{tag-suffix}: a regular expression for matching a suffix of
+the tag name.
+
+@item @code{tag-version-delimiter}: a string used as the delimiter in
+the tag name for separating the numbers of the version.
+@end itemize
+
+@lisp
+(package
+  (name "foo")
+  ;; ...
+  (properties
+    '((tag-prefix . "^release0-")
+      (tag-suffix . "[a-z]?$")
+      (tag-version-delimiter . ":"))))
+@end lisp=20=20=20=20=20=20
+
 @end table
=20
 For instance, the following command only checks for updates of Emacs
diff --git a/guix/git.scm b/guix/git.scm
index 9c6f326c36..c5d0d2da8e 100644
=2D-- a/guix/git.scm
+++ b/guix/git.scm
@@ -56,6 +56,8 @@
             commit-difference
             commit-relation
=20
+            ls-remote-refs
+
             git-checkout
             git-checkout?
             git-checkout-url
@@ -556,6 +558,37 @@ objects: 'ancestor (meaning that OLD is an ancestor of=
 NEW), 'descendant, or
               (if (set-contains? oldest new)
                   'descendant
                   'unrelated))))))
+
+;;
+;;; Remote operations.
+;;;
+
+(define* (ls-remote-refs url #:key tags?)
+  "Return the list of references advertised at Git repository URL.  If TAG=
S?
+is true, limit to only refs/tags."
+  (define (ref? ref)
+    ;; Like `git ls-remote --refs', only show actual references.
+    (and (string-prefix? "refs/" ref)
+         (not (string-suffix? "^{}" ref))))
+
+  (define (tag? ref)
+    (string-prefix? "refs/tags/" ref))
+
+  (define (include? ref)
+    (and (ref? ref)
+         (or (not tags?) (tag? ref))))
+
+  (with-libgit2
+   (call-with-temporary-directory
+    (lambda (cache-directory)
+      (let* ((repository (repository-init cache-directory))
+             ;; Create an in-memory remote so we don't touch disk.
+             (remote (remote-create-anonymous repository url)))
+        (remote-connect remote)
+        (remote-disconnect remote)
+        (repository-close! repository)
+
+        (filter include? (map remote-head-name (remote-ls remote))))))))
=20
 
 ;;;
diff --git a/guix/import/git.scm b/guix/import/git.scm
new file mode 100644
index 0000000000..52c98de197
=2D-- /dev/null
+++ b/guix/import/git.scm
@@ -0,0 +1,217 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN>
+;;; Copyright =C2=A9 2021 Sarah Morgensen <iskarian@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import git)
+  #:use-module (guix build utils)
+  #:use-module (guix diagnostics)
+  #:use-module (guix git)
+  #:use-module (guix git-download)
+  #:use-module (guix i18n)
+  #:use-module (guix packages)
+  #:use-module (guix upstream)
+  #:use-module (guix utils)
+  #:use-module (ice-9 format)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 rdelim)
+  #:use-module (ice-9 regex)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
+  #:export (%generic-git-updater))
+
+;;; Commentary:
+;;;
+;;; This module provides a generic package updater for packages hosted on =
Git
+;;; repositories.
+;;;
+;;; It tries to be smart about tag names, but if it is not automatically a=
ble
+;;; to parse the tag names correctly, users can set the `tag-prefix',
+;;; `tag-suffix' and `tag-version-delimiter' properties of the package to =
make
+;;; the updater parse the Git tag name correctly.
+;;;
+;;; Code:
+
+;;; Errors & warnings
+
+(define-condition-type &git-no-valid-tags-error &error
+  git-no-valid-tags-error?)
+
+(define (git-no-valid-tags-error)
+  (raise (condition (&message (message "no valid tags found"))
+                    (&git-no-valid-tags-error))))
+
+(define-condition-type &git-no-tags-error &error
+  git-no-tags-error?)
+
+(define (git-no-tags-error)
+  (raise (condition (&message (message "no tags were found"))
+                    (&git-no-tags-error))))
+
+
+;;; Updater
+
+(define %pre-release-words
+  '("alpha" "beta" "rc" "dev" "test"))
+
+(define* (get-version-mapping tags #:key prefix suffix delim pre-releases?)
+  (define (guess-delim)
+    (let ((total (length tags))
+          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
+          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
+          (underscores (reduce + 0 (map (cut string-count <> #\_) tags))))
+      (display (format #t "total: ~d, dots: ~d, dashes ~d, underscores ~d~=
%"
+              total dots dashes underscores))
+      (cond
+       ((>=3D dots (* total 0.35)) ".")
+       ((>=3D dashes (* total 0.8)) "-")
+       ((>=3D underscores (* total 0.8)) "_")
+       (else ""))))
+
+  (define delim-rx (regexp-quote (or delim (guess-delim))))
+  (define suffix-rx
+    (string-append
+     (or suffix
+         (if pre-releases?
+             (string-append ".*(" (string-join %pre-release-words "|") ").=
*")
+             ""))
+     "$"))
+=20=20
+  (define prefix-rx (string-append "^" (or prefix "[^[:digit:]]*")))
+  (pk delim-rx)
+
+  (define tag-rx
+     (string-append "([[:digit:]][^" delim-rx "[:punct:]]*"
+                    "(" delim-rx "[^[:punct:]" delim-rx "]+)"
+                    ;; If there is are no delimiters, it could mean that t=
he
+                    ;; version just contains one number (e.g., "2"), thus,=
 use
+                    ;; "*" instead of "+" to match zero or more numbers.
+                    (if (string=3D? delim-rx "") "*" "+")
+                    ")" suffix-rx))
+
+  (define (get-version tag)
+    (let ((tag-match (regexp-exec (make-regexp tag-rx) tag)))
+      (pk tag)
+      (pk tag-match)
+      (pk tag-rx)
+      (and tag-match
+           (regexp-exec (make-regexp prefix-rx) (match:prefix tag-match))
+           (regexp-substitute/global
+            #f delim-rx (match:substring tag-match)
+            ;; Don't insert "." if there aren't any delimiters in the first
+            ;; place.
+            'pre (if (string=3D? delim-rx "") "" ".") 'post))))
+
+  (define (entry<? a b)
+    (eq? (version-compare (car a) (car b)) '<))
+
+  (let ((mapping (fold alist-cons '() (map get-version tags) tags)))
+    (stable-sort! (filter car mapping) entry<?)))
+
+(define* (get-latest-tag url #:key prefix suffix delim pre-releases?)
+  "Return the latest tag available from the Git repository at URL."
+  (define (pre-release? tag)
+    (any (lambda (rx) (regexp-exec (make-regexp rx regexp/icase) tag))
+              %pre-release-words))
+
+  (let* ((tags (map (cut string-drop <> (string-length "refs/tags/"))
+                    (ls-remote-refs url #:tags? #t)))
+         (versions->tags
+          (get-version-mapping (if pre-releases?
+                                   tags
+                                   (filter (negate pre-release?) tags))
+                               #:prefix prefix
+                               #:suffix suffix
+                               #:delim delim
+                               #:pre-releases? pre-releases?)))
+    (display versions->tags)
+    (newline)
+    (cond
+     ((null? tags)
+      (git-no-tags-error))
+     ((null? versions->tags)
+      (git-no-valid-tags-error))
+     (else
+      (match (last versions->tags)
+        ((version . tag)
+         (values version tag)))))))
+
+(define (latest-git-tag-version package tag-prefix tag-suffix
+                                tag-version-delimiter refresh-pre-releases=
?)
+  "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, TAG-VERSION-DELIMITER, and
+REFRESH-PRE-RELEASES?  properties of PACKAGE, returns the latest version of
+PACKAGE."
+  (guard (c ((or (git-no-tags-error? c) (git-no-valid-tags-error? c))
+             (warning (or (package-field-location package 'source)
+                          (package-location package))
+                      (G_ "~a for ~a~%")
+                      (condition-message c)
+                      (package-name package))
+             #f)
+            ((eq? (exception-kind c) 'git-error)
+             (warning (or (package-field-location package 'source)
+                          (package-location package))
+                      (G_ "failed to fetch Git repository for ~a~%")
+                      (package-name package))
+             #f))
+    (let* ((source (package-source package))
+           (url (git-reference-url (origin-uri source))))
+      ;;(format #t "~a~%" (package-name package))
+      (get-latest-tag url #:prefix tag-prefix #:suffix tag-suffix
+                      #:delim tag-version-delimiter
+                      #:pre-releases? refresh-pre-releases?))))
+
+(define (git-package? package)
+  "Whether the origin of PACKAGE is a Git repostiory."
+  (match (package-source package)
+    ((? origin? origin)
+     (and (eq? (origin-method origin) git-fetch)
+          (git-reference? (origin-uri origin))))
+    (_ #f)))
+
+(define (latest-git-release package)
+  "Return the latest release of PACKAGE."
+  (let* ((name (package-name package))
+         (properties (package-properties package))
+         (tag-prefix (assq-ref properties 'tag-prefix))
+         (tag-suffix (assq-ref properties 'tag-suffix))
+         (tag-version-delimiter (assq-ref properties 'tag-version-delimite=
r))
+         (refresh-pre-releases? (assq-ref properties 'refresh-pre-releases=
?))
+         (old-version (package-version package))
+         (url (git-reference-url (origin-uri (package-source package))))
+         (new-version (latest-git-tag-version package
+                                              tag-prefix
+                                              tag-suffix
+                                              tag-version-delimiter
+                                              refresh-pre-releases?)))
+
+    (if new-version
+        (upstream-source
+         (package name)
+         (version new-version)
+         (urls (list url)))
+        ;; No new release or no tags available.
+        #f)))
+
+(define %generic-git-updater
+  (upstream-updater
+   (name 'generic-git)
+   (description "Updater for packages hosted on Git repositories")
+   (pred git-package?)
+   (latest latest-git-release)))
=2D-=20
2.33.0


--=-=-=--

--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE5ANcVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5QjoP/A7+M+zpmxPhDcR69CHICY0j850j
eljZ0arJ38IiGAonrsQDd7uGsSoFqFfwN5sks/J1HlpAUiYpkYIJ+TX9f5NzIEoR
FKdKO+IDScFsoesBnp1JYg4YvUYq0JgktZaBEAL1a6LsLlRqJ7lVgE0oWXbpcuKh
uKW3c50/QFglL91avSfLU/BPc1fSGrLF12f8+pE/yD5WivpUH5+/xv+apk2xiLzD
+/c6KVGsVvA5CvMl2IEQ9riui0SKbnhw48HGRrMArvxbh/zAlN7gaYTOuRmwlOoR
pM00dlNTZn3yJFu9TzaY05yhD2FFsNmZVH9FBfmy6h1AqJhjL92JuqiQnUfaPGVM
oek6NE/JT7TRAVWEhbh0ks5m4pBppYyxDKseK/A3EF/ZqeYe+QQSQajS3W0zhDOc
dtbeqrHuxgawrvTyT3PdzA7T3BCJRgtseISwie4a5sziZb8fxTORE3qHx3CTN+0f
50UiiLmC+FBdak4LpZnCnzwQsct2bFrFH1kvDLuze1OJ9OpK5xd9UWfXXHLRbSv0
bAwxO350RKPIg6+e2Bfw3NP+zmUUNyMVp/KkqVRxZ78lp4r6F/xkW4n64u4zZezR
Oe9M4OWiVg+byX7DPybbJybeafNXIq/PWW46hZ4vrD5dgub2Iow7Ctzt9t16aiAr
ywZ5GSLN7s5M/Hgc
=xwBb
-----END PGP SIGNATURE-----
--==-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 10 Sep 2021 08:37:02 +0000
Resent-Message-ID: <handler.50359.B50359.163126298917734 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: Sarah Morgensen <iskarian@HIDDEN>, 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163126298917734
          (code B ref 50359); Fri, 10 Sep 2021 08:37:02 +0000
Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 08:36:29 +0000
Received: from localhost ([127.0.0.1]:36865 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mOc1M-0004by-Mm
	for submit <at> debbugs.gnu.org; Fri, 10 Sep 2021 04:36:29 -0400
Received: from eggs.gnu.org ([209.51.188.92]:56760)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1mOc1K-0004bk-Rd
 for 50359 <at> debbugs.gnu.org; Fri, 10 Sep 2021 04:36:27 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:37190)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1mOc1A-0006OB-P7; Fri, 10 Sep 2021 04:36:17 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=47218 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1mOc1A-0004Ru-1e; Fri, 10 Sep 2021 04:36:16 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
 <86y28ai7ns.fsf@HIDDEN> <87y28928vh.fsf@HIDDEN>
 <86pmtli4hn.fsf@HIDDEN> <87wnnsyzal.fsf@HIDDEN>
 <87mtomzzu1.fsf@HIDDEN>
Date: Fri, 10 Sep 2021 10:36:12 +0200
In-Reply-To: <87mtomzzu1.fsf@HIDDEN> (Xinglu Chen's message of "Wed, 08
 Sep 2021 20:28:38 +0200")
Message-ID: <87o890ddz7.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.3 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.3 (-)

Hello,

This looks very cool!

Xinglu Chen <public@HIDDEN> skribis:

> From f924dbb835425f6b9a5796918125592870391405 Mon Sep 17 00:00:00 2001
> Message-Id: <f924dbb835425f6b9a5796918125592870391405.1631125652.git.publ=
ic@HIDDEN>
> From: Xinglu Chen <public@HIDDEN>
> Date: Fri, 3 Sep 2021 17:50:56 +0200
> Subject: [PATCH] import: Add 'generic-git' updater.
>
> * guix/import/git.scm: New file.
> * doc/guix.texi (Invoking guix refresh): Document it.
> * Makefile.am (MODULES): Register it.
> * guix/git.scm (ls-remote-refs): New procedure.
>
> Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>

Overall LGTM; comments below.

> diff --git a/doc/guix.texi b/doc/guix.texi
> index 36a0c7f5ec..26afb1607a 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, Lau=
nchpad} packages.
>  @item generic-html
>  a generic updater that crawls the HTML page where the source tarball of
>  the package is hosted, when applicable.
> +@item generic-git

Please add a newline above.

> +@lisp
> +(package
> +  (name "foo")
> +  ;; ...
> +  (properties
> +    '((tag-prefix . "^release0-")
> +      (tag-suffix . "[a-z]?$")
> +      (tag-version-delimiter . ":"))))
> +@end lisp=20=20=20=20=20=20

Very nice.  Perhaps s/tag-/release-tag-/ for clarity?

> +(define* (ls-remote-refs url #:key tags?)
> +  "Return the list of references advertised at Git repository URL.  If T=
AGS?
> +is true, limit to only refs/tags."

To remain consistent with existing naming conventions, I=E2=80=99d call it
=E2=80=98remote-refs=E2=80=99.

> +  (with-libgit2
> +   (call-with-temporary-directory
> +    (lambda (cache-directory)
> +      (let* ((repository (repository-init cache-directory))
> +             ;; Create an in-memory remote so we don't touch disk.
> +             (remote (remote-create-anonymous repository url)))

Too bad we need to create an empty repo; hopefully it costs next to
nothing though.

> +        (remote-connect remote)
> +        (remote-disconnect remote)
> +        (repository-close! repository)
> +
> +        (filter include? (map remote-head-name (remote-ls remote))))))))

Use =E2=80=98filter-map=E2=80=99.

> +(define* (get-version-mapping tags #:key prefix suffix delim pre-release=
s?)

Please add a docstring and remove =E2=80=98get-=E2=80=99 from the name.  :-)

> +  (define (guess-delim)
> +    (let ((total (length tags))
> +          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
> +          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
> +          (underscores (reduce + 0 (map (cut string-count <> #\_) tags))=
))
> +      (display (format #t "total: ~d, dots: ~d, dashes ~d, underscores ~=
d~%"
> +              total dots dashes underscores))

Leftover?  (Also display + format.)

Please spell out =E2=80=98delimiter=E2=80=99 (info "(guix) Formatting Code"=
).

> +      (cond
> +       ((>=3D dots (* total 0.35)) ".")
> +       ((>=3D dashes (* total 0.8)) "-")
> +       ((>=3D underscores (* total 0.8)) "_")
> +       (else ""))))

That=E2=80=99s a fancy heuristic.  :-)

> +  (let ((mapping (fold alist-cons '() (map get-version tags) tags)))
> +    (stable-sort! (filter car mapping) entry<?)))

It=E2=80=99s perhaps clearer written like this:

  (stable-sort (filter-map (lambda (tag)
                             (let ((version (get-version tag)))
                               (and version (cons version tag))))
                            tags)
               entry<?)

> +(define* (get-latest-tag url #:key prefix suffix delim pre-releases?)
> +  "Return the latest tag available from the Git repository at URL."

Maybe =E2=80=9Cthe tag corresponding to the latest version=E2=80=9D.

s/get-latest-tag/latest-tag/

> +  (define (pre-release? tag)
> +    (any (lambda (rx) (regexp-exec (make-regexp rx regexp/icase) tag))
> +              %pre-release-words))

Better call =E2=80=98make-regexp=E2=80=99 only once; so you could change
=E2=80=98%pre-release-words=E2=80=99 to be a list of regexp objects instead=
 of a list of
strings.

> +(define (latest-git-tag-version package tag-prefix tag-suffix
> +                                tag-version-delimiter refresh-pre-releas=
es?)
> +  "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, TAG-VERSION-DELIMITER, a=
nd
> +REFRESH-PRE-RELEASES?  properties of PACKAGE, returns the latest version=
 of
> +PACKAGE."

Maybe s/refresh-pre-releases?/accept-pre-preleases?/

Since this procedure takes a package, it probably doesn=E2=80=99t need the =
other
arguments: it can extract them from the package properties, rather than
doing it at the call site.

> +(define (latest-git-release package)
> +  "Return the latest release of PACKAGE."
> +  (let* ((name (package-name package))
> +         (properties (package-properties package))
> +         (tag-prefix (assq-ref properties 'tag-prefix))
> +         (tag-suffix (assq-ref properties 'tag-suffix))
> +         (tag-version-delimiter (assq-ref properties 'tag-version-delimi=
ter))
> +         (refresh-pre-releases? (assq-ref properties 'refresh-pre-releas=
es?))
> +         (old-version (package-version package))
> +         (url (git-reference-url (origin-uri (package-source package))))
> +         (new-version (latest-git-tag-version package
> +                                              tag-prefix
> +                                              tag-suffix
> +                                              tag-version-delimiter
> +                                              refresh-pre-releases?)))
> +
> +    (if new-version
> +        (upstream-source
> +         (package name)
> +         (version new-version)
> +         (urls (list url)))
> +        ;; No new release or no tags available.
> +        #f)))

Simply: (and new-version (upstream-source =E2=80=A6)).

It would have been nice to have tests.  I think testing
=E2=80=98latest-git-release=E2=80=99 should be feasible without too much ha=
ssle using
the (guix tests git) infrastructure, as is done in tests/git.scm, with a
package referring to a locally-created repo using a git-reference with a
file:// URL.

Could you send an updated patch?

Thanks,
Ludo=E2=80=99.




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 10 Sep 2021 13:25:01 +0000
Resent-Message-ID: <handler.50359.B50359.163128025626212 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: Sarah Morgensen <iskarian@HIDDEN>, 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163128025626212
          (code B ref 50359); Fri, 10 Sep 2021 13:25:01 +0000
Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 13:24:16 +0000
Received: from localhost ([127.0.0.1]:37260 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mOgVs-0006oh-CH
	for submit <at> debbugs.gnu.org; Fri, 10 Sep 2021 09:24:16 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:32840
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mOgVl-0006oN-Dg
 for 50359 <at> debbugs.gnu.org; Fri, 10 Sep 2021 09:24:14 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631280241;
 bh=dA/Av/IPp+kgHuUFg4yuSP5fWmSIuSvL7H81Ahln9XU=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=SqoVnPZ8gwW5ffYbExWKbL2MUVKaazlkzjU5Oqcir88ToTs3VH3ROEeZYwcDFM4tQ
 n9stD1FGmfyHwPqviIMKHcY/Y6WJUBwYNrub5jkxXUqNyTe9l/UZbiI+ecL8uTFHDj
 EbH4cWejKq+d3yVbfVcGLvXFDmKvqb+1HFcDLFDY=
In-Reply-To: <87o890ddz7.fsf_-_@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <86k0jvkh5v.fsf@HIDDEN> <87h7ez48d3.fsf@HIDDEN>
 <86y28ai7ns.fsf@HIDDEN> <87y28928vh.fsf@HIDDEN>
 <86pmtli4hn.fsf@HIDDEN> <87wnnsyzal.fsf@HIDDEN>
 <87mtomzzu1.fsf@HIDDEN> <87o890ddz7.fsf_-_@HIDDEN>
Date: Fri, 10 Sep 2021 15:23:46 +0200
Message-ID: <87bl50zhr1.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Fri, Sep 10 2021, Ludovic =?UTF-8?Q?Court=C3=A8s?= wrote: > Hello, > >
   This looks very cool! Thanks for taking a look! 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Fri, Sep 10 2021, Ludovic =?UTF-8?Q?Court=C3=A8s?= wrote: > Hello, > >
   This looks very cool! Thanks for taking a look! 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Fri, Sep 10 2021, Ludovic Court=C3=A8s wrote:

> Hello,
>
> This looks very cool!

Thanks for taking a look!

It=E2=80=99s still a WIP, but I think it=E2=80=99s getting there.  :-)

> Xinglu Chen <public@HIDDEN> skribis:
>
>> From f924dbb835425f6b9a5796918125592870391405 Mon Sep 17 00:00:00 2001
>> Message-Id: <f924dbb835425f6b9a5796918125592870391405.1631125652.git.pub=
lic@HIDDEN>
>> From: Xinglu Chen <public@HIDDEN>
>> Date: Fri, 3 Sep 2021 17:50:56 +0200
>> Subject: [PATCH] import: Add 'generic-git' updater.
>>
>> * guix/import/git.scm: New file.
>> * doc/guix.texi (Invoking guix refresh): Document it.
>> * Makefile.am (MODULES): Register it.
>> * guix/git.scm (ls-remote-refs): New procedure.
>>
>> Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>
>
> Overall LGTM; comments below.
>
>> diff --git a/doc/guix.texi b/doc/guix.texi
>> index 36a0c7f5ec..26afb1607a 100644
>> --- a/doc/guix.texi
>> +++ b/doc/guix.texi
>> @@ -11920,6 +11920,33 @@ the updater for @uref{https://launchpad.net, La=
unchpad} packages.
>>  @item generic-html
>>  a generic updater that crawls the HTML page where the source tarball of
>>  the package is hosted, when applicable.
>> +@item generic-git
>
> Please add a newline above.

Noted.

>> +@lisp
>> +(package
>> +  (name "foo")
>> +  ;; ...
>> +  (properties
>> +    '((tag-prefix . "^release0-")
>> +      (tag-suffix . "[a-z]?$")
>> +      (tag-version-delimiter . ":"))))
>> +@end lisp=20=20=20=20=20=20
>
> Very nice.  Perhaps s/tag-/release-tag-/ for clarity?

Good idea.

>> +(define* (ls-remote-refs url #:key tags?)
>> +  "Return the list of references advertised at Git repository URL.  If =
TAGS?
>> +is true, limit to only refs/tags."
>
> To remain consistent with existing naming conventions, I=E2=80=99d call it
> =E2=80=98remote-refs=E2=80=99.
>
>> +  (with-libgit2
>> +   (call-with-temporary-directory
>> +    (lambda (cache-directory)
>> +      (let* ((repository (repository-init cache-directory))
>> +             ;; Create an in-memory remote so we don't touch disk.
>> +             (remote (remote-create-anonymous repository url)))
>
> Too bad we need to create an empty repo; hopefully it costs next to
> nothing though.
>
>> +        (remote-connect remote)
>> +        (remote-disconnect remote)
>> +        (repository-close! repository)
>> +
>> +        (filter include? (map remote-head-name (remote-ls remote))))))))
>
> Use =E2=80=98filter-map=E2=80=99.
>
>> +(define* (get-version-mapping tags #:key prefix suffix delim pre-releas=
es?)
>
> Please add a docstring and remove =E2=80=98get-=E2=80=99 from the name.  =
:-)
>
>> +  (define (guess-delim)
>> +    (let ((total (length tags))
>> +          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
>> +          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
>> +          (underscores (reduce + 0 (map (cut string-count <> #\_) tags)=
)))
>> +      (display (format #t "total: ~d, dots: ~d, dashes ~d, underscores =
~d~%"
>> +              total dots dashes underscores))
>
> Leftover?  (Also display + format.)

Yep.  :-)

> Please spell out =E2=80=98delimiter=E2=80=99 (info "(guix) Formatting Cod=
e").
>
>> +      (cond
>> +       ((>=3D dots (* total 0.35)) ".")
>> +       ((>=3D dashes (* total 0.8)) "-")
>> +       ((>=3D underscores (* total 0.8)) "_")
>> +       (else ""))))
>
> That=E2=80=99s a fancy heuristic.  :-)

Yeah, it was suggested by Sarah, and in my testing it seems to work
pretty well.  :-)

>> +  (let ((mapping (fold alist-cons '() (map get-version tags) tags)))
>> +    (stable-sort! (filter car mapping) entry<?)))
>
> It=E2=80=99s perhaps clearer written like this:
>
>   (stable-sort (filter-map (lambda (tag)
>                              (let ((version (get-version tag)))
>                                (and version (cons version tag))))
>                             tags)
>                entry<?)

Agreed, I will use your suggested version.

>> +(define* (get-latest-tag url #:key prefix suffix delim pre-releases?)
>> +  "Return the latest tag available from the Git repository at URL."
>
> Maybe =E2=80=9Cthe tag corresponding to the latest version=E2=80=9D.

Yeah, as the latest tag might not correspond to a release...

> s/get-latest-tag/latest-tag/
>
>> +  (define (pre-release? tag)
>> +    (any (lambda (rx) (regexp-exec (make-regexp rx regexp/icase) tag))
>> +              %pre-release-words))
>
> Better call =E2=80=98make-regexp=E2=80=99 only once; so you could change
> =E2=80=98%pre-release-words=E2=80=99 to be a list of regexp objects inste=
ad of a list of
> strings.

Noted.

>> +(define (latest-git-tag-version package tag-prefix tag-suffix
>> +                                tag-version-delimiter refresh-pre-relea=
ses?)
>> +  "Given a PACKAGE, the TAG-PREFIX, TAG-SUFFIX, TAG-VERSION-DELIMITER, =
and
>> +REFRESH-PRE-RELEASES?  properties of PACKAGE, returns the latest versio=
n of
>> +PACKAGE."
>
> Maybe s/refresh-pre-releases?/accept-pre-preleases?/

=E2=80=98accept-pre-releases?=E2=80=99 ;-)

> Since this procedure takes a package, it probably doesn=E2=80=99t need th=
e other
> arguments: it can extract them from the package properties, rather than
> doing it at the call site.

Good point.

>> +(define (latest-git-release package)
>> +  "Return the latest release of PACKAGE."
>> +  (let* ((name (package-name package))
>> +         (properties (package-properties package))
>> +         (tag-prefix (assq-ref properties 'tag-prefix))
>> +         (tag-suffix (assq-ref properties 'tag-suffix))
>> +         (tag-version-delimiter (assq-ref properties 'tag-version-delim=
iter))
>> +         (refresh-pre-releases? (assq-ref properties 'refresh-pre-relea=
ses?))
>> +         (old-version (package-version package))
>> +         (url (git-reference-url (origin-uri (package-source package))))
>> +         (new-version (latest-git-tag-version package
>> +                                              tag-prefix
>> +                                              tag-suffix
>> +                                              tag-version-delimiter
>> +                                              refresh-pre-releases?)))
>> +
>> +    (if new-version
>> +        (upstream-source
>> +         (package name)
>> +         (version new-version)
>> +         (urls (list url)))
>> +        ;; No new release or no tags available.
>> +        #f)))
>
> Simply: (and new-version (upstream-source =E2=80=A6)).
>
> It would have been nice to have tests.  I think testing
> =E2=80=98latest-git-release=E2=80=99 should be feasible without too much =
hassle using
> the (guix tests git) infrastructure, as is done in tests/git.scm, with a
> package referring to a locally-created repo using a git-reference with a
> file:// URL.

Thanks for the pointers!  I will look into it.

> Could you send an updated patch?

Sure!  Thanks for the review!  :-)


--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE7XGMVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5WHkQALL76RuYAn3al9OovUTR6OM0PY80
icGunDKkIgsAPba0SWL+3SmohhwfKwtaKlOn7+LSCdTNmV9D9USS/58oGCwQ8z+o
HX3SCHLGhAP66pHPbT/LKv5AZL2yvt/bPP6N8VKGuwoihWF3zVMmH2yjDl8ux8se
hN4Orie8H1b8u2g+edApe7CpZWIX8K2caxy7yDKuyh7YeMOMtPgroy/03o2ucA0m
odIRZXuTr4m/YD6O6gqKn7tRaIfzFPIMfF2FiZqHAB+g+pdRMdLFxLFA5QaW+3+W
J3fRe4YQA08lA/YOobb1sOVT357pT4jVpAK9N3E6yulNil+D/jQfCkAhDulCDRLP
aEO6SWntd8lBBDOvWnbUSuvZvQOKNED9qIOj/OXYmcfNBRze2YUpmchLiGv12Mai
xrQfp6xOplPtDTci5ZYYSp/Y8ygMn+a6ClzA6e6rdru2BMYVjpvGe1fjvzz7laeP
dFBQOpiblgL7K/xVF/xia0LvHmjFKDpm6zwccGZ9p+F/cNUeU4Mh+so3Y4V/Zjvk
PWjktr9Yx+yK5Ue9bbLkEyXPHUAzDWRzz8p85FcKcIOkWUTKFHRWdqZK3wlB8hJ/
9MastbBgN3dIIC9fBTWA0MSPOQN0VtUt8pkFxSxuZiYzVsADTblUbASYB/zWKeAY
92cDAVngc5i36LDp
=RW8F
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 0/3] Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 10 Sep 2021 16:21:01 +0000
Resent-Message-ID: <handler.50359.B50359.163129083721350 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163129083721350
          (code B ref 50359); Fri, 10 Sep 2021 16:21:01 +0000
Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 16:20:37 +0000
Received: from localhost ([127.0.0.1]:39056 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mOjGW-0005YI-Tu
	for submit <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:20:37 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:37004
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mOjGU-0005Y3-Q4
 for 50359 <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:20:35 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631290825;
 bh=uT6ySe8o8ok3i8wheFT5owmUIcivPHnPmsMQpTOrNPU=;
 h=From:To:Subject:In-Reply-To:References:Date;
 b=cUWTgaHAeCQhBsIa+3dcc0F7MaGTJoTFuY8ldZgE8sQ9wusEe6E+oqWx+5AieRfYS
 /kTE19At5wznCfdBRbfAEa/skDb9+IGBrn/r3Tyv+2XZqFHVC9/14R7/ehS5/fWc8R
 gETl5RNU9d72UJzllf/kEIkaiTGefKAU+GdHFK/M=
In-Reply-To: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Message-Id: <cover.1631290349.git.public@HIDDEN>
Date: Fri, 10 Sep 2021 18:20:23 +0200
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  Changes since v1: * Add =?UTF-8?Q?=E2=80=98remote-refs=E2=80=99?= procedure to (guix
    git) (written by Sarah Morgensen). Add tests for it too. * Make the updater
    try to guess the delimiter if none was provided (also written by Sarah). 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.9 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  Changes since v1: * Add =?UTF-8?Q?=E2=80=98remote-refs=E2=80=99?= procedure to (guix
    git) (written by Sarah Morgensen). Add tests for it too. * Make the updater
    try to guess the delimiter if none was provided (also written by Sarah). 
 
 Content analysis details:   (1.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Changes since v1:

* Add =E2=80=98remote-refs=E2=80=99 procedure to (guix git) (written by Sar=
ah
  Morgensen).  Add tests for it too.

* Make the updater try to guess the delimiter if none was provided (also
  written by Sarah).

* Honor the =E2=80=98accept-pre-releases?=E2=80=99 property to include pre-=
releases when
  looking for tags.

* Various regexp improvements.

* Add tests for the updater.

* Some fixes to (guix tests git).

Xinglu Chen (3):
  tests: git: Don't read from the users global Git config file.
  tests: git: Make 'tag' directive non-interactive.
  import: Add 'generic-git' updater.

 Makefile.am          |   2 +
 doc/guix.texi        |  32 +++++++
 guix/git.scm         |  37 ++++++++
 guix/import/git.scm  | 218 +++++++++++++++++++++++++++++++++++++++++++
 guix/tests/git.scm   |   6 +-
 tests/git.scm        |  26 ++++++
 tests/import-git.scm | 204 ++++++++++++++++++++++++++++++++++++++++
 7 files changed, 523 insertions(+), 2 deletions(-)
 create mode 100644 guix/import/git.scm
 create mode 100644 tests/import-git.scm


base-commit: 9875f9bca3976bf3576eab9be42164fde454597e
=2D-=20
2.33.0




--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmE7hccVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5fKwP/ilcewnTalQmrsgdluSaqBHUIReu
PparJfqAWqkKbrwgQOxxo7Y9Qm+qqc6OPpp97IGJ2lciWtnnF5Dz9UOi37hfuNdi
xJDWedHesXVOPx4UlHN71c0679MxW/kbmpnRBrmS64/dvzgBk5LYj3W9BxhJSSKa
lm90431vkZrdn9yf7CCaF2yuYCJQ37Z1OD4EXbvJlwOj+urdJJPL2+bXMxOQK37b
SOO/lRH0+d1/M2TOcKEnYTvak/3VkhNvLI3y4NYJj0OkEJnkYBMXiKQ8ualH/f17
UX7BAcZTg1Dm9G7/i0vafa9AjepHrzYTCJnjrXiU53ERi+8kwhO08ocFB47N8W+q
04DiWGtxCwIwEWvHHsdi4bn2+K3cVc3wFUiVcXaXpLOYeG31fQ2826Sj0Gl3BpCd
vlG9uqQvYks2/REFJtL2D7ue7G0S16NdpjIayP6kL6r8INvMdwYNIoeIePpg/OsB
ou/bMHhXGShnRon10Gx5O+1N4k+5o6OSZe/3/Bkx3Bt5YeazHaZSyTJHaCSYRPUn
gBI6KAvUG62lfCATPgGLhUfvYiMgu079K9xQIvjYDhHiLCsGVudJVcywY8RGBeRe
PK+dgpxreG7TEHPUH1g122UpK58r4LNx8o7DYp122uEqjzDYt5fOWh+ewMxKNW7y
CEZsJ+Xc9rtMxTN/
=KcN9
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 1/3] tests: git: Don't read from the users global Git config file.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 10 Sep 2021 16:22:01 +0000
Resent-Message-ID: <handler.50359.B50359.163129088921505 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 50359 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Sarah Morgensen <iskarian@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163129088921505
          (code B ref 50359); Fri, 10 Sep 2021 16:22:01 +0000
Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 16:21:29 +0000
Received: from localhost ([127.0.0.1]:39064 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mOjHN-0005an-64
	for submit <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:21:29 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:37028
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mOjHK-0005aU-VP
 for 50359 <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:21:27 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631290879;
 bh=D1fehBRccifXBpnZDbd2xCB6H0KHAxZb9ggk4wYluzo=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=VzHe2oGMqjxV6gRWm3m2hRccE4+9farsMJ69LkE8NdVQ/jCig24BtW+Jc45ca8Bne
 l9lAfXeTPlbjaYyOoCFXoI+nRRfRyoU/Z8pOn2Gm1lGF/Ru3u1dDtaM0NPBbjU67gq
 rziRMTdkN9oeUilib174bvDBOf/Qd49Ko8cv7Ijg=
In-Reply-To: <cover.1631290349.git.public@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Message-Id: <04db258c6eb1b90dca9f2b32a7bc40ed8e39c471.1631290349.git.public@HIDDEN>
Date: Fri, 10 Sep 2021 18:21:17 +0200
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: * guix/tests/git (populate-git-repository): Set the
 GIT_CONFIG_GLOBAL
 environment variable to the temporary Git config file. --- Without this,
 Git would try to sign the commits when running tests/git. [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.9 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  * guix/tests/git (populate-git-repository): Set the GIT_CONFIG_GLOBAL
    environment variable to the temporary Git config file. --- Without this,
   Git would try to sign the commits when running tests/git. [...] 
 
 Content analysis details:   (1.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

* guix/tests/git (populate-git-repository): Set the GIT_CONFIG_GLOBAL
environment variable to the temporary Git config file.
---
Without this, Git would try to sign the commits when running
tests/git.scm, this was beacuse I had set =E2=80=98gpgSign =3D true=E2=80=
=99 in
~/.config/git/config.  Setting the environment variable would remove
this impurity.

 guix/tests/git.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/guix/tests/git.scm b/guix/tests/git.scm
index b8e5f7e643..e11541e83b 100644
--- a/guix/tests/git.scm
+++ b/guix/tests/git.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright =C2=A9 2019, 2020 Ludovic Court=C3=A8s <ludo@HIDDEN>
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -53,6 +54,7 @@ Return DIRECTORY on success."
        (with-environment-variables
         `(("GIT_CONFIG_NOSYSTEM" "1")
           ("GIT_ATTR_NOSYSTEM" "1")
+          ("GIT_CONFIG_GLOBAL" ,(string-append home "/.gitconfig"))
           ("HOME" ,home))
         (apply invoke (git-command) "-C" directory
                command args)))))
--=20
2.33.0







Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 2/3] tests: git: Make 'tag' directive non-interactive.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 10 Sep 2021 16:22:02 +0000
Resent-Message-ID: <handler.50359.B50359.163129089421526 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 50359 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Sarah Morgensen <iskarian@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163129089421526
          (code B ref 50359); Fri, 10 Sep 2021 16:22:02 +0000
Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 16:21:34 +0000
Received: from localhost ([127.0.0.1]:39067 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mOjHS-0005b8-D9
	for submit <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:21:34 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:37040
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mOjHQ-0005al-VA
 for 50359 <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:21:33 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631290885;
 bh=dcGafiE09F0lvz11QdAsd9lHlaLZq97KCXKZkOyaaJ4=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=a/iuZOJiR5lcoTA4R1FQ3KzfcJtqOguytFqboetM8r2KE+Wb71lGRIBQJqOA3IkyZ
 B48zgld6EkKMoBoY5Vfoqa/6uMeA1jnqbgou21If2wpUryGbotyeajk9O9QGn3016/
 Ydmy08qRAsTKF1GvYv/f+I/ELc3q3twcGgOwvuRs=
In-Reply-To: <cover.1631290349.git.public@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Message-Id: <29b4b579f8fe8fc3406a48224c5af28d9e70b865.1631290349.git.public@HIDDEN>
Date: Fri, 10 Sep 2021 18:21:25 +0200
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  When running 'git tag TAGNAME', Git will open up the user's
 default text editor to make them write a message. This is not desirable when
 running tests. * guix/tests/git.scm (populate-git-repository): Make the 'tag'
 directive take an additional argument, and pass it to the '-m' flag. ---
 guix/tests/git.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 d [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.9 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  When running 'git tag TAGNAME', Git will open up the user's
    default text editor to make them write a message. This is not desirable when
    running tests. * guix/tests/git.scm (populate-git-repository): Make the 'tag'
    directive take an additional argument, and pass it to the '-m' flag. ---
   guix/tests/git.scm | 4 ++-- 1 file changed, 2 insertions(+), 2 d [...] 
 
 Content analysis details:   (1.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

When running 'git tag TAGNAME', Git will open up the user's default text
editor to make them write a message.  This is not desirable when running
tests.

* guix/tests/git.scm (populate-git-repository): Make the 'tag' directive take
an additional argument, and pass it to the '-m' flag.
---
 guix/tests/git.scm | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/guix/tests/git.scm b/guix/tests/git.scm
index e11541e83b..e8d4946e87 100644
--- a/guix/tests/git.scm
+++ b/guix/tests/git.scm
@@ -87,8 +87,8 @@ Return DIRECTORY on success."
       ((('commit text ('signer fingerprint)) rest ...)
        (git "commit" "-m" text (string-append "--gpg-sign=" fingerprint))
        (loop rest))
-      ((('tag name) rest ...)
-       (git "tag" name)
+      ((('tag name text) rest ...)
+       (git "tag" "-m" text name)
        (loop rest))
       ((('branch name) rest ...)
        (git "branch" name)
-- 
2.33.0







Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 3/3] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 10 Sep 2021 16:22:02 +0000
Resent-Message-ID: <handler.50359.B50359.163129090521550 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: 50359 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, Sarah Morgensen <iskarian@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163129090521550
          (code B ref 50359); Fri, 10 Sep 2021 16:22:02 +0000
Received: (at 50359) by debbugs.gnu.org; 10 Sep 2021 16:21:45 +0000
Received: from localhost ([127.0.0.1]:39070 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mOjHc-0005bV-Ji
	for submit <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:21:45 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:37048
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mOjHW-0005az-6X
 for 50359 <at> debbugs.gnu.org; Fri, 10 Sep 2021 12:21:43 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631290890;
 bh=oLzOkXkhGcUbIClE8yUyURsLSov7O8LsDzC18JfE0lI=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=frdrsRGRkIXVUBikuAd5Xo1n4l49fHjJ5/tzM+W0Jhmg1y9beC9KJQvOFLQRQ5WiD
 Si/I4zdrAJ0yWIdmAPViXmcr7mbIGUfdD2hRCQcoe4KA32jsUvUcSWkwM8+PDv8eE1
 ADVs1bM5GD/qR911pWLNR9pvX1sdXpmZA6ZNjapA=
In-Reply-To: <cover.1631290349.git.public@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
Message-Id: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
Date: Fri, 10 Sep 2021 18:21:30 +0200
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview: * guix/git.scm (ls-remote-refs): New procedure. *
 tests/git.scm
 ("remote-refs" "remote-refs: only tags"): New tests. * guix/import/git.scm:
 New file. * doc/guix.texi (Invoking guix refresh): Document [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 T_SPF_HELO_TEMPERROR   SPF: test of HELO record failed (temperror)
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.9 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  * guix/git.scm (ls-remote-refs): New procedure. * tests/git.scm
    ("remote-refs" "remote-refs: only tags"): New tests. * guix/import/git.scm:
    New file. * doc/guix.texi (Invoking guix refresh): Document [...] 
 
 Content analysis details:   (1.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

* guix/git.scm (ls-remote-refs): New procedure.
* tests/git.scm ("remote-refs" "remote-refs: only tags"): New tests.
* guix/import/git.scm: New file.
* doc/guix.texi (Invoking guix refresh): Document it.
* tests/import-git.scm: New test file.
* Makefile.am (MODULES, SCM_TESTS): Register the new files.

Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>
---
 Makefile.am          |   2 +
 doc/guix.texi        |  32 +++++++
 guix/git.scm         |  37 ++++++++
 guix/import/git.scm  | 218 +++++++++++++++++++++++++++++++++++++++++++
 tests/git.scm        |  26 ++++++
 tests/import-git.scm | 204 ++++++++++++++++++++++++++++++++++++++++
 6 files changed, 519 insertions(+)
 create mode 100644 guix/import/git.scm
 create mode 100644 tests/import-git.scm

diff --git a/Makefile.am b/Makefile.am
index dd40a5ad9c..c71d9a29e2 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -254,6 +254,7 @@ MODULES =3D					\
   guix/import/egg.scm   			\
   guix/import/elpa.scm   			\
   guix/import/gem.scm				\
+  guix/import/git.scm                           \
   guix/import/github.scm   			\
   guix/import/gnome.scm				\
   guix/import/gnu.scm				\
@@ -473,6 +474,7 @@ SCM_TESTS =3D					\
   tests/graph.scm				\
   tests/gremlin.scm				\
   tests/hackage.scm				\
+  tests/import-git.scm				\
   tests/import-utils.scm			\
   tests/inferior.scm				\
   tests/lint.scm				\
diff --git a/doc/guix.texi b/doc/guix.texi
index 220499503d..dbaa000006 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -11921,6 +11921,38 @@ the updater for @uref{https://launchpad.net, Launc=
hpad} packages.
 @item generic-html
 a generic updater that crawls the HTML page where the source tarball of
 the package is hosted, when applicable.
+
+@item generic-git
+a generic updater for packages hosted on Git repositories.  It tries to
+be smart about parsing Git tag names, but if it is not able to parse the
+tag name and compare tags correctly, users can define the following
+properties for a package.
+
+@itemize
+@item @code{release-tag-prefix}: a regular expression for matching a prefi=
x of
+the tag name.
+
+@item @code{release-tag-suffix}: a regular expression for matching a suffi=
x of
+the tag name.
+
+@item @code{release-tag-version-delimiter}: a string used as the delimiter=
 in
+the tag name for separating the numbers of the version.
+@end itemize
+
+@lisp
+(package
+  (name "foo")
+  ;; ...
+  (properties
+    '((release-tag-prefix . "^release0-")
+      (release-tag-suffix . "[a-z]?$")
+      (release-tag-version-delimiter . ":"))))
+@end lisp
+
+By default, the updater will ignore pre-releases; to make it also look
+for pre-releases, set the @code{accept-pre-releases?} property to
+@code{#t}.
+
 @end table
=20
 For instance, the following command only checks for updates of Emacs
diff --git a/guix/git.scm b/guix/git.scm
index acc48fd12f..dc3d3afd02 100644
--- a/guix/git.scm
+++ b/guix/git.scm
@@ -57,6 +57,8 @@
             commit-difference
             commit-relation
=20
+            remote-refs
+
             git-checkout
             git-checkout?
             git-checkout-url
@@ -571,6 +573,41 @@ objects: 'ancestor (meaning that OLD is an ancestor of=
 NEW), 'descendant, or
               (if (set-contains? oldest new)
                   'descendant
                   'unrelated))))))
+
+;;
+;;; Remote operations.
+;;;
+
+(define* (remote-refs url #:key tags?)
+  "Return the list of references advertised at Git repository URL.  If TAG=
S?
+is true, limit to only refs/tags."
+  (define (ref? ref)
+    ;; Like `git ls-remote --refs', only show actual references.
+    (and (string-prefix? "refs/" ref)
+         (not (string-suffix? "^{}" ref))))
+
+  (define (tag? ref)
+    (string-prefix? "refs/tags/" ref))
+
+  (define (include? ref)
+    (and (ref? ref)
+         (or (not tags?) (tag? ref))))
+
+  (with-libgit2
+   (call-with-temporary-directory
+    (lambda (cache-directory)
+      (let* ((repository (repository-init cache-directory))
+             ;; Create an in-memory remote so we don't touch disk.
+             (remote (remote-create-anonymous repository url)))
+        (remote-connect remote)
+        (remote-disconnect remote)
+        (repository-close! repository)
+
+        (filter-map (lambda (remote)
+                      (let ((name (remote-head-name remote)))
+                        (and (include? name)
+                             name)))
+                    (remote-ls remote)))))))
=20
 
 ;;;
diff --git a/guix/import/git.scm b/guix/import/git.scm
new file mode 100644
index 0000000000..b69f9d70f2
--- /dev/null
+++ b/guix/import/git.scm
@@ -0,0 +1,218 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN>
+;;; Copyright =C2=A9 2021 Sarah Morgensen <iskarian@HIDDEN>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import git)
+  #:use-module (guix build utils)
+  #:use-module (guix diagnostics)
+  #:use-module (guix git)
+  #:use-module (guix git-download)
+  #:use-module (guix i18n)
+  #:use-module (guix packages)
+  #:use-module (guix upstream)
+  #:use-module (guix utils)
+  #:use-module (ice-9 format)
+  #:use-module (ice-9 match)
+  #:use-module (ice-9 rdelim)
+  #:use-module (ice-9 regex)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-26)
+  #:use-module (srfi srfi-34)
+  #:use-module (srfi srfi-35)
+  #:export (%generic-git-updater
+
+            ;; For tests.
+            latest-git-tag-version))
+
+;;; Commentary:
+;;;
+;;; This module provides a generic package updater for packages hosted on =
Git
+;;; repositories.
+;;;
+;;; It tries to be smart about tag names, but if it is not automatically a=
ble
+;;; to parse the tag names correctly, users can set the `release-tag-prefi=
x',
+;;; `release-tag-suffix' and `release-tag-version-delimiter' properties of=
 the
+;;; package to make the updater parse the Git tag name correctly.
+;;;
+;;; Code:
+
+;;; Errors & warnings
+
+(define-condition-type &git-no-valid-tags-error &error
+  git-no-valid-tags-error?)
+
+(define (git-no-valid-tags-error)
+  (raise (condition (&message (message "no valid tags found"))
+                    (&git-no-valid-tags-error))))
+
+(define-condition-type &git-no-tags-error &error
+  git-no-tags-error?)
+
+(define (git-no-tags-error)
+  (raise (condition (&message (message "no tags were found"))
+                    (&git-no-tags-error))))
+
+
+;;; Updater
+
+(define %pre-release-words
+  '("alpha" "beta" "rc" "dev" "test"))
+
+(define %pre-release-rx
+  (map (cut make-regexp <> regexp/icase) %pre-release-words))
+
+(define* (version-mapping tags #:key prefix suffix delim pre-releases?)
+  "Given a list of Git TAGS, return a association list where the car is the
+version corresponding to the tag, and the cdr is the name of the tag."
+  (define (guess-delimiter)
+    (let ((total (length tags))
+          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
+          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
+          (underscores (reduce + 0 (map (cut string-count <> #\_) tags))))
+      (cond
+       ((>=3D dots (* total 0.35)) ".")
+       ((>=3D dashes (* total 0.8)) "-")
+       ((>=3D underscores (* total 0.8)) "_")
+       (else ""))))
+
+  (define delim-rx (regexp-quote (or delim (guess-delimiter))))
+  (define suffix-rx  (string-append (or suffix "") "$"))
+
+=20=20
+  (define prefix-rx (string-append "^" (or prefix "[^[:digit:]]*")))
+  (define pre-release-rx
+    (if pre-releases?
+        (string-append ".*(" (string-join %pre-release-words "|") ").*")
+        ""))
+
+  (define tag-rx
+    (string-append prefix-rx "([[:digit:]][^" delim-rx "[:punct:]]*"
+                   "(" delim-rx "[^[:punct:]" delim-rx "]+)"
+                   ;; If there is are no delimiters, it could mean that the
+                   ;; version just contains one number (e.g., "2"), thus, =
use
+                   ;; "*" instead of "+" to match zero or more numbers.
+                   (if (string=3D? delim-rx "") "*" "+")
+                   pre-release-rx ")" suffix-rx))
+
+  (define (get-version tag)
+    (let ((tag-match (regexp-exec (make-regexp tag-rx) tag)))
+      (and tag-match
+           (regexp-substitute/global
+            #f delim-rx (match:substring tag-match 1)
+            ;; Don't insert "." if there aren't any delimiters in the first
+            ;; place.
+            'pre (if (string=3D? delim-rx "") "" ".") 'post))))
+
+  (define (entry<? a b)
+    (eq? (version-compare (car a) (car b)) '<))
+
+  (stable-sort (filter-map (lambda (tag)
+                             (let ((version (get-version tag)))
+                               (and version (cons version tag))))
+                           tags)
+               entry<?))
+
+(define* (latest-tag url #:key prefix suffix delim pre-releases?)
+  "Return the latest tag available from the Git repository at URL."
+  (define (pre-release? tag)
+    (any (cut regexp-exec <> tag)
+         %pre-release-rx))
+
+  (let* ((tags (map (cut string-drop <> (string-length "refs/tags/"))
+                    (remote-refs url #:tags? #t)))
+         (versions->tags
+          (version-mapping (if pre-releases?
+                               tags
+                               (filter (negate pre-release?) tags))
+                           #:prefix prefix
+                           #:suffix suffix
+                           #:delim delim
+                           #:pre-releases? pre-releases?)))
+    (cond
+     ((null? tags)
+      (git-no-tags-error))
+     ((null? versions->tags)
+      (git-no-valid-tags-error))
+     (else
+      (match (last versions->tags)
+        ((version . tag)
+         (values version tag)))))))
+
+(define (latest-git-tag-version package)
+  "Given a PACKAGE, return the latest version of it, or #f if the latest v=
ersion
+could not be determined."
+  (guard (c ((or (git-no-tags-error? c) (git-no-valid-tags-error? c))
+             (warning (or (package-field-location package 'source)
+                          (package-location package))
+                      (G_ "~a for ~a~%")
+                      (condition-message c)
+                      (package-name package))
+             #f)
+            ((eq? (exception-kind c) 'git-error)
+             (warning (or (package-field-location package 'source)
+                          (package-location package))
+                      (G_ "failed to fetch Git repository for ~a~%")
+                      (package-name package))
+             #f))
+    (let* ((source (package-source package))
+           (url (git-reference-url (origin-uri source)))
+           (properties (package-properties package))
+           (tag-prefix (assq-ref properties 'release-tag-prefix))
+           (tag-suffix (assq-ref properties 'release-tag-suffix))
+           (tag-version-delimiter (assq-ref properties 'release-tag-versio=
n-delimiter))
+           (refresh-pre-releases? (assq-ref properties 'accept-pre-release=
s?)))
+      (latest-tag url
+                  #:prefix tag-prefix
+                  #:suffix tag-suffix
+                  #:delim tag-version-delimiter
+                  #:pre-releases? refresh-pre-releases?))))
+
+(define (git-package? package)
+  "Whether the origin of PACKAGE is a Git repostiory."
+  (match (package-source package)
+    ((? origin? origin)
+     (and (eq? (origin-method origin) git-fetch)
+          (git-reference? (origin-uri origin))))
+    (_ #f)))
+
+(define (latest-git-release package)
+  "Return the latest release of PACKAGE."
+  (let* ((name (package-name package))
+         (old-version (package-version package))
+         (url (git-reference-url (origin-uri (package-source package))))
+         (new-version (latest-git-tag-version package)))
+
+    (and new-version
+         (upstream-source
+          (package name)
+          (version new-version)
+          (urls (list url))))))
+
+(define %generic-git-updater
+  (upstream-updater
+   (name 'generic-git)
+   (description "Updater for packages hosted on Git repositories")
+   (pred git-package?)
+   (latest latest-git-release)))
diff --git a/tests/git.scm b/tests/git.scm
index aa4f03ca62..1f4fbb9adb 100644
--- a/tests/git.scm
+++ b/tests/git.scm
@@ -1,5 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
 ;;; Copyright =C2=A9 2019, 2020 Ludovic Court=C3=A8s <ludo@HIDDEN>
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -161,4 +162,29 @@
               (commit-relation master1 merge)
               (commit-relation merge master1))))))
=20
+(test-equal "remote-refs"
+  '("refs/heads/develop" "refs/heads/master"
+    "refs/tags/v1.0" "refs/tags/v1.1")
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "v1.0" "release-1.0")
+        (branch "develop")
+        (checkout "develop")
+        (add "b.txt" "B")
+        (commit "Second commit")
+        (tag "v1.1" "release-1.1"))
+    (remote-refs directory)))
+
+(test-equal "remote-refs: only tags"
+ '("refs/tags/v1.0" "refs/tags/v1.1")
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "v1.0" "Release 1.0")
+        (add "b.txt" "B")
+        (commit "Second commit")
+        (tag "v1.1" "Release 1.1"))
+    (remote-refs directory #:tags? #t)))
+
 (test-end "git")
diff --git a/tests/import-git.scm b/tests/import-git.scm
new file mode 100644
index 0000000000..9ef724c09c
--- /dev/null
+++ b/tests/import-git.scm
@@ -0,0 +1,204 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (test-import-git)
+  #:use-module (git)
+  #:use-module (guix git)
+  #:use-module (guix tests)
+  #:use-module (guix packages)
+  #:use-module (guix import git)
+  #:use-module (guix git-download)
+  #:use-module (guix tests git)
+  #:use-module (guix build utils)
+  #:use-module (srfi srfi-1)
+  #:use-module (srfi srfi-64))
+
+;; Test the (guix import git) tools.
+
+(test-begin "git")
+
+(define* (make-package directory version #:optional (properties '()))
+  (dummy-package "test-package"
+    (version version)
+    (properties properties)
+    (source
+     (origin
+       (method git-fetch)
+       (uri (git-reference
+             (url (string-append "file://" directory))
+             (commit version)))
+       (sha256
+        (base32
+         "0000000000000000000000000000000000000000000000000000"))))))
+
+(test-equal "latest-git-tag-version: no custom prefix, suffix, and delimit=
er"
+  "1.0.1"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "1.0.1" "Release 1.0.1"))
+    (let ((package (make-package directory "1.0.0")))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: custom prefix, no suffix and delimite=
r"
+  "1.0.1"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "prefix-1.0.1" "Release 1.0.1"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((release-tag-prefix . "prefix-")))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: custom suffix, no prefix and delimite=
r"
+  "1.0.1"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "1.0.1-suffix-123" "Release 1.0.1"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((release-tag-suffix . "-suffix-[0-9]*")=
))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: custom delimiter, no prefix and suffi=
x"
+  "2021.09.07"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "2021-09-07" "Release 2021-09-07"))
+    (let ((package (make-package directory "2021-09-06"
+                                 '((release-tag-version-delimiter . "-")))=
))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: empty delimiter, no prefix and suffix"
+  "20210907"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "20210907" "Release 20210907"))
+    (let ((package (make-package directory "20210906"
+                                 '((release-tag-version-delimiter . "")))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: custom prefix and suffix, no delimite=
r"
+  "2.0.0"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "Release-2.0.0suffix-1" "Release 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((release-tag-prefix . "Release-")
+                                   (release-tag-suffix . "suffix-[0-9]")))=
))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: custom prefix, suffix, and delimiter"
+  "2.0.0"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "Release-2_0_0suffix-1" "Release 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((release-tag-prefix . "Release-")
+                                   (release-tag-suffix . "suffix-[0-9]")
+                                   (release-tag-version-delimiter . "_")))=
))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: only pre-releases available"
+  #f
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "2.0.0-rc1" "Release candidate for 2.0.0"))
+    (let ((package (make-package directory "1.0.0")))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: accept pre-releases"
+  "2.0.0-rc1"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "2.0.0-rc1" "Release candidate for 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((accept-pre-releases? . #t)))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: accept pre-releases, and custom prefi=
x"
+  "2.0.0-rc1"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "version-2.0.0-rc1" "Release candidate for 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((accept-pre-releases? . #t)
+                                   (release-tag-prefix . "version-")))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: accept pre-releases, and custom suffi=
x"
+  "2.0.0-rc1"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "2.0.0-rc1-suffix" "Release candidate for 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((accept-pre-releases? . #t)
+                                   (release-tag-suffix . "-suffix")))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: accept pre-releases, and custom suffi=
x and prefix"
+  "2.0.0-alpha"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "prefix123-2.0.0-alpha-suffix" "Alpha release for 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((accept-pre-releases? . #t)
+                                   (release-tag-prefix . "prefix[0-9]{3}-")
+                                   (release-tag-suffix . "-suffix")))))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: accept pre-releases, and custom suffi=
x, prefix, and delimiter"
+  "2.0.0-alpha"
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "prefix123-2_0_0-alpha-suffix" "Alpha release for 2.0.0"))
+    (let ((package (make-package directory "1.0.0"
+                                 '((accept-pre-releases? . #t)
+                                   (release-tag-prefix . "prefix[0-9]{3}-")
+                                   (release-tag-suffix . "-suffix")
+                                   (release-tag-version-delimiter . "_")))=
))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: no tags found"
+  #f
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit"))
+    (let ((package (make-package directory "1.0.0")))
+      (latest-git-tag-version package))))
+
+(test-equal "latest-git-tag-version: no valid tags found"
+  #f
+  (with-temporary-git-repository directory
+      '((add "a.txt" "A")
+        (commit "First commit")
+        (tag "Test" "Test tag"))
+    (let ((package (make-package directory "1.0.0")))
+      (latest-git-tag-version package))))
+
+(test-end "git")
--=20
2.33.0







Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 2/3] tests: git: Make 'tag' directive non-interactive.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 13 Sep 2021 08:04:02 +0000
Resent-Message-ID: <handler.50359.B50359.163152021211948 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org, Sarah Morgensen <iskarian@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163152021211948
          (code B ref 50359); Mon, 13 Sep 2021 08:04:02 +0000
Received: (at 50359) by debbugs.gnu.org; 13 Sep 2021 08:03:32 +0000
Received: from localhost ([127.0.0.1]:44163 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mPgw7-00036e-MO
	for submit <at> debbugs.gnu.org; Mon, 13 Sep 2021 04:03:31 -0400
Received: from eggs.gnu.org ([209.51.188.92]:60514)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1mPgw5-00036L-NA
 for 50359 <at> debbugs.gnu.org; Mon, 13 Sep 2021 04:03:30 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:57106)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1mPgvy-0008Dp-SV; Mon, 13 Sep 2021 04:03:22 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=47238 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1mPgvy-00017K-Gc; Mon, 13 Sep 2021 04:03:22 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <29b4b579f8fe8fc3406a48224c5af28d9e70b865.1631290349.git.public@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 27 Fructidor an 229 de la =?UTF-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 13 Sep 2021 10:03:20 +0200
In-Reply-To: <29b4b579f8fe8fc3406a48224c5af28d9e70b865.1631290349.git.public@HIDDEN>
 (Xinglu Chen's message of "Fri, 10 Sep 2021 18:21:25 +0200")
Message-ID: <87bl4wc37b.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.3 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.3 (-)

Hello,

Xinglu Chen <public@HIDDEN> skribis:

> When running 'git tag TAGNAME', Git will open up the user's default text
> editor to make them write a message.  This is not desirable when running
> tests.
>
> * guix/tests/git.scm (populate-git-repository): Make the 'tag' directive =
take
> an additional argument, and pass it to the '-m' flag.

[...]

> -      ((('tag name) rest ...)
> -       (git "tag" name)
> +      ((('tag name text) rest ...)
> +       (git "tag" "-m" text name)

LGTM, but you need to update at least tests/channels.scm accordingly.

Thanks,
Ludo=E2=80=99.




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 3/3] import: Add 'generic-git' updater.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 13 Sep 2021 08:08:02 +0000
Resent-Message-ID: <handler.50359.B50359.163152043312338 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: 50359 <at> debbugs.gnu.org, Sarah Morgensen <iskarian@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163152043312338
          (code B ref 50359); Mon, 13 Sep 2021 08:08:02 +0000
Received: (at 50359) by debbugs.gnu.org; 13 Sep 2021 08:07:13 +0000
Received: from localhost ([127.0.0.1]:44175 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mPgzg-0003Cw-MV
	for submit <at> debbugs.gnu.org; Mon, 13 Sep 2021 04:07:12 -0400
Received: from eggs.gnu.org ([209.51.188.92]:33508)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1mPgze-0003Ci-Du
 for 50359 <at> debbugs.gnu.org; Mon, 13 Sep 2021 04:07:11 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:57238)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1mPgzY-0003AR-Py; Mon, 13 Sep 2021 04:07:04 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=47240 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1mPgzY-00029Y-Fx; Mon, 13 Sep 2021 04:07:04 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 27 Fructidor an 229 de la =?UTF-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 13 Sep 2021 10:07:03 +0200
In-Reply-To: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
 (Xinglu Chen's message of "Fri, 10 Sep 2021 18:21:30 +0200")
Message-ID: <875yv4c314.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -0.3 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.3 (-)

Xinglu Chen <public@HIDDEN> skribis:

> * guix/git.scm (ls-remote-refs): New procedure.
> * tests/git.scm ("remote-refs" "remote-refs: only tags"): New tests.
> * guix/import/git.scm: New file.
> * doc/guix.texi (Invoking guix refresh): Document it.
> * tests/import-git.scm: New test file.
> * Makefile.am (MODULES, SCM_TESTS): Register the new files.
>
> Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>

Nice, thanks for writing the tests!

> +++ b/tests/git.scm
> @@ -1,5 +1,6 @@
>  ;;; GNU Guix --- Functional package management for GNU
>  ;;; Copyright =C2=A9 2019, 2020 Ludovic Court=C3=A8s <ludo@HIDDEN>
> +;;; Copyright =C2=A9 2021 Xinglu Chen <public@HIDDEN
>  ;;;
>  ;;; This file is part of GNU Guix.
>  ;;;
> @@ -161,4 +162,29 @@
>                (commit-relation master1 merge)
>                (commit-relation merge master1))))))
>=20=20
> +(test-equal "remote-refs"
> +  '("refs/heads/develop" "refs/heads/master"
> +    "refs/tags/v1.0" "refs/tags/v1.1")
> +  (with-temporary-git-repository directory
> +      '((add "a.txt" "A")
> +        (commit "First commit")
> +        (tag "v1.0" "release-1.0")
> +        (branch "develop")
> +        (checkout "develop")
> +        (add "b.txt" "B")
> +        (commit "Second commit")
> +        (tag "v1.1" "release-1.1"))
> +    (remote-refs directory)))

[...]

> +(test-equal "latest-git-tag-version: no custom prefix, suffix, and delim=
iter"
> +  "1.0.1"
> +  (with-temporary-git-repository directory
> +      '((add "a.txt" "A")
> +        (commit "First commit")
> +        (tag "1.0.1" "Release 1.0.1"))
> +    (let ((package (make-package directory "1.0.0")))
> +      (latest-git-tag-version package))))

I think that for each of these tests that uses the =E2=80=98git=E2=80=99 co=
mmand under
the hood, you=E2=80=99ll need something like what =E2=80=98tests/git.scm=E2=
=80=99 does:

  (unless (which (git-command)) (test-skip 1))
  (test-equal =E2=80=A6)

It=E2=80=99d admittedly annoying to have this boilerplate, but I can=E2=80=
=99t think of
a better solution.

Could you send an updated version?  Then we=E2=80=99ll be all set!

Thank you,
Ludo=E2=80=99.




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 3/3] import: Add 'generic-git' updater.
Resent-From: iskarian@HIDDEN
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 15 Sep 2021 08:45:02 +0000
Resent-Message-ID: <handler.50359.B50359.16316954813295 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: "Xinglu Chen" <public@HIDDEN>, 50359 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.16316954813295
          (code B ref 50359); Wed, 15 Sep 2021 08:45:02 +0000
Received: (at 50359) by debbugs.gnu.org; 15 Sep 2021 08:44:41 +0000
Received: from localhost ([127.0.0.1]:50831 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mQQX3-0000r5-4S
	for submit <at> debbugs.gnu.org; Wed, 15 Sep 2021 04:44:41 -0400
Received: from out2.migadu.com ([188.165.223.204]:54869)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mQQX0-0000qv-SA
 for 50359 <at> debbugs.gnu.org; Wed, 15 Sep 2021 04:44:40 -0400
MIME-Version: 1.0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1631695477;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=/560hck3gl4aDjQNlJGLO5RXT9aQriQgkBb0VWZc53k=;
 b=jfkmYfJ1AQBid/roEbS9JshRdMUEFvHjvLCgRARFtMi/kw+BRKy5cn3firqZgVzdUwpimg
 5XbGgm/JDzgdRwSLpcHra562cqhHA0npRoJyFNSh6IDWSwmHHhoS6QGpLU/3QM/bs0NUvp
 QLEk/kuwseIn+BzK6ni78bmNSW6u3P4=
Date: Wed, 15 Sep 2021 08:44:35 +0000
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
From: iskarian@HIDDEN
Message-ID: <44519ae5dfdf3702dc73903b80314a6a@HIDDEN>
In-Reply-To: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
References: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
 <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 2.0 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hi, September 10, 2021 9:21 AM,
 "Xinglu Chen" <public@HIDDEN>
 wrote: > * guix/git.scm (ls-remote-refs): New procedure. > * tests/git.scm
 ("remote-refs" "remote-refs: only tags"): New tests. > * guix/import/git.scm:
 New file. > * doc/guix.texi (Invoking guix refresh): D [...] 
 Content analysis details:   (2.0 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [188.165.223.204 listed in wl.mailspike.net]
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 1.0 (+)

Hi,=0A=0ASeptember 10, 2021 9:21 AM, "Xinglu Chen" <public@HIDDEN>=
 wrote:=0A=0A> * guix/git.scm (ls-remote-refs): New procedure.=0A> * test=
s/git.scm ("remote-refs" "remote-refs: only tags"): New tests.=0A> * guix=
/import/git.scm: New file.=0A> * doc/guix.texi (Invoking guix refresh): D=
ocument it.=0A> * tests/import-git.scm: New test file.=0A> * Makefile.am =
(MODULES, SCM_TESTS): Register the new files.=0A> =0A> Co-authored-by: Sa=
rah Morgensen <iskarian@HIDDEN>=0A=0AOverall this is looking good.  Tha=
nk you for adding tests (for remote-refs as well!), much appreciated.  It=
 looks like you've done some good polishing.  I see a few nits, which I'l=
l point out in a separate email when I'm not on mobile.  I'll also give i=
t a good test.=0A=0ABut... I've been thinking about the overall approach =
for a couple days, because I'm not very happy with the complexity of my h=
euristic.=0A=0AThere can be a lot of weird tags in a repository--look at =
the one for xf86-video-intel for example.  My heuristic attempts to captu=
re the assumption that repostories tend to move from using "_" or "-" to =
"." but it does fail to account for moving to or from dates (because date=
s don't compare with normal versions).=0A=0AI also realized that we are n=
ot using a very useful piece of information--the previous version/tag com=
bo.  I expect that in the vast majority of cases, the version delimiter f=
or the newest version will be the same as the version delimiter for the l=
ast known version.  (Perhaps the prefix as well?)  Can we use this inform=
ation to make our guesses better?  What do you think?=0A=0ADespite saying=
 all that, it's probably better to not try to get it perfect on the first=
 go--we can always adjust the internals later.  We just want to avoid sho=
wing bogus updates.=0A=0A(Later, I think I'll put together a dataset of t=
ags and current versions to see if I can test how well a particular algor=
ithm works.)=0A=0A--=0ASarah (mobile)




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 3/3] import: Add 'generic-git' updater.
Resent-From: Xinglu Chen <public@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 15 Sep 2021 12:00:01 +0000
Resent-Message-ID: <handler.50359.B50359.163170717631714 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: iskarian@HIDDEN, 50359 <at> debbugs.gnu.org
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163170717631714
          (code B ref 50359); Wed, 15 Sep 2021 12:00:01 +0000
Received: (at 50359) by debbugs.gnu.org; 15 Sep 2021 11:59:36 +0000
Received: from localhost ([127.0.0.1]:51174 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mQTZf-0008FS-Ko
	for submit <at> debbugs.gnu.org; Wed, 15 Sep 2021 07:59:35 -0400
Received: from h87-96-130-155.cust.a3fiber.se ([87.96.130.155]:55490
 helo=mail.yoctocell.xyz) by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <public@HIDDEN>) id 1mQTZe-0008FG-Gz
 for 50359 <at> debbugs.gnu.org; Wed, 15 Sep 2021 07:59:35 -0400
From: Xinglu Chen <public@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=yoctocell.xyz;
 s=mail; t=1631707166;
 bh=ZX+NNxys7tt8H0ouS4ghq4wpDvK3m7uJ8o1mi9v3u2U=;
 h=From:To:Cc:Subject:In-Reply-To:References:Date;
 b=lu6s1iYmRv9l0rnC5t6Kp55+Wt7/bx63x9/mxrGxAgWr78LZi2/zMXEKkbRjbs3oj
 SzaxF6CyB9VH5IaWFFK/Yb4xsatvi7EVYNxFcrHKbv/cOjP4cE3TR5n3RKOyBgWq4U
 oJ9FQ2FDqkxiKScu9T8mhYdTBqLrRH+O8BXmgePM=
In-Reply-To: <44519ae5dfdf3702dc73903b80314a6a@HIDDEN>
References: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
 <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <44519ae5dfdf3702dc73903b80314a6a@HIDDEN>
Date: Wed, 15 Sep 2021 13:59:22 +0200
Message-ID: <87sfy6rqw5.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  On Wed, Sep 15 2021, iskarian@HIDDEN wrote: > Hi,
 > > September 10, 2021 9:21 AM,
 "Xinglu Chen" <public@HIDDEN> wrote: > >> * guix/git.scm
 (ls-remote-refs): New procedure. >> * tests/git.scm ("remote-refs"
 "remote-refs: only tags"): New [...] 
 Content analysis details:   (2.9 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
 0.4 RDNS_DYNAMIC           Delivered to internal network by host with
 dynamic-looking rDNS
 0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 2.9 (++)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 
 Content preview:  On Wed, Sep 15 2021, iskarian@HIDDEN wrote: > Hi, > > September
    10, 2021 9:21 AM, "Xinglu Chen" <public@HIDDEN> wrote: > >> * guix/git.scm
    (ls-remote-refs): New procedure. >> * tests/git.scm ("remote-refs" "remote-refs:
    only tags"): New [...] 
 
 Content analysis details:   (2.9 points, 10.0 required)
 
  pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 SPF_PASS               SPF: sender matches SPF record
  2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
                             [URI: yoctocell.xyz (xyz)]
  0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
  0.5 FROM_SUSPICIOUS_NTLD   From abused NTLD
  0.4 RDNS_DYNAMIC           Delivered to internal network by host with
                             dynamic-looking rDNS
  1.0 BULK_RE_SUSP_NTLD      Precedence bulk and RE: from a suspicious TLD
 -1.0 MAILING_LIST_MULTI     Multiple indicators imply a widely-seen list
                             manager
  0.0 PDS_RDNS_DYNAMIC_FP    RDNS_DYNAMIC with FP steps

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

On Wed, Sep 15 2021, iskarian@HIDDEN wrote:

> Hi,
>
> September 10, 2021 9:21 AM, "Xinglu Chen" <public@HIDDEN> wrote:
>
>> * guix/git.scm (ls-remote-refs): New procedure.
>> * tests/git.scm ("remote-refs" "remote-refs: only tags"): New tests.
>> * guix/import/git.scm: New file.
>> * doc/guix.texi (Invoking guix refresh): Document it.
>> * tests/import-git.scm: New test file.
>> * Makefile.am (MODULES, SCM_TESTS): Register the new files.
>>=20
>> Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>
>
> Overall this is looking good.  Thank you for adding tests (for
> remote-refs as well!), much appreciated.  It looks like you've done
> some good polishing.  I see a few nits, which I'll point out in a
> separate email when I'm not on mobile.  I'll also give it a good test.
>
> But... I've been thinking about the overall approach for a couple
> days, because I'm not very happy with the complexity of my heuristic.
>
> There can be a lot of weird tags in a repository--look at the one for
> xf86-video-intel for example.  My heuristic attempts to capture the
> assumption that repostories tend to move from using "_" or "-" to "."
> but it does fail to account for moving to or from dates (because dates
> don't compare with normal versions).

But if a repo moved from using versions to tags, or vice-versa, we still
wouldn=E2=80=99t know if say =E2=80=9C3.0.1=E2=80=9D is newer than =E2=80=
=9C2021.03.02=E2=80=9D.  We would have
to know when the =E2=80=9C3.0.1=E2=80=9D tag was created.

Maybe we could have a =E2=80=98release-tag-date-scheme?=E2=80=99 property, =
that way we
could just try to match dates?

> I also realized that we are not using a very useful piece of
> information--the previous version/tag combo.  I expect that in the
> vast majority of cases, the version delimiter for the newest version
> will be the same as the version delimiter for the last known version.
> (Perhaps the prefix as well?)  Can we use this information to make our
> guesses better?  What do you think?

That sounds like a good idea.  What should happen if the delimiter from
the previous version/tag combo is different from the one that the
=E2=80=98guess-delimiter=E2=80=99 procedure returns?  Should the one from t=
he previous
version/tag combo take precedence.

> Despite saying all that, it's probably better to not try to get it
> perfect on the first go--we can always adjust the internals later.  We
> just want to avoid showing bogus updates.

Yeah, we can always improve things later.
=20=20
> (Later, I think I'll put together a dataset of tags and current
> versions to see if I can test how well a particular algorithm works.)

Cool, looking forward to that.  :-)

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQJJBAEBCAAzFiEEAVhh4yyK5+SEykIzrPUJmaL7XHkFAmFB4BoVHHB1YmxpY0B5
b2N0b2NlbGwueHl6AAoJEKz1CZmi+1x5F/sQAKkyjf7qyW/HkRcAfoZG3oGyTfLi
5DNEy2XtqrxV07gZNy9pJOhdz+x+utSchutn33xRjJLo0N5eg9uj9FNTyjVsTpMV
6P31JCLU3fB6fQZiStONuHTksTLP3YrKoNMR31BYRCQ7EQ6EysxAbCLApcz/E+vn
jHQvPcLxIeZj4NfTw9qH3jCpk3JKEn+fyEu50H/XXxE/loXVXCSGxP1aWKSCo0Ii
+TpFS6CSYLCHjxW5M0j0JOVrFliFhqUV3L+EKr6ooTDt0FmZyWf1/iD5DBH9D+Vm
0iD8wiZ0sjRRqL8955en44NnEU+g63iTBdR3UvX8pOLQZn/SbBHLzQdi3Gpvx2HF
BTvEm3wmsRQlt18e5yyn43wmxofJfksQI6/sYEF8ULDWzqC3rKiZL1r7Vt2+I9pn
1vW7YC8Zy9mYWYx9vDdWai8i9ltIrxXkSSZKmiLXDlNB0cq1LSDoQv4OQXmuUiMA
rcilETL3RNT1RdeFGGzPhmTHEHlMnpZG1ZCtgUiG8tjJkRRSz2fvL25M09i89SLK
9pLGUYwXoYJeykQ7rhcUnLPekHzPveC+AnY7JbpFwpXqlVI7wTNceVG4NXXxjEY0
vRxNiMGrSwPg1K7uRTDangD5rLFndt7nq5ANhUj1B4p30jiemoWwQM5xJsAO3ER0
b1afQNrw2MAsEgk3
=5CXA
-----END PGP SIGNATURE-----
--=-=-=--




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 3/3] import: Add 'generic-git' updater.
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Thu, 16 Sep 2021 09:10:02 +0000
Resent-Message-ID: <handler.50359.B50359.163178338928420 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.163178338928420
          (code B ref 50359); Thu, 16 Sep 2021 09:10:02 +0000
Received: (at 50359) by debbugs.gnu.org; 16 Sep 2021 09:09:49 +0000
Received: from localhost ([127.0.0.1]:53807 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mQnOu-0007OJ-Nf
	for submit <at> debbugs.gnu.org; Thu, 16 Sep 2021 05:09:49 -0400
Received: from out0.migadu.com ([94.23.1.103]:33137)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mQnOn-0007O5-Qm
 for 50359 <at> debbugs.gnu.org; Thu, 16 Sep 2021 05:09:47 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1631783379;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 in-reply-to:in-reply-to:references:references;
 bh=+4Td9ov4xszFVhGp2zWH36ghExMeFtjA/PLoyjKvFGo=;
 b=V1lRUYIkBVYPIrNh38PMkwtxAIO5d5rpIfzBP5kiJSTcrI5yvZNPLwlxByXbq2G9TMM0Oh
 Jj4a1F1YrMIfNojwBp02Dmee8k1Wg1nVIo5liypWQKYGqJMY2MVhptVi1wXe9mbp5WAbuq
 m947wjhYsSNEOLyZq5z/tYdvmJVSZ0Q=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
Date: Thu, 16 Sep 2021 02:09:32 -0700
In-Reply-To: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
 (Xinglu Chen's message of "Fri, 10 Sep 2021 18:21:30 +0200 (5 days, 15
 hours, 34 minutes ago)")
Message-ID: <86czp8j38z.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 1.3 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hello,
 Here are my promised nits. Xinglu Chen <public@HIDDEN>
 writes: Content analysis details:   (1.3 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [94.23.1.103 listed in wl.mailspike.net]
 -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at https://www.dnswl.org/,
 low trust [94.23.1.103 listed in list.dnswl.org]
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
 0.0 T_SPF_TEMPERROR        SPF: test of record failed (temperror)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.3 (/)

Hello,

Here are my promised nits.

Xinglu Chen <public@HIDDEN> writes:

> * guix/git.scm (ls-remote-refs): New procedure.
> * tests/git.scm ("remote-refs" "remote-refs: only tags"): New tests.
> * guix/import/git.scm: New file.
> * doc/guix.texi (Invoking guix refresh): Document it.
> * tests/import-git.scm: New test file.
> * Makefile.am (MODULES, SCM_TESTS): Register the new files.
>
> Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>

Again, much thanks for writing tests.

> +@item generic-git
> +a generic updater for packages hosted on Git repositories.  It tries to
> +be smart about parsing Git tag names, but if it is not able to parse the
> +tag name and compare tags correctly, users can define the following
> +properties for a package.
> +
> +@itemize
> +@item @code{release-tag-prefix}: a regular expression for matching a prefix of
> +the tag name.
> +
> +@item @code{release-tag-suffix}: a regular expression for matching a suffix of
> +the tag name.
> +
> +@item @code{release-tag-version-delimiter}: a string used as the delimiter in
> +the tag name for separating the numbers of the version.
> +@end itemize
> +
> +@lisp
> +(package
> +  (name "foo")
> +  ;; ...
> +  (properties
> +    '((release-tag-prefix . "^release0-")
> +      (release-tag-suffix . "[a-z]?$")
> +      (release-tag-version-delimiter . ":"))))
> +@end lisp
> +
> +By default, the updater will ignore pre-releases; to make it also look
> +for pre-releases, set the @code{accept-pre-releases?} property to
> +@code{#t}.

Should this be itemized above?

> +
> +;;
> +;;; Remote operations.
> +;;;
> +
> +(define* (remote-refs url #:key tags?)
> +  "Return the list of references advertised at Git repository URL.  If TAGS?
> +is true, limit to only refs/tags."
> +  (define (ref? ref)
> +    ;; Like `git ls-remote --refs', only show actual references.
> +    (and (string-prefix? "refs/" ref)
> +         (not (string-suffix? "^{}" ref))))
> +
> +  (define (tag? ref)
> +    (string-prefix? "refs/tags/" ref))
> +
> +  (define (include? ref)
> +    (and (ref? ref)
> +         (or (not tags?) (tag? ref))))
> +
> +  (with-libgit2
> +   (call-with-temporary-directory
> +    (lambda (cache-directory)
> +      (let* ((repository (repository-init cache-directory))
> +             ;; Create an in-memory remote so we don't touch disk.
> +             (remote (remote-create-anonymous repository url)))
> +        (remote-connect remote)
> +        (remote-disconnect remote)
> +        (repository-close! repository)
> +
> +        (filter-map (lambda (remote)
> +                      (let ((name (remote-head-name remote)))
> +                        (and (include? name)
> +                             name)))
> +                    (remote-ls remote)))))))

I discovered that this can segfault unless 'remote-disconnect' and
possibly 'repository-close!' are called *after* copying the data out.
I've attached a diff for this.

> +
> +;;; Updater
> +
> +(define %pre-release-words
> +  '("alpha" "beta" "rc" "dev" "test"))

I found a few packages that use "pre" as well.

> +
> +(define %pre-release-rx
> +  (map (cut make-regexp <> regexp/icase) %pre-release-words))
> +
> +(define* (version-mapping tags #:key prefix suffix delim pre-releases?)
> +  "Given a list of Git TAGS, return a association list where the car is the
                                       ^ an

> +version corresponding to the tag, and the cdr is the name of the tag."
> +  (define (guess-delimiter)
> +    (let ((total (length tags))
> +          (dots (reduce + 0 (map (cut string-count <> #\.) tags)))
> +          (dashes (reduce + 0 (map (cut string-count <> #\-) tags)))
> +          (underscores (reduce + 0 (map (cut string-count <> #\_) tags))))
> +      (cond
> +       ((>= dots (* total 0.35)) ".")
> +       ((>= dashes (* total 0.8)) "-")
> +       ((>= underscores (* total 0.8)) "_")
> +       (else ""))))
> +
> +  (define delim-rx (regexp-quote (or delim (guess-delimiter))))
> +  (define suffix-rx  (string-append (or suffix "") "$"))
> +
> +  
> +  (define prefix-rx (string-append "^" (or prefix "[^[:digit:]]*")))
> +  (define pre-release-rx
> +    (if pre-releases?
> +        (string-append ".*(" (string-join %pre-release-words "|") ").*")
> +        ""))
> +
> +  (define tag-rx
> +    (string-append prefix-rx "([[:digit:]][^" delim-rx "[:punct:]]*"
> +                   "(" delim-rx "[^[:punct:]" delim-rx "]+)"
> +                   ;; If there is are no delimiters, it could mean that the
                                  ^ no "is"

> +                   ;; version just contains one number (e.g., "2"), thus, use
> +                   ;; "*" instead of "+" to match zero or more numbers.
> +                   (if (string=? delim-rx "") "*" "+")

Good catch.

> +                   pre-release-rx ")" suffix-rx))
> +
> +  (define (get-version tag)
> +    (let ((tag-match (regexp-exec (make-regexp tag-rx) tag)))
> +      (and tag-match
> +           (regexp-substitute/global
> +            #f delim-rx (match:substring tag-match 1)
> +            ;; Don't insert "." if there aren't any delimiters in the first

Nit: "if there were no delimiters", to be consistent with above comment.

> +            ;; place.
> +            'pre (if (string=? delim-rx "") "" ".") 'post))))

One issue with returning a different delimiter than the package
currently uses is that the automatic updater won't really work as-is.

Hmmm.  When things are modified so the updater gets both the version and
the git-reference, it should be able to reverse-engineer things well
enough there.

I imagine this is really only going to be an issue with dates currently
written as "2017-01-01", anyway.  I'll put my comments on that in reply
to the other email.

> +
> +  (define (entry<? a b)
> +    (eq? (version-compare (car a) (car b)) '<))
> +
> +  (stable-sort (filter-map (lambda (tag)
> +                             (let ((version (get-version tag)))
> +                               (and version (cons version tag))))
> +                           tags)
> +               entry<?))
> +
> +(define* (latest-tag url #:key prefix suffix delim pre-releases?)
> +  "Return the latest tag available from the Git repository at URL."

This returns two values (in preparation for the above-mentioned switch),
so maybe something like "Return the latest version and corresponding tag
available from..."

> +  (define (pre-release? tag)
> +    (any (cut regexp-exec <> tag)
> +         %pre-release-rx))
> +
> +  (let* ((tags (map (cut string-drop <> (string-length "refs/tags/"))

Should be "cute" so string-length is only evaluated once -- though it's
probably optimized like that anyway.

> +                    (remote-refs url #:tags? #t)))
> +         (versions->tags
> +          (version-mapping (if pre-releases?
> +                               tags
> +                               (filter (negate pre-release?) tags))
> +                           #:prefix prefix
> +                           #:suffix suffix
> +                           #:delim delim
> +                           #:pre-releases? pre-releases?)))
> +    (cond
> +     ((null? tags)
> +      (git-no-tags-error))
> +     ((null? versions->tags)
> +      (git-no-valid-tags-error))
> +     (else
> +      (match (last versions->tags)
> +        ((version . tag)
> +         (values version tag)))))))
> +
> +(define (latest-git-tag-version package)
> +  "Given a PACKAGE, return the latest version of it, or #f if the latest version
> +could not be determined."
> +  (guard (c ((or (git-no-tags-error? c) (git-no-valid-tags-error? c))
> +             (warning (or (package-field-location package 'source)
> +                          (package-location package))
> +                      (G_ "~a for ~a~%")
> +                      (condition-message c)
> +                      (package-name package))
> +             #f)
> +            ((eq? (exception-kind c) 'git-error)
> +             (warning (or (package-field-location package 'source)
> +                          (package-location package))
> +                      (G_ "failed to fetch Git repository for ~a~%")
> +                      (package-name package))
> +             #f))
> +    (let* ((source (package-source package))
> +           (url (git-reference-url (origin-uri source)))
> +           (properties (package-properties package))
> +           (tag-prefix (assq-ref properties 'release-tag-prefix))
> +           (tag-suffix (assq-ref properties 'release-tag-suffix))
> +           (tag-version-delimiter (assq-ref properties 'release-tag-version-delimiter))
> +           (refresh-pre-releases? (assq-ref properties 'accept-pre-releases?)))
> +      (latest-tag url
> +                  #:prefix tag-prefix
> +                  #:suffix tag-suffix
> +                  #:delim tag-version-delimiter
> +                  #:pre-releases? refresh-pre-releases?))))

This is entirely a style preference, so only take this suggestion if you
like it :)

    (let* ((source (package-source package))
           (url (git-reference-url (origin-uri source)))
           (property (cute assq-ref (package-properties package) <>)))
      (latest-tag url
                  #:prefix (property 'release-tag-prefix)
                  #:suffix (property 'release-tag-suffix)
                  #:delim (property 'release-tag-version-delimiter)
                  #:pre-releases? (property 'accept-pre-releases?)))))

> +
> +(define (git-package? package)
> +  "Whether the origin of PACKAGE is a Git repostiory."

"Return true if PACKAGE is..."

> +  (match (package-source package)
> +    ((? origin? origin)
> +     (and (eq? (origin-method origin) git-fetch)
> +          (git-reference? (origin-uri origin))))
> +    (_ #f)))
> +
> +(define (latest-git-release package)
> +  "Return the latest release of PACKAGE."

"Return an <upstream-source> for the latest...", to match the other
updaters.

> +  (let* ((name (package-name package))
> +         (old-version (package-version package))
> +         (url (git-reference-url (origin-uri (package-source package))))
> +         (new-version (latest-git-tag-version package)))
> +
> +    (and new-version
> +         (upstream-source
> +          (package name)
> +          (version new-version)
> +          (urls (list url))))))
> +
> +(define %generic-git-updater
> +  (upstream-updater
> +   (name 'generic-git)
> +   (description "Updater for packages hosted on Git repositories")
> +   (pred git-package?)
> +   (latest latest-git-release)))

I tested this updater on all packages in .scm files starting with f
through z, and I found the following packages with possibly bogus
updates:

--8<---------------cut here---------------start------------->8---
javaxom
luakit
ocproxy
pitivi
eid-mw
libhomfly
gnuradio
welle-io
racket-minimal
milkytracker
cl-portal
kodi-cli
openjdk
java-bouncycastle
hurd
opencsg
povray
gpsbabel
go
stepmania
ocaml-mcl

many minetest packages (minetest will have its own updater, though)

ocaml4.07-core-kernel, ocamlbuild and many other ocaml packages
(they seem to be covered by the github updater)
--8<---------------cut here---------------end--------------->8---

The following packages suggest a version -> date update, which may or
may not be bogus:

--8<---------------cut here---------------start------------->8---
cataclysm-dda
autotrace
lbalgtk
nheko
libqalculate
cl-antik
cl-antik-base
cl-hu.dwim.stefil
cl-stefil
cl-gsll
sbcl-cl-gserver
--8<---------------cut here---------------end--------------->8---

--
Sarah




Message sent to guix-patches@HIDDEN:


X-Loop: help-debbugs@HIDDEN
Subject: [bug#50359] [PATCH 3/3] import: Add 'generic-git' updater.
Resent-From: Sarah Morgensen <iskarian@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Thu, 16 Sep 2021 09:47:02 +0000
Resent-Message-ID: <handler.50359.B50359.16317855783598 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 50359
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Xinglu Chen <public@HIDDEN>
Cc: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>, 50359 <at> debbugs.gnu.org
Received: via spool by 50359-submit <at> debbugs.gnu.org id=B50359.16317855783598
          (code B ref 50359); Thu, 16 Sep 2021 09:47:02 +0000
Received: (at 50359) by debbugs.gnu.org; 16 Sep 2021 09:46:18 +0000
Received: from localhost ([127.0.0.1]:53862 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1mQnyD-0000vY-SJ
	for submit <at> debbugs.gnu.org; Thu, 16 Sep 2021 05:46:18 -0400
Received: from out1.migadu.com ([91.121.223.63]:37057)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <iskarian@HIDDEN>) id 1mQnyA-0000s3-J4
 for 50359 <at> debbugs.gnu.org; Thu, 16 Sep 2021 05:46:16 -0400
X-Report-Abuse: Please report any abuse attempt to abuse@HIDDEN and
 include these headers.
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=mgsn.dev; s=key1;
 t=1631785572;
 h=from:from:reply-to:subject:subject:date:date:message-id:message-id:
 to:to:cc:cc:mime-version:mime-version:content-type:content-type:
 content-transfer-encoding:content-transfer-encoding:
 in-reply-to:in-reply-to:references:references;
 bh=l1BG7SqJ9jopQRULDRoJuwtszJ5w4cSIwejutiGBDDg=;
 b=VDR6iEJjNqcrioAgV9Q9htR0Oj1S5vcvfB6O2XF3PJq46jlNN0dBNFD4sI3aOsfW0052Pb
 i+TykrhZRQuDUuJaIxzBZQCgS/qprl8dFtG23Q5NQ8ghDtf31sPWG49xg2hJvwpU5KzBBB
 aHt+e2D88dGOvMGn+r9poqXBVtoe3bo=
From: Sarah Morgensen <iskarian@HIDDEN>
References: <5d10dd1e65b0a65ada4a8102310c10de42f53e8d.1631290349.git.public@HIDDEN>
 <e2bec21e5757204c45b8a06308b4ec183acabe86.1630683779.git.public@HIDDEN>
 <44519ae5dfdf3702dc73903b80314a6a@HIDDEN> <87sfy6rqw5.fsf@HIDDEN>
Date: Thu, 16 Sep 2021 02:46:10 -0700
In-Reply-To: <87sfy6rqw5.fsf@HIDDEN> (Xinglu Chen's message of "Wed, 15
 Sep 2021 13:59:22 +0200 (20 hours, 25 minutes, 52 seconds ago)")
Message-ID: <86a6kcj1jx.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Migadu-Flow: FLOW_OUT
X-Migadu-Auth-User: iskarian@HIDDEN
X-Spam-Score: 1.3 (+)
X-Spam-Report: Spam detection software, running on the system "debbugs.gnu.org",
 has NOT identified this incoming email as spam.  The original
 message has been attached to this so you can view it or label
 similar future email.  If you have any questions, see
 the administrator of that system for details.
 Content preview:  Hi, Xinglu Chen <public@HIDDEN> writes: > On Wed, Sep
 15 2021, iskarian@HIDDEN wrote: > >> Hi, >> >> September 10, 2021 9:21
 AM,
 "Xinglu Chen" <public@HIDDEN> wrote: >> >>> * guix/git.scm
 (ls-remote-refs): New procedure. >>> * test [...] 
 Content analysis details:   (1.3 points, 10.0 required)
 pts rule name              description
 ---- ---------------------- --------------------------------------------------
 -0.7 RCVD_IN_DNSWL_LOW      RBL: Sender listed at https://www.dnswl.org/,
 low trust [91.121.223.63 listed in list.dnswl.org]
 -0.0 RCVD_IN_MSPIKE_H2      RBL: Average reputation (+2)
 [91.121.223.63 listed in wl.mailspike.net]
 -0.0 SPF_PASS               SPF: sender matches SPF record
 2.0 PDS_OTHER_BAD_TLD      Untrustworthy TLDs
 [URI: yoctocell.xyz (xyz)]
 0.0 SPF_HELO_NONE          SPF: HELO does not publish an SPF Record
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.3 (/)

Hi,

Xinglu Chen <public@HIDDEN> writes:

> On Wed, Sep 15 2021, iskarian@HIDDEN wrote:
>
>> Hi,
>>
>> September 10, 2021 9:21 AM, "Xinglu Chen" <public@HIDDEN> wrote:
>>
>>> * guix/git.scm (ls-remote-refs): New procedure.
>>> * tests/git.scm ("remote-refs" "remote-refs: only tags"): New tests.
>>> * guix/import/git.scm: New file.
>>> * doc/guix.texi (Invoking guix refresh): Document it.
>>> * tests/import-git.scm: New test file.
>>> * Makefile.am (MODULES, SCM_TESTS): Register the new files.
>>>=20
>>> Co-authored-by: Sarah Morgensen <iskarian@HIDDEN>
>>
>> Overall this is looking good.  Thank you for adding tests (for
>> remote-refs as well!), much appreciated.  It looks like you've done
>> some good polishing.  I see a few nits, which I'll point out in a
>> separate email when I'm not on mobile.  I'll also give it a good test.
>>
>> But... I've been thinking about the overall approach for a couple
>> days, because I'm not very happy with the complexity of my heuristic.
>>
>> There can be a lot of weird tags in a repository--look at the one for
>> xf86-video-intel for example.  My heuristic attempts to capture the
>> assumption that repostories tend to move from using "_" or "-" to "."
>> but it does fail to account for moving to or from dates (because dates
>> don't compare with normal versions).
>
> But if a repo moved from using versions to tags, or vice-versa, we still
> wouldn=E2=80=99t know if say =E2=80=9C3.0.1=E2=80=9D is newer than =E2=80=
=9C2021.03.02=E2=80=9D.  We would have
> to know when the =E2=80=9C3.0.1=E2=80=9D tag was created.

You're right; I thought of that afterwards.

> Maybe we could have a =E2=80=98release-tag-date-scheme?=E2=80=99 property=
, that way we
> could just try to match dates?

That seems like it might be the only way to handle it in some cases (if
they have both versions and dates with a "." delimiter).  (Though, we
are actually interested in the *lack* of a date scheme.  If they use a
date scheme now, other versions will be disregarded, so we're fine; but
if they use versions now and used a date scheme before, the versions
will be discarded.)

>> I also realized that we are not using a very useful piece of
>> information--the previous version/tag combo.  I expect that in the
>> vast majority of cases, the version delimiter for the newest version
>> will be the same as the version delimiter for the last known version.
>> (Perhaps the prefix as well?)  Can we use this information to make our
>> guesses better?  What do you think?
>
> That sounds like a good idea.  What should happen if the delimiter from
> the previous version/tag combo is different from the one that the
> =E2=80=98guess-delimiter=E2=80=99 procedure returns?  Should the one from=
 the previous
> version/tag combo take precedence.

Consider:

  prefix :=3D 'tag-prefix or guess-prefix-from-current-version+tag or defau=
lt
  delim :=3D 'tag-delim or guess-delim-from-current-version+tag or guess-de=
limiter
  suffix :=3D 'tag-suffix or default

This should cover:

  1. Format stayed the same (including date formats)
  2. Format changed from (git-version ...) to proper version

This does not otherwise cover a complete change in format, such as "_"
-> ".", date(-) -> version, or version -> date(.), for which I could
argue requiring a manual update is reasonable.  It also does not cover
when the tags have both versions and dates with the same delimiter.

Though it would be nice to see when such updates are available, is it
worth some bogus results?  Are false positives better or false negatives
better?

Unless you/we want to pursue one or both of the above changes now, the
latest patch LGTM (modulo my nits).

Thanks for your work,
--
Sarah





Last modified: Thu, 16 Sep 2021 09:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.