GNU logs - #64036, boring messages

Message sent to guix-patches@HIDDEN:

Subject: [bug#64036] [PATH] fix a bug on importing go packages.
Resent-From: Elbek <el@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Tue, 13 Jun 2023 06:33:03 +0000
Resent-Message-ID: <handler.64036.B.168663793130188 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 64036 <at> debbugs.gnu.org
Date: Mon, 12 Jun 2023 23:20:34 +0500
From: Elbek <el@HIDDEN>
Message-Id: <188b0d69a95.da3e871b642333.4245430848638815620@HIDDEN>
In-Reply-To: 
MIME-Version: 1.0
Content-Type: multipart/mixed; 
 boundary="----=_Part_2002805_481489749.1686594034325"
Importance: Medium
User-Agent: Zoho Mail
Received-SPF: pass client-ip=136.143.188.54; envelope-from=el@HIDDEN;
 helo=sender4-of-o54.zoho.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001,
 T_SCC_BODY_TEXT_LINE=-0.01 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Mailman-Approved-At: Tue, 13 Jun 2023 02:32:05 -0400
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

------=_Part_2002805_481489749.1686594034325
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 7bit

Hello.

This patch addresses the issue https://issues.guix.gnu.org/63001

Currently "guix import go ..." looks to the repository depending on the version of a package.

For example, if package version is $1.11.0$, the importer tries to look to the repository's tag named v1.11.0,
but some packages are using tags that are prefixed by the module name.

For example, the package cloud.google.com/go/workflows use the tag workflows/v1.11.0

This patch changes go importer, such that it looks to the module-name prefixed tags if they exists, and behaves as before otherwise.

------=_Part_2002805_481489749.1686594034325
Content-Type: application/octet-stream; 
	name=guix-import-go--tags-may-prefixed-by-module-name.patch
Content-Transfer-Encoding: 7bit
X-ZM_AttachId: 139058868343260050
Content-Disposition: attachment; 
	filename=guix-import-go--tags-may-prefixed-by-module-name.patch

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 0357e6a1eb..962ff4c76f 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -448,6 +448,13 @@ (define* (go-module->guix-package-name module-path #:optional version)
                                         (string-append "-" version)
                                         "")))))

+(define* (go-module->package-name module-path version)
+  "Converts a module's path to the go package name."
+  (let ((name (basename module-path)))
+    (if (string-prefix? name version)
+        (basename (dirname module-path))
+        name)))
+
 (define (strip-.git-suffix/maybe repo-url)
   "Strip a repository URL '.git' suffix from REPO-URL if hosted at GitHub."
   (match repo-url
@@ -513,13 +520,18 @@ (define cache
                                           `(tag-or-commit . ,reference)))))
     (file-hash* checkout #:algorithm algorithm #:recursive? #true)))

-(define (vcs->origin vcs-type vcs-repo-url version)
+(define (vcs->origin vcs-type vcs-repo-url version module-name)
   "Generate the `origin' block of a package depending on what type of source
 control system is being used."
   (case vcs-type
     ((git)
-     (let ((plain-version? (string=? version (go-version->git-ref version)))
-           (v-prefixed?    (string-prefix? "v" version)))
+     (let* ((plain-version? (string=? version (go-version->git-ref version)))
+            (v-prefixed?    (string-prefix? "v" version))
+            (prefixed-version (string-append module-name "/" version))
+            (refs (remote-refs vcs-repo-url))
+            (use-prefixed-version? (member
+                                    (string-append "refs/tags/" prefixed-version)
+                                    refs)))
        `(origin
           (method git-fetch)
           (uri (git-reference
@@ -528,13 +540,26 @@ (define (vcs->origin vcs-type vcs-repo-url version)
                 ;; which the generated quoted expression refers to, has been
                 ;; stripped of any 'v' prefixed.
                 (commit ,(if (and plain-version? v-prefixed?)
-                             '(string-append "v" version)
-                             '(go-version->git-ref version)))))
+                             (if use-prefixed-version?
+                                 `(string-append
+                                   ,(string-append module-name "/v")
+                                   version)
+                                 '(string-append "v" version))
+                             (if use-prefixed-version?
+                                 `(go-version->git-ref
+                                   (string-append
+                                    ,(string-append module-name "/")
+                                    version))
+                                 '(go-version->git-ref version))))))
           (file-name (git-file-name name version))
           (sha256
            (base32
             ,(bytevector->nix-base32-string
-              (git-checkout-hash vcs-repo-url (go-version->git-ref version)
+              (git-checkout-hash vcs-repo-url
+                                 (if use-prefixed-version?
+                                     (go-version->git-ref
+                                      (string-append module-name "/" version))
+                                     (go-version->git-ref version))
                                  (hash-algorithm sha256))))))))
     ((hg)
      `(origin
@@ -605,6 +630,7 @@ (define* (go-module->guix-package module-path #:key
                         (go-module-version-string goproxy module-path)) ;latest
                     available-versions
                     module-path))
+         (module-name (go-module->package-name module-path version*))
          (content (fetch-go.mod goproxy module-path version*))
          (dependencies+versions (go.mod-requirements (parse-go.mod content)))
          (dependencies (if pin-versions?
@@ -627,7 +653,7 @@ (define* (go-module->guix-package module-path #:key
         (name ,guix-name)
         (version ,(strip-v-prefix version*))
         (source
-         ,(vcs->origin vcs-type vcs-repo-url version*))
+         ,(vcs->origin vcs-type vcs-repo-url version* module-name))
         (build-system go-build-system)
         (arguments
          '(#:import-path ,module-path

------=_Part_2002805_481489749.1686594034325--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Elbek <el@HIDDEN>
Subject: bug#64036: Acknowledgement ([PATH] fix a bug on importing go
 packages.)
Message-ID: <handler.64036.B.168663793130188.ack <at> debbugs.gnu.org>
References: <188b0d69a95.da3e871b642333.4245430848638815620@HIDDEN>
X-Gnu-PR-Message: ack 64036
X-Gnu-PR-Package: guix-patches
Reply-To: 64036 <at> debbugs.gnu.org
Date: Tue, 13 Jun 2023 06:33:04 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 guix-patches@HIDDEN

If you wish to submit further information on this problem, please
send it to 64036 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
64036: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D64036
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to guix-patches@HIDDEN:

Subject: [bug#64036] bug#63631: [PATCH] import: go: Handle subpackage versioning correctly.
Resent-From: "Timo Wilken" <guix@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sat, 17 Jun 2023 15:14:04 +0000
Resent-Message-ID: <handler.64036.B64036.1687014839950 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 63631 <at> debbugs.gnu.org, 64036 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune@HIDDEN>, 63647 <at> debbugs.gnu.org, 64035 <at> debbugs.gnu.org, 63001 <at> debbugs.gnu.org, 54097 <at> debbugs.gnu.org, wolf@HIDDEN
Content-Type: multipart/signed;
 boundary=dc3bfcd0ce40fc9a1a65ded3476be064f73442d113438739c89741eeba4a;
 micalg=pgp-sha256; protocol="application/pgp-signature"
Date: Sat, 17 Jun 2023 17:12:58 +0200
From: "Timo Wilken" <guix@HIDDEN>
Message-Id: <CTF06XBYWPT0.1MV6QA1B2OB98@HIDDEN>
References: <6dd1de3dd4d968876fa55f5126056834c77b0244.1684703258.git.guix@HIDDEN>
 <87pm5xrbsg.fsf@HIDDEN>
In-Reply-To: <87pm5xrbsg.fsf@HIDDEN>
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

--dc3bfcd0ce40fc9a1a65ded3476be064f73442d113438739c89741eeba4a
Mime-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8

Hi Ludo', (hi everyone,)

On Wed Jun 14, 2023 at 11:09 PM CEST, Ludovic Court=C3=A8s wrote:
> Timo Wilken <guix@HIDDEN> skribis:
> > Here's a patch that fixes the reported issue (bug#54097) for me. I've o=
nly
> > tested this on the github.com/googleapis/google-cloud-go/compute packag=
e so
> > far, though it seems to work there. Perhaps others have more testcases?
> >
> > I don't know enough about Go tooling to use it, so I've just patched th=
e Guile
> > logic of the importer. (I don't write Go, I just want to package stuff =
written
> > in it.) In terms of performance, at least the repo contents are apparen=
tly
> > cached by the first `git-checkout-hash' call, even if it fails, so the =
second
> > call doesn't have to redownload them.

I've been testing my patch further this weekend, and I have a couple more
patches in the pipeline; I suppose I ought to clean those up and submit the=
m.

In particular, I've got fixes for the following queued up locally:

1. Finding the `module-path-subdir' needs another case for e.g.
   cloud.google.com/go/*.

2. My patch sometimes generates an unnecessary `go-version->git-ref' call.

3. Go versions need to be parsed from go.mod, since some packages require a
   newer Go compiler than our default. This I've got a patch for, but this =
Go
   version also ought to propagate up the dependency tree. I haven't found =
an
   easy way to do that, since the importer seems to generate top-level
   packages first, before descending the dep tree...

4. `fetch-module-meta-data' ought to ignore 4xx HTTP errors to follow the
   spec; gonum.org/v1/gonum specifically depends on this behaviour.

I've been trying to recursively import github.com/matrix-org/dendrite, whic=
h
has a particularly large and hairy dependency tree. While I can now import =
it
without crashes, I can't build it from the imported package definitions yet=
 --
mainly because of lots of dependency cycles in the generated packages, but
there may be more issues hidden beneath that.

Still, I can recommend it as a test of everyone's importer patches, since
it'll find a lot of edge cases in importing alone!

> What you propose looks similar to part of the work Simon Tournier
> submitted at <https://issues.guix.gnu.org/63647>.

It seems lots of people have been working on the same problem -- in additio=
n
to Simon's patches, I found a patch submitted by Elbek (issues 64035 & 6403=
6;
Cc'd). I also forgot about the issue I submitted months ago (63001)...

> What would you suggest?  Simon?

Here's a brief comparison between Simon's patches and mine -- Simon's seem =
to
contain fixes for a couple more things than mine currently does:

1. Simon sorts available versions in an error message; this can presumably =
be
   merged independently since it doesn't conflict with other patches.

2. Simon always prepends a "SUBDIR/" prefix to the tag if found, whereas I =
try
   to find the plain "vX" tag first, then fall back to "SUBDIR/vX". Judging=
 by
   https://go.dev/ref/mod#vcs-version, Simon's approach seems more correct.
   I'll change my implementation to match and try it out.

3. For detecting the `module-path-subdirectory' in Simon's patches: that's =
the
   same approach I used initially, but I found I have to try `(substring
   module-path (string-length import-prefix))' first (to handle e.g.
   cloud.google.com/go/*). This is one of the things I haven't submitted
   yet...

> Thanks for the patch, Timo!

Thanks for your work in sorting through all of this, Ludo'!

Cheers,
Timo

--dc3bfcd0ce40fc9a1a65ded3476be064f73442d113438739c89741eeba4a
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iQIzBAABCAAdFiEEU+w8BoVog92SNVvCL8eFBGgfabAFAmSNzaUACgkQL8eFBGgf
abDEaA//Y6k/Uckhvq+wllES0TjACxVG+2bY2E/wLZNoQShXF5Tn3ZQ9o2HOU7Gr
p8er/t/JkPLYm9w2MidUtvTms6qxp8H/YIZ12gzZ+GP/eoKa/iiqt42+KZvLS/Is
B2JWEcaJLFYy0w9nuubeuQpgcWEmxyk7/zb++GovD0tfYgczII3N1Z5fhRmeAj0N
FbKqfkBocxruYvBfcI5KhMV6Q1TjCwuAH+j2iwNmsiSnm7NTKbJxfCyO9+p5UF9w
O/em01p9J+Cg6EEFunwoi1pjoEAkVZCRtgaiP1SMgapq74UKKagvsWqMoW+JHpDf
uuiaVn1sLuTt+u9KGamYqaCj1NRcCfN7SjPxS7oTM4hsytVDmUB/eIHD6t3HntCP
Ev43fJKJwLoybQV0ptajnJggEEaEY1cegQG683jsD0ycrToh4YEhLe5fQsGhaLMA
7MSLjjf0K73BoFyZi+wYyfDPYqXRQmpb3Z6cKJPlF3v4IF13UtSxP0yBRdz930aI
1b1Q5khLQzQBI0fvxMNcZGPeA2RjPyNsXE8pKn9zEkIunzX9VYN8az++EjX928Ox
mmgjEUsLWMPCSMwGcEoZye6R5wJUG34LEkoQTbySf+314hlELPuGcLCatFgZZbXq
1Hn1upt13XYRYVCX/SXIpBJL9gdNbi2sSg8FVm4Aq+sQQqZzAKs=
=fjJn
-----END PGP SIGNATURE-----

--dc3bfcd0ce40fc9a1a65ded3476be064f73442d113438739c89741eeba4a--

Message sent to guix-patches@HIDDEN:

Subject: [bug#64036] bug#63001: bug#63631: [PATCH] import: go: Handle subpackage versioning correctly.
Resent-From: Simon Tournier <zimon.toutoune@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 16 Aug 2023 17:12:05 +0000
Resent-Message-ID: <handler.64036.B64036.169220591920854 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Timo Wilken <guix@HIDDEN>, Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 63631 <at> debbugs.gnu.org, 64036 <at> debbugs.gnu.org, 63647 <at> debbugs.gnu.org, 64035 <at> debbugs.gnu.org, 63001 <at> debbugs.gnu.org, 54097 <at> debbugs.gnu.org, wolf@HIDDEN
From: Simon Tournier <zimon.toutoune@HIDDEN>
In-Reply-To: <CTF06XBYWPT0.1MV6QA1B2OB98@HIDDEN>
References: <6dd1de3dd4d968876fa55f5126056834c77b0244.1684703258.git.guix@HIDDEN>
 <87pm5xrbsg.fsf@HIDDEN> <CTF06XBYWPT0.1MV6QA1B2OB98@HIDDEN>
Date: Wed, 16 Aug 2023 17:59:53 +0200
Message-ID: <875y5ff05i.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hi Timo,

On Sat, 17 Jun 2023 at 17:12, "Timo Wilken" <guix@HIDDEN> wrote:

>> What would you suggest?  Simon?
>
> Here's a brief comparison between Simon's patches and mine -- Simon's seem to
> contain fixes for a couple more things than mine currently does:
>
> 1. Simon sorts available versions in an error message; this can presumably be
>    merged independently since it doesn't conflict with other patches.
>
> 2. Simon always prepends a "SUBDIR/" prefix to the tag if found, whereas I try
>    to find the plain "vX" tag first, then fall back to "SUBDIR/vX". Judging by
>    https://go.dev/ref/mod#vcs-version, Simon's approach seems more correct.
>    I'll change my implementation to match and try it out.
>
> 3. For detecting the `module-path-subdirectory' in Simon's patches: that's the
>    same approach I used initially, but I found I have to try `(substring
>    module-path (string-length import-prefix))' first (to handle e.g.
>    cloud.google.com/go/*). This is one of the things I haven't submitted
>    yet...

Sorry if I have missed some patches or overlooked something.  Do you
plan to send another patch series handling all?


Cheers,
simon

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 14 Sep 2023 21:13:32 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Sep 14 17:13:32 2023
Received: from localhost ([127.0.0.1]:41293 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1qgteV-00055L-Qm
	for submit <at> debbugs.gnu.org; Thu, 14 Sep 2023 17:13:32 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:40014)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1qgteU-000554-0F
 for control <at> debbugs.gnu.org; Thu, 14 Sep 2023 17:13:30 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1qgteI-0006OO-Me
 for control <at> debbugs.gnu.org; Thu, 14 Sep 2023 17:13:18 -0400
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=WTdiu/lEdEDs62CWzXWqqU3FqnbD1yFDaPgT9NFMWvA=; b=l9rNazm8UPi13H
 +z7IL2ixhE5tO+sCUBXVeGhlL1F5/aesps/hHuVnMZyoqGR7qGBLUxirix9hEgPM8YAxZM3KQhqBD
 056LMore2/Immcd+s0H4sKK4ZYcIvBeJ9lRJZLH7Od8GOFOPjEQOYHydhsWswpRRqLCN/zg1hRkfc
 2dEdEpYKZW+u3LJZEPPwgkcTxp7VQ41bCmK4/f1xgpnwTwcU+fY847q65XqFjRN5Se6kXqnxqmYQ2
 49Sr7CtG34SeD8lKYDSn+FKvyZRuO+rg3phkJ9/e1XFlSwOHzUjX2J6eT0Pevuxd1zph/pAN7dwNj
 KPLqL9s7DuufJoieDsBg==;
Date: Thu, 14 Sep 2023 23:13:15 +0200
Message-Id: <8734zg5uhg.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #64035
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

merge 64035 64036
quit

Last modified: Thu, 14 Sep 2023 21:15:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.