GNU bug report logs - #67503
[PATCH 0/2] Improvements to the go importer

Previous Next

To reply to this bug, email your comments to 67503 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: guix-patches <at> gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 0/2] Improvements to the go importer
Date: Tue, 28 Nov 2023 12:18:47 +0200

I had intended to do some go packaging but found myself working on the
go importer instead.

According to the upstream go documentation, go.mod IS supposed to list
the indirect dependencies so that when someone runs 'go build' or 'go
install' all the dependencies are already listed.  For us, the indirect
dependencies are already listed as propagated-inputs for the packages
which actually use them.

For the second patch I'm not super happy about using 'second' (yes,
(flatten (go.mod-directives go.mod 'go)) does return (version "1.18"))
but I couldn't find a nice way to return just the version string, and it
seemed better than cadr.  Using version>? was the simplest way to do the
version compare, but I can copy the logic over if we'd rather not import
(guix utils).

Efraim Flashner (2):
  guix: import: Don't include indirect dependencies in go.
  guix: import: Report go version for go importer.

 guix/import/go.scm | 37 +++++++++++++++++++++++++++++++------
 1 file changed, 31 insertions(+), 6 deletions(-)


base-commit: 62376e3eb67644454bc655bed56be4be965bd13e
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #8 received at 67503 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 67503 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 1/2] guix: import: Don't include indirect dependencies in go.
Date: Tue, 28 Nov 2023 12:21:06 +0200

* guix/import/go.scm (parse-go.mod)[define-peg-patern require]: Adjust
the peg pattern to reject lines with the 'indirect' comment in them.

Change-Id: I9618bbaa1cb8c6549ced875e3c8d32afc72c3b9b
---
 guix/import/go.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 0357e6a1eb..940cdac4b0 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2021 Xinglu Chen <public <at> yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian <at> mgsn.dev>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune <at> gmail.com>
+;;; Copyright © 2023 Efraim Flashner <efraim <at> flashner.co.il>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -293,7 +294,10 @@ (define (parse-go.mod content)
 
   ;; The following directives may all be used solo or in a block
   ;; RequireSpec = ModulePath Version newline .
-  (define-peg-pattern require all (and module-path version EOL))
+  (define-peg-pattern require all
+    (and module-path version
+         ;; We don't want the transitive dependencies.
+         (not-followed-by (and (* WS) "//" (* WS) "indirect")) EOL))
   (define-peg-pattern require-top body
     (and (ignore "require")
          (or (and block-start (* (or require block-line)) block-end) require)))
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #11 received at 67503 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 67503 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH 2/2] guix: import: Report go version for go importer.
Date: Tue, 28 Nov 2023 12:21:07 +0200

* guix/import/go.scm (go-package, go.mod-go-version): New procedures.
(go-module->guix-package): Add the #:go keyword in the generated package
definition if the required go is newer than the default go.

Change-Id: I8d005740a442330ac307a40a53764c803ceffc4f
---
 guix/import/go.scm | 31 ++++++++++++++++++++++++++-----
 1 file changed, 26 insertions(+), 5 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 940cdac4b0..dd9298808d 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -29,6 +29,7 @@ (define-module (guix import go)
   #:use-module (guix git)
   #:use-module (guix hash)
   #:use-module (guix i18n)
+  #:use-module ((guix utils) #:select (version>?))
   #:use-module (guix diagnostics)
   #:use-module (guix import utils)
   #:use-module (guix import json)
@@ -93,6 +94,11 @@ (define-module (guix import go)
 
 ;;; Code:
 
+(define (go-package)
+  "Return the 'go' package.  This is a lazy reference so that we don't
+depend on (gnu packages golang)."
+  (module-ref (resolve-interface '(gnu packages golang)) 'go))
+
 (define http-fetch*
   ;; Like http-fetch, but memoized and returning the body as a string.
   (memoize (lambda args
@@ -314,7 +320,7 @@ (define (parse-go.mod content)
   (define-peg-pattern with all (or (and module-path version) file-path))
   (define-peg-pattern replace all (and original => with EOL))
   (define-peg-pattern replace-top body
-    (and (ignore "replace") 
+    (and (ignore "replace")
          (or (and block-start (* (or replace block-line)) block-end) replace)))
 
   ;; RetractSpec = ( Version | "[" Version "," Version "]" ) newline .
@@ -378,6 +384,17 @@ (define (go.mod-requirements go.mod)
 ;; Prevent inlining of this procedure, which is accessed by unit tests.
 (set! go.mod-requirements go.mod-requirements)
 
+(define (go.mod-go-version go.mod)
+  "Return the minimum version of go required to specified by GO.MOD."
+  (let ((go-version (go.mod-directives go.mod 'go)))
+    (if (null? go-version)
+      ;; If the go directive is missing, go 1.16 is assumed.
+      '(version "1.16")
+      (flatten go-version))))
+
+;; Prevent inlining of this procedure, which is accessed by unit tests.
+(set! go.mod-go-version go.mod-go-version)
+
 (define-record-type <vcs>
   (%make-vcs url-prefix root-regex type)
   vcs?
@@ -610,6 +627,7 @@ (define* (go-module->guix-package module-path #:key
                     available-versions
                     module-path))
          (content (fetch-go.mod goproxy module-path version*))
+         (min-go-version (second (go.mod-go-version (parse-go.mod content))))
          (dependencies+versions (go.mod-requirements (parse-go.mod content)))
          (dependencies (if pin-versions?
                            dependencies+versions
@@ -634,10 +652,13 @@ (define* (go-module->guix-package module-path #:key
          ,(vcs->origin vcs-type vcs-repo-url version*))
         (build-system go-build-system)
         (arguments
-         '(#:import-path ,module-path
-           ,@(if (string=? module-path-sans-suffix root-module-path)
-                 '()
-                 `(#:unpack-path ,root-module-path))))
+         (list ,@(if (version>? min-go-version (package-version (go-package)))
+                     `(#:go ,(string->number min-go-version))
+                     '())
+               #:import-path ,module-path
+               ,@(if (string=? module-path-sans-suffix root-module-path)
+                     '()
+                     `(#:unpack-path ,root-module-path))))
         ,@(maybe-propagated-inputs
            (map (match-lambda
                   ((name version)
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #14 received at 67503 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 67503 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH v2 2/2] guix: import: Report go version for go importer.
Date: Thu,  7 Dec 2023 13:05:26 +0200

* guix/import/go.scm (go-package, go.mod-go-version): New procedures.
(go-module->guix-package): Add the #:go keyword in the generated package
definition if the required go is newer than the default go.
* tests/go.scm (mock-http-get): Use gexps for package arguments.

Change-Id: I8d005740a442330ac307a40a53764c803ceffc4f
---
 guix/import/go.scm | 31 ++++++++++++++++++++++++++-----
 tests/go.scm       |  6 +++---
 2 files changed, 29 insertions(+), 8 deletions(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 940cdac4b0..dd9298808d 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -29,6 +29,7 @@ (define-module (guix import go)
   #:use-module (guix git)
   #:use-module (guix hash)
   #:use-module (guix i18n)
+  #:use-module ((guix utils) #:select (version>?))
   #:use-module (guix diagnostics)
   #:use-module (guix import utils)
   #:use-module (guix import json)
@@ -93,6 +94,11 @@ (define-module (guix import go)
 
 ;;; Code:
 
+(define (go-package)
+  "Return the 'go' package.  This is a lazy reference so that we don't
+depend on (gnu packages golang)."
+  (module-ref (resolve-interface '(gnu packages golang)) 'go))
+
 (define http-fetch*
   ;; Like http-fetch, but memoized and returning the body as a string.
   (memoize (lambda args
@@ -314,7 +320,7 @@ (define (parse-go.mod content)
   (define-peg-pattern with all (or (and module-path version) file-path))
   (define-peg-pattern replace all (and original => with EOL))
   (define-peg-pattern replace-top body
-    (and (ignore "replace") 
+    (and (ignore "replace")
          (or (and block-start (* (or replace block-line)) block-end) replace)))
 
   ;; RetractSpec = ( Version | "[" Version "," Version "]" ) newline .
@@ -378,6 +384,17 @@ (define (go.mod-requirements go.mod)
 ;; Prevent inlining of this procedure, which is accessed by unit tests.
 (set! go.mod-requirements go.mod-requirements)
 
+(define (go.mod-go-version go.mod)
+  "Return the minimum version of go required to specified by GO.MOD."
+  (let ((go-version (go.mod-directives go.mod 'go)))
+    (if (null? go-version)
+      ;; If the go directive is missing, go 1.16 is assumed.
+      '(version "1.16")
+      (flatten go-version))))
+
+;; Prevent inlining of this procedure, which is accessed by unit tests.
+(set! go.mod-go-version go.mod-go-version)
+
 (define-record-type <vcs>
   (%make-vcs url-prefix root-regex type)
   vcs?
@@ -610,6 +627,7 @@ (define* (go-module->guix-package module-path #:key
                     available-versions
                     module-path))
          (content (fetch-go.mod goproxy module-path version*))
+         (min-go-version (second (go.mod-go-version (parse-go.mod content))))
          (dependencies+versions (go.mod-requirements (parse-go.mod content)))
          (dependencies (if pin-versions?
                            dependencies+versions
@@ -634,10 +652,13 @@ (define* (go-module->guix-package module-path #:key
          ,(vcs->origin vcs-type vcs-repo-url version*))
         (build-system go-build-system)
         (arguments
-         '(#:import-path ,module-path
-           ,@(if (string=? module-path-sans-suffix root-module-path)
-                 '()
-                 `(#:unpack-path ,root-module-path))))
+         (list ,@(if (version>? min-go-version (package-version (go-package)))
+                     `(#:go ,(string->number min-go-version))
+                     '())
+               #:import-path ,module-path
+               ,@(if (string=? module-path-sans-suffix root-module-path)
+                     '()
+                     `(#:unpack-path ,root-module-path))))
         ,@(maybe-propagated-inputs
            (map (match-lambda
                   ((name version)
diff --git a/tests/go.scm b/tests/go.scm
index a70a0ddbf5..d2e8846b30 100644
--- a/tests/go.scm
+++ b/tests/go.scm
@@ -1,6 +1,6 @@
 ;;; GNU Guix --- Functional package management for GNU
-;;; Copyright � 2021 Fran�ois Joulaud <francois.joulaud <at> radiofrance.com>
-;;; Copyright � 2021 Sarah Morgensen <iskarian <at> mgsn.dev>
+;;; Copyright © 2021 François Joulaud <francois.joulaud <at> radiofrance.com>
+;;; Copyright © 2021 Sarah Morgensen <iskarian <at> mgsn.dev>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -387,7 +387,7 @@ (define (mock-http-get testcase)
           "0sjjj9z1dhilhpc8pq4154czrb79z9cm044jvn75kxcjv6v5l2m5"))))
      (build-system go-build-system)
      (arguments
-      '(#:import-path "github.com/go-check/check"))
+      (list #:import-path "github.com/go-check/check"))
      (propagated-inputs
       `(("go-github-com-kr-pretty" ,go-github-com-kr-pretty)))
      (home-page "https://github.com/go-check/check")
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Message #17 received at 67503 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: 67503 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: [PATCH v2 1/2] guix: import: Don't include indirect dependencies in
 go.
Date: Thu,  7 Dec 2023 13:05:25 +0200

* guix/import/go.scm (parse-go.mod)[define-peg-patern require]: Adjust
the peg pattern to reject lines with the 'indirect' comment in them.

Change-Id: I9618bbaa1cb8c6549ced875e3c8d32afc72c3b9b
---
 guix/import/go.scm | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/guix/import/go.scm b/guix/import/go.scm
index 0357e6a1eb..940cdac4b0 100644
--- a/guix/import/go.scm
+++ b/guix/import/go.scm
@@ -7,6 +7,7 @@
 ;;; Copyright © 2021 Xinglu Chen <public <at> yoctocell.xyz>
 ;;; Copyright © 2021 Sarah Morgensen <iskarian <at> mgsn.dev>
 ;;; Copyright © 2021 Simon Tournier <zimon.toutoune <at> gmail.com>
+;;; Copyright © 2023 Efraim Flashner <efraim <at> flashner.co.il>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -293,7 +294,10 @@ (define (parse-go.mod content)
 
   ;; The following directives may all be used solo or in a block
   ;; RequireSpec = ModulePath Version newline .
-  (define-peg-pattern require all (and module-path version EOL))
+  (define-peg-pattern require all
+    (and module-path version
+         ;; We don't want the transitive dependencies.
+         (not-followed-by (and (* WS) "//" (* WS) "indirect")) EOL))
   (define-peg-pattern require-top body
     (and (ignore "require")
          (or (and block-start (* (or require block-line)) block-end) require)))
-- 
Efraim Flashner   <efraim <at> flashner.co.il>   רנשלפ םירפא
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.