Package: guix-patches;
Reported by: Hilton Chain <hako <at> ultrarare.space>
Date: Tue, 18 Mar 2025 07:18:02 UTC
Severity: normal
Tags: patch
To reply to this bug, email your comments to 77093 AT debbugs.gnu.org.
Toggle the display of automated, internal messages from the tracker.
View this report as an mbox folder, status mbox, maintainer mbox
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:18:02 GMT) Full text and rfc822 format available.Hilton Chain <hako <at> ultrarare.space>:ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org.
(Tue, 18 Mar 2025 07:18:02 GMT) Full text and rfc822 format available.Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: guix-patches <at> gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 00/18] New Rust packaging workflow based on lockfile importer. Date: Tue, 18 Mar 2025 15:16:05 +0800
Hi Guix!
This series:
* integrates cargo2guix (https://git.sr.ht/~look/cargo2guix) as a crate importer
option, ‘-f / --lockfile=’.
* adds Cargo workspace support to cargo-build-system, via two new arguments:
#:cargo-package-crates and #:cargo-install-paths.
* introduces a new Rust packaging workflow, as described in contributing.texi
change.
This change is compatible with existing cargo-build-system users.
Thanks
Efraim Flashner (2):
build/cargo: Don't try to unpack sanity-check.py.
build/cargo: Use system jemalloc.
Hilton Chain (16):
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
scripts: import: Document argument for ‘--insert’ option in help
message.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build-system: cargo: Add ‘cargo-inputs’.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
doc: Document lockfile importer based Rust packaging workflow.
Makefile.am | 1 +
doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
doc/guix.texi | 25 ++
etc/teams/rust/cleanup-crates.sh | 38 +++
etc/teams/rust/rust-crates.tmpl | 42 +++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 +++
gnu/packages/rust-sources.scm | 29 +++
guix/build-system/cargo.scm | 57 +++-
guix/build/cargo-build-system.scm | 163 +++++++++---
guix/import/crate.scm | 81 +++++-
guix/import/crate/cargo-lock.scm | 105 ++++++++
guix/scripts/import.scm | 54 ++--
guix/scripts/import/crate.scm | 58 ++++-
guix/utils.scm | 27 +-
tests/crate.scm | 88 +++++++
16 files changed, 1121 insertions(+), 106 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:04 GMT) Full text and rfc822 format available.Message #8 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 01/18] build/cargo: Pass ‘--offline’ to cargo. Date: Tue, 18 Mar 2025 15:24:14 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build,check,package,install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:06 GMT) Full text and rfc822 format available.Message #11 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 02/18] build/cargo: Print out all non-empty binary files. Date: Tue, 18 Mar 2025 15:24:15 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
'unpack-rust-crates after 'unpack.
Move 'check-for-pregenerated-files after 'configure.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 31 +++++++++++++++++++++++++------
1 file changed, 25 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..58238992b8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -29,9 +29,11 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build utils) #:hide (delete))
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +113,29 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (filename)
+ (call-with-ascii-input-file filename
+ (lambda (p)
+ (eqv? #\0 (read-char p)))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ (unless (empty-file? file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file))))
+ (find-files "."))))
(define* (configure #:key inputs
target system
@@ -380,8 +399,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:08 GMT) Full text and rfc822 format available.Message #14 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 03/18] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 15:24:16 +0800
* guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 28 ++++++++++++++++++++++++++--
3 files changed, 37 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0ca109a214..322512fd36 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9363,6 +9363,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
+names of library crates to package in @code{package} phase. Specified
+crates are packaged from left to right, in case there's dependency among
+them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
+package @code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..4f6d46e70c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -94,6 +94,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -122,6 +123,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -154,6 +156,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -185,6 +188,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 58238992b8..001e9c7adc 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -300,11 +300,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -322,7 +327,26 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ ;; Use unstable feature ‘--registry’.
+ (begin
+ (setenv "RUSTC_BOOTSTRAP" "1")
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ "--registry" "crates-io" "-Z" "package-workspace"
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (begin
+ (delete-file-recursively "target/package/tmp-registry")
+ (find-files "target/package" "\\.crate$")))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates)
+ (unsetenv "RUSTC_BOOTSTRAP")))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:26:09 GMT) Full text and rfc822 format available.Message #17 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 04/18] build-system: cargo: Support installing Cargo workspace. Date: Tue, 18 Mar 2025 15:24:17 +0800
*guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 322512fd36..c20e1d7f9c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9369,7 +9369,10 @@ Build Systems
names of library crates to package in @code{package} phase. Specified
crates are packaged from left to right, in case there's dependency among
them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
-package @code{"pcre2-sys"} first and then @code{"pcre2"}.
+package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
+paths of binary crates to install in @code{install} phase,
+@code{''("crates/atuin")}, for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4f6d46e70c..4486c706a1 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -96,6 +96,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -125,6 +126,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -190,6 +193,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 001e9c7adc..bcee10837e 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -384,6 +384,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -398,10 +399,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:04 GMT) Full text and rfc822 format available.Message #20 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 05/18] build/cargo: Set default value of arguments for build phases. Date: Tue, 18 Mar 2025 15:24:18 +0800
This makes it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates,configure,build,package)
(install): Set default value of arguments.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index bcee10837e..fdc9ae89c8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -76,7 +76,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -139,7 +140,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -258,7 +259,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -298,8 +299,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -381,9 +382,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:06 GMT) Full text and rfc822 format available.Message #23 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team 06/18] build/cargo: Don't try to unpack sanity-check.py. Date: Tue, 18 Mar 2025 15:24:19 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index fdc9ae89c8..ea00ff3fbe 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -64,6 +64,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:07 GMT) Full text and rfc822 format available.Message #26 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 07/18] build/cargo: Support non-workspace directory source inputs. Date: Tue, 18 Mar 2025 15:24:20 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index ea00ff3fbe..a1c4f556bb 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -33,6 +33,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -60,22 +61,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -159,14 +173,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:07 GMT) Full text and rfc822 format available.Message #29 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team 08/18] build/cargo: Use system jemalloc. Date: Tue, 18 Mar 2025 15:24:21 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (configure): Set environment variables to
use system jemalloc.
Change-Id: Ibb1c025c7354ba2befd2770e3f1ccd785baa06c3
Modified-by: Hilton Chain <hako <at> ultrarare.space>
---
guix/build/cargo-build-system.scm | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index a1c4f556bb..b547421b88 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -248,6 +248,12 @@ (define* (configure #:key inputs
(setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
(setenv "SODIUM_USE_PKG_CONFIG" "1")
(setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
+ ;; This flag is needed when not using the bundled jemalloc.
+ ;; https://github.com/tikv/jemallocator/issues/19
+ (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
+ (when (assoc-ref inputs "jemalloc")
+ (setenv "JEMALLOC_OVERRIDE"
+ (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
(when (assoc-ref inputs "openssl")
(setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
(when (assoc-ref inputs "gettext")
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:09 GMT) Full text and rfc822 format available.Message #32 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 09/18] scripts: import: Document argument for ‘--insert’ option in help message. Date: Tue, 18 Mar 2025 15:24:22 +0800
* guix/scripts/import.scm (show-help): Add missing ‘FILE’. Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d --- guix/scripts/import.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index bbf31baa15..4fad329cd3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -69,7 +69,7 @@ (define (show-help) (display (G_ " -h, --help display this help and exit")) (display (G_ " - -i, --insert insert packages into file alphabetically")) + -i, --insert=FILE insert packages into FILE alphabetically")) (display (G_ " -V, --version display version information and exit")) (newline) -- 2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:10 GMT) Full text and rfc822 format available.Message #35 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 15:24:24 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 20 +++++++++++++++-----
guix/utils.scm | 27 +++++++++++++++++++--------
2 files changed, 34 insertions(+), 13 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 58a84d0db7..aaa3d26673 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -117,13 +120,20 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((definer?
+ (cut member
+ <>
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public)))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? definer? definer) term _ ...)
(let ((source-properties
(find-definition-insertion-location
- file term)))
+ file term #:definer definer)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index c7c23d9d5b..77ec6d992a 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (definer 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term comparing to TERM through PRED. Return the location if PRED returns #t,
+or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (definer? (cut eq? definer <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? definer?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key (definer 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term alphabetically succeeds TERM. Return the location if found, or #f
+otherwise."
+ (find-definition-location file term #:definer definer #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:12 GMT) Full text and rfc822 format available.Message #38 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 10/18] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 15:24:23 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines when inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4fad329cd3..58a84d0db7 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -129,6 +129,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:12 GMT) Full text and rfc822 format available.Message #41 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 12/18] scripts: import: Pass "--insert" to importers. Date: Tue, 18 Mar 2025 15:24:25 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index aaa3d26673..1b2c35f3b1 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -141,7 +147,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:13 GMT) Full text and rfc822 format available.Message #44 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 13/18] scripts: import: Skip existing definition for ‘--insert’ option. Date: Tue, 18 Mar 2025 15:24:26 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 1b2c35f3b1..d8e1ed3e6f 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -137,16 +137,18 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? definer? definer) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
+ ;; Skip existing definition.
+ (unless (find-definition-location file term #:definer definer)
+ (let ((source-properties
+ (find-definition-insertion-location
file term #:definer definer)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:15 GMT) Full text and rfc822 format available.Message #47 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 14/18] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Tue, 18 Mar 2025 15:24:27 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4486c706a1..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:17 GMT) Full text and rfc822 format available.Message #50 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 15/18] build-system: cargo: Add ‘cargo-inputs’. Date: Tue, 18 Mar 2025 15:24:28 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/cleanup-crates.sh | 38 +++++++++++++++++++++++++++
etc/teams/rust/rust-crates.tmpl | 42 +++++++++++++++++++++++++++++
gnu/local.mk | 2 ++
gnu/packages/rust-crates.scm | 42 +++++++++++++++++++++++++++++
gnu/packages/rust-sources.scm | 29 ++++++++++++++++++++
guix/build-system/cargo.scm | 45 +++++++++++++++++++++++++++++++-
guix/import/crate.scm | 2 +-
7 files changed, 198 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..cd4c2462b9
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:definer 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/local.mk b/gnu/local.mk
index 02de02e65f..8d8bd6ebbe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..bf9b91a671
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust dependencies requiring external inputs to
+;;; unbundle.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..47d4f10969 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -31,6 +31,7 @@ (define-module (guix build-system cargo)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +46,9 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +65,46 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
+ (sources-module '(gnu packages rust-sources)))
+ "Given NAME, resolve input list 'NAME-cargo-inputs' in CRATES-MODULE, return
+its copy with #f removed and symbols resolved to variables defined in
+SOURCES-MODULE, if the list exists, otherwise return an empty list."
+ (let loop ((inputs
+ (catch #t
+ (lambda ()
+ (module-ref (resolve-interface crates-module)
+ (symbol-append name '-cargo-inputs)))
+ (const '())))
+ (result '()))
+ (if (null? inputs)
+ result
+ (match inputs
+ ((input . rest)
+ (loop rest
+ (cond
+ ;; #f, remove it.
+ ((not input)
+ result)
+ ;; Symbol, resolve it in SOURCES-MODULE.
+ ((symbol? input)
+ (cons (module-ref (resolve-interface sources-module) input)
+ result))
+ ;; Else: no change.
+ (else
+ (cons input result)))))))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.48.1
hako <at> ultrarare.space, divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:17 GMT) Full text and rfc822 format available.Message #53 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH rust-team 16/18] import: crate: Add Cargo.lock parser. Date: Tue, 18 Mar 2025 15:24:29 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* guix/import/crate.scm (cargo-lock->definitions): New procedure.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
3 files changed, 184 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 4091a52387..a04e461ea8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..ef3a6d0b4b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->definitions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->definitions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, convert its content into source
+definitions. Return a list of sources and 'PACKAGE-NAME-cargo-inputs', an input
+list referencing all imported sources."
+ (define (crate->definition crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Workspace member.
+ (else #f)))
+
+ (let ((definitions
+ (filter-map crate->definition
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all)))))
+ (values definitions
+ `(define-public
+ ,(string->symbol (string-append package-name "-cargo-inputs"))
+ (list ,@(map second definitions))))))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
--
2.48.1
hako <at> ultrarare.space, guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:18 GMT) Full text and rfc822 format available.Message #56 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 17/18] import: crate: Add ‘--lockfile’ option. Date: Tue, 18 Mar 2025 15:24:30 +0800
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
* guix/scripts/import/crate.scm (show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 15 ++++++
guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 151 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index c20e1d7f9c..029c8cf59c 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14703,6 +14703,21 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, importer will ignore other
+options and won't output package definitions, instead generating source
+definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
+For example:
+
+@example
+guix import crate --lockfile=/path/to/Cargo.lock my-package
+@end example
+
+generates sources from @file{/path/to/Cargo.lock}, and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated
+list is intended for package's base @code{inputs}, replacing
+@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..3b536e135a 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,44 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((crate-definitions input-list)
+ (cargo-lock->definitions lockfile name)))
+ (if file-to-insert
+ ;; Adjusted from ‘--insert’ option of guix-import.
+ (let ((term (second input-list)))
+ (begin
+ ;; Remove existing input list definition.
+ (and=> (find-definition-location file-to-insert term)
+ delete-expression)
+ ;; Insert input list alphabetically.
+ (or (and=> (find-definition-insertion-location
+ file-to-insert term)
+ (cut insert-expression <> input-list))
+ (let ((port (open-file file-to-insert "a")))
+ (newline port)
+ (pretty-print-with-comments port input-list)
+ (newline port)
+ (newline port)
+ (close-port port))))
+ crate-definitions)
+ `(,@crate-definitions
+ ,input-list))))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..d46f753f9c 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-utils-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((crates-definitions input-list)
+ (cargo-lock->definitions temp-file "test")))
+ (and
+ (match crates-definitions
+ (((define 'rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define 'rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define 'rust-smithay-0.4.0.0cd3345
+ ($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match input-list
+ ((define-public 'test-cargo-inputs
+ (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.48.1
hako <at> ultrarare.space, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 07:27:18 GMT) Full text and rfc822 format available.Message #59 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team 18/18] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 15:24:31 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
1 file changed, 390 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..e7c5116a3d 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,399 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+As currently there's no achievable way to reuse Rust packages as pre-compiled
+inputs for other packages at a distribution scale, and our previous approach on
+making all Rust dependencies as full packages doesn't cope well with Rust
+ecosystem thus causing issues on both contribution and maintenance sides, we
+have switched to a different packaging model.
+
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
+
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense to dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, crate, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+contributing to Guix. Naturally, this module serves as a store for both sources
+and unbundling strategies.
+
+This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
+version containing all changes from other branches, so that the maintained
+version can be used directly in case of merge conflicts, thus coordination is
+required for other committers to modify it.
+
+Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
+script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
+For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix. Examples@:
+@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Let's demonstrate the packaging workflow by examples, note that package-specific
+issues are not involved here.
+
+In preparation, we'll adding the following packages to our environment:
+
+@example
+guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
+repository @uref{https://crates.io, crates.io}.
+
+@enumerate
+@item
+We can generate a draft definition via the crates.io importer (@pxref{Invoking
+guix import, crate}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
+facilitating dependency modification. @code{'cargo-audit} used here must be a
+unique identifier, usually same to the program's variable name.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+cargo generate-lockfile
+cargo audit
+cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining Packages,
+@code{license}}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them in Guix command-line interface through expression:
+
+@example
+guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+point the imported definition to a symbol with the same name.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to point to it:
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+
+Such modifications are processed by @code{cargo-inputs} procedure.
+
+@code{cargo-inputs} can also be configured by keywoard arguments
+@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
+resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
+packages rust-sources)}, module to resolve modified symbols), which are useful
+to maintain an independent channel.
+@end enumerate
+
+Example 2: @code{niri}, which is not available on crates.io and depends on
+development snapshots (also Cargo workspaces in this example).
+
+@enumerate
+@item
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+in a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in build environment, the substitution pattern is
+package-specific.
+
+@code{cargo-inputs} returns a list, all list operations apply to it too.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3)))
+@end lisp
+
+@item
+@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
+workspace, build argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+@end enumerate
+
+Example 3: @code{fish}, which combines two build systems.
+
+@enumerate
+@item
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+@end enumerate
@node Elm Packages
--
2.48.1
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:04 GMT) Full text and rfc822 format available.Message #62 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Steve George <steve <at> futurile.net>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [bug#77093] [PATCH rust-team 03/18] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 13:49:15 +0200
[Message part 1 (text/plain, inline)]
I've attached a diff that removes the need for RUSTC_BOOTSTRAP that
seems to work when I rebuilt rust-pubgrub.
Some links I followed:
https://doc.rust-lang.org/cargo/commands/cargo-publish.html#package-selection
https://github.com/rust-lang/cargo/issues/10948#issuecomment-1229236664
"One background knowledge: cargo publish is roughly `cargo package` +
call crates.io API."
Before and after:
(ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70 | head
/gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70
└── share
├── cargo
│ ├── registry
│ │ ├── pubgrub-0.3.0.crate
│ │ └── version-ranges-0.1.1.crate
│ └── src
│ ├── pubgrub-0.3.0
│ │ ├── benches
│ │ │ ├── backtracking.rs
(ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70 | head
/gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70
└── share
├── cargo
│ ├── registry
│ │ ├── pubgrub-0.3.0.crate
│ │ └── version-ranges-0.1.1.crate
│ └── src
│ ├── pubgrub-0.3.0
│ │ ├── benches
│ │ │ ├── backtracking.rs
On Tue, Mar 18, 2025 at 03:24:16PM +0800, Hilton Chain wrote:
> * guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
> [#:cargo-package-crates]: New argument.
> * guix/build/cargo-build-system.scm (package): Use it.
> * doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
>
> Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
> ---
> doc/guix.texi | 7 +++++++
> guix/build-system/cargo.scm | 4 ++++
> guix/build/cargo-build-system.scm | 28 ++++++++++++++++++++++++++--
> 3 files changed, 37 insertions(+), 2 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 0ca109a214..322512fd36 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9363,6 +9363,13 @@ Build Systems
> the binaries defined by the crate. Unless @code{install-source? #f} is
> defined it will also install a source crate repository of itself and unpacked
> sources, to ease in future hacking on rust packages.
> +
> +This build system supports cargo workspaces. Parameter
> +@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
> +names of library crates to package in @code{package} phase. Specified
> +crates are packaged from left to right, in case there's dependency among
> +them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> +package @code{"pcre2-sys"} first and then @code{"pcre2"}.
> @end defvar
>
> @defvar chicken-build-system
> diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
> index 452f7f78d0..4f6d46e70c 100644
> --- a/guix/build-system/cargo.scm
> +++ b/guix/build-system/cargo.scm
> @@ -94,6 +94,7 @@ (define* (cargo-build name inputs
> (vendor-dir "guix-vendor")
> (cargo-build-flags ''("--release"))
> (cargo-test-flags ''())
> + (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> (features ''())
> (skip-build? #f)
> @@ -122,6 +123,7 @@ (define* (cargo-build name inputs
> #:vendor-dir #$vendor-dir
> #:cargo-build-flags #$(sexp->gexp cargo-build-flags)
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> + #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> #:cargo-target #$(cargo-triplet system)
> #:features #$(sexp->gexp features)
> @@ -154,6 +156,7 @@ (define* (cargo-cross-build name
> (vendor-dir "guix-vendor")
> (cargo-build-flags ''("--release"))
> (cargo-test-flags ''())
> + (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> (cargo-target (cargo-triplet (or target system)))
> (features ''())
> @@ -185,6 +188,7 @@ (define* (cargo-cross-build name
> #:vendor-dir #$vendor-dir
> #:cargo-build-flags #$(sexp->gexp cargo-build-flags)
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> + #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> #:cargo-target #$(cargo-triplet (or target system))
> #:features #$(sexp->gexp features)
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index 58238992b8..001e9c7adc 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -300,11 +300,16 @@ (define* (package #:key
> source
> skip-build?
> install-source?
> + (cargo-package-crates '())
> (cargo-package-flags '("--no-metadata" "--no-verify"))
> + (vendor-dir "guix-vendor")
> #:allow-other-keys)
> "Run 'cargo-package' for a given Cargo package."
> (if install-source?
> - (if skip-build?
> + ;; NOTE: Cargo workspace packaging support:
> + ;; #:install-source? #t + #:cargo-package-crates.
> + (if (and (null? cargo-package-crates)
> + skip-build?)
> (begin
> (install-file source "target/package")
> (with-directory-excursion "target/package"
> @@ -322,7 +327,26 @@ (define* (package #:key
> ;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
> (when (file-exists? "Cargo.toml.orig")
> (delete-file "Cargo.toml.orig"))
> - (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> +
> + (if (null? cargo-package-crates)
> + (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> + ;; Use unstable feature ‘--registry’.
> + (begin
> + (setenv "RUSTC_BOOTSTRAP" "1")
> + (for-each
> + (lambda (pkg)
> + (apply invoke "cargo" "package" "--offline" "--package" pkg
> + "--registry" "crates-io" "-Z" "package-workspace"
> + cargo-package-flags)
> + (for-each
> + (lambda (crate)
> + (invoke "tar" "xzf" crate "-C" vendor-dir))
> + (begin
> + (delete-file-recursively "target/package/tmp-registry")
> + (find-files "target/package" "\\.crate$")))
> + (patch-cargo-checksums #:vendor-dir vendor-dir))
> + cargo-package-crates)
> + (unsetenv "RUSTC_BOOTSTRAP")))
>
> ;; Then unpack the crate, reset the timestamp of all contained files, and
> ;; repack them. This is necessary to ensure that they are reproducible.
> --
> 2.48.1
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[0003-no-unstable-flags (text/plain, attachment)]
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:06 GMT) Full text and rfc822 format available.Message #65 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team 08/18] build/cargo: Use system jemalloc. Date: Tue, 18 Mar 2025 13:49:21 +0200
[Message part 1 (text/plain, inline)]
I'm not sure how I feel about this one. On one hand the unprefixed_malloc flag is definitely necessary. On the other hand sometimes we use libjemalloc.so and sometimes libjemalloc_pic.a, and I don't remember when/why we use one over the other, other than "it just works" On Tue, Mar 18, 2025 at 03:24:21PM +0800, Hilton Chain wrote: > From: Efraim Flashner <efraim <at> flashner.co.il> > > * guix/build/cargo-build-system.scm (configure): Set environment variables to > use system jemalloc. > > Change-Id: Ibb1c025c7354ba2befd2770e3f1ccd785baa06c3 > Modified-by: Hilton Chain <hako <at> ultrarare.space> > --- > guix/build/cargo-build-system.scm | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm > index a1c4f556bb..b547421b88 100644 > --- a/guix/build/cargo-build-system.scm > +++ b/guix/build/cargo-build-system.scm > @@ -248,6 +248,12 @@ (define* (configure #:key inputs > (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1") > (setenv "SODIUM_USE_PKG_CONFIG" "1") > (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1") > + ;; This flag is needed when not using the bundled jemalloc. > + ;; https://github.com/tikv/jemallocator/issues/19 > + (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1") > + (when (assoc-ref inputs "jemalloc") > + (setenv "JEMALLOC_OVERRIDE" > + (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so"))) > (when (assoc-ref inputs "openssl") > (setenv "OPENSSL_DIR" (assoc-ref inputs "openssl"))) > (when (assoc-ref inputs "gettext") > -- > 2.48.1 > -- Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:08 GMT) Full text and rfc822 format available.Message #68 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Steve George <steve <at> futurile.net>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [bug#77093] [PATCH rust-team 04/18] build-system: cargo: Support installing Cargo workspace. Date: Tue, 18 Mar 2025 13:49:27 +0200
[Message part 1 (text/plain, inline)]
I don't love the name cargo-install-paths. I had to build atuin to see
what it did. The last few lines of the 'install phase:
Compiling atuin-daemon v18.4.0 (/tmp/guix-build-atuin-18.4.0.drv-0/source/crates/atuin-daemon)
Compiling atuin v18.4.0 (/tmp/guix-build-atuin-18.4.0.drv-0/source/crates/atuin)
Finished `release` profile [optimized] target(s) in 22.73s
Installing /gnu/store/yqfi454wkc1lv455m7n5i7l5lwn1la8v-atuin-18.4.0/bin/atuin
Installed package `atuin v18.4.0 (/tmp/guix-build-atuin-18.4.0.drv-0/source/crates/atuin)` (executable `atuin`)
warning: be sure to add `/gnu/store/yqfi454wkc1lv455m7n5i7l5lwn1la8v-atuin-18.4.0/bin` to your PATH to be able to run the installed binaries
With #:cargo-install-paths removed I got an error that it was a virtual
manifest and to find the actual package. Looking at the Cargo.toml I
don't see an obvious way to figure out which package(s) in the workspace
has an installable executable.
So obviously 'paths-to-installable-crates-in-the-worktree' is too long.
Looking at the code it's obvious how the name cargo-install-path came to
be.
On Tue, Mar 18, 2025 at 03:24:17PM +0800, Hilton Chain wrote:
> *guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
> [#:cargo-install-paths]: New argument.
> * guix/build/cargo-build-system.scm (install): Use it.
> * doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
>
> Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
> ---
> doc/guix.texi | 5 ++++-
> guix/build-system/cargo.scm | 4 ++++
> guix/build/cargo-build-system.scm | 17 +++++++++++++----
> 3 files changed, 21 insertions(+), 5 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 322512fd36..c20e1d7f9c 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9369,7 +9369,10 @@ Build Systems
> names of library crates to package in @code{package} phase. Specified
> crates are packaged from left to right, in case there's dependency among
> them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> -package @code{"pcre2-sys"} first and then @code{"pcre2"}.
> +package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> +@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
> +paths of binary crates to install in @code{install} phase,
paths of binary crates to install in the @code{install} phase,
> +@code{''("crates/atuin")}, for example.
> @end defvar
>
> @defvar chicken-build-system
> diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
> index 4f6d46e70c..4486c706a1 100644
> --- a/guix/build-system/cargo.scm
> +++ b/guix/build-system/cargo.scm
> @@ -96,6 +96,7 @@ (define* (cargo-build name inputs
> (cargo-test-flags ''())
> (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> + (cargo-install-paths ''())
> (features ''())
> (skip-build? #f)
> (parallel-build? #t)
> @@ -125,6 +126,7 @@ (define* (cargo-build name inputs
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> + #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
> #:cargo-target #$(cargo-triplet system)
> #:features #$(sexp->gexp features)
> #:skip-build? #$skip-build?
> @@ -158,6 +160,7 @@ (define* (cargo-cross-build name
> (cargo-test-flags ''())
> (cargo-package-crates ''())
> (cargo-package-flags ''("--no-metadata" "--no-verify"))
> + (cargo-install-paths ''())
> (cargo-target (cargo-triplet (or target system)))
> (features ''())
> (skip-build? #f)
> @@ -190,6 +193,7 @@ (define* (cargo-cross-build name
> #:cargo-test-flags #$(sexp->gexp cargo-test-flags)
> #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
> #:cargo-package-flags #$(sexp->gexp cargo-package-flags)
> + #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
> #:cargo-target #$(cargo-triplet (or target system))
> #:features #$(sexp->gexp features)
> #:skip-build? #$skip-build?
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index 001e9c7adc..bcee10837e 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -384,6 +384,7 @@ (define* (install #:key
> skip-build?
> install-source?
> features
> + (cargo-install-paths '())
> #:allow-other-keys)
> "Install a given Cargo package."
> (let* ((out (assoc-ref outputs "out"))
> @@ -398,10 +399,18 @@ (define* (install #:key
> ;; Only install crates which include binary targets,
> ;; otherwise cargo will raise an error.
> (or skip-build?
> - (not (has-executable-target?))
> - (invoke "cargo" "install" "--offline" "--no-track"
> - "--path" "." "--root" out
> - "--features" (string-join features)))
> + ;; NOTE: Cargo workspace installation support:
> + ;; #:skip-build? #f + #:cargo-install-paths.
> + (and (null? cargo-install-paths)
> + (not (has-executable-target?)))
> + (for-each
> + (lambda (path)
> + (invoke "cargo" "install" "--offline" "--no-track"
> + "--path" path "--root" out
> + "--features" (string-join features)))
> + (if (null? cargo-install-paths)
> + '(".")
> + cargo-install-paths)))
>
> (when install-source?
> ;; Install crate tarballs and unpacked sources for later use.
> --
> 2.48.1
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:50:12 GMT) Full text and rfc822 format available.Message #71 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 10/18] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 13:49:33 +0200
[Message part 1 (text/plain, inline)]
Is this only relevant for packages crated with 'guix import cargo'? If so we should probably just change the importer to add a newline to the misbehaving importer. On Tue, Mar 18, 2025 at 03:24:23PM +0800, Hilton Chain wrote: > For consistency with ‘insert-expression’. > > * guix/scripts/import.scm (guix-import): Add two newlines when inserting. > > Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d > --- > guix/scripts/import.scm | 1 + > 1 file changed, 1 insertion(+) > > diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm > index 4fad329cd3..58a84d0db7 100644 > --- a/guix/scripts/import.scm > +++ b/guix/scripts/import.scm > @@ -129,6 +129,7 @@ (define-command (guix-import . args) > (let ((port (open-file file "a"))) > (pretty-print-with-comments port expr) > (newline port) > + (newline port) > (close-port port))))))))) > (import-as-definitions importer args find-and-insert))) > ((importer args ...) > -- > 2.48.1 > > > > -- Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351 Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:51:03 GMT) Full text and rfc822 format available.Message #74 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 13:49:39 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 03:24:24PM +0800, Hilton Chain wrote:
> * guix/utils.scm (find-definition-location): New procedure.
> (find-definition-insertion-location): Define with it.
> * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> expressions defined by 'define.
>
> Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> ---
> guix/scripts/import.scm | 20 +++++++++++++++-----
> guix/utils.scm | 27 +++++++++++++++++++--------
> 2 files changed, 34 insertions(+), 13 deletions(-)
>
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index 58a84d0db7..aaa3d26673 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> #:use-module (guix read-print)
> #:use-module (guix utils)
> #:use-module (srfi srfi-1)
> + #:use-module (srfi srfi-26)
> #:use-module (ice-9 format)
> #:use-module (ice-9 match)
> #:export (%standard-import-options
> @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))
> ((expressions ...)
> (for-each (lambda (expr)
> @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))))
> expressions))
> (x
> @@ -117,13 +120,20 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let ((find-and-insert
> + (let* ((definer?
> + (cut member
> + <>
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public)))
This part above seems like it would break the option to use 'guix import
crate' with the --insert flag for "normal" packages. I question how
useful it currently is in that scenario since we normally have to strip
the rust- prefix from those packages anyway.
> + (find-and-insert
> (lambda (expr)
> (match expr
> - (('define-public term _ ...)
> + (((? definer? definer) term _ ...)
> (let ((source-properties
> (find-definition-insertion-location
> - file term)))
> + file term #:definer definer)))
> (if source-properties
> (insert-expression source-properties expr)
> (let ((port (open-file file "a")))
> diff --git a/guix/utils.scm b/guix/utils.scm
> index c7c23d9d5b..77ec6d992a 100644
> --- a/guix/utils.scm
> +++ b/guix/utils.scm
> @@ -154,6 +154,7 @@ (define-module (guix utils)
> edit-expression
> delete-expression
> insert-expression
> + find-definition-location
> find-definition-insertion-location
>
> filtered-port
> @@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
> (string-append expr "\n\n" str))))
> (edit-expression source-properties insert)))
>
> -(define (find-definition-insertion-location file term)
> - "Search in FILE for a top-level public definition whose defined term
> -alphabetically succeeds TERM. Return the location if found, or #f
> -otherwise."
> - (let ((search-term (symbol->string term)))
> +(define* (find-definition-location file term
> + #:key (definer 'define-public)
> + (pred string=))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term comparing to TERM through PRED. Return the location if PRED returns #t,
> +or #f otherwise."
> + (let ((search-term (symbol->string term))
> + (definer? (cut eq? definer <>)))
> (call-with-input-file file
> (lambda (port)
> (do ((syntax (read-syntax port)
> (read-syntax port)))
> ((match (syntax->datum syntax)
> - (('define-public current-term _ ...)
> - (string> (symbol->string current-term)
> - search-term))
> + (((? definer?) current-term _ ...)
> + (pred (symbol->string current-term)
> + search-term))
> ((? eof-object?) #t)
> (_ #f))
> (and (not (eof-object? syntax))
> (syntax-source syntax))))))))
>
> +(define* (find-definition-insertion-location file term
> + #:key (definer 'define-public))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term alphabetically succeeds TERM. Return the location if found, or #f
> +otherwise."
> + (find-definition-location file term #:definer definer #:pred string>))
> +
>
> ;;;
> ;;; Keyword arguments.
> --
> 2.48.1
>
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:51:04 GMT) Full text and rfc822 format available.Message #77 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Ludovic Courtès <ludo <at> gnu.org>, 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: Re: [bug#77093] [PATCH rust-team 18/18] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 13:49:54 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 03:24:31PM +0800, Hilton Chain wrote:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
> doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..e7c5116a3d 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
> @subsection Rust Crates
>
> @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
I'm not sure the first paragraph is necessary.
> +
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately. We put main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with manual focus on unbundling
> +vendored dependencies. The following paragraphs will explain them and give
> +several examples.
Rust programs (binary crates) and dependencies (library crates) are treated
separately. We put our main efforts into programs and only package Rust
dependencies as sources, utilizing automation with a manual focus on unbundling
vendored dependencies. The following paragraphs will explain them and give
several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name. When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense to dependencies.
argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix. The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix. Naturally, this module serves as a store for both sources
> +and unbundling strategies.
> +
> +This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
This module is managed by the Rust team (@pxref{Teams}) to ensure there's always one
> +version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
> +
> +Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> +script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
script @file{etc/teams/rust/cleanup-crates.sh} for this module.
> +
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages. This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
These dependencies should have the @code{#:skip-build?} argument set to @code{#t}.
> +For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix. The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>
> In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix. The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course). This follows the ``caret'' version scheme
> -intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments. It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}. If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible. Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix. The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course). This follows the ``caret'' version scheme intended
> +by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> +
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix. Examples@:
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
> +
> +In preparation, we'll adding the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
> +
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}). In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> + (package
> + (name "cargo-audit")
> + (version "0.21.2")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (crate-uri "cargo-audit" version))
> + (file-name (string-append name "-" version ".tar.gz"))
> + (sha256
> + (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> + (build-system cargo-build-system)
> + (arguments (list #:install-source? #f))
> + (inputs (cargo-inputs 'cargo-audit))
> + (home-page "https://rustsec.org/")
> + (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> + (description
> + "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> + (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification. @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> + crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources. The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> + (crate-source "bzip2-sys" "0.1.13+1.0.8"
> + "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> + #:snippet
> + '(begin
> + (delete-file-recursively "bzip2-1.0.8")
> + (delete-file "build.rs")
> + ;; Inspired by Debian's patch.
> + (with-output-to-file "build.rs"
> + (lambda _
> + (format #t "fn main() @{~@@
> + println!(\"cargo:rustc-link-lib=bz2\");~@@
> + @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
> +
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
> +
> +@lisp
> +(define-public niri
> + (package
> + (name "niri")
> + (version "25.02")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/YaLTeR/niri")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:install-source? #f
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("# version =.*")
> + "version = \"*\"")
> + (("git.*optional")
> + "version = \"*\", optional")
> + (("^git = .*")
> + "")))))))
> + (native-inputs
> + (list pkg-config))
> + (inputs
> + (cons* clang
> + libdisplay-info
> + libinput-minimal
> + libseat
> + libxkbcommon
> + mesa
> + pango
> + pipewire
> + wayland
> + (cargo-inputs 'niri)))
> + (home-page "https://github.com/YaLTeR/niri")
> + (synopsis "Scrollable-tiling Wayland compositor")
> + (description
> + "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format. It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> + (license license:gpl3)))
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
> +
> +@lisp
> +(define-public rust-pipewire-for-niri
> + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> + (revision "0"))
> + (package
> + (name "rust-pipewire")
> + (version (git-version "0.8.0" revision commit))
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:skip-build? #t
> + #:cargo-package-crates
> + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> + (inputs (cargo-inputs 'rust-pipewire-for-niri))
> + (home-page "https://pipewire.org/")
> + (synopsis "Rust bindings for PipeWire")
> + (description "This package provides Rust bindings for PipeWire.")
> + (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> + (package
> + (name "fish")
> + (version "4.0.0")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://github.com/fish-shell/fish-shell/"
> + "releases/download/" version "/"
> + "fish-" version ".tar.xz"))
> + (sha256
> + (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> + (build-system cmake-build-system)
> + (inputs
> + (cons* fish-foreign-env ncurses pcre2
> + python ;for fish_config and manpage completions
> + (cargo-inputs 'fish)))
> + (native-inputs
> + (list doxygen groff ;for 'fish --help'
> + pkg-config
> + procps ;for the test suite
> + rust
> + `(,rust "cargo")))
> + (arguments
> + (list #:imported-modules
> + (append %cargo-build-system-modules
> + %cmake-build-system-modules)
> + #:modules
> + (((guix build cargo-build-system) #:prefix cargo:)
> + (guix build cmake-build-system)
> + (guix build utils))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("git.*tag.*,")
> + "version = \"*\","))))
> + (add-after 'unpack 'prepare-cargo-build-system
> + (lambda args
> + (for-each
> + (lambda (phase)
> + (format #t "Running cargo phase: ~a~%" phase)
> + (apply (assoc-ref cargo:%standard-phases phase)
> + args))
> + '(unpack-rust-crates
> + configure
> + check-for-pregenerated-files
> + patch-cargo-checksums)))))))
> + (synopsis "The friendly interactive shell")
> + (description
> + "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness. Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands. It also
> +has extensive and discoverable help. A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser. Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> + (home-page "https://fishshell.com/")
> + (license license:gpl2)))
> +@end lisp
> +@end enumerate
>
>
> @node Elm Packages
> --
> 2.48.1
>
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 11:51:05 GMT) Full text and rfc822 format available.Message #80 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Steve George <steve <at> futurile.net>, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 17/18] import: crate: Add ‘--lockfile’ option. Date: Tue, 18 Mar 2025 13:49:46 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 03:24:30PM +0800, Hilton Chain wrote:
> * guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
> * guix/scripts/import/crate.scm (show-help): Add it.
> (guix-import-crate): Use it.
> * doc/guix.texi (Invoking guix import): Document it.
> * tests/crate.scm (temp-file): New variable.
> ("crate-lockfile-import"): New test.
>
> Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
> ---
> doc/guix.texi | 15 ++++++
> guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
> tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
> 3 files changed, 151 insertions(+), 10 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index c20e1d7f9c..029c8cf59c 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -14703,6 +14703,21 @@ Invoking guix import
> If a crate dependency is not (yet) packaged, make the corresponding
> input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
> a comment.
> +@item --lockfile=@var{file}
> +@itemx -f @var{file}
> +When @option{--lockfile} is specified, importer will ignore other
> +options and won't output package definitions, instead generating source
> +definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
When @option{--lockfile} is specified, the importer will ignore other
options and won't output package definitions, instead generating source
definitions for all dependencies in @var{file}, a @file{Cargo.lock} file.
> +For example:
> +
> +@example
> +guix import crate --lockfile=/path/to/Cargo.lock my-package
> +@end example
> +
> +generates sources from @file{/path/to/Cargo.lock}, and a list
> +@code{my-package-cargo-inputs} referencing these sources. The generated
> +list is intended for package's base @code{inputs}, replacing
> +@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
generates sources from @file{/path/to/Cargo.lock} and a list
@code{my-package-cargo-inputs} referencing these sources. The generated
list is intended for the package's @code{inputs}, replacing
@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
> @end table
>
> @item elm
> diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
> index 723cbb3665..3b536e135a 100644
> --- a/guix/scripts/import/crate.scm
> +++ b/guix/scripts/import/crate.scm
> @@ -25,11 +25,13 @@
> (define-module (guix scripts import crate)
> #:use-module (guix ui)
> #:use-module (guix utils)
> + #:use-module (guix read-print)
> #:use-module (guix scripts)
> #:use-module (guix import crate)
> #:use-module (guix scripts import)
> #:use-module (srfi srfi-1)
> #:use-module (srfi srfi-11)
> + #:use-module (srfi srfi-26)
> #:use-module (srfi srfi-37)
> #:use-module (ice-9 match)
> #:use-module (ice-9 format)
> @@ -60,6 +62,9 @@ (define (show-help)
> sufficient package exists for it"))
> (newline)
> (display (G_ "
> + -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
> + (newline)
> + (display (G_ "
> -h, --help display this help and exit"))
> (display (G_ "
> -V, --version display version information and exit"))
> @@ -87,6 +92,9 @@ (define %options
> (option '("mark-missing") #f #f
> (lambda (opt name arg result)
> (alist-cons 'mark-missing #t result)))
> + (option '(#\f "lockfile") #f #t
> + (lambda (opt name arg result)
> + (alist-cons 'lockfile arg result)))
> %standard-import-options))
>
>
> @@ -101,6 +109,8 @@ (define (guix-import-crate . args)
> #:build-options? #f))
>
> (let* ((opts (parse-options))
> + (lockfile (assoc-ref opts 'lockfile))
> + (file-to-insert (assoc-ref opts 'file-to-insert))
> (args (filter-map (match-lambda
> (('argument . value)
> value)
> @@ -111,16 +121,44 @@ (define (guix-import-crate . args)
> (define-values (name version)
> (package-name->name+version spec))
>
> - (match (if (assoc-ref opts 'recursive)
> - (crate-recursive-import
> - name #:version version
> - #:recursive-dev-dependencies?
> - (assoc-ref opts 'recursive-dev-dependencies)
> - #:allow-yanked? (assoc-ref opts 'allow-yanked))
> - (crate->guix-package
> - name #:version version #:include-dev-deps? #t
> - #:allow-yanked? (assoc-ref opts 'allow-yanked)
> - #:mark-missing? (assoc-ref opts 'mark-missing)))
> + (match (cond
> + ((and=> lockfile
> + (lambda (file)
> + (or (file-exists? file)
> + (leave (G_ "file '~a' does not exist~%") file))))
> + (let-values (((crate-definitions input-list)
> + (cargo-lock->definitions lockfile name)))
> + (if file-to-insert
> + ;; Adjusted from ‘--insert’ option of guix-import.
> + (let ((term (second input-list)))
> + (begin
> + ;; Remove existing input list definition.
> + (and=> (find-definition-location file-to-insert term)
> + delete-expression)
> + ;; Insert input list alphabetically.
> + (or (and=> (find-definition-insertion-location
> + file-to-insert term)
> + (cut insert-expression <> input-list))
> + (let ((port (open-file file-to-insert "a")))
> + (newline port)
> + (pretty-print-with-comments port input-list)
> + (newline port)
> + (newline port)
> + (close-port port))))
> + crate-definitions)
> + `(,@crate-definitions
> + ,input-list))))
> + ((assoc-ref opts 'recursive)
> + (crate-recursive-import
> + name #:version version
> + #:recursive-dev-dependencies?
> + (assoc-ref opts 'recursive-dev-dependencies)
> + #:allow-yanked? (assoc-ref opts 'allow-yanked)))
> + (else
> + (crate->guix-package
> + name #:version version #:include-dev-deps? #t
> + #:allow-yanked? (assoc-ref opts 'allow-yanked)
> + #:mark-missing? (assoc-ref opts 'mark-missing))))
> ((or #f '())
> (leave (G_ "failed to download meta-data for package '~a'~%")
> (if version
> diff --git a/tests/crate.scm b/tests/crate.scm
> index d0dc779cd2..d46f753f9c 100644
> --- a/tests/crate.scm
> +++ b/tests/crate.scm
> @@ -34,6 +34,7 @@ (define-module (test-crate)
> #:use-module (gnu packages)
> #:use-module (ice-9 iconv)
> #:use-module (ice-9 match)
> + #:use-module (srfi srfi-11)
> #:use-module (srfi srfi-64))
>
>
> @@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
> (description #f)
> (license #f)))
>
> +(define temp-file
> + (string-append "t-utils-" (number->string (getpid))))
> +
>
> (test-begin "crate")
>
> @@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
> (x
> (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
>
> +
> +(test-assert "crate-lockfile-import"
> + (begin
> + (call-with-output-file temp-file
> + (lambda (port)
> + (display "\
> +# This file is automatically @generated by Cargo.
> +# It is not intended for manual editing.
> +version = 3
> +
> +[[package]]
> +name = \"adler2\"
> +version = \"2.0.0\"
> +source = \"registry+https://github.com/rust-lang/crates.io-index\"
> +checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
> +
> +[[package]]
> +name = \"aho-corasick\"
> +version = \"1.1.3\"
> +source = \"registry+https://github.com/rust-lang/crates.io-index\"
> +checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
> +dependencies = [
> + \"memchr\",
> +]
> +
> +[[package]]
> +name = \"smithay\"
> +version = \"0.4.0\"
> +source = \"git+https://github.com/Smithay/smithay.git?rev=\
> +0cd3345c59f7cb139521f267956a1a4e33248393#\
> +0cd3345c59f7cb139521f267956a1a4e33248393\"
> +dependencies = [
> + \"appendlist\",
> +]
> +
> +[[package]]
> +name = \"test\"
> +version = \"25.2.0\"\n" port)))
> + (mock
> + ((guix scripts download) guix-download
> + (lambda _
> + (format #t "~a~%~a~%"
> + "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
> + "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
> + (let-values
> + (((crates-definitions input-list)
> + (cargo-lock->definitions temp-file "test")))
> + (and
> + (match crates-definitions
> + (((define 'rust-adler2-2.0.0
> + (crate-source
> + "adler2" "2.0.0"
> + "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
> + (define 'rust-aho-corasick-1.1.3
> + (crate-source
> + "aho-corasick" "1.1.3"
> + "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
> + (define 'rust-smithay-0.4.0.0cd3345
> + ($ <comment>
> + ";; TODO: Define standalone package if this is a workspace.\n"
> + #f)
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/Smithay/smithay.git")
> + (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
> + (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
> + (sha256
> + (base32
> + "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
> + #t)
> + (x
> + (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
> + (match input-list
> + ((define-public 'test-cargo-inputs
> + (list rust-adler2-2.0.0
> + rust-aho-corasick-1.1.3
> + rust-smithay-0.4.0.0cd3345))
> + #t)
> + (x
> + (pk 'fail x #f))))))))
> +
> (test-end "crate")
> +
> +(false-if-exception (delete-file temp-file))
> --
> 2.48.1
>
>
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:02 GMT) Full text and rfc822 format available.Message #83 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Divya Ranjan Pattanaik <divya <at> subvertising.org>, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net> Subject: Re: [bug#77093] [PATCH rust-team 03/18] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 20:07:26 +0800
On Tue, 18 Mar 2025 19:49:15 +0800, Efraim Flashner wrote: > > [1 <multipart/mixed (7bit)>] > [1.1 <text/plain; utf-8 (quoted-printable)>] > I've attached a diff that removes the need for RUSTC_BOOTSTRAP that > seems to work when I rebuilt rust-pubgrub. > > Some links I followed: > https://doc.rust-lang.org/cargo/commands/cargo-publish.html#package-selection > https://github.com/rust-lang/cargo/issues/10948#issuecomment-1229236664 > "One background knowledge: cargo publish is roughly `cargo package` + > call crates.io API." > > Before and after: > > (ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70 | head > /gnu/store/zk1n23l2wdca8snz1wqllmjxv83hccl5-rust-pubgrub-0.3.0-0.b70cf70 > └── share > ├── cargo > │ ├── registry > │ │ ├── pubgrub-0.3.0.crate > │ │ └── version-ranges-0.1.1.crate > │ └── src > │ ├── pubgrub-0.3.0 > │ │ ├── benches > │ │ │ ├── backtracking.rs > (ins)efraim <at> 3900XT ~/workspace/guix-rust-team$ tree /gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70 | head > /gnu/store/9310sgn182cgqy4p1xkn2hi8x4wg0nn8-rust-pubgrub-0.3.0-0.b70cf70 > └── share > ├── cargo > │ ├── registry > │ │ ├── pubgrub-0.3.0.crate > │ │ └── version-ranges-0.1.1.crate > │ └── src > │ ├── pubgrub-0.3.0 > │ │ ├── benches > │ │ │ ├── backtracking.rs Thanks! It works for rust-pipewire too.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:02 GMT) Full text and rfc822 format available.Message #86 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team 08/18] build/cargo: Use system jemalloc. Date: Tue, 18 Mar 2025 20:09:07 +0800
On Tue, 18 Mar 2025 19:49:21 +0800, Efraim Flashner wrote: > > [1 <text/plain; utf-8 (quoted-printable)>] > I'm not sure how I feel about this one. On one hand the > unprefixed_malloc flag is definitely necessary. On the other hand > sometimes we use libjemalloc.so and sometimes libjemalloc_pic.a, and I > don't remember when/why we use one over the other, other than "it just > works" > > On Tue, Mar 18, 2025 at 03:24:21PM +0800, Hilton Chain wrote: > > From: Efraim Flashner <efraim <at> flashner.co.il> > > > > * guix/build/cargo-build-system.scm (configure): Set environment variables to > > use system jemalloc. > > > > Change-Id: Ibb1c025c7354ba2befd2770e3f1ccd785baa06c3 > > Modified-by: Hilton Chain <hako <at> ultrarare.space> > > --- > > guix/build/cargo-build-system.scm | 6 ++++++ > > 1 file changed, 6 insertions(+) I'll drop this one for now then, I didn't search the source tree :P
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:03 GMT) Full text and rfc822 format available.Message #89 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Christopher Baines <guix <at> cbaines.net>, Josselin Poiret <dev <at> jpoiret.xyz>, Ludovic Courtès <ludo <at> gnu.org>, Mathieu Othacehe <othacehe <at> gnu.org>, Simon Tournier <zimon.toutoune <at> gmail.com>, Tobias Geerinckx-Rice <me <at> tobias.gr> Subject: Re: [bug#77093] [PATCH rust-team 10/18] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 20:18:43 +0800
On Tue, 18 Mar 2025 19:49:33 +0800, Efraim Flashner wrote: > > [1 <text/plain; utf-8 (quoted-printable)>] > Is this only relevant for packages crated with 'guix import cargo'? If > so we should probably just change the importer to add a newline to the > misbehaving importer. > > On Tue, Mar 18, 2025 at 03:24:23PM +0800, Hilton Chain wrote: > > For consistency with ‘insert-expression’. > > > > * guix/scripts/import.scm (guix-import): Add two newlines when inserting. > > > > Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d > > --- > > guix/scripts/import.scm | 1 + > > 1 file changed, 1 insertion(+) This is a general issue, adding one newline doesn't work well with appending multiple definitions. For appending n definitions, (n - 1) lines need manual addition: --8<---------------cut here---------------start------------->8--- a b c --8<---------------cut here---------------end--------------->8--- By adding two newlines, only one line needs manual deletion: --8<---------------cut here---------------start------------->8--- a b c --8<---------------cut here---------------end--------------->8---
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:03 GMT) Full text and rfc822 format available.Message #92 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Christopher Baines <guix <at> cbaines.net>, Josselin Poiret <dev <at> jpoiret.xyz>, Ludovic Courtès <ludo <at> gnu.org>, Mathieu Othacehe <othacehe <at> gnu.org>, Simon Tournier <zimon.toutoune <at> gmail.com>, Tobias Geerinckx-Rice <me <at> tobias.gr> Subject: Re: [bug#77093] [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 20:29:49 +0800
On Tue, 18 Mar 2025 19:49:39 +0800,
Efraim Flashner wrote:
>
> [1 <text/plain; utf-8 (quoted-printable)>]
> On Tue, Mar 18, 2025 at 03:24:24PM +0800, Hilton Chain wrote:
> > * guix/utils.scm (find-definition-location): New procedure.
> > (find-definition-insertion-location): Define with it.
> > * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> > expressions defined by 'define.
> >
> > Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> > ---
> > guix/scripts/import.scm | 20 +++++++++++++++-----
> > guix/utils.scm | 27 +++++++++++++++++++--------
> > 2 files changed, 34 insertions(+), 13 deletions(-)
> >
> > diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> > index 58a84d0db7..aaa3d26673 100644
> > --- a/guix/scripts/import.scm
> > +++ b/guix/scripts/import.scm
> > @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> > #:use-module (guix read-print)
> > #:use-module (guix utils)
> > #:use-module (srfi srfi-1)
> > + #:use-module (srfi srfi-26)
> > #:use-module (ice-9 format)
> > #:use-module (ice-9 match)
> > #:export (%standard-import-options
> > @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> > ((and expr (or ('package _ ...)
> > ('let _ ...)))
> > (proc (package->definition expr)))
> > - ((and expr ('define-public _ ...))
> > + ((and expr (or ('define-public _ ...)
> > + ('define _ ...)))
> > (proc expr))
> > ((expressions ...)
> > (for-each (lambda (expr)
> > @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> > ((and expr (or ('package _ ...)
> > ('let _ ...)))
> > (proc (package->definition expr)))
> > - ((and expr ('define-public _ ...))
> > + ((and expr (or ('define-public _ ...)
> > + ('define _ ...)))
> > (proc expr))))
> > expressions))
> > (x
> > @@ -117,13 +120,20 @@ (define-command (guix-import . args)
> > (show-version-and-exit "guix import"))
> > ((or ("-i" file importer args ...)
> > ("--insert" file importer args ...))
> > - (let ((find-and-insert
> > + (let* ((definer?
> > + (cut member
> > + <>
> > + `(,@(if (member importer '("crate"))
> > + '(define)
> > + '())
> > + define-public)))
>
> This part above seems like it would break the option to use 'guix import
> crate' with the --insert flag for "normal" packages. I question how
> useful it currently is in that scenario since we normally have to strip
> the rust- prefix from those packages anyway.
Hmmm, since the crate importer now checks both ‘define’ and ‘define-public’,
there might be ordering issue when the module to insert uses ‘define’. But it's
still managable I think.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 12:39:04 GMT) Full text and rfc822 format available.Message #95 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il>, Hilton Chain <hako <at> ultrarare.space>, 77093 <at> debbugs.gnu.org, Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: Re: [bug#77093] [PATCH rust-team 18/18] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 20:36:17 +0800
On Tue, 18 Mar 2025 19:49:54 +0800, Efraim Flashner wrote: > > [1 <text/plain; utf-8 (quoted-printable)>] > On Tue, Mar 18, 2025 at 03:24:31PM +0800, Hilton Chain wrote: > > * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update > > documentation. > > > > Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc > > --- > > doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++--- > > 1 file changed, 390 insertions(+), 25 deletions(-) > > > > diff --git a/doc/contributing.texi b/doc/contributing.texi > > index ab4f30d54b..e7c5116a3d 100644 > > --- a/doc/contributing.texi > > +++ b/doc/contributing.texi > > @@ -1600,34 +1600,399 @@ Rust Crates > > @subsection Rust Crates > > > > @cindex rust > > -Rust programs standing for themselves are named as any other package, using the > > -lowercase upstream name. > > +As currently there's no achievable way to reuse Rust packages as pre-compiled > > +inputs for other packages at a distribution scale, and our previous approach on > > +making all Rust dependencies as full packages doesn't cope well with Rust > > +ecosystem thus causing issues on both contribution and maintenance sides, we > > +have switched to a different packaging model. > > I'm not sure the first paragraph is necessary. I think we need to clarify why do we only do this for Rust, since similar workflow can be applied to other ecosystems with a package manager as well. Thanks for the review, I'll prepare V2 soon.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 13:17:04 GMT) Full text and rfc822 format available.Message #98 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: Josselin Poiret <dev <at> jpoiret.xyz>, 77093 <at> debbugs.gnu.org, Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès <ludo <at> gnu.org>, Tobias Geerinckx-Rice <me <at> tobias.gr>, Christopher Baines <guix <at> cbaines.net> Subject: Re: [bug#77093] [PATCH rust-team 11/18] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 15:16:18 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 08:29:49PM +0800, Hilton Chain wrote:
> On Tue, 18 Mar 2025 19:49:39 +0800,
> Efraim Flashner wrote:
> >
> > [1 <text/plain; utf-8 (quoted-printable)>]
> > On Tue, Mar 18, 2025 at 03:24:24PM +0800, Hilton Chain wrote:
> > > * guix/utils.scm (find-definition-location): New procedure.
> > > (find-definition-insertion-location): Define with it.
> > > * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> > > expressions defined by 'define.
> > >
> > > Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> > > ---
> > > guix/scripts/import.scm | 20 +++++++++++++++-----
> > > guix/utils.scm | 27 +++++++++++++++++++--------
> > > 2 files changed, 34 insertions(+), 13 deletions(-)
> > >
> > > diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> > > index 58a84d0db7..aaa3d26673 100644
> > > --- a/guix/scripts/import.scm
> > > +++ b/guix/scripts/import.scm
> > > @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> > > #:use-module (guix read-print)
> > > #:use-module (guix utils)
> > > #:use-module (srfi srfi-1)
> > > + #:use-module (srfi srfi-26)
> > > #:use-module (ice-9 format)
> > > #:use-module (ice-9 match)
> > > #:export (%standard-import-options
> > > @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> > > ((and expr (or ('package _ ...)
> > > ('let _ ...)))
> > > (proc (package->definition expr)))
> > > - ((and expr ('define-public _ ...))
> > > + ((and expr (or ('define-public _ ...)
> > > + ('define _ ...)))
> > > (proc expr))
> > > ((expressions ...)
> > > (for-each (lambda (expr)
> > > @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> > > ((and expr (or ('package _ ...)
> > > ('let _ ...)))
> > > (proc (package->definition expr)))
> > > - ((and expr ('define-public _ ...))
> > > + ((and expr (or ('define-public _ ...)
> > > + ('define _ ...)))
> > > (proc expr))))
> > > expressions))
> > > (x
> > > @@ -117,13 +120,20 @@ (define-command (guix-import . args)
> > > (show-version-and-exit "guix import"))
> > > ((or ("-i" file importer args ...)
> > > ("--insert" file importer args ...))
> > > - (let ((find-and-insert
> > > + (let* ((definer?
> > > + (cut member
> > > + <>
> > > + `(,@(if (member importer '("crate"))
> > > + '(define)
> > > + '())
> > > + define-public)))
> >
> > This part above seems like it would break the option to use 'guix import
> > crate' with the --insert flag for "normal" packages. I question how
> > useful it currently is in that scenario since we normally have to strip
> > the rust- prefix from those packages anyway.
>
> Hmmm, since the crate importer now checks both ‘define’ and ‘define-public’,
> there might be ordering issue when the module to insert uses ‘define’. But it's
> still managable I think.
Reading it over again I think I was in need of more coffee. I thought
that the importer for cargo would only search for define, not for
define-public also.
As for needing to strip the rust- prefix, there's no change from our
current setup, so we can ignore that for now.
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:28:05 GMT) Full text and rfc822 format available.Message #101 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 01/17] build/cargo: Pass ‘--offline’ to cargo. Date: Tue, 18 Mar 2025 22:26:44 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build,check,package,install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.48.1
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:28:07 GMT) Full text and rfc822 format available.Message #104 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 00/17] New Rust packaging workflow based on lockfile importer. Date: Tue, 18 Mar 2025 22:26:43 +0800
V1 -> V2:
* Drop jemalloc environment variable patch.
* Remove use of RUSTC_BOOTSTRAP.
* Adjust documentation.
Diff inserted at end.
Efraim Flashner (1):
build/cargo: Don't try to unpack sanity-check.py.
Hilton Chain (16):
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
scripts: import: Document argument for ‘--insert’ option in help
message.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build-system: cargo: Add ‘cargo-inputs’.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
doc: Document lockfile importer based Rust packaging workflow.
Makefile.am | 1 +
doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
doc/guix.texi | 24 ++
etc/teams/rust/cleanup-crates.sh | 38 +++
etc/teams/rust/rust-crates.tmpl | 42 +++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 +++
gnu/packages/rust-sources.scm | 29 +++
guix/build-system/cargo.scm | 57 +++-
guix/build/cargo-build-system.scm | 151 ++++++++---
guix/import/crate.scm | 81 +++++-
guix/import/crate/cargo-lock.scm | 105 ++++++++
guix/scripts/import.scm | 53 ++--
guix/scripts/import/crate.scm | 58 ++++-
guix/utils.scm | 27 +-
tests/crate.scm | 88 +++++++
16 files changed, 1107 insertions(+), 106 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
--
2.48.1
--8<---------------cut here---------------start------------->8---
diff --git a/doc/contributing.texi b/doc/contributing.texi
index e7c5116a3d..837074dead 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1607,15 +1607,15 @@ Rust Crates
have switched to a different packaging model.
Rust programs (binary crates) and dependencies (library crates) are treated
-separately. We put main efforts into programs and only package Rust
-dependencies as sources, utilizing automation with manual focus on unbundling
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
vendored dependencies. The following paragraphs will explain them and give
several examples.
Rust programs are treated like any other package and named using the lowercase
upstream name. When using the Cargo build system (@pxref{Build Systems,
@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
-argument set to @code{#f}, as this argument only makes sense to dependencies.
+argument set to @code{#f}, as this argument only makes sense for dependencies.
When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
be set to enable relevant support.
@@ -1631,21 +1631,21 @@ Rust Crates
contributing to Guix. Naturally, this module serves as a store for both sources
and unbundling strategies.
-This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
-version containing all changes from other branches, so that the maintained
+This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
+one version containing all changes from other branches, so that the maintained
version can be used directly in case of merge conflicts, thus coordination is
required for other committers to modify it.
Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
-script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
+script @file{etc/teams/rust/cleanup-crates.sh} for this module.
@item
@code{(gnu packages rust-sources)}, storing more complex definitions that need
to be full packages. This includes Rust dependencies requiring external inputs
to unbundle and Cargo workspaces.
-These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
-For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
Since they are added manually, the following naming convention applies:
@@ -1668,7 +1668,7 @@ Rust Crates
Let's demonstrate the packaging workflow by examples, note that package-specific
issues are not involved here.
-In preparation, we'll adding the following packages to our environment:
+In preparation, we'll add the following packages to our environment:
@example
guix shell rust rust:cargo cargo-audit cargo-license
@@ -1747,7 +1747,7 @@ Rust Crates
@code{check-for-pregenerated-files} phase:
@example
-$ guix build cargo-audit
+$ ./pre-inst-env guix build cargo-audit
@dots{}
starting phase `check-for-pregenerated-files'
Searching for binary files...
diff --git a/doc/guix.texi b/doc/guix.texi
index 029c8cf59c..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9365,14 +9365,14 @@ Build Systems
sources, to ease in future hacking on rust packages.
This build system supports cargo workspaces. Parameter
-@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
-names of library crates to package in @code{package} phase. Specified
-crates are packaged from left to right, in case there's dependency among
-them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
-package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
-@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
-paths of binary crates to install in @code{install} phase,
-@code{''("crates/atuin")}, for example.
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
@@ -14705,19 +14705,18 @@ Invoking guix import
a comment.
@item --lockfile=@var{file}
@itemx -f @var{file}
-When @option{--lockfile} is specified, importer will ignore other
-options and won't output package definitions, instead generating source
-definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
-For example:
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
@example
guix import crate --lockfile=/path/to/Cargo.lock my-package
@end example
-generates sources from @file{/path/to/Cargo.lock}, and a list
-@code{my-package-cargo-inputs} referencing these sources. The generated
-list is intended for package's base @code{inputs}, replacing
-@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index b547421b88..41adc03752 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -248,12 +248,6 @@ (define* (configure #:key inputs
(setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
(setenv "SODIUM_USE_PKG_CONFIG" "1")
(setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
- ;; This flag is needed when not using the bundled jemalloc.
- ;; https://github.com/tikv/jemallocator/issues/19
- (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
- (when (assoc-ref inputs "jemalloc")
- (setenv "JEMALLOC_OVERRIDE"
- (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
(when (assoc-ref inputs "openssl")
(setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
(when (assoc-ref inputs "gettext")
@@ -356,23 +350,17 @@ (define* (package #:key
(if (null? cargo-package-crates)
(apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
- ;; Use unstable feature ‘--registry’.
(begin
- (setenv "RUSTC_BOOTSTRAP" "1")
(for-each
(lambda (pkg)
(apply invoke "cargo" "package" "--offline" "--package" pkg
- "--registry" "crates-io" "-Z" "package-workspace"
cargo-package-flags)
(for-each
(lambda (crate)
(invoke "tar" "xzf" crate "-C" vendor-dir))
- (begin
- (delete-file-recursively "target/package/tmp-registry")
- (find-files "target/package" "\\.crate$")))
+ (find-files "target/package" "\\.crate$"))
(patch-cargo-checksums #:vendor-dir vendor-dir))
- cargo-package-crates)
- (unsetenv "RUSTC_BOOTSTRAP")))
+ cargo-package-crates)))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index d8e1ed3e6f..f6e2985ed9 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -126,13 +126,12 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let* ((definer?
- (cut member
- <>
- `(,@(if (member importer '("crate"))
- '(define)
- '())
- define-public)))
+ (let* ((importer-definers
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (definer? (cut member <> importer-definers))
(find-and-insert
(lambda (expr)
(match expr
diff --git a/tests/crate.scm b/tests/crate.scm
index d46f753f9c..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -478,7 +478,7 @@ (define rust-leaf-bob-3.0.2-yanked
(license #f)))
(define temp-file
- (string-append "t-utils-" (number->string (getpid))))
+ (string-append "t-crate-" (number->string (getpid))))
(test-begin "crate")
--8<---------------cut here---------------end--------------->8---
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:28:10 GMT) Full text and rfc822 format available.Message #107 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 02/17] build/cargo: Print out all non-empty binary files. Date: Tue, 18 Mar 2025 22:26:45 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
'unpack-rust-crates after 'unpack.
Move 'check-for-pregenerated-files after 'configure.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 31 +++++++++++++++++++++++++------
1 file changed, 25 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..58238992b8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -29,9 +29,11 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build utils) #:hide (delete))
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +113,29 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (filename)
+ (call-with-ascii-input-file filename
+ (lambda (p)
+ (eqv? #\0 (read-char p)))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ (unless (empty-file? file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file))))
+ (find-files "."))))
(define* (configure #:key inputs
target system
@@ -380,8 +399,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:04 GMT) Full text and rfc822 format available.Message #110 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 05/17] build/cargo: Set default value of arguments for build phases. Date: Tue, 18 Mar 2025 22:26:48 +0800
This makes it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates,configure,build,package)
(install): Set default value of arguments.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 85f10a7ac4..85a9698680 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -76,7 +76,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -139,7 +140,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -258,7 +259,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -298,8 +299,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -375,9 +376,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:05 GMT) Full text and rfc822 format available.Message #113 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 03/17] build-system: cargo: Support packaging Cargo workspace. Date: Tue, 18 Mar 2025 22:26:46 +0800
* guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 22 ++++++++++++++++++++--
3 files changed, 31 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0ca109a214..0089f9e60f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9363,6 +9363,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..4f6d46e70c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -94,6 +94,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -122,6 +123,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -154,6 +156,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -185,6 +188,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 58238992b8..979f93aef4 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -300,11 +300,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -322,7 +327,20 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ (begin
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates)))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:05 GMT) Full text and rfc822 format available.Message #116 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 04/17] build-system: cargo: Support installing Cargo workspace. Date: Tue, 18 Mar 2025 22:26:47 +0800
*guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0089f9e60f..7fa986b4b9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9369,7 +9369,10 @@ Build Systems
library crates to package in the @code{package} phase. Specified crates are
packaged from left to right, in case there's dependency among them. For
example, specifying @code{''("pcre2-sys" "pcre2")} will package
-@code{"pcre2-sys"} first and then @code{"pcre2"}.
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4f6d46e70c..4486c706a1 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -96,6 +96,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -125,6 +126,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -190,6 +193,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 979f93aef4..85f10a7ac4 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -378,6 +378,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -392,10 +393,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:06 GMT) Full text and rfc822 format available.Message #119 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 06/17] build/cargo: Don't try to unpack sanity-check.py. Date: Tue, 18 Mar 2025 22:26:49 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 85a9698680..32a756ad99 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -64,6 +64,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:06 GMT) Full text and rfc822 format available.Message #122 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 07/17] build/cargo: Support non-workspace directory source inputs. Date: Tue, 18 Mar 2025 22:26:50 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 32a756ad99..41adc03752 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -33,6 +33,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -60,22 +61,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -159,14 +173,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:29:08 GMT) Full text and rfc822 format available.Message #125 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 08/17] scripts: import: Document argument for ‘--insert’ option in help message. Date: Tue, 18 Mar 2025 22:26:51 +0800
* guix/scripts/import.scm (show-help): Add missing ‘FILE’. Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d --- guix/scripts/import.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index bbf31baa15..4fad329cd3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -69,7 +69,7 @@ (define (show-help) (display (G_ " -h, --help display this help and exit")) (display (G_ " - -i, --insert insert packages into file alphabetically")) + -i, --insert=FILE insert packages into FILE alphabetically")) (display (G_ " -V, --version display version information and exit")) (newline) -- 2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:03 GMT) Full text and rfc822 format available.Message #128 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 09/17] scripts: import: Add two newlines for ‘--insert’ option. Date: Tue, 18 Mar 2025 22:26:52 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines when inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4fad329cd3..58a84d0db7 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -129,6 +129,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:05 GMT) Full text and rfc822 format available.Message #131 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 14/17] build-system: cargo: Add ‘cargo-inputs’. Date: Tue, 18 Mar 2025 22:26:57 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/cleanup-crates.sh | 38 +++++++++++++++++++++++++++
etc/teams/rust/rust-crates.tmpl | 42 +++++++++++++++++++++++++++++
gnu/local.mk | 2 ++
gnu/packages/rust-crates.scm | 42 +++++++++++++++++++++++++++++
gnu/packages/rust-sources.scm | 29 ++++++++++++++++++++
guix/build-system/cargo.scm | 45 +++++++++++++++++++++++++++++++-
guix/import/crate.scm | 2 +-
7 files changed, 198 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..cd4c2462b9
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,38 @@
+#!/bin/sh
+
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:definer 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/local.mk b/gnu/local.mk
index 02de02e65f..8d8bd6ebbe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..bf9b91a671
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust dependencies requiring external inputs to
+;;; unbundle.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..47d4f10969 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -31,6 +31,7 @@ (define-module (guix build-system cargo)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +46,9 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +65,46 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
+ (sources-module '(gnu packages rust-sources)))
+ "Given NAME, resolve input list 'NAME-cargo-inputs' in CRATES-MODULE, return
+its copy with #f removed and symbols resolved to variables defined in
+SOURCES-MODULE, if the list exists, otherwise return an empty list."
+ (let loop ((inputs
+ (catch #t
+ (lambda ()
+ (module-ref (resolve-interface crates-module)
+ (symbol-append name '-cargo-inputs)))
+ (const '())))
+ (result '()))
+ (if (null? inputs)
+ result
+ (match inputs
+ ((input . rest)
+ (loop rest
+ (cond
+ ;; #f, remove it.
+ ((not input)
+ result)
+ ;; Symbol, resolve it in SOURCES-MODULE.
+ ((symbol? input)
+ (cons (module-ref (resolve-interface sources-module) input)
+ result))
+ ;; Else: no change.
+ (else
+ (cons input result)))))))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:07 GMT) Full text and rfc822 format available.Message #134 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH rust-team v2 15/17] import: crate: Add Cargo.lock parser. Date: Tue, 18 Mar 2025 22:26:58 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* guix/import/crate.scm (cargo-lock->definitions): New procedure.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
3 files changed, 184 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 4091a52387..a04e461ea8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..ef3a6d0b4b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->definitions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->definitions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, convert its content into source
+definitions. Return a list of sources and 'PACKAGE-NAME-cargo-inputs', an input
+list referencing all imported sources."
+ (define (crate->definition crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Workspace member.
+ (else #f)))
+
+ (let ((definitions
+ (filter-map crate->definition
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all)))))
+ (values definitions
+ `(define-public
+ ,(string->symbol (string-append package-name "-cargo-inputs"))
+ (list ,@(map second definitions))))))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
--
2.48.1
guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:11 GMT) Full text and rfc822 format available.Message #137 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 16/17] import: crate: Add ‘--lockfile’ option. Date: Tue, 18 Mar 2025 22:26:59 +0800
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
* guix/scripts/import/crate.scm (show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 14 ++++++
guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 150 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 7fa986b4b9..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14703,6 +14703,20 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
+
+@example
+guix import crate --lockfile=/path/to/Cargo.lock my-package
+@end example
+
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..3b536e135a 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,44 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((crate-definitions input-list)
+ (cargo-lock->definitions lockfile name)))
+ (if file-to-insert
+ ;; Adjusted from ‘--insert’ option of guix-import.
+ (let ((term (second input-list)))
+ (begin
+ ;; Remove existing input list definition.
+ (and=> (find-definition-location file-to-insert term)
+ delete-expression)
+ ;; Insert input list alphabetically.
+ (or (and=> (find-definition-insertion-location
+ file-to-insert term)
+ (cut insert-expression <> input-list))
+ (let ((port (open-file file-to-insert "a")))
+ (newline port)
+ (pretty-print-with-comments port input-list)
+ (newline port)
+ (newline port)
+ (close-port port))))
+ crate-definitions)
+ `(,@crate-definitions
+ ,input-list))))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-crate-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((crates-definitions input-list)
+ (cargo-lock->definitions temp-file "test")))
+ (and
+ (match crates-definitions
+ (((define 'rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define 'rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define 'rust-smithay-0.4.0.0cd3345
+ ($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match input-list
+ ((define-public 'test-cargo-inputs
+ (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:15 GMT) Full text and rfc822 format available.Message #140 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 10/17] scripts: import: Support expressions defined by 'define. Date: Tue, 18 Mar 2025 22:26:53 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 19 ++++++++++++++-----
guix/utils.scm | 27 +++++++++++++++++++--------
2 files changed, 33 insertions(+), 13 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 58a84d0db7..e53028e747 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -117,13 +120,19 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((importer-definers
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (definer? (cut member <> importer-definers))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? definer? definer) term _ ...)
(let ((source-properties
(find-definition-insertion-location
- file term)))
+ file term #:definer definer)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index c7c23d9d5b..77ec6d992a 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (definer 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term comparing to TERM through PRED. Return the location if PRED returns #t,
+or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (definer? (cut eq? definer <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? definer?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key (definer 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINER with defined
+term alphabetically succeeds TERM. Return the location if found, or #f
+otherwise."
+ (find-definition-location file term #:definer definer #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:17 GMT) Full text and rfc822 format available.Message #143 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 11/17] scripts: import: Pass "--insert" to importers. Date: Tue, 18 Mar 2025 22:26:54 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index e53028e747..e99796a430 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -140,7 +146,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.48.1
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:20 GMT) Full text and rfc822 format available.Message #146 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 12/17] scripts: import: Skip existing definition for ‘--insert’ option. Date: Tue, 18 Mar 2025 22:26:55 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 20 +++++++++++---------
1 file changed, 11 insertions(+), 9 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index e99796a430..f6e2985ed9 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -136,16 +136,18 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? definer? definer) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
+ ;; Skip existing definition.
+ (unless (find-definition-location file term #:definer definer)
+ (let ((source-properties
+ (find-definition-insertion-location
file term #:definer definer)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.48.1
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:22 GMT) Full text and rfc822 format available.Message #149 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 13/17] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Tue, 18 Mar 2025 22:26:56 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4486c706a1..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.48.1
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 18 Mar 2025 14:30:24 GMT) Full text and rfc822 format available.Message #152 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space>, Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Tue, 18 Mar 2025 22:27:00 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
1 file changed, 390 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..837074dead 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,399 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+As currently there's no achievable way to reuse Rust packages as pre-compiled
+inputs for other packages at a distribution scale, and our previous approach on
+making all Rust dependencies as full packages doesn't cope well with Rust
+ecosystem thus causing issues on both contribution and maintenance sides, we
+have switched to a different packaging model.
+
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
+
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense for dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, crate, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+contributing to Guix. Naturally, this module serves as a store for both sources
+and unbundling strategies.
+
+This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
+one version containing all changes from other branches, so that the maintained
+version can be used directly in case of merge conflicts, thus coordination is
+required for other committers to modify it.
+
+Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
+script @file{etc/teams/rust/cleanup-crates.sh} for this module.
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix. Examples@:
+@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Let's demonstrate the packaging workflow by examples, note that package-specific
+issues are not involved here.
+
+In preparation, we'll add the following packages to our environment:
+
+@example
+guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
+repository @uref{https://crates.io, crates.io}.
+
+@enumerate
+@item
+We can generate a draft definition via the crates.io importer (@pxref{Invoking
+guix import, crate}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
+facilitating dependency modification. @code{'cargo-audit} used here must be a
+unique identifier, usually same to the program's variable name.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+cargo generate-lockfile
+cargo audit
+cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining Packages,
+@code{license}}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them in Guix command-line interface through expression:
+
+@example
+guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+point the imported definition to a symbol with the same name.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to point to it:
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+
+Such modifications are processed by @code{cargo-inputs} procedure.
+
+@code{cargo-inputs} can also be configured by keywoard arguments
+@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
+resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
+packages rust-sources)}, module to resolve modified symbols), which are useful
+to maintain an independent channel.
+@end enumerate
+
+Example 2: @code{niri}, which is not available on crates.io and depends on
+development snapshots (also Cargo workspaces in this example).
+
+@enumerate
+@item
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+in a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in build environment, the substitution pattern is
+package-specific.
+
+@code{cargo-inputs} returns a list, all list operations apply to it too.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3)))
+@end lisp
+
+@item
+@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
+workspace, build argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+@end enumerate
+
+Example 3: @code{fish}, which combines two build systems.
+
+@enumerate
+@item
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+@end enumerate
@node Elm Packages
--
2.48.1
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 06:53:03 GMT) Full text and rfc822 format available.Message #155 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: steve <at> futurile.net, 77093 <at> debbugs.gnu.org, divya <at> subvertising.org Subject: Re: [PATCH rust-team v2 00/17] New Rust packaging workflow based on lockfile importer. Date: Wed, 19 Mar 2025 08:52:19 +0200
[Message part 1 (text/plain, inline)]
+CC the other rust-team members, not sure how they got left out.
guix/import/crate/cargo-lock.scm should be added to etc/teams.scm.
I was thinking some more about the rust-crates.tmpl file and I think it
makes sense to keep it since it'll likely be useful for people who want
rust packages in their own channel or for working into a guix.scm file.
Some of the patches, specifically some of the importer ones, seem like
they could go straight to master.
On Tue, Mar 18, 2025 at 10:26:43PM +0800, Hilton Chain wrote:
> V1 -> V2:
> * Drop jemalloc environment variable patch.
> * Remove use of RUSTC_BOOTSTRAP.
> * Adjust documentation.
> Diff inserted at end.
>
> Efraim Flashner (1):
> build/cargo: Don't try to unpack sanity-check.py.
>
> Hilton Chain (16):
> build/cargo: Pass ‘--offline’ to cargo.
> build/cargo: Print out all non-empty binary files.
> build-system: cargo: Support packaging Cargo workspace.
> build-system: cargo: Support installing Cargo workspace.
> build/cargo: Set default value of arguments for build phases.
> build/cargo: Support non-workspace directory source inputs.
> scripts: import: Document argument for ‘--insert’ option in help
> message.
> scripts: import: Add two newlines for ‘--insert’ option.
> scripts: import: Support expressions defined by 'define.
> scripts: import: Pass "--insert" to importers.
> scripts: import: Skip existing definition for ‘--insert’ option.
> import: crate: crate-name->package-name: Move to (guix build-system
> cargo).
> build-system: cargo: Add ‘cargo-inputs’.
> import: crate: Add Cargo.lock parser.
> import: crate: Add ‘--lockfile’ option.
> doc: Document lockfile importer based Rust packaging workflow.
>
> Makefile.am | 1 +
> doc/contributing.texi | 415 ++++++++++++++++++++++++++++--
> doc/guix.texi | 24 ++
> etc/teams/rust/cleanup-crates.sh | 38 +++
> etc/teams/rust/rust-crates.tmpl | 42 +++
> gnu/local.mk | 2 +
> gnu/packages/rust-crates.scm | 42 +++
> gnu/packages/rust-sources.scm | 29 +++
> guix/build-system/cargo.scm | 57 +++-
> guix/build/cargo-build-system.scm | 151 ++++++++---
> guix/import/crate.scm | 81 +++++-
> guix/import/crate/cargo-lock.scm | 105 ++++++++
> guix/scripts/import.scm | 53 ++--
> guix/scripts/import/crate.scm | 58 ++++-
> guix/utils.scm | 27 +-
> tests/crate.scm | 88 +++++++
> 16 files changed, 1107 insertions(+), 106 deletions(-)
> create mode 100755 etc/teams/rust/cleanup-crates.sh
> create mode 100644 etc/teams/rust/rust-crates.tmpl
> create mode 100644 gnu/packages/rust-crates.scm
> create mode 100644 gnu/packages/rust-sources.scm
> create mode 100644 guix/import/crate/cargo-lock.scm
>
>
> base-commit: b54a9ca849f013300c633fb79d80bc754f6b28a2
> --
> 2.48.1
>
> --8<---------------cut here---------------start------------->8---
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index e7c5116a3d..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1607,15 +1607,15 @@ Rust Crates
> have switched to a different packaging model.
>
> Rust programs (binary crates) and dependencies (library crates) are treated
> -separately. We put main efforts into programs and only package Rust
> -dependencies as sources, utilizing automation with manual focus on unbundling
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> vendored dependencies. The following paragraphs will explain them and give
> several examples.
>
> Rust programs are treated like any other package and named using the lowercase
> upstream name. When using the Cargo build system (@pxref{Build Systems,
> @code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> -argument set to @code{#f}, as this argument only makes sense to dependencies.
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> be set to enable relevant support.
>
> @@ -1631,21 +1631,21 @@ Rust Crates
> contributing to Guix. Naturally, this module serves as a store for both sources
> and unbundling strategies.
>
> -This module is managed by Rust team (@pxref{Teams}) to ensure there's always one
> -version containing all changes from other branches, so that the maintained
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> version can be used directly in case of merge conflicts, thus coordination is
> required for other committers to modify it.
>
> Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> -script @file{etc/teams/rust/cleanup-crates.sh} for this moudle.
> +script @file{etc/teams/rust/cleanup-crates.sh} for this module.
>
> @item
> @code{(gnu packages rust-sources)}, storing more complex definitions that need
> to be full packages. This includes Rust dependencies requiring external inputs
> to unbundle and Cargo workspaces.
>
> -These dependencies should have @code{#:skip-build?} argument set to @code{#t}.
> -For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
>
> Since they are added manually, the following naming convention applies:
>
> @@ -1668,7 +1668,7 @@ Rust Crates
> Let's demonstrate the packaging workflow by examples, note that package-specific
> issues are not involved here.
>
> -In preparation, we'll adding the following packages to our environment:
> +In preparation, we'll add the following packages to our environment:
>
> @example
> guix shell rust rust:cargo cargo-audit cargo-license
> @@ -1747,7 +1747,7 @@ Rust Crates
> @code{check-for-pregenerated-files} phase:
>
> @example
> -$ guix build cargo-audit
> +$ ./pre-inst-env guix build cargo-audit
> @dots{}
> starting phase `check-for-pregenerated-files'
> Searching for binary files...
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 029c8cf59c..218c2ba630 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -9365,14 +9365,14 @@ Build Systems
> sources, to ease in future hacking on rust packages.
>
> This build system supports cargo workspaces. Parameter
> -@code{#:cargo-package-crates} (default: @code{''()}) allows specifying
> -names of library crates to package in @code{package} phase. Specified
> -crates are packaged from left to right, in case there's dependency among
> -them. For example, specifying @code{''("pcre2-sys" "pcre2")} will
> -package @code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> -@code{#:cargo-install-paths} (default: @code{''()}) allows specifying
> -paths of binary crates to install in @code{install} phase,
> -@code{''("crates/atuin")}, for example.
> +@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
> +library crates to package in the @code{package} phase. Specified crates are
> +packaged from left to right, in case there's dependency among them. For
> +example, specifying @code{''("pcre2-sys" "pcre2")} will package
> +@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
> +@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
> +binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
> +for example.
> @end defvar
>
> @defvar chicken-build-system
> @@ -14705,19 +14705,18 @@ Invoking guix import
> a comment.
> @item --lockfile=@var{file}
> @itemx -f @var{file}
> -When @option{--lockfile} is specified, importer will ignore other
> -options and won't output package definitions, instead generating source
> -definition for all dependencies in @var{file}, a @file{Cargo.lock} file.
> -For example:
> +When @option{--lockfile} is specified, the importer will ignore other options
> +and won't output package definitions, instead generating source definitions for
> +all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
>
> @example
> guix import crate --lockfile=/path/to/Cargo.lock my-package
> @end example
>
> -generates sources from @file{/path/to/Cargo.lock}, and a list
> -@code{my-package-cargo-inputs} referencing these sources. The generated
> -list is intended for package's base @code{inputs}, replacing
> -@code{#:cargo-inputs} and @code{#:cargo-development-inputs}.
> +generates sources from @file{/path/to/Cargo.lock} and a list
> +@code{my-package-cargo-inputs} referencing these sources. The generated list is
> +intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
> +@code{#:cargo-development-inputs}.
> @end table
>
> @item elm
> diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
> index b547421b88..41adc03752 100644
> --- a/guix/build/cargo-build-system.scm
> +++ b/guix/build/cargo-build-system.scm
> @@ -248,12 +248,6 @@ (define* (configure #:key inputs
> (setenv "LIBSSH2_SYS_USE_PKG_CONFIG" "1")
> (setenv "SODIUM_USE_PKG_CONFIG" "1")
> (setenv "ZSTD_SYS_USE_PKG_CONFIG" "1")
> - ;; This flag is needed when not using the bundled jemalloc.
> - ;; https://github.com/tikv/jemallocator/issues/19
> - (setenv "CARGO_FEATURE_UNPREFIXED_MALLOC_ON_SUPPORTED_PLATFORMS" "1")
> - (when (assoc-ref inputs "jemalloc")
> - (setenv "JEMALLOC_OVERRIDE"
> - (string-append (assoc-ref inputs "jemalloc") "/lib/libjemalloc.so")))
> (when (assoc-ref inputs "openssl")
> (setenv "OPENSSL_DIR" (assoc-ref inputs "openssl")))
> (when (assoc-ref inputs "gettext")
> @@ -356,23 +350,17 @@ (define* (package #:key
>
> (if (null? cargo-package-crates)
> (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
> - ;; Use unstable feature ‘--registry’.
> (begin
> - (setenv "RUSTC_BOOTSTRAP" "1")
> (for-each
> (lambda (pkg)
> (apply invoke "cargo" "package" "--offline" "--package" pkg
> - "--registry" "crates-io" "-Z" "package-workspace"
> cargo-package-flags)
> (for-each
> (lambda (crate)
> (invoke "tar" "xzf" crate "-C" vendor-dir))
> - (begin
> - (delete-file-recursively "target/package/tmp-registry")
> - (find-files "target/package" "\\.crate$")))
> + (find-files "target/package" "\\.crate$"))
> (patch-cargo-checksums #:vendor-dir vendor-dir))
> - cargo-package-crates)
> - (unsetenv "RUSTC_BOOTSTRAP")))
> + cargo-package-crates)))
>
> ;; Then unpack the crate, reset the timestamp of all contained files, and
> ;; repack them. This is necessary to ensure that they are reproducible.
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index d8e1ed3e6f..f6e2985ed9 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -126,13 +126,12 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let* ((definer?
> - (cut member
> - <>
> - `(,@(if (member importer '("crate"))
> - '(define)
> - '())
> - define-public)))
> + (let* ((importer-definers
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public))
> + (definer? (cut member <> importer-definers))
> (find-and-insert
> (lambda (expr)
> (match expr
> diff --git a/tests/crate.scm b/tests/crate.scm
> index d46f753f9c..9b7066c3b1 100644
> --- a/tests/crate.scm
> +++ b/tests/crate.scm
> @@ -478,7 +478,7 @@ (define rust-leaf-bob-3.0.2-yanked
> (license #f)))
>
> (define temp-file
> - (string-append "t-utils-" (number->string (getpid))))
> + (string-append "t-crate-" (number->string (getpid))))
>
>
> (test-begin "crate")
> --8<---------------cut here---------------end--------------->8---
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 06:54:07 GMT) Full text and rfc822 format available.Message #158 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Wed, 19 Mar 2025 08:52:32 +0200
[Message part 1 (text/plain, inline)]
On Tue, Mar 18, 2025 at 10:27:00PM +0800, Hilton Chain wrote:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
> doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
> @subsection Rust Crates
>
> @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
> +
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> +vendored dependencies. The following paragraphs will explain them and give
> +several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name. When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix. The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix. Naturally, this module serves as a store for both sources
> +and unbundling strategies.
> +
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
> +
> +Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
> +script @file{etc/teams/rust/cleanup-crates.sh} for this module.
> +
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages. This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix. The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>
> In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix. The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course). This follows the ``caret'' version scheme
> -intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments. It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}. If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible. Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix. The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course). This follows the ``caret'' version scheme intended
> +by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> +
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix. Examples@:
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
> +
> +In preparation, we'll add the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
I was able to run cargo-audit as 'cargo-audit audit ...' but I wouldn't
actually recommend that to anyone. I'm not sure we actually need rust
in the shell but I don't have a strong preference. More thinking aloud,
perhaps we should adjust rust:cargo to automatically pull in rust:out?
I also find it amusing to suggest using cargo-audit when it's not (yet)
in guix and we immediately follow up with importing it.
I could see an argument for putting this part in the cookbook instead of
here but I feel strongly that it should be here.
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
> +
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}). In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> + (package
> + (name "cargo-audit")
> + (version "0.21.2")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (crate-uri "cargo-audit" version))
> + (file-name (string-append name "-" version ".tar.gz"))
> + (sha256
> + (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> + (build-system cargo-build-system)
> + (arguments (list #:install-source? #f))
> + (inputs (cargo-inputs 'cargo-audit))
> + (home-page "https://rustsec.org/")
> + (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> + (description
> + "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> + (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification. @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> + crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources. The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ ./pre-inst-env guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> + (crate-source "bzip2-sys" "0.1.13+1.0.8"
> + "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> + #:snippet
> + '(begin
> + (delete-file-recursively "bzip2-1.0.8")
> + (delete-file "build.rs")
> + ;; Inspired by Debian's patch.
> + (with-output-to-file "build.rs"
> + (lambda _
> + (format #t "fn main() @{~@@
> + println!(\"cargo:rustc-link-lib=bz2\");~@@
> + @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
> +
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
> +
> +@lisp
> +(define-public niri
> + (package
> + (name "niri")
> + (version "25.02")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/YaLTeR/niri")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:install-source? #f
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("# version =.*")
> + "version = \"*\"")
> + (("git.*optional")
> + "version = \"*\", optional")
> + (("^git = .*")
> + "")))))))
> + (native-inputs
> + (list pkg-config))
> + (inputs
> + (cons* clang
> + libdisplay-info
> + libinput-minimal
> + libseat
> + libxkbcommon
> + mesa
> + pango
> + pipewire
> + wayland
> + (cargo-inputs 'niri)))
> + (home-page "https://github.com/YaLTeR/niri")
> + (synopsis "Scrollable-tiling Wayland compositor")
> + (description
> + "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format. It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> + (license license:gpl3)))
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
> +
> +@lisp
> +(define-public rust-pipewire-for-niri
> + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> + (revision "0"))
> + (package
> + (name "rust-pipewire")
> + (version (git-version "0.8.0" revision commit))
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:skip-build? #t
> + #:cargo-package-crates
> + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> + (inputs (cargo-inputs 'rust-pipewire-for-niri))
should these be inputs or propagated-inputs? in rust-ring-0.17 in
rust-sources they are propagated. I would think inputs would be enough
since the Cargo.lock file will let us know which other crates are
necessary for actually building it as a dependency of that package.
> + (home-page "https://pipewire.org/")
> + (synopsis "Rust bindings for PipeWire")
> + (description "This package provides Rust bindings for PipeWire.")
> + (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> + (package
> + (name "fish")
> + (version "4.0.0")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://github.com/fish-shell/fish-shell/"
> + "releases/download/" version "/"
> + "fish-" version ".tar.xz"))
> + (sha256
> + (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> + (build-system cmake-build-system)
> + (inputs
> + (cons* fish-foreign-env ncurses pcre2
> + python ;for fish_config and manpage completions
> + (cargo-inputs 'fish)))
> + (native-inputs
> + (list doxygen groff ;for 'fish --help'
> + pkg-config
> + procps ;for the test suite
> + rust
> + `(,rust "cargo")))
> + (arguments
> + (list #:imported-modules
> + (append %cargo-build-system-modules
> + %cmake-build-system-modules)
> + #:modules
> + (((guix build cargo-build-system) #:prefix cargo:)
> + (guix build cmake-build-system)
> + (guix build utils))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("git.*tag.*,")
> + "version = \"*\","))))
> + (add-after 'unpack 'prepare-cargo-build-system
> + (lambda args
> + (for-each
> + (lambda (phase)
> + (format #t "Running cargo phase: ~a~%" phase)
> + (apply (assoc-ref cargo:%standard-phases phase)
> + args))
> + '(unpack-rust-crates
> + configure
> + check-for-pregenerated-files
> + patch-cargo-checksums)))))))
> + (synopsis "The friendly interactive shell")
> + (description
> + "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness. Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands. It also
> +has extensive and discoverable help. A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser. Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> + (home-page "https://fishshell.com/")
> + (license license:gpl2)))
> +@end lisp
> +@end enumerate
>
>
> @node Elm Packages
> --
> 2.48.1
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 06:54:11 GMT) Full text and rfc822 format available.Message #161 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Efraim Flashner <efraim <at> flashner.co.il> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team v2 10/17] scripts: import: Support expressions defined by 'define. Date: Wed, 19 Mar 2025 08:52:40 +0200
[Message part 1 (text/plain, inline)]
How about 'define-prefix' instead of 'definer'? It seems more
descriptive.
On Tue, Mar 18, 2025 at 10:26:53PM +0800, Hilton Chain wrote:
> * guix/utils.scm (find-definition-location): New procedure.
> (find-definition-insertion-location): Define with it.
> * guix/scripts/import.scm (import-as-definitions, guix-import): Support
> expressions defined by 'define.
>
> Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
> ---
> guix/scripts/import.scm | 19 ++++++++++++++-----
> guix/utils.scm | 27 +++++++++++++++++++--------
> 2 files changed, 33 insertions(+), 13 deletions(-)
>
> diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
> index 58a84d0db7..e53028e747 100644
> --- a/guix/scripts/import.scm
> +++ b/guix/scripts/import.scm
> @@ -30,6 +30,7 @@ (define-module (guix scripts import)
> #:use-module (guix read-print)
> #:use-module (guix utils)
> #:use-module (srfi srfi-1)
> + #:use-module (srfi srfi-26)
> #:use-module (ice-9 format)
> #:use-module (ice-9 match)
> #:export (%standard-import-options
> @@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))
> ((expressions ...)
> (for-each (lambda (expr)
> @@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
> ((and expr (or ('package _ ...)
> ('let _ ...)))
> (proc (package->definition expr)))
> - ((and expr ('define-public _ ...))
> + ((and expr (or ('define-public _ ...)
> + ('define _ ...)))
> (proc expr))))
> expressions))
> (x
> @@ -117,13 +120,19 @@ (define-command (guix-import . args)
> (show-version-and-exit "guix import"))
> ((or ("-i" file importer args ...)
> ("--insert" file importer args ...))
> - (let ((find-and-insert
> + (let* ((importer-definers
> + `(,@(if (member importer '("crate"))
> + '(define)
> + '())
> + define-public))
> + (definer? (cut member <> importer-definers))
> + (find-and-insert
> (lambda (expr)
> (match expr
> - (('define-public term _ ...)
> + (((? definer? definer) term _ ...)
> (let ((source-properties
> (find-definition-insertion-location
> - file term)))
> + file term #:definer definer)))
> (if source-properties
> (insert-expression source-properties expr)
> (let ((port (open-file file "a")))
> diff --git a/guix/utils.scm b/guix/utils.scm
> index c7c23d9d5b..77ec6d992a 100644
> --- a/guix/utils.scm
> +++ b/guix/utils.scm
> @@ -154,6 +154,7 @@ (define-module (guix utils)
> edit-expression
> delete-expression
> insert-expression
> + find-definition-location
> find-definition-insertion-location
>
> filtered-port
> @@ -520,24 +521,34 @@ (define (insert-expression source-properties expr)
> (string-append expr "\n\n" str))))
> (edit-expression source-properties insert)))
>
> -(define (find-definition-insertion-location file term)
> - "Search in FILE for a top-level public definition whose defined term
> -alphabetically succeeds TERM. Return the location if found, or #f
> -otherwise."
> - (let ((search-term (symbol->string term)))
> +(define* (find-definition-location file term
> + #:key (definer 'define-public)
> + (pred string=))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term comparing to TERM through PRED. Return the location if PRED returns #t,
> +or #f otherwise."
> + (let ((search-term (symbol->string term))
> + (definer? (cut eq? definer <>)))
> (call-with-input-file file
> (lambda (port)
> (do ((syntax (read-syntax port)
> (read-syntax port)))
> ((match (syntax->datum syntax)
> - (('define-public current-term _ ...)
> - (string> (symbol->string current-term)
> - search-term))
> + (((? definer?) current-term _ ...)
> + (pred (symbol->string current-term)
> + search-term))
> ((? eof-object?) #t)
> (_ #f))
> (and (not (eof-object? syntax))
> (syntax-source syntax))))))))
>
> +(define* (find-definition-insertion-location file term
> + #:key (definer 'define-public))
> + "Search in FILE for a top-level definition defined by DEFINER with defined
> +term alphabetically succeeds TERM. Return the location if found, or #f
> +otherwise."
> + (find-definition-location file term #:definer definer #:pred string>))
> +
>
> ;;;
> ;;; Keyword arguments.
> --
> 2.48.1
>
--
Efraim Flashner <efraim <at> flashner.co.il> אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D 14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted
[signature.asc (application/pgp-signature, inline)]
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 12:44:02 GMT) Full text and rfc822 format available.Message #164 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Efraim Flashner <efraim <at> flashner.co.il> Cc: 77093 <at> debbugs.gnu.org Subject: Re: [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Wed, 19 Mar 2025 19:34:46 +0800
On Wed, 19 Mar 2025 14:52:32 +0800,
Efraim Flashner wrote:
>
> > +Let's demonstrate the packaging workflow by examples, note that package-specific
> > +issues are not involved here.
> > +
> > +In preparation, we'll add the following packages to our environment:
> > +
> > +@example
> > +guix shell rust rust:cargo cargo-audit cargo-license
> > +@end example
>
> I was able to run cargo-audit as 'cargo-audit audit ...' but I wouldn't
> actually recommend that to anyone. I'm not sure we actually need rust
> in the shell but I don't have a strong preference. More thinking aloud,
‘cargo generate-lockfile’ sometimes requires rust.
> perhaps we should adjust rust:cargo to automatically pull in rust:out?
Sounds reasonable.
> I also find it amusing to suggest using cargo-audit when it's not (yet)
> in guix and we immediately follow up with importing it.
XD yes, it's funny to use cargo-audit audit cargo-audit, and cargo-license to
check license of cargo-license.
> I could see an argument for putting this part in the cookbook instead of
> here but I feel strongly that it should be here.
> > [...]
> > +@item
> > +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> > +workspace, build argument @code{#:cargo-package-crates} is required.
> > +
> > +@lisp
> > +(define-public rust-pipewire-for-niri
> > + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> > + (revision "0"))
> > + (package
> > + (name "rust-pipewire")
> > + (version (git-version "0.8.0" revision commit))
> > + (source
> > + (origin
> > + (method git-fetch)
> > + (uri (git-reference
> > + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> > + (commit commit)))
> > + (file-name (git-file-name name version))
> > + (sha256
> > + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> > + (build-system cargo-build-system)
> > + (arguments
> > + (list #:skip-build? #t
> > + #:cargo-package-crates
> > + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> > + (inputs (cargo-inputs 'rust-pipewire-for-niri))
>
> should these be inputs or propagated-inputs? in rust-ring-0.17 in
> rust-sources they are propagated. I would think inputs would be enough
> since the Cargo.lock file will let us know which other crates are
> necessary for actually building it as a dependency of that package.
For rust-ring-0.17 the propagation is necessary, I think this is because our
packaged version is a bit older than the one used in generated lockfile.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Wed, 19 Mar 2025 13:43:01 GMT) Full text and rfc822 format available.Message #167 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> To: Hilton Chain <hako <at> ultrarare.space> Cc: Ludovic Courtès <ludo <at> gnu.org>, 77093 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [bug#77093] [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Wed, 19 Mar 2025 22:41:39 +0900
Hi,
Hilton Chain <hako <at> ultrarare.space> writes:
> * doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
> documentation.
>
> Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
> ---
> doc/contributing.texi | 415 +++++++++++++++++++++++++++++++++++++++---
> 1 file changed, 390 insertions(+), 25 deletions(-)
>
> diff --git a/doc/contributing.texi b/doc/contributing.texi
> index ab4f30d54b..837074dead 100644
> --- a/doc/contributing.texi
> +++ b/doc/contributing.texi
> @@ -1600,34 +1600,399 @@ Rust Crates
> @subsection Rust Crates
>
> @cindex rust
> -Rust programs standing for themselves are named as any other package, using the
> -lowercase upstream name.
> +As currently there's no achievable way to reuse Rust packages as pre-compiled
> +inputs for other packages at a distribution scale, and our previous approach on
> +making all Rust dependencies as full packages doesn't cope well with Rust
> +ecosystem thus causing issues on both contribution and maintenance sides, we
> +have switched to a different packaging model.
I'd drop the above paragraph; it won't age well.
> +Rust programs (binary crates) and dependencies (library crates) are treated
> +separately. We put our main efforts into programs and only package Rust
> +dependencies as sources, utilizing automation with a manual focus on unbundling
> +vendored dependencies. The following paragraphs will explain them and give
> +several examples.
> +
> +Rust programs are treated like any other package and named using the lowercase
> +upstream name. When using the Cargo build system (@pxref{Build Systems,
> +@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
> +argument set to @code{#f}, as this argument only makes sense for dependencies.
> +When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
> +be set to enable relevant support.
> +
> +Rust dependencies are managed in two modules:
>
> -To prevent namespace collisions we prefix all other Rust packages with the
> -@code{rust-} prefix. The name should be changed to lowercase as appropriate and
> -dashes should remain in place.
> +@enumerate
> +@item
> +@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
> +programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
> +import, crate, @code{--lockfile=@var{file}}}).
> +
> +Imported definitions must be checked and have vendored sources unbundled before
> +contributing to Guix. Naturally, this module serves as a store for both sources
s/contributing/being contributed/. The next sentence (Naturally, ...)
seems superfluous to me.
> +This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
> +one version containing all changes from other branches, so that the maintained
> +version can be used directly in case of merge conflicts, thus coordination is
> +required for other committers to modify it.
I think the above could be simplified like: This module is managed by
the Rust team, via the @file{etc/teams/rust/rust-crates.tmpl} template
file and @file{etc/teams/rust/rust-crates.tmpl} cleanup script.
[...]
> +@item
> +@code{(gnu packages rust-sources)}, storing more complex definitions that need
> +to be full packages. This includes Rust dependencies requiring external inputs
> +to unbundle and Cargo workspaces.
> +
> +These dependencies should have the @code{#:skip-build?} argument set to
> +@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
> +
> +Since they are added manually, the following naming convention applies:
> +
> +To prevent namespace collisions they are named with @code{rust-} prefix. The
> +name should be changed to lowercase as appropriate and dashes should remain in
> +place.
>
> In the rust ecosystem it is common for multiple incompatible versions of a
> -package to be used at any given time, so all package definitions should have a
> -versioned suffix. The versioned suffix is the left-most non-zero digit (and
> -any leading zeros, of course). This follows the ``caret'' version scheme
> -intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
> -
> -Because of the difficulty in reusing rust packages as pre-compiled inputs for
> -other packages the Cargo build system (@pxref{Build Systems,
> -@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
> -@code{cargo-development-inputs} keywords as build system arguments. It would be
> -helpful to think of these as similar to @code{propagated-inputs} and
> -@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
> -should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
> -@code{#:cargo-development-inputs}. If a Rust package links to other libraries
> -then the standard placement in @code{inputs} and the like should be used.
> -
> -Care should be taken to ensure the correct version of dependencies are used; to
> -this end we try to refrain from skipping the tests or using @code{#:skip-build?}
> -when possible. Of course this is not always possible, as the package may be
> -developed for a different Operating System, depend on features from the Nightly
> -Rust compiler, or the test suite may have atrophied since it was released.
> +package to be used at any given time, so all dependencies should have a
> +versioned suffix. The versioned suffix is the left-most non-zero digit (and any
> +leading zeros, of course). This follows the ``caret'' version scheme intended
> +by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
@: is not needed before a colon. Refer to (info "(texinfo) Not Ending a Sentence").
> +In practice we are usually packaging development snapshots of Rust dependencies
> +specifically for some Rust programs, and can't simply identity them by version.
> +In this case we can use a @code{for-@var{program}} suffix. Examples@:
I'd join the last sentence with the previous one, e.g. [...] suffix, for
example: [...]
> +@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
> +@end enumerate
> +
> +Let's demonstrate the packaging workflow by examples, note that package-specific
> +issues are not involved here.
I'd drop the part after examples.
> +In preparation, we'll add the following packages to our environment:
> +
> +@example
> +guix shell rust rust:cargo cargo-audit cargo-license
> +@end example
> +
> +Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
> +repository @uref{https://crates.io, crates.io}.
I'd replace the textual crates.io with the later @uref, end place the
period after 'repository'.
This verbose how-to Rust packaging, while useful, would better belong in
the cookbook, in my opinion. The GNU Guix Cookbook document section can
then be referenced from here, for example:
@ref{Rust Packaging Examples,,,guix-cookbook,GNU Guix Cookbook} for real
life examples of packaging Rust applications.
> +@enumerate
> +@item
> +We can generate a draft definition via the crates.io importer (@pxref{Invoking
> +guix import, crate}). In the end we'll have the following definiton:
> +
> +@lisp
> +(define-public cargo-audit
> + (package
> + (name "cargo-audit")
> + (version "0.21.2")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (crate-uri "cargo-audit" version))
> + (file-name (string-append name "-" version ".tar.gz"))
> + (sha256
> + (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
> + (build-system cargo-build-system)
> + (arguments (list #:install-source? #f))
> + (inputs (cargo-inputs 'cargo-audit))
> + (home-page "https://rustsec.org/")
> + (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
> + (description
> + "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
> +audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
> + (license (list license:asl2.0 license:expat))))
> +@end lisp
> +
> +@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
> +facilitating dependency modification. @code{'cargo-audit} used here must be a
> +unique identifier, usually same to the program's variable name.
I'd reword to: [...], usually matching the variable name of the package.
> +
> +@item
> +Unpack package source and navigate to the unpacked directory, then execute the
> +following commands:
> +
> +@example
> +cargo generate-lockfile
> +cargo audit
> +cargo license
> +@end example
> +
> +@command{cargo generate-lockfile} updates dependencies to compatible versions,
> +@command{cargo audit} checks known vulnerabilities and @command{cargo license}
> +checks licenses of all dependencies.
> +
> +We must have an acceptable output of @command{cargo audit} and ensure all
> +dependencies are licensed with our supported licenses (@pxref{Defining Packages,
> +@code{license}}).
> +
> +@item
> +Import dependencies from previously generated lockfile:
> +
> +@example
> +guix import --insert=gnu/packages/rust-crates.scm \
> + crate --lockfile=/path/to/Cargo.lock cargo-audit
> +@end example
> +
> +@code{cargo-audit} used here must be consistent with the identifier used for
> +@code{cargo-inputs}.
> +
> +At this stage, @code{cargo-audit} is buildable.
> +
> +@item
> +Finally we'll unbundle vendored sources. The lockfile importer inserts
> +@code{TODO:} comments to dependencies with high probability of bundled sources.
> +@code{cargo-build-system} also performs additional check in its
> +@code{check-for-pregenerated-files} phase:
> +
> +@example
> +$ ./pre-inst-env guix build cargo-audit
> +@dots{}
> +starting phase `check-for-pregenerated-files'
> +Searching for binary files...
> +./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
> +./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
> +./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
> +@dots{}
> +@end example
> +
> +Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
> +can still select them in Guix command-line interface through expression:
select them *via* the Guix command-line interface [...]
> +
> +@example
> +guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
> +@end example
> +
> +For most dependencies, a snippet is sufficient:
> +
> +@lisp
> +(define rust-bzip2-sys-0.1.13+1.0.8
> + (crate-source "bzip2-sys" "0.1.13+1.0.8"
> + "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
> + #:snippet
> + '(begin
> + (delete-file-recursively "bzip2-1.0.8")
> + (delete-file "build.rs")
> + ;; Inspired by Debian's patch.
> + (with-output-to-file "build.rs"
> + (lambda _
> + (format #t "fn main() @{~@@
> + println!(\"cargo:rustc-link-lib=bz2\");~@@
> + @}~%"))))))
> +@end lisp
> +
> +In a more complex case, where unbundling one dependency requires other packages,
> +we should package the dependency in @code{(gnu packages rust-sources)} first and
> +point the imported definition to a symbol with the same name.
> +
> +For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
> +rust-sources)}, then the imported definition in @code{(gnu packages
> +rust-crates)} should be modified to point to it:
> +
> +@lisp
> +(define rust-ring-0.17.14 'rust-ring-0.17)
> +@end lisp
> +
> +When one dependency can be safely removed, modify it to @code{#f}.
> +
> +@lisp
> +(define rust-openssl-src-300.4.2+3.4.1 #f)
> +@end lisp
> +
> +Such modifications are processed by @code{cargo-inputs} procedure.
> +
> +@code{cargo-inputs} can also be configured by keywoard arguments
> +@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
> +resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
> +packages rust-sources)}, module to resolve modified symbols), which are useful
> +to maintain an independent channel.
> +@end enumerate
> +
> +Example 2: @code{niri}, which is not available on crates.io and depends on
It seems to me these examples should have their own structure node
(subsection or subsubsection, as appropriate).
> +development snapshots (also Cargo workspaces in this example).
> +
> +@enumerate
> +@item
> +As we can't ensure compatibility of a development snapshot, before executing
> +@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
> +in a known working revision.
> +
> +To use our packaged development snapshots, it's also necessary to modify
> +@file{Cargo.toml} in build environment, the substitution pattern is
> +package-specific.
The above sentence appears incomplete or disconnected. Perhaps,
To use our packaged development snapshots, it's also necessary to modify
@file{Cargo.toml} in a phase, with a package-specific substitution
pattern.
> +@code{cargo-inputs} returns a list, all list operations apply to it too.
This appears to be out of context?
> +
> +@lisp
> +(define-public niri
> + (package
> + (name "niri")
> + (version "25.02")
> + (source (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://github.com/YaLTeR/niri")
> + (commit (string-append "v" version))))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32
> + "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:install-source? #f
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("# version =.*")
> + "version = \"*\"")
> + (("git.*optional")
> + "version = \"*\", optional")
> + (("^git = .*")
> + "")))))))
> + (native-inputs
> + (list pkg-config))
> + (inputs
> + (cons* clang
> + libdisplay-info
> + libinput-minimal
> + libseat
> + libxkbcommon
> + mesa
> + pango
> + pipewire
> + wayland
> + (cargo-inputs 'niri)))
> + (home-page "https://github.com/YaLTeR/niri")
> + (synopsis "Scrollable-tiling Wayland compositor")
> + (description
> + "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
> +scrollable format. It is considered stable for daily use and performs most
> +functions expected of a Wayland compositor.")
> + (license license:gpl3)))
I think this should be gpl3+, as the license file carries the original
'or any later version' and apparently no further directions to limit
gpl3 only.
> +@end lisp
> +
> +@item
> +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> +workspace, build argument @code{#:cargo-package-crates} is required.
Hm. Does that mean we're still in a situation where some inputs are not
true Guix inputs, and some Rust-packaging oddity that our tooling
doesn't handle (e.g. 'guix graph' ?).
> +@lisp
> +(define-public rust-pipewire-for-niri
> + (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
> + (revision "0"))
> + (package
> + (name "rust-pipewire")
> + (version (git-version "0.8.0" revision commit))
> + (source
> + (origin
> + (method git-fetch)
> + (uri (git-reference
> + (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
> + (commit commit)))
> + (file-name (git-file-name name version))
> + (sha256
> + (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
> + (build-system cargo-build-system)
> + (arguments
> + (list #:skip-build? #t
> + #:cargo-package-crates
> + ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
> + (inputs (cargo-inputs 'rust-pipewire-for-niri))
> + (home-page "https://pipewire.org/")
> + (synopsis "Rust bindings for PipeWire")
> + (description "This package provides Rust bindings for PipeWire.")
> + (license license:expat))))
> +@end lisp
> +
> +Don't forget to modify all workspace members in @code{(gnu packages
> +rust-crates)}:
> +
> +@lisp
> +(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@dots{}
> +(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
> +@end lisp
> +@end enumerate
> +
> +Example 3: @code{fish}, which combines two build systems.
> +
> +@enumerate
> +@item
> +When building Rust packages in other build systems, we need to add @code{rust}
> +and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
> +build systems, and apply necessary build phases from @code{cargo-build-system}.
> +
> +@lisp
> +(define-public fish
> + (package
> + (name "fish")
> + (version "4.0.0")
> + (source
> + (origin
> + (method url-fetch)
> + (uri (string-append "https://github.com/fish-shell/fish-shell/"
> + "releases/download/" version "/"
> + "fish-" version ".tar.xz"))
> + (sha256
> + (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
> + (build-system cmake-build-system)
> + (inputs
> + (cons* fish-foreign-env ncurses pcre2
> + python ;for fish_config and manpage completions
> + (cargo-inputs 'fish)))
> + (native-inputs
> + (list doxygen groff ;for 'fish --help'
> + pkg-config
> + procps ;for the test suite
> + rust
> + `(,rust "cargo")))
> + (arguments
> + (list #:imported-modules
> + (append %cargo-build-system-modules
> + %cmake-build-system-modules)
> + #:modules
> + (((guix build cargo-build-system) #:prefix cargo:)
> + (guix build cmake-build-system)
> + (guix build utils))
> + #:phases
> + #~(modify-phases %standard-phases
> + (add-after 'unpack 'use-guix-vendored-dependencies
> + (lambda _
> + (substitute* "Cargo.toml"
> + (("git.*tag.*,")
> + "version = \"*\","))))
> + (add-after 'unpack 'prepare-cargo-build-system
> + (lambda args
> + (for-each
> + (lambda (phase)
> + (format #t "Running cargo phase: ~a~%" phase)
> + (apply (assoc-ref cargo:%standard-phases phase)
> + args))
> + '(unpack-rust-crates
> + configure
> + check-for-pregenerated-files
> + patch-cargo-checksums)))))))
> + (synopsis "The friendly interactive shell")
> + (description
> + "Fish (friendly interactive shell) is a shell focused on interactive use,
> +discoverability, and friendliness. Fish has very user-friendly and powerful
> +tab-completion, including descriptions of every completion, completion of
> +strings with wildcards, and many completions for specific commands. It also
> +has extensive and discoverable help. A special @@command@{help@} command gives
> +access to all the fish documentation in your web browser. Other features
> +include smart terminal handling based on terminfo, an easy to search history,
> +and syntax highlighting.")
> + (home-page "https://fishshell.com/")
> + (license license:gpl2)))
> +@end lisp
> +@end enumerate
Thanks for taking the time to author all of this!
--
Maxim
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Thu, 20 Mar 2025 12:04:02 GMT) Full text and rfc822 format available.Message #170 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Cc: Ludovic Courtès <ludo <at> gnu.org>, 77093 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il> Subject: Re: [bug#77093] [PATCH rust-team v2 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Thu, 20 Mar 2025 20:02:37 +0800
On Wed, 19 Mar 2025 21:41:39 +0800,
Maxim Cournoyer wrote:
>
> [...]
> > +@end lisp
> > +
> > +@item
> > +@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
> > +workspace, build argument @code{#:cargo-package-crates} is required.
>
> Hm. Does that mean we're still in a situation where some inputs are not
> true Guix inputs, and some Rust-packaging oddity that our tooling
> doesn't handle (e.g. 'guix graph' ?).
This argument is meant to specify a order, workspace members may depend on each
other, so the dependencies should be packaged and put into "guix-vendor" first.
The naming might be confusing but I can't come up with a better one :(
cargo- : cargo-build-system specific
package- : package phase
crates : list of crate names
Rust <package>s will have all their Rust dependencies in inputs field but most
of them are origins, so there won't be much difference in terms of ‘guix graph’.
--8<---------------cut here---------------start------------->8---
$ guix graph niri | grep '"rust-.*'
"140032307245408" [label = "rust-bindgen-cli <at> 0.71.1", shape = box, fontname = sans];
"140032307245936" [label = "rust-cbindgen <at> 0.26.0", shape = box, fontname = sans];
"140032240396352" [label = "rust-ring <at> 0.17.8", shape = box, fontname = sans];
"140032307276192" [label = "rust-cargo-c <at> 0.10.9+cargo-0.85.0", shape = box, fontname = sans];
"140032240396176" [label = "rust-smithay <at> 0.4.0-0.0cd3345", shape = box, fontname = sans];
"140032240396880" [label = "rust-pipewire <at> 0.8.0-0.fd3d8f7", shape = box, fontname = sans];
$ guix graph niri --type=bag-with-origins | grep '"rust-.*'
"/gnu/store/kzl3bv8yfczwg18kcx649p7nmf8n2m55-rust-bindgen-cli-0.71.1.drv" [label = "rust-bindgen-cli <at> 0.71.1", shape = box, fontname = sans];
"/gnu/store/d28511hifkbpzr8hrnngb5hlsy7js2na-rust-windows-x86-64-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/7kh3innc661lf8fcya6nziglv93hy2yl-rust-windows-x86-64-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/fc73asklfa1q29g46bc8gpkggqwpc83w-rust-windows-x86-64-gnu-0.52.6.tar.zst.drv" [label = "rust-windows-x86-64-gnu-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/ca8laa955hw96r49dg5pzsl5pjxqs37j-rust-windows-i686-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-i686-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/9czqdmfglvmag8xi69a4jyabblhff72j-rust-windows-i686-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-i686-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/wjmvcdbqjm1zxqaih1c5f52d7990h40p-rust-windows-i686-gnu-0.52.6.tar.zst.drv" [label = "rust-windows-i686-gnu-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/w16561c65nzsqmif3ww634mijfk835gl-rust-windows-aarch64-msvc-0.52.6.tar.zst.drv" [label = "rust-windows-aarch64-msvc-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/aaw14iijajsxv5s3p6g58py1ippbr3ql-rust-windows-aarch64-gnullvm-0.52.6.tar.zst.drv" [label = "rust-windows-aarch64-gnullvm-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/xp4i48sdij8qmq2xwpq8dq5h4ax0mzgs-rust-windows-targets-0.52.6.tar.gz.drv" [label = "rust-windows-targets-0.52.6.tar.gz", shape = box, fontname = sans];
"/gnu/store/wlfg3iy55ph6ffwp96sqyzg22isx8573-rust-windows-sys-0.59.0.tar.gz.drv" [label = "rust-windows-sys-0.59.0.tar.gz", shape = box, fontname = sans];
"/gnu/store/q8x8721s5r6rb8nrb19jkwbszrqih26f-rust-winapi-util-0.1.9.tar.gz.drv" [label = "rust-winapi-util-0.1.9.tar.gz", shape = box, fontname = sans];
"/gnu/store/qynjbqr3md18m6q4ynrpvglfa8xjys0d-rust-utf8parse-0.2.2.tar.gz.drv" [label = "rust-utf8parse-0.2.2.tar.gz", shape = box, fontname = sans];
[...]
--8<---------------cut here---------------end--------------->8---
Thanks for the review! I'll prepare v3 soon :)
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:01 GMT) Full text and rfc822 format available.Message #173 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 01/17] build/cargo: Pass ‘--offline’ to cargo. Date: Sun, 23 Mar 2025 15:23:05 +0800
This will make error messages more helpful.
* guix/build/cargo-build-system.scm (build,check,package,install): Pass
‘--offline’ to cargo.
Change-Id: Ic95f603b793319f99c9c1fbce43f773bfc8126c0
---
guix/build/cargo-build-system.scm | 9 +++++----
1 file changed, 5 insertions(+), 4 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 5ef6b23abd..41f54f42b7 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -246,7 +246,7 @@ (define* (build #:key
"Build a given Cargo package."
(or skip-build?
(apply invoke
- `("cargo" "build"
+ `("cargo" "build" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -264,7 +264,7 @@ (define* (check #:key
"Run tests for a given Cargo package."
(when tests?
(apply invoke
- `("cargo" "test"
+ `("cargo" "test" "--offline"
,@(if parallel-build?
(list "-j" (number->string (parallel-job-count)))
(list "-j" "1"))
@@ -303,7 +303,7 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" ,@cargo-package-flags))
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
@@ -356,7 +356,8 @@ (define* (install #:key
;; otherwise cargo will raise an error.
(or skip-build?
(not (has-executable-target?))
- (invoke "cargo" "install" "--no-track" "--path" "." "--root" out
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" "." "--root" out
"--features" (string-join features)))
(when install-source?
--
2.49.0
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:02 GMT) Full text and rfc822 format available.Message #176 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 02/17] build/cargo: Print out all non-empty binary files. Date: Sun, 23 Mar 2025 15:23:06 +0800
* guix/build/cargo-build-system.scm (%standard-phases): Move
'unpack-rust-crates after 'unpack.
Move 'check-for-pregenerated-files after 'configure.
(check-for-pregenerated-files): Only check non-empty files.
Print out binary files.
Run in parallel.
Don't fail to keep compatibility for phase order change.
Change-Id: I0a332fe843e97687324bd908fa111422a63e475d
---
guix/build/cargo-build-system.scm | 31 +++++++++++++++++++++++++------
1 file changed, 25 insertions(+), 6 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41f54f42b7..58238992b8 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -29,9 +29,11 @@ (define-module (guix build cargo-build-system)
#:use-module ((guix build utils) #:hide (delete))
#:use-module (ice-9 popen)
#:use-module (ice-9 rdelim)
+ #:use-module (ice-9 regex)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
#:export (%standard-phases
@@ -111,12 +113,29 @@ (define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
(define (rust-package? name)
(string-prefix? "rust-" name))
-(define* (check-for-pregenerated-files #:rest _)
+(define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
"Check the source code for files which are known to generally be bundled
libraries or executables."
- (let ((pregenerated-files (find-files "." "\\.(a|dll|dylib|exe|lib)$")))
- (when (not (null-list? pregenerated-files))
- (error "Possible pre-generated files found:" pregenerated-files))))
+ (format #t "Searching for binary files...~%")
+ (let ((known-pattern (make-regexp "\\.(a|dll|dylib|exe|lib)$"))
+ (empty-file?
+ (lambda (filename)
+ (call-with-ascii-input-file filename
+ (lambda (p)
+ (eqv? #\0 (read-char p)))))))
+ (n-par-for-each
+ (if parallel-build?
+ (parallel-job-count)
+ 1)
+ (lambda (file)
+ (unless (empty-file? file)
+ ;; Print out binary files.
+ (false-if-exception (invoke "grep" "-IL" "." file))
+ ;; Warn about known pre-generated files.
+ ;; Not failing here for compatibility with existing packages.
+ (when (regexp-exec known-pattern file)
+ (format #t "error: Possible pre-generated file found: ~a~%" file))))
+ (find-files "."))))
(define* (configure #:key inputs
target system
@@ -380,8 +399,8 @@ (define %standard-phases
(replace 'check check)
(replace 'install install)
(add-after 'build 'package package)
- (add-after 'unpack 'check-for-pregenerated-files check-for-pregenerated-files)
- (add-after 'check-for-pregenerated-files 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'unpack 'unpack-rust-crates unpack-rust-crates)
+ (add-after 'configure 'check-for-pregenerated-files check-for-pregenerated-files)
(add-after 'patch-generated-file-shebangs 'patch-cargo-checksums patch-cargo-checksums)))
(define* (cargo-build #:key inputs (phases %standard-phases)
--
2.49.0
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, ludo <at> gnu.org, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:02 GMT) Full text and rfc822 format available.Message #179 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sun, 23 Mar 2025 15:23:04 +0800
V2 -> V3:
* Add Efraim's etc/teams/rust/audit-rust-crates script.
* Add (guix import crate cargo-lock) to Rust team's scope.
* ‘definer’ -> ‘define-prefix’
* Adjust documentation and move examples to Guix Cookbook.
* Document ‘cargo-inputs’ and remove mentioning of files under etc/teams/rust.
Diff inserted at end.
Efraim Flashner (1):
build/cargo: Don't try to unpack sanity-check.py.
Hilton Chain (16):
build/cargo: Pass ‘--offline’ to cargo.
build/cargo: Print out all non-empty binary files.
build-system: cargo: Support packaging Cargo workspace.
build-system: cargo: Support installing Cargo workspace.
build/cargo: Set default value of arguments for build phases.
build/cargo: Support non-workspace directory source inputs.
scripts: import: Document argument for ‘--insert’ option in help
message.
scripts: import: Add two newlines for ‘--insert’ option.
scripts: import: Support expressions defined by 'define.
scripts: import: Pass "--insert" to importers.
scripts: import: Skip existing definition for ‘--insert’ option.
import: crate: crate-name->package-name: Move to (guix build-system
cargo).
build-system: cargo: Add ‘cargo-inputs’.
import: crate: Add Cargo.lock parser.
import: crate: Add ‘--lockfile’ option.
doc: Document lockfile importer based Rust packaging workflow.
Makefile.am | 1 +
doc/contributing.texi | 91 +++++---
doc/guix-cookbook.texi | 352 ++++++++++++++++++++++++++++++
doc/guix.texi | 24 ++
etc/teams.scm | 1 +
etc/teams/rust/audit-rust-crates | 70 ++++++
etc/teams/rust/cleanup-crates.sh | 37 ++++
etc/teams/rust/rust-crates.tmpl | 42 ++++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 ++++
gnu/packages/rust-sources.scm | 29 +++
guix/build-system/cargo.scm | 59 ++++-
guix/build/cargo-build-system.scm | 150 +++++++++----
guix/import/crate.scm | 81 ++++++-
guix/import/crate/cargo-lock.scm | 105 +++++++++
guix/scripts/import.scm | 54 +++--
guix/scripts/import/crate.scm | 58 ++++-
guix/utils.scm | 29 ++-
tests/crate.scm | 88 ++++++++
19 files changed, 1209 insertions(+), 106 deletions(-)
create mode 100755 etc/teams/rust/audit-rust-crates
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
create mode 100644 guix/import/crate/cargo-lock.scm
base-commit: 127e1e89a83785f4ad8362bdc76b0d850279cf2f
--
2.49.0
diff --git a/doc/contributing.texi b/doc/contributing.texi
index 837074dead..17f22390d5 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,12 +1600,6 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-As currently there's no achievable way to reuse Rust packages as pre-compiled
-inputs for other packages at a distribution scale, and our previous approach on
-making all Rust dependencies as full packages doesn't cope well with Rust
-ecosystem thus causing issues on both contribution and maintenance sides, we
-have switched to a different packaging model.
-
Rust programs (binary crates) and dependencies (library crates) are treated
separately. We put our main efforts into programs and only package Rust
dependencies as sources, utilizing automation with a manual focus on unbundling
@@ -1625,19 +1619,11 @@ Rust Crates
@item
@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
-import, crate, @code{--lockfile=@var{file}}}).
+import, @code{crate}, @code{--lockfile=@var{file}}}).
Imported definitions must be checked and have vendored sources unbundled before
-contributing to Guix. Naturally, this module serves as a store for both sources
-and unbundling strategies.
-
-This module is managed by the Rust team (@pxref{Teams}) to ensure there's always
-one version containing all changes from other branches, so that the maintained
-version can be used directly in case of merge conflicts, thus coordination is
-required for other committers to modify it.
-
-Guix source ships template @file{etc/teams/rust/rust-crates.tmpl} and cleanup
-script @file{etc/teams/rust/cleanup-crates.sh} for this module.
+being contributed to Guix. This module is managed by the Rust team
+(@pxref{Teams}).
@item
@code{(gnu packages rust-sources)}, storing more complex definitions that need
@@ -1661,338 +1647,28 @@ Rust Crates
In practice we are usually packaging development snapshots of Rust dependencies
specifically for some Rust programs, and can't simply identity them by version.
-In this case we can use a @code{for-@var{program}} suffix. Examples@:
-@code{rust-pipewire-for-niri}, @code{rust-pubgrub-for-uv}.
-@end enumerate
-
-Let's demonstrate the packaging workflow by examples, note that package-specific
-issues are not involved here.
-
-In preparation, we'll add the following packages to our environment:
-
-@example
-guix shell rust rust:cargo cargo-audit cargo-license
-@end example
-
-Example 1: @code{cargo-audit}, which is published on the crates.io Rust package
-repository @uref{https://crates.io, crates.io}.
-
-@enumerate
-@item
-We can generate a draft definition via the crates.io importer (@pxref{Invoking
-guix import, crate}). In the end we'll have the following definiton:
-
-@lisp
-(define-public cargo-audit
- (package
- (name "cargo-audit")
- (version "0.21.2")
- (source
- (origin
- (method url-fetch)
- (uri (crate-uri "cargo-audit" version))
- (file-name (string-append name "-" version ".tar.gz"))
- (sha256
- (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
- (build-system cargo-build-system)
- (arguments (list #:install-source? #f))
- (inputs (cargo-inputs 'cargo-audit))
- (home-page "https://rustsec.org/")
- (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
- (description
- "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
-audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
- (license (list license:asl2.0 license:expat))))
-@end lisp
-
-@code{cargo-inputs} is a procedure defined in @code{guix build-system cargo},
-facilitating dependency modification. @code{'cargo-audit} used here must be a
-unique identifier, usually same to the program's variable name.
-
-@item
-Unpack package source and navigate to the unpacked directory, then execute the
-following commands:
-
-@example
-cargo generate-lockfile
-cargo audit
-cargo license
-@end example
-
-@command{cargo generate-lockfile} updates dependencies to compatible versions,
-@command{cargo audit} checks known vulnerabilities and @command{cargo license}
-checks licenses of all dependencies.
-
-We must have an acceptable output of @command{cargo audit} and ensure all
-dependencies are licensed with our supported licenses (@pxref{Defining Packages,
-@code{license}}).
-
-@item
-Import dependencies from previously generated lockfile:
-
-@example
-guix import --insert=gnu/packages/rust-crates.scm \
- crate --lockfile=/path/to/Cargo.lock cargo-audit
-@end example
-
-@code{cargo-audit} used here must be consistent with the identifier used for
-@code{cargo-inputs}.
-
-At this stage, @code{cargo-audit} is buildable.
-
-@item
-Finally we'll unbundle vendored sources. The lockfile importer inserts
-@code{TODO:} comments to dependencies with high probability of bundled sources.
-@code{cargo-build-system} also performs additional check in its
-@code{check-for-pregenerated-files} phase:
-
-@example
-$ ./pre-inst-env guix build cargo-audit
-@dots{}
-starting phase `check-for-pregenerated-files'
-Searching for binary files...
-./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
-./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
-./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
-@dots{}
-@end example
-
-Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
-can still select them in Guix command-line interface through expression:
-
-@example
-guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
-@end example
-
-For most dependencies, a snippet is sufficient:
-
-@lisp
-(define rust-bzip2-sys-0.1.13+1.0.8
- (crate-source "bzip2-sys" "0.1.13+1.0.8"
- "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
- #:snippet
- '(begin
- (delete-file-recursively "bzip2-1.0.8")
- (delete-file "build.rs")
- ;; Inspired by Debian's patch.
- (with-output-to-file "build.rs"
- (lambda _
- (format #t "fn main() @{~@@
- println!(\"cargo:rustc-link-lib=bz2\");~@@
- @}~%"))))))
-@end lisp
-
-In a more complex case, where unbundling one dependency requires other packages,
-we should package the dependency in @code{(gnu packages rust-sources)} first and
-point the imported definition to a symbol with the same name.
-
-For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
-rust-sources)}, then the imported definition in @code{(gnu packages
-rust-crates)} should be modified to point to it:
-
-@lisp
-(define rust-ring-0.17.14 'rust-ring-0.17)
-@end lisp
-
-When one dependency can be safely removed, modify it to @code{#f}.
-
-@lisp
-(define rust-openssl-src-300.4.2+3.4.1 #f)
-@end lisp
-
-Such modifications are processed by @code{cargo-inputs} procedure.
-
-@code{cargo-inputs} can also be configured by keywoard arguments
-@code{#:crates-module} (default: @code{(gnu packages rust-crates)}, module to
-resolve imported definitions) and @code{#:sources-module} (default: @code{(gnu
-packages rust-sources)}, module to resolve modified symbols), which are useful
-to maintain an independent channel.
+In this case we can use a @code{for-@var{program}} suffix, for example,
+@code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
@end enumerate
-Example 2: @code{niri}, which is not available on crates.io and depends on
-development snapshots (also Cargo workspaces in this example).
+Rust dependencies are not referenced directly. @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to create an input list, in combination
+with the lockfile importer.
-@enumerate
-@item
-As we can't ensure compatibility of a development snapshot, before executing
-@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
-in a known working revision.
+@cindex cargo-inputs
+@deffn {Procedure} cargo-inputs @var{name} @
+ [#:crates-module '(gnu packages rust-crates)] @
+ [#:sources-module '(gnu packages rust-sources)]
+Given symbol @code{'@var{name}}, resolve variable
+@code{@var{name}-cargo-inputs}, an input list, in @var{crates-module}, return
+its copy with @code{#f} removed and symbols resolved to variables defined in
+@var{sources-module} if the input list exists, otherwise return an empty list.
-To use our packaged development snapshots, it's also necessary to modify
-@file{Cargo.toml} in build environment, the substitution pattern is
-package-specific.
-
-@code{cargo-inputs} returns a list, all list operations apply to it too.
-
-@lisp
-(define-public niri
- (package
- (name "niri")
- (version "25.02")
- (source (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://github.com/YaLTeR/niri")
- (commit (string-append "v" version))))
- (file-name (git-file-name name version))
- (sha256
- (base32
- "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
- (build-system cargo-build-system)
- (arguments
- (list #:install-source? #f
- #:phases
- #~(modify-phases %standard-phases
- (add-after 'unpack 'use-guix-vendored-dependencies
- (lambda _
- (substitute* "Cargo.toml"
- (("# version =.*")
- "version = \"*\"")
- (("git.*optional")
- "version = \"*\", optional")
- (("^git = .*")
- "")))))))
- (native-inputs
- (list pkg-config))
- (inputs
- (cons* clang
- libdisplay-info
- libinput-minimal
- libseat
- libxkbcommon
- mesa
- pango
- pipewire
- wayland
- (cargo-inputs 'niri)))
- (home-page "https://github.com/YaLTeR/niri")
- (synopsis "Scrollable-tiling Wayland compositor")
- (description
- "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
-scrollable format. It is considered stable for daily use and performs most
-functions expected of a Wayland compositor.")
- (license license:gpl3)))
-@end lisp
-
-@item
-@code{niri} also has Cargo workspace dependencies. When packaging a Cargo
-workspace, build argument @code{#:cargo-package-crates} is required.
-
-@lisp
-(define-public rust-pipewire-for-niri
- (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
- (revision "0"))
- (package
- (name "rust-pipewire")
- (version (git-version "0.8.0" revision commit))
- (source
- (origin
- (method git-fetch)
- (uri (git-reference
- (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
- (commit commit)))
- (file-name (git-file-name name version))
- (sha256
- (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
- (build-system cargo-build-system)
- (arguments
- (list #:skip-build? #t
- #:cargo-package-crates
- ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
- (inputs (cargo-inputs 'rust-pipewire-for-niri))
- (home-page "https://pipewire.org/")
- (synopsis "Rust bindings for PipeWire")
- (description "This package provides Rust bindings for PipeWire.")
- (license license:expat))))
-@end lisp
-
-Don't forget to modify all workspace members in @code{(gnu packages
-rust-crates)}:
-
-@lisp
-(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-@dots{}
-(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
-@end lisp
-@end enumerate
-
-Example 3: @code{fish}, which combines two build systems.
-
-@enumerate
-@item
-When building Rust packages in other build systems, we need to add @code{rust}
-and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
-build systems, and apply necessary build phases from @code{cargo-build-system}.
+@var{name} must be consistent with the one used in lockfile importer invocation.
+@end deffn
-@lisp
-(define-public fish
- (package
- (name "fish")
- (version "4.0.0")
- (source
- (origin
- (method url-fetch)
- (uri (string-append "https://github.com/fish-shell/fish-shell/"
- "releases/download/" version "/"
- "fish-" version ".tar.xz"))
- (sha256
- (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
- (build-system cmake-build-system)
- (inputs
- (cons* fish-foreign-env ncurses pcre2
- python ;for fish_config and manpage completions
- (cargo-inputs 'fish)))
- (native-inputs
- (list doxygen groff ;for 'fish --help'
- pkg-config
- procps ;for the test suite
- rust
- `(,rust "cargo")))
- (arguments
- (list #:imported-modules
- (append %cargo-build-system-modules
- %cmake-build-system-modules)
- #:modules
- (((guix build cargo-build-system) #:prefix cargo:)
- (guix build cmake-build-system)
- (guix build utils))
- #:phases
- #~(modify-phases %standard-phases
- (add-after 'unpack 'use-guix-vendored-dependencies
- (lambda _
- (substitute* "Cargo.toml"
- (("git.*tag.*,")
- "version = \"*\","))))
- (add-after 'unpack 'prepare-cargo-build-system
- (lambda args
- (for-each
- (lambda (phase)
- (format #t "Running cargo phase: ~a~%" phase)
- (apply (assoc-ref cargo:%standard-phases phase)
- args))
- '(unpack-rust-crates
- configure
- check-for-pregenerated-files
- patch-cargo-checksums)))))))
- (synopsis "The friendly interactive shell")
- (description
- "Fish (friendly interactive shell) is a shell focused on interactive use,
-discoverability, and friendliness. Fish has very user-friendly and powerful
-tab-completion, including descriptions of every completion, completion of
-strings with wildcards, and many completions for specific commands. It also
-has extensive and discoverable help. A special @@command@{help@} command gives
-access to all the fish documentation in your web browser. Other features
-include smart terminal handling based on terminfo, an easy to search history,
-and syntax highlighting.")
- (home-page "https://fishshell.com/")
- (license license:gpl2)))
-@end lisp
-@end enumerate
+For a more detailed packaging workflow, @pxref{Packaging Rust Crates,,,
+guix-cookbook, GNU Guix Cookbook}.
@node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index d9b98a2ab3..1f0e5eee54 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -103,6 +103,7 @@ Top
Packaging
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
Packaging Tutorial
@@ -127,6 +128,16 @@ Top
* Automatic update::
* Inheritance::
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+
System Configuration
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
@@ -514,6 +525,7 @@ Packaging
@menu
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
@end menu
@node Packaging Tutorial
@@ -1593,6 +1605,346 @@ References
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
@end itemize
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+@end menu
+
+@node Common Workflow for Packaging Rust Crates
+@subsubsection Common Workflow for Packaging Rust Crates
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository. All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, We can generate a draft
+definition via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU
+Guix Reference Manual}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The symbol used in @code{cargo-inputs}, @code{'cargo-audit} here, must be a
+unique identifier, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining
+Packages,,, guix, GNU Guix Reference Manual}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them via the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+reference it by a symbol in the imported definition.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to a matching symbol.
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+@end enumerate
+
+@node Development Snapshots and Cargo Workspaces
+@subsubsection Development Snapshots and Cargo Workspaces
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies. When packaging a Cargo workspace,
+argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+
+@node Rust Programs With Mixed Build Systems
+@subsubsection Rust Programs With Mixed Build Systems
+
+In this example, we'll package @code{fish}, which combines two build systems.
+
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+
+
@c *********************************************************************
@node System Configuration
@chapter System Configuration
diff --git a/etc/teams.scm b/etc/teams.scm
index 6bddbd91fa..c4bee8d3d9 100755
--- a/etc/teams.scm
+++ b/etc/teams.scm
@@ -389,6 +389,7 @@ (define-team rust
"guix/build/cargo-utils.scm"
"guix/build-system/cargo.scm"
"guix/import/crate.scm"
+ "guix/import/crate/cargo-lock.scm"
"guix/scripts/import/crate.scm"
"tests/crate.scm")))
diff --git a/etc/teams/rust/audit-rust-crates b/etc/teams/rust/audit-rust-crates
new file mode 100755
index 0000000000..d5546fd1e1
--- /dev/null
+++ b/etc/teams/rust/audit-rust-crates
@@ -0,0 +1,70 @@
+#!/usr/bin/env -S gawk -f
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Efraim Flashner <efraim <at> flashner.co.il>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+# To run:
+# ./etc/teams/rust/audit-rust-crates ./path/to/file.scm
+# Prints the output of cargo-audit to the shell.
+
+# Make sure we have cargo-audit in our PATH
+BEGIN {
+ if (system("which cargo-audit 1> /dev/null"))
+ exit 1;
+ # Parse a record at a time.
+ RS = "\n\n"
+ cargoAudit = "cargo-audit audit --file -"
+}
+
+# Check the crate-source origin-only inputs
+/crate-source/ {
+ for(i=3; i <= NF-2; i++) {
+ if($i == "(crate-source") {
+ cargoLock = cargoLock "[[package]]\nname = " $(i+1) "\nversion = " $(i+2) "\n"
+ next
+ }
+ }
+}
+
+# Check the crates packaged from crates.io tarballs
+/crate-uri/ {
+ for(i=3; i <= NF; i++) {
+ if($i == "(version")
+ crateVersion = $(i+1)
+ if($i == "(crate-uri")
+ crateName = $(i+1)
+ }
+ gsub(/)/, "", crateVersion)
+ cargoLock = cargoLock "[[package]]\nname = " crateName "\nversion = " crateVersion "\n"
+}
+
+# The xxxx-cargo-input variables have a set style
+# TODO: Replace the last dash between the name and the version with a space!
+# This doesn't take into account swapping between "-" and "_" so we skip it.
+#( $2 ~ /-cargo-inputs/ ) {
+# sub(/-cargo-inputs/, "", $2)
+# gsub(/)/, "", $0)
+# gsub(/rust-/, "", $0)
+# #gensub(/([[:alpha:]])-([[:digit:]]+)/, "\\1 \\2", "g", $i)
+# print "[[package]]\nname = \"" $2 "\"\nversion = \"1.0.0\"\ndependencies = ["
+# for (i = 4; i <= NF; i++) {
+# print "\"" $i "\","
+# }
+# print "]"
+#}
+
+END { print cargoLock | cargoAudit }
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
index cd4c2462b9..eca37ca00c 100755
--- a/etc/teams/rust/cleanup-crates.sh
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -1,5 +1,4 @@
#!/bin/sh
-
# GNU Guix --- Functional package management for GNU
# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
#
@@ -28,7 +27,7 @@ do
(begin
(use-modules (guix utils))
(let ((source-properties
- (find-definition-location \"$FILE\" '$crate #:definer 'define)))
+ (find-definition-location \"$FILE\" '$crate #:define-prefix 'define)))
(and=> source-properties delete-expression)))" |
guix repl -t machine
fi
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 47d4f10969..bbfa2f933b 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -78,9 +78,11 @@ (define* (crate-source name version hash #:key (patches '()) (snippet #f))
(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
(sources-module '(gnu packages rust-sources)))
- "Given NAME, resolve input list 'NAME-cargo-inputs' in CRATES-MODULE, return
-its copy with #f removed and symbols resolved to variables defined in
-SOURCES-MODULE, if the list exists, otherwise return an empty list."
+
+ "Given symbol NAME, resolve variable 'NAME-cargo-inputs', an input list, in
+CRATES-MODULE, return its copy with #f removed and symbols resolved to
+variables defined in SOURCES-MODULE if the input list exists, otherwise return
+an empty list."
(let loop ((inputs
(catch #t
(lambda ()
@@ -101,7 +103,7 @@ (define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
((symbol? input)
(cons (module-ref (resolve-interface sources-module) input)
result))
- ;; Else: no change.
+ ;; Else: keep it.
(else
(cons input result)))))))))
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 41adc03752..1012d7f401 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -350,17 +350,16 @@ (define* (package #:key
(if (null? cargo-package-crates)
(apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
- (begin
- (for-each
- (lambda (pkg)
- (apply invoke "cargo" "package" "--offline" "--package" pkg
- cargo-package-flags)
- (for-each
- (lambda (crate)
- (invoke "tar" "xzf" crate "-C" vendor-dir))
- (find-files "target/package" "\\.crate$"))
- (patch-cargo-checksums #:vendor-dir vendor-dir))
- cargo-package-crates)))
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index f6e2985ed9..1bddac15c6 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -126,21 +126,22 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let* ((importer-definers
+ (let* ((define-prefixes
`(,@(if (member importer '("crate"))
'(define)
'())
define-public))
- (definer? (cut member <> importer-definers))
+ (define-prefix? (cut member <> define-prefixes))
(find-and-insert
(lambda (expr)
(match expr
- (((? definer? definer) term _ ...)
+ (((? define-prefix? define-prefix) term _ ...)
;; Skip existing definition.
- (unless (find-definition-location file term #:definer definer)
+ (unless (find-definition-location
+ file term #:define-prefix define-prefix)
(let ((source-properties
(find-definition-insertion-location
- file term #:definer definer)))
+ file term #:define-prefix define-prefix)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index 77ec6d992a..3f85320845 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -522,19 +522,19 @@ (define (insert-expression source-properties expr)
(edit-expression source-properties insert)))
(define* (find-definition-location file term
- #:key (definer 'define-public)
+ #:key (define-prefix 'define-public)
(pred string=))
- "Search in FILE for a top-level definition defined by DEFINER with defined
-term comparing to TERM through PRED. Return the location if PRED returns #t,
-or #f otherwise."
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term comparing to TERM through PRED. Return the location if PRED
+returns #t, or #f otherwise."
(let ((search-term (symbol->string term))
- (definer? (cut eq? definer <>)))
+ (define-prefix? (cut eq? define-prefix <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (((? definer?) current-term _ ...)
+ (((? define-prefix?) current-term _ ...)
(pred (symbol->string current-term)
search-term))
((? eof-object?) #t)
@@ -543,11 +543,13 @@ (define* (find-definition-location file term
(syntax-source syntax))))))))
(define* (find-definition-insertion-location file term
- #:key (definer 'define-public))
- "Search in FILE for a top-level definition defined by DEFINER with defined
-term alphabetically succeeds TERM. Return the location if found, or #f
+ #:key
+ (define-prefix 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term alphabetically succeeds TERM. Return the location if found, or #f
otherwise."
- (find-definition-location file term #:definer definer #:pred string>))
+ (find-definition-location
+ file term #:define-prefix define-prefix #:pred string>))
;;;
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, ludo <at> gnu.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:03 GMT) Full text and rfc822 format available.Message #182 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 03/17] build-system: cargo: Support packaging Cargo workspace. Date: Sun, 23 Mar 2025 15:23:07 +0800
* guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-package-crates]: New argument.
* guix/build/cargo-build-system.scm (package): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I45ccd95e90827d47127015cb0bda2d41f792335b
---
doc/guix.texi | 7 +++++++
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 21 +++++++++++++++++++--
3 files changed, 30 insertions(+), 2 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0ca109a214..0089f9e60f 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9363,6 +9363,13 @@ Build Systems
the binaries defined by the crate. Unless @code{install-source? #f} is
defined it will also install a source crate repository of itself and unpacked
sources, to ease in future hacking on rust packages.
+
+This build system supports cargo workspaces. Parameter
+@code{#:cargo-package-crates} (default: @code{''()}) allows specifying names of
+library crates to package in the @code{package} phase. Specified crates are
+packaged from left to right, in case there's dependency among them. For
+example, specifying @code{''("pcre2-sys" "pcre2")} will package
+@code{"pcre2-sys"} first and then @code{"pcre2"}.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 452f7f78d0..4f6d46e70c 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -94,6 +94,7 @@ (define* (cargo-build name inputs
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(features ''())
(skip-build? #f)
@@ -122,6 +123,7 @@ (define* (cargo-build name inputs
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
@@ -154,6 +156,7 @@ (define* (cargo-cross-build name
(vendor-dir "guix-vendor")
(cargo-build-flags ''("--release"))
(cargo-test-flags ''())
+ (cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
(cargo-target (cargo-triplet (or target system)))
(features ''())
@@ -185,6 +188,7 @@ (define* (cargo-cross-build name
#:vendor-dir #$vendor-dir
#:cargo-build-flags #$(sexp->gexp cargo-build-flags)
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
+ #:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 58238992b8..51a45b4eab 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -300,11 +300,16 @@ (define* (package #:key
source
skip-build?
install-source?
+ (cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
+ (vendor-dir "guix-vendor")
#:allow-other-keys)
"Run 'cargo-package' for a given Cargo package."
(if install-source?
- (if skip-build?
+ ;; NOTE: Cargo workspace packaging support:
+ ;; #:install-source? #t + #:cargo-package-crates.
+ (if (and (null? cargo-package-crates)
+ skip-build?)
(begin
(install-file source "target/package")
(with-directory-excursion "target/package"
@@ -322,7 +327,19 @@ (define* (package #:key
;;error: invalid inclusion of reserved file name Cargo.toml.orig in package source
(when (file-exists? "Cargo.toml.orig")
(delete-file "Cargo.toml.orig"))
- (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+
+ (if (null? cargo-package-crates)
+ (apply invoke `("cargo" "package" "--offline" ,@cargo-package-flags))
+ (for-each
+ (lambda (pkg)
+ (apply invoke "cargo" "package" "--offline" "--package" pkg
+ cargo-package-flags)
+ (for-each
+ (lambda (crate)
+ (invoke "tar" "xzf" crate "-C" vendor-dir))
+ (find-files "target/package" "\\.crate$"))
+ (patch-cargo-checksums #:vendor-dir vendor-dir))
+ cargo-package-crates))
;; Then unpack the crate, reset the timestamp of all contained files, and
;; repack them. This is necessary to ensure that they are reproducible.
--
2.49.0
hako <at> ultrarare.space, efraim <at> flashner.co.il, maxim.cournoyer <at> gmail.com, divya <at> subvertising.org, ludo <at> gnu.org, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:26:03 GMT) Full text and rfc822 format available.Message #185 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 04/17] build-system: cargo: Support installing Cargo workspace. Date: Sun, 23 Mar 2025 15:23:08 +0800
*guix/build-system/cargo.scm (cargo-build,cargo-cross-build)
[#:cargo-install-paths]: New argument.
* guix/build/cargo-build-system.scm (install): Use it.
* doc/guix.texi (Build Systems)[cargo-build-system]: Document it.
Change-Id: I74ed1972a5716da05afeac8edb2b0e4b6834bf40
---
doc/guix.texi | 5 ++++-
guix/build-system/cargo.scm | 4 ++++
guix/build/cargo-build-system.scm | 17 +++++++++++++----
3 files changed, 21 insertions(+), 5 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 0089f9e60f..7fa986b4b9 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -9369,7 +9369,10 @@ Build Systems
library crates to package in the @code{package} phase. Specified crates are
packaged from left to right, in case there's dependency among them. For
example, specifying @code{''("pcre2-sys" "pcre2")} will package
-@code{"pcre2-sys"} first and then @code{"pcre2"}.
+@code{"pcre2-sys"} first and then @code{"pcre2"}. Parameter
+@code{#:cargo-install-paths} (default: @code{''()}) allows specifying paths of
+binary crates to install in the @code{install} phase, @code{''("crates/atuin")},
+for example.
@end defvar
@defvar chicken-build-system
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4f6d46e70c..4486c706a1 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -96,6 +96,7 @@ (define* (cargo-build name inputs
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(features ''())
(skip-build? #f)
(parallel-build? #t)
@@ -125,6 +126,7 @@ (define* (cargo-build name inputs
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet system)
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
@@ -158,6 +160,7 @@ (define* (cargo-cross-build name
(cargo-test-flags ''())
(cargo-package-crates ''())
(cargo-package-flags ''("--no-metadata" "--no-verify"))
+ (cargo-install-paths ''())
(cargo-target (cargo-triplet (or target system)))
(features ''())
(skip-build? #f)
@@ -190,6 +193,7 @@ (define* (cargo-cross-build name
#:cargo-test-flags #$(sexp->gexp cargo-test-flags)
#:cargo-package-crates #$(sexp->gexp cargo-package-crates)
#:cargo-package-flags #$(sexp->gexp cargo-package-flags)
+ #:cargo-install-paths #$(sexp->gexp cargo-install-paths)
#:cargo-target #$(cargo-triplet (or target system))
#:features #$(sexp->gexp features)
#:skip-build? #$skip-build?
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 51a45b4eab..67311fc850 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -377,6 +377,7 @@ (define* (install #:key
skip-build?
install-source?
features
+ (cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
(let* ((out (assoc-ref outputs "out"))
@@ -391,10 +392,18 @@ (define* (install #:key
;; Only install crates which include binary targets,
;; otherwise cargo will raise an error.
(or skip-build?
- (not (has-executable-target?))
- (invoke "cargo" "install" "--offline" "--no-track"
- "--path" "." "--root" out
- "--features" (string-join features)))
+ ;; NOTE: Cargo workspace installation support:
+ ;; #:skip-build? #f + #:cargo-install-paths.
+ (and (null? cargo-install-paths)
+ (not (has-executable-target?)))
+ (for-each
+ (lambda (path)
+ (invoke "cargo" "install" "--offline" "--no-track"
+ "--path" path "--root" out
+ "--features" (string-join features)))
+ (if (null? cargo-install-paths)
+ '(".")
+ cargo-install-paths)))
(when install-source?
;; Install crate tarballs and unpacked sources for later use.
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:03 GMT) Full text and rfc822 format available.ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:04 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:06 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:08 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:10 GMT) Full text and rfc822 format available.divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:11 GMT) Full text and rfc822 format available.Message #203 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 05/17] build/cargo: Set default value of arguments for build phases. Date: Sun, 23 Mar 2025 15:28:27 +0800
This makes it easier to use these phases in other bulid systems.
* guix/build/cargo-build-system.scm (unpack-rust-crates,configure,build,package)
(install): Set default value of arguments.
Change-Id: I1dde1b063d8eee57967903abd2fce94574211a0a
---
guix/build/cargo-build-system.scm | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 67311fc850..15eda8396c 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -76,7 +76,8 @@ (define (crate-src? path)
" | cut -d/ -f2"
" | grep -q '^Cargo.toml$'")))))
-(define* (unpack-rust-crates #:key inputs vendor-dir #:allow-other-keys)
+(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
+ #:allow-other-keys)
(define (inputs->rust-inputs inputs)
"Filter using the label part from INPUTS."
(filter (lambda (input)
@@ -139,7 +140,7 @@ (define* (check-for-pregenerated-files #:key parallel-build? #:allow-other-keys)
(define* (configure #:key inputs
target system
- cargo-target
+ (cargo-target #f)
(vendor-dir "guix-vendor")
#:allow-other-keys)
"Vendor Cargo.toml dependencies as guix inputs."
@@ -258,7 +259,7 @@ (define* (patch-cargo-checksums #:key
(define* (build #:key
parallel-build?
- skip-build?
+ (skip-build? #f)
(features '())
(cargo-build-flags '("--release"))
#:allow-other-keys)
@@ -298,8 +299,8 @@ (define* (check #:key
(define* (package #:key
source
- skip-build?
- install-source?
+ (skip-build? #f)
+ (install-source? #t)
(cargo-package-crates '())
(cargo-package-flags '("--no-metadata" "--no-verify"))
(vendor-dir "guix-vendor")
@@ -374,9 +375,9 @@ (define* (package #:key
(define* (install #:key
inputs
outputs
- skip-build?
- install-source?
- features
+ (skip-build? #f)
+ (install-source? #t)
+ (features '())
(cargo-install-paths '())
#:allow-other-keys)
"Install a given Cargo package."
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:13 GMT) Full text and rfc822 format available.Message #206 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Efraim Flashner <efraim <at> flashner.co.il> Subject: [PATCH rust-team v3 06/17] build/cargo: Don't try to unpack sanity-check.py. Date: Sun, 23 Mar 2025 15:28:28 +0800
From: Efraim Flashner <efraim <at> flashner.co.il>
* guix/build/cargo-build-system.scm (crate-src?): Also don't try to
check python scripts for Cargo.toml.
Change-Id: I001a89b83d2e472706b1263007be45d1153c140f
---
guix/build/cargo-build-system.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index 15eda8396c..f2a7323748 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -64,6 +64,7 @@ (define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
(and (not (directory-exists? path)) ; not a tarball
+ (not (string-suffix? "py" path)) ; sanity-check.py
;; First we print out all file names within the tarball to see if it
;; looks like the source of a crate. However, the tarball will include
;; an extra path component which we would like to ignore (since we're
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:15 GMT) Full text and rfc822 format available.Message #209 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 07/17] build/cargo: Support non-workspace directory source inputs. Date: Sun, 23 Mar 2025 15:28:29 +0800
* guix/build/cargo-build-system.scm (single-crate?): New procedure.
(crate-src?): Support non-workspace directory source inputs.
(configure): Likewise.
Change-Id: If2bd318b44ed765baec69309f25d320edceee116
---
guix/build/cargo-build-system.scm | 60 ++++++++++++++++++++-----------
1 file changed, 39 insertions(+), 21 deletions(-)
diff --git a/guix/build/cargo-build-system.scm b/guix/build/cargo-build-system.scm
index f2a7323748..1012d7f401 100644
--- a/guix/build/cargo-build-system.scm
+++ b/guix/build/cargo-build-system.scm
@@ -33,6 +33,7 @@ (define-module (guix build cargo-build-system)
#:use-module (ice-9 ftw)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
+ #:use-module (ice-9 textual-ports)
#:use-module (ice-9 threads)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -60,22 +61,35 @@ (define (has-executable-target?)
(bin-dep? (lambda (dep) (find bin? (get-kinds dep)))))
(find bin-dep? (manifest-targets))))
+(define (single-crate? dir)
+ "Check if directory DIR contains 'Cargo.toml' and is not a workspace."
+ (let ((manifest-file (in-vicinity dir "Cargo.toml")))
+ (and (file-exists? manifest-file)
+ (not (string-contains
+ (call-with-input-file manifest-file get-string-all)
+ "[workspace]")))))
+
(define (crate-src? path)
"Check if PATH refers to a crate source, namely a gzipped tarball with a
Cargo.toml file present at its root."
- (and (not (directory-exists? path)) ; not a tarball
- (not (string-suffix? "py" path)) ; sanity-check.py
- ;; First we print out all file names within the tarball to see if it
- ;; looks like the source of a crate. However, the tarball will include
- ;; an extra path component which we would like to ignore (since we're
- ;; interested in checking if a Cargo.toml exists at the root of the
- ;; archive, but not nested anywhere else). We do this by cutting up
- ;; each output line and only looking at the second component. We then
- ;; check if it matches Cargo.toml exactly and short circuit if it does.
- (apply invoke (list "sh" "-c"
- (string-append "tar -tf " path
- " | cut -d/ -f2"
- " | grep -q '^Cargo.toml$'")))))
+ (if (directory-exists? path)
+ ;; The build system only handles sources containing single crate.
+ ;; Workspaces should be packaged into crates (via 'package phase)
+ ;; and used in inputs.
+ (single-crate? path)
+ (and (not (string-suffix? "py" path)) ;sanity-check.py
+ ;; First we print out all file names within the tarball to see
+ ;; if it looks like the source of a crate. However, the tarball
+ ;; will include an extra path component which we would like to
+ ;; ignore (since we're interested in checking if a Cargo.toml
+ ;; exists at the root of the archive, but not nested anywhere
+ ;; else). We do this by cutting up each output line and only
+ ;; looking at the second component. We then check if it matches
+ ;; Cargo.toml exactly and short circuit if it does.
+ (invoke "sh" "-c"
+ (string-append "tar -tf " path
+ " | cut -d/ -f2"
+ " | grep -q '^Cargo.toml$'")))))
(define* (unpack-rust-crates #:key inputs (vendor-dir "guix-vendor")
#:allow-other-keys)
@@ -159,14 +173,18 @@ (define* (configure #:key inputs
(and (crate-src? path)
;; Gracefully handle duplicate inputs
(not (file-exists? crate-dir))
- (mkdir-p crate-dir)
- ;; Cargo crates are simply gzipped tarballs but with a .crate
- ;; extension. We expand the source to a directory name we control
- ;; so that we can generate any cargo checksums.
- ;; The --strip-components argument is needed to prevent creating
- ;; an extra directory within `crate-dir`.
- (format #t "Unpacking ~a~%" name)
- (invoke "tar" "xf" path "-C" crate-dir "--strip-components" "1")))))
+ (if (directory-exists? path)
+ (copy-recursively path crate-dir)
+ (begin
+ (mkdir-p crate-dir)
+ ;; Cargo crates are simply gzipped tarballs but with a
+ ;; .crate extension. We expand the source to a directory
+ ;; name we control so that we can generate any cargo
+ ;; checksums. The --strip-components argument is needed to
+ ;; prevent creating an extra directory within `crate-dir`.
+ (format #t "Unpacking ~a~%" name)
+ (invoke "tar" "xf" path "-C" crate-dir
+ "--strip-components" "1")))))))
inputs)
;; For cross-building
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:18 GMT) Full text and rfc822 format available.Message #212 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 08/17] scripts: import: Document argument for ‘--insert’ option in help message. Date: Sun, 23 Mar 2025 15:28:30 +0800
* guix/scripts/import.scm (show-help): Add missing ‘FILE’. Change-Id: I540d5feae3fe49c00e9bd6f7a8649ffe0d6e006d --- guix/scripts/import.scm | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm index bbf31baa15..4fad329cd3 100644 --- a/guix/scripts/import.scm +++ b/guix/scripts/import.scm @@ -69,7 +69,7 @@ (define (show-help) (display (G_ " -h, --help display this help and exit")) (display (G_ " - -i, --insert insert packages into file alphabetically")) + -i, --insert=FILE insert packages into FILE alphabetically")) (display (G_ " -V, --version display version information and exit")) (newline) -- 2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:21 GMT) Full text and rfc822 format available.Message #215 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 09/17] scripts: import: Add two newlines for ‘--insert’ option. Date: Sun, 23 Mar 2025 15:28:31 +0800
For consistency with ‘insert-expression’.
* guix/scripts/import.scm (guix-import): Add two newlines when inserting.
Change-Id: I55b45ca137d175fdf6ee5c0bb2b6b1ca8385750d
---
guix/scripts/import.scm | 1 +
1 file changed, 1 insertion(+)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 4fad329cd3..58a84d0db7 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -129,6 +129,7 @@ (define-command (guix-import . args)
(let ((port (open-file file "a")))
(pretty-print-with-comments port expr)
(newline port)
+ (newline port)
(close-port port)))))))))
(import-as-definitions importer args find-and-insert)))
((importer args ...)
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:23 GMT) Full text and rfc822 format available.Message #218 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 10/17] scripts: import: Support expressions defined by 'define. Date: Sun, 23 Mar 2025 15:28:32 +0800
* guix/utils.scm (find-definition-location): New procedure.
(find-definition-insertion-location): Define with it.
* guix/scripts/import.scm (import-as-definitions, guix-import): Support
expressions defined by 'define.
Change-Id: I03118e1a3372028b4f0530964aba871b4a1a4d25
---
guix/scripts/import.scm | 21 +++++++++++++++------
guix/utils.scm | 29 +++++++++++++++++++++--------
2 files changed, 36 insertions(+), 14 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 58a84d0db7..6f6354928e 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -30,6 +30,7 @@ (define-module (guix scripts import)
#:use-module (guix read-print)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
+ #:use-module (srfi srfi-26)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -83,7 +84,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))
((expressions ...)
(for-each (lambda (expr)
@@ -91,7 +93,8 @@ (define (import-as-definitions importer args proc)
((and expr (or ('package _ ...)
('let _ ...)))
(proc (package->definition expr)))
- ((and expr ('define-public _ ...))
+ ((and expr (or ('define-public _ ...)
+ ('define _ ...)))
(proc expr))))
expressions))
(x
@@ -117,13 +120,19 @@ (define-command (guix-import . args)
(show-version-and-exit "guix import"))
((or ("-i" file importer args ...)
("--insert" file importer args ...))
- (let ((find-and-insert
+ (let* ((define-prefixes
+ `(,@(if (member importer '("crate"))
+ '(define)
+ '())
+ define-public))
+ (define-prefix? (cut member <> define-prefixes))
+ (find-and-insert
(lambda (expr)
(match expr
- (('define-public term _ ...)
+ (((? define-prefix? define-prefix) term _ ...)
(let ((source-properties
- (find-definition-insertion-location
- file term)))
+ (find-definition-insertion-location
+ file term #:define-prefix define-prefix)))
(if source-properties
(insert-expression source-properties expr)
(let ((port (open-file file "a")))
diff --git a/guix/utils.scm b/guix/utils.scm
index c7c23d9d5b..3f85320845 100644
--- a/guix/utils.scm
+++ b/guix/utils.scm
@@ -154,6 +154,7 @@ (define-module (guix utils)
edit-expression
delete-expression
insert-expression
+ find-definition-location
find-definition-insertion-location
filtered-port
@@ -520,24 +521,36 @@ (define (insert-expression source-properties expr)
(string-append expr "\n\n" str))))
(edit-expression source-properties insert)))
-(define (find-definition-insertion-location file term)
- "Search in FILE for a top-level public definition whose defined term
-alphabetically succeeds TERM. Return the location if found, or #f
-otherwise."
- (let ((search-term (symbol->string term)))
+(define* (find-definition-location file term
+ #:key (define-prefix 'define-public)
+ (pred string=))
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term comparing to TERM through PRED. Return the location if PRED
+returns #t, or #f otherwise."
+ (let ((search-term (symbol->string term))
+ (define-prefix? (cut eq? define-prefix <>)))
(call-with-input-file file
(lambda (port)
(do ((syntax (read-syntax port)
(read-syntax port)))
((match (syntax->datum syntax)
- (('define-public current-term _ ...)
- (string> (symbol->string current-term)
- search-term))
+ (((? define-prefix?) current-term _ ...)
+ (pred (symbol->string current-term)
+ search-term))
((? eof-object?) #t)
(_ #f))
(and (not (eof-object? syntax))
(syntax-source syntax))))))))
+(define* (find-definition-insertion-location file term
+ #:key
+ (define-prefix 'define-public))
+ "Search in FILE for a top-level definition defined by DEFINE-PREFIX with
+defined term alphabetically succeeds TERM. Return the location if found, or #f
+otherwise."
+ (find-definition-location
+ file term #:define-prefix define-prefix #:pred string>))
+
;;;
;;; Keyword arguments.
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:25 GMT) Full text and rfc822 format available.Message #221 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 11/17] scripts: import: Pass "--insert" to importers. Date: Sun, 23 Mar 2025 15:28:33 +0800
So that importers can adjust the file before inserting.
* guix/scripts/import.scm (%standard-import-options): Add ‘--file-to-insert’.
(guix-import): Pass it to importers when ‘--insert’ is set.
Change-Id: I8e7a18ee8e0f96d7fc5688a207a7a5390ad2fa30
---
guix/scripts/import.scm | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 6f6354928e..38349bff3a 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -31,6 +31,7 @@ (define-module (guix scripts import)
#:use-module (guix utils)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
+ #:use-module (srfi srfi-37)
#:use-module (ice-9 format)
#:use-module (ice-9 match)
#:export (%standard-import-options
@@ -41,7 +42,12 @@ (define-module (guix scripts import)
;;; Command line options.
;;;
-(define %standard-import-options '())
+(define %standard-import-options
+ (list
+ ;; Hidden option for importer-specific file preprocessing.
+ (option '("file-to-insert") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'file-to-insert arg result)))))
;;;
@@ -140,7 +146,10 @@ (define-command (guix-import . args)
(newline port)
(newline port)
(close-port port)))))))))
- (import-as-definitions importer args find-and-insert)))
+ (import-as-definitions importer
+ (cons (string-append "--file-to-insert=" file)
+ args)
+ find-and-insert)))
((importer args ...)
(let ((print (lambda (expr)
(leave-on-EPIPE
--
2.49.0
guix <at> cbaines.net, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, zimon.toutoune <at> gmail.com, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:26 GMT) Full text and rfc822 format available.Message #224 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 12/17] scripts: import: Skip existing definition for ‘--insert’ option. Date: Sun, 23 Mar 2025 15:28:34 +0800
* guix/scripts/import.scm (guix-import): Skip existing definition for
‘--insert’ option.
Change-Id: I2c4242669f974b263a018ab0cf56538bd7c81d06
---
guix/scripts/import.scm | 23 +++++++++++++----------
1 file changed, 13 insertions(+), 10 deletions(-)
diff --git a/guix/scripts/import.scm b/guix/scripts/import.scm
index 38349bff3a..1bddac15c6 100644
--- a/guix/scripts/import.scm
+++ b/guix/scripts/import.scm
@@ -136,16 +136,19 @@ (define-command (guix-import . args)
(lambda (expr)
(match expr
(((? define-prefix? define-prefix) term _ ...)
- (let ((source-properties
- (find-definition-insertion-location
- file term #:define-prefix define-prefix)))
- (if source-properties
- (insert-expression source-properties expr)
- (let ((port (open-file file "a")))
- (pretty-print-with-comments port expr)
- (newline port)
- (newline port)
- (close-port port)))))))))
+ ;; Skip existing definition.
+ (unless (find-definition-location
+ file term #:define-prefix define-prefix)
+ (let ((source-properties
+ (find-definition-insertion-location
+ file term #:define-prefix define-prefix)))
+ (if source-properties
+ (insert-expression source-properties expr)
+ (let ((port (open-file file "a")))
+ (pretty-print-with-comments port expr)
+ (newline port)
+ (newline port)
+ (close-port port))))))))))
(import-as-definitions importer
(cons (string-append "--file-to-insert=" file)
args)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:26 GMT) Full text and rfc822 format available.Message #227 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 13/17] import: crate: crate-name->package-name: Move to (guix build-system cargo). Date: Sun, 23 Mar 2025 15:28:35 +0800
* guix/import/crate.scm (crate-name->package-name): Move to...
* guix/build-system/cargo.scm (crate-name->package-name): ...here and export.
Change-Id: Ie9813179d6c40d159956cc1e2ae59a74dea0a42d
---
guix/build-system/cargo.scm | 6 +++++-
guix/import/crate.scm | 5 -----
2 files changed, 5 insertions(+), 6 deletions(-)
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 4486c706a1..7a07003262 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -44,7 +44,8 @@ (define-module (guix build-system cargo)
%crate-base-url
crate-url
crate-url?
- crate-uri))
+ crate-uri
+ crate-name->package-name))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -58,6 +59,9 @@ (define (crate-uri name version)
to NAME and VERSION."
(string-append crate-url name "/" version "/download"))
+(define (crate-name->package-name name)
+ (downstream-package-name "rust-" name))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 5a8caeb3e1..14e6e28c5b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -481,10 +481,6 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
-(define (crate-name->package-name name)
- (downstream-package-name "rust-" name))
-
-
;;;
;;; Updater
@@ -528,4 +524,3 @@ (define %crate-updater
(description "Updater for crates.io packages")
(pred crate-package?)
(import import-release)))
-
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:27 GMT) Full text and rfc822 format available.Message #230 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 14/17] build-system: cargo: Add ‘cargo-inputs’. Date: Sun, 23 Mar 2025 15:28:36 +0800
* gnu/packages/rust-crates.scm: New file.
* gnu/packages/rust-sources.scm: New file.
* gnu/local.mk (GNU_SYSTEM_MODULES): Regisiter them.
* guix/build-system/cargo.scm (crate-source,cargo-inputs): New procedures.
* guix/import/crate.scm: Hide ‘crate-source’ from (guix build-system cargo).
* etc/teams/rust/audit-rust-crates: New file.
* etc/teams/rust/cleanup-crates.sh: New file.
* etc/teams/rust/rust-crates.tmpl: New file.
Change-Id: I2f2d705a3e376ed3c646f31b824052a2278d4fb3
---
etc/teams/rust/audit-rust-crates | 70 ++++++++++++++++++++++++++++++++
etc/teams/rust/cleanup-crates.sh | 37 +++++++++++++++++
etc/teams/rust/rust-crates.tmpl | 42 +++++++++++++++++++
gnu/local.mk | 2 +
gnu/packages/rust-crates.scm | 42 +++++++++++++++++++
gnu/packages/rust-sources.scm | 29 +++++++++++++
guix/build-system/cargo.scm | 47 ++++++++++++++++++++-
guix/import/crate.scm | 2 +-
8 files changed, 269 insertions(+), 2 deletions(-)
create mode 100755 etc/teams/rust/audit-rust-crates
create mode 100755 etc/teams/rust/cleanup-crates.sh
create mode 100644 etc/teams/rust/rust-crates.tmpl
create mode 100644 gnu/packages/rust-crates.scm
create mode 100644 gnu/packages/rust-sources.scm
diff --git a/etc/teams/rust/audit-rust-crates b/etc/teams/rust/audit-rust-crates
new file mode 100755
index 0000000000..d5546fd1e1
--- /dev/null
+++ b/etc/teams/rust/audit-rust-crates
@@ -0,0 +1,70 @@
+#!/usr/bin/env -S gawk -f
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Efraim Flashner <efraim <at> flashner.co.il>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+# To run:
+# ./etc/teams/rust/audit-rust-crates ./path/to/file.scm
+# Prints the output of cargo-audit to the shell.
+
+# Make sure we have cargo-audit in our PATH
+BEGIN {
+ if (system("which cargo-audit 1> /dev/null"))
+ exit 1;
+ # Parse a record at a time.
+ RS = "\n\n"
+ cargoAudit = "cargo-audit audit --file -"
+}
+
+# Check the crate-source origin-only inputs
+/crate-source/ {
+ for(i=3; i <= NF-2; i++) {
+ if($i == "(crate-source") {
+ cargoLock = cargoLock "[[package]]\nname = " $(i+1) "\nversion = " $(i+2) "\n"
+ next
+ }
+ }
+}
+
+# Check the crates packaged from crates.io tarballs
+/crate-uri/ {
+ for(i=3; i <= NF; i++) {
+ if($i == "(version")
+ crateVersion = $(i+1)
+ if($i == "(crate-uri")
+ crateName = $(i+1)
+ }
+ gsub(/)/, "", crateVersion)
+ cargoLock = cargoLock "[[package]]\nname = " crateName "\nversion = " crateVersion "\n"
+}
+
+# The xxxx-cargo-input variables have a set style
+# TODO: Replace the last dash between the name and the version with a space!
+# This doesn't take into account swapping between "-" and "_" so we skip it.
+#( $2 ~ /-cargo-inputs/ ) {
+# sub(/-cargo-inputs/, "", $2)
+# gsub(/)/, "", $0)
+# gsub(/rust-/, "", $0)
+# #gensub(/([[:alpha:]])-([[:digit:]]+)/, "\\1 \\2", "g", $i)
+# print "[[package]]\nname = \"" $2 "\"\nversion = \"1.0.0\"\ndependencies = ["
+# for (i = 4; i <= NF; i++) {
+# print "\"" $i "\","
+# }
+# print "]"
+#}
+
+END { print cargoLock | cargoAudit }
diff --git a/etc/teams/rust/cleanup-crates.sh b/etc/teams/rust/cleanup-crates.sh
new file mode 100755
index 0000000000..eca37ca00c
--- /dev/null
+++ b/etc/teams/rust/cleanup-crates.sh
@@ -0,0 +1,37 @@
+#!/bin/sh
+# GNU Guix --- Functional package management for GNU
+# Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+#
+# This file is part of GNU Guix.
+#
+# GNU Guix is free software; you can redistribute it and/or modify it
+# under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 3 of the License, or (at
+# your option) any later version.
+#
+# GNU Guix is distributed in the hope that it will be useful, but
+# WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+FILE=gnu/packages/rust-crates.scm
+PATTERN='^(define rust-'
+
+grep "$PATTERN" $FILE | cut -d' ' -f2 | while IFS= read -r crate
+do
+ if [ "$(grep -wc "$crate" $FILE)" -eq 1 ]; then
+ echo "\
+(begin
+ (use-modules (guix utils))
+ (let ((source-properties
+ (find-definition-location \"$FILE\" '$crate #:define-prefix 'define)))
+ (and=> source-properties delete-expression)))" |
+ guix repl -t machine
+ fi
+done
+
+# Delete extra newlines.
+sed --in-place ':a;N;$!ba;s/\n\n\+/\n\n/g' $FILE
diff --git a/etc/teams/rust/rust-crates.tmpl b/etc/teams/rust/rust-crates.tmpl
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/etc/teams/rust/rust-crates.tmpl
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/local.mk b/gnu/local.mk
index 02de02e65f..8d8bd6ebbe 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -616,6 +616,8 @@ GNU_SYSTEM_MODULES = \
%D%/packages/rush.scm \
%D%/packages/rust.scm \
%D%/packages/rust-apps.scm \
+ %D%/packages/rust-crates.scm \
+ %D%/packages/rust-sources.scm \
%D%/packages/samba.scm \
%D%/packages/sagemath.scm \
%D%/packages/sawfish.scm \
diff --git a/gnu/packages/rust-crates.scm b/gnu/packages/rust-crates.scm
new file mode 100644
index 0000000000..98053b7151
--- /dev/null
+++ b/gnu/packages/rust-crates.scm
@@ -0,0 +1,42 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-crates)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; This file is managed by ‘guix import’. DO NOT add definitions manually.
+;;;
+
+;;;
+;;; Rust dependencies fetched from crates.io and non-workspace development
+;;; snapshots.
+;;;
+
+(define qqqq-separator 'begin-of-crates)
+
+(define ssss-separator 'end-of-crates)
+
+
+;;;
+;;; Cargo inputs.
+;;;
diff --git a/gnu/packages/rust-sources.scm b/gnu/packages/rust-sources.scm
new file mode 100644
index 0000000000..bf9b91a671
--- /dev/null
+++ b/gnu/packages/rust-sources.scm
@@ -0,0 +1,29 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu packages rust-sources)
+ #:use-module (guix gexp)
+ #:use-module (guix packages)
+ #:use-module (guix download)
+ #:use-module (guix git-download)
+ #:use-module (guix build-system cargo))
+
+;;;
+;;; Cargo workspaces and Rust dependencies requiring external inputs to
+;;; unbundle.
+;;;
diff --git a/guix/build-system/cargo.scm b/guix/build-system/cargo.scm
index 7a07003262..bbfa2f933b 100644
--- a/guix/build-system/cargo.scm
+++ b/guix/build-system/cargo.scm
@@ -31,6 +31,7 @@ (define-module (guix build-system cargo)
#:use-module (guix gexp)
#:use-module (guix monads)
#:use-module (guix packages)
+ #:use-module (guix download)
#:use-module (guix platform)
#:use-module (guix build-system)
#:use-module (guix build-system gnu)
@@ -45,7 +46,9 @@ (define-module (guix build-system cargo)
crate-url
crate-url?
crate-uri
- crate-name->package-name))
+ crate-name->package-name
+ crate-source
+ cargo-inputs))
(define %crate-base-url
(make-parameter "https://crates.io"))
@@ -62,6 +65,48 @@ (define (crate-uri name version)
(define (crate-name->package-name name)
(downstream-package-name "rust-" name))
+(define* (crate-source name version hash #:key (patches '()) (snippet #f))
+ (origin
+ (method url-fetch)
+ (uri (crate-uri name version))
+ (file-name
+ (string-append (crate-name->package-name name) "-" version ".tar.gz"))
+ (sha256 (base32 hash))
+ (modules '((guix build utils)))
+ (patches patches)
+ (snippet snippet)))
+
+(define* (cargo-inputs name #:key (crates-module '(gnu packages rust-crates))
+ (sources-module '(gnu packages rust-sources)))
+
+ "Given symbol NAME, resolve variable 'NAME-cargo-inputs', an input list, in
+CRATES-MODULE, return its copy with #f removed and symbols resolved to
+variables defined in SOURCES-MODULE if the input list exists, otherwise return
+an empty list."
+ (let loop ((inputs
+ (catch #t
+ (lambda ()
+ (module-ref (resolve-interface crates-module)
+ (symbol-append name '-cargo-inputs)))
+ (const '())))
+ (result '()))
+ (if (null? inputs)
+ result
+ (match inputs
+ ((input . rest)
+ (loop rest
+ (cond
+ ;; #f, remove it.
+ ((not input)
+ result)
+ ;; Symbol, resolve it in SOURCES-MODULE.
+ ((symbol? input)
+ (cons (module-ref (resolve-interface sources-module) input)
+ result))
+ ;; Else: keep it.
+ (else
+ (cons input result)))))))))
+
(define (default-rust target)
"Return the default Rust package."
;; Lazily resolve the binding to avoid a circular dependency.
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index 14e6e28c5b..a6f247bbae 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -27,7 +27,7 @@
(define-module (guix import crate)
#:use-module (guix base32)
- #:use-module (guix build-system cargo)
+ #:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
--
2.49.0
divya <at> subvertising.org, efraim <at> flashner.co.il, steve <at> futurile.net, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:28 GMT) Full text and rfc822 format available.Message #233 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Murilo <murilo <at> disroot.org>, Hilton Chain <hako <at> ultrarare.space>, Luis Guilherme Coelho <lgcoelho <at> disroot.org> Subject: [PATCH rust-team v3 15/17] import: crate: Add Cargo.lock parser. Date: Sun, 23 Mar 2025 15:28:37 +0800
* guix/import/crate/cargo-lock.scm: New file.
* Makefile.am (MODULES): Regisiter it.
* etc/teams.scm (rust)[#:scope]: Add it.
* guix/import/crate.scm (cargo-lock->definitions): New procedure.
Co-authored-by: Murilo <murilo <at> disroot.org>
Co-authored-by: Luis Guilherme Coelho <lgcoelho <at> disroot.org>
Change-Id: I95421e9e2ba11a671b4bc4e1323c6d31a1b012c5
---
Makefile.am | 1 +
etc/teams.scm | 1 +
guix/import/crate.scm | 78 +++++++++++++++++++++++
guix/import/crate/cargo-lock.scm | 105 +++++++++++++++++++++++++++++++
4 files changed, 185 insertions(+)
create mode 100644 guix/import/crate/cargo-lock.scm
diff --git a/Makefile.am b/Makefile.am
index 4091a52387..a04e461ea8 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -295,6 +295,7 @@ MODULES = \
guix/import/cpan.scm \
guix/import/cran.scm \
guix/import/crate.scm \
+ guix/import/crate/cargo-lock.scm \
guix/import/egg.scm \
guix/import/elm.scm \
guix/import/elpa.scm \
diff --git a/etc/teams.scm b/etc/teams.scm
index 6bddbd91fa..c4bee8d3d9 100755
--- a/etc/teams.scm
+++ b/etc/teams.scm
@@ -389,6 +389,7 @@ (define-team rust
"guix/build/cargo-utils.scm"
"guix/build-system/cargo.scm"
"guix/import/crate.scm"
+ "guix/import/crate/cargo-lock.scm"
"guix/scripts/import/crate.scm"
"tests/crate.scm")))
diff --git a/guix/import/crate.scm b/guix/import/crate.scm
index a6f247bbae..ef3a6d0b4b 100644
--- a/guix/import/crate.scm
+++ b/guix/import/crate.scm
@@ -9,6 +9,9 @@
;;; Copyright © 2023, 2024 David Elsing <david.elsing <at> posteo.net>
;;; Copyright © 2024 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
;;; Copyright © 2025 Herman Rimm <herman <at> rimm.ee>
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024-2025 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;; Copyright © 2025 Hilton Chain <hako <at> ultrarare.space>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -26,12 +29,14 @@
;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
(define-module (guix import crate)
+ #:use-module (guix base16)
#:use-module (guix base32)
#:use-module ((guix build-system cargo) #:hide (crate-source))
#:use-module (guix diagnostics)
#:use-module (gcrypt hash)
#:use-module (guix http-client)
#:use-module (guix i18n)
+ #:use-module (guix import crate cargo-lock)
#:use-module (guix import json)
#:use-module (guix import utils)
#:use-module (guix memoization)
@@ -39,9 +44,11 @@ (define-module (guix import crate)
#:use-module (guix read-print)
#:use-module (guix upstream)
#:use-module (guix utils)
+ #:use-module (guix scripts download)
#:use-module (gnu packages)
#:use-module (ice-9 match)
#:use-module (ice-9 regex)
+ #:use-module (ice-9 textual-ports)
#:use-module (json)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-2)
@@ -52,6 +59,7 @@ (define-module (guix import crate)
guix-package->crate-name
string->license
crate-recursive-import
+ cargo-lock->definitions
%crate-updater))
@@ -481,6 +489,76 @@ (define (guix-package->crate-name package)
(match parts
((name _ ...) name))))
+
+;;;
+;;; Convert ‘Cargo.lock’ to Guix sources.
+;;;
+
+(define (cargo-lock->definitions lockfile package-name)
+ "Given LOCKFILE, a 'Cargo.lock' file, convert its content into source
+definitions. Return a list of sources and 'PACKAGE-NAME-cargo-inputs', an input
+list referencing all imported sources."
+ (define (crate->definition crate)
+ (match crate
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source _)
+ ('crate-checksum checksum))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,@(if (or (string-suffix? "src" name)
+ (string-suffix? "sys" name))
+ (list (comment ";; TODO: Check bundled sources.\n" #f))
+ '())
+ (crate-source ,name ,version
+ ,(bytevector->nix-base32-string
+ (base16-string->bytevector checksum)))))
+ ;; Git snapshot.
+ (('crate
+ ('crate-name name)
+ ('crate-version version)
+ ('crate-source source))
+ (begin
+ (let* ((src (string-split source (char-set #\+ #\? #\#)))
+ (url (second src))
+ (commit (last src))
+ (version (string-append version "." (string-take commit 7)))
+ (checksum
+ (second
+ (string-split
+ (with-output-to-string
+ (lambda _
+ (guix-download "-g" url
+ (string-append "--commit=" commit))))
+ #\newline))))
+ `(define
+ ,(string->symbol
+ (string-append (crate-name->package-name name) "-" version))
+ ,(comment
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url ,url)
+ (commit ,commit)))
+ (file-name
+ (git-file-name ,(crate-name->package-name name) ,version))
+ (sha256 (base32 ,checksum)))))))
+ ;; Workspace member.
+ (else #f)))
+
+ (let ((definitions
+ (filter-map crate->definition
+ (cargo-lock-string->scm
+ (call-with-input-file lockfile get-string-all)))))
+ (values definitions
+ `(define-public
+ ,(string->symbol (string-append package-name "-cargo-inputs"))
+ (list ,@(map second definitions))))))
+
;;;
;;; Updater
diff --git a/guix/import/crate/cargo-lock.scm b/guix/import/crate/cargo-lock.scm
new file mode 100644
index 0000000000..b8a6ab20c0
--- /dev/null
+++ b/guix/import/crate/cargo-lock.scm
@@ -0,0 +1,105 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2024 Murilo <murilo <at> disroot.org>
+;;; Copyright © 2024 Luis Guilherme Coelho <lgcoelho <at> disroot.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix. If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (guix import crate cargo-lock)
+ #:use-module (ice-9 peg)
+ #:export (cargo-lock-string->scm
+
+ crate-name
+ crate-version
+ crate-source
+ crate-checksum
+ crate-dependencies
+ cargo-lock))
+
+;;;
+;;; PEG parser for ‘Cargo.lock’.
+;;;
+
+(define (cargo-lock-string->scm str)
+ (peg:tree (search-for-pattern cargo-lock str)))
+
+;; Auxiliar peg patterns
+(define-peg-pattern numeric-char body
+ (range #\0 #\9))
+
+(define-peg-pattern lowercase-char body
+ (range #\a #\z))
+
+(define-peg-pattern uppercase-char body
+ (range #\A #\Z))
+
+(define-peg-pattern alphabetic-char body
+ (or lowercase-char uppercase-char))
+
+(define-peg-pattern alphanumeric-char body
+ (or alphabetic-char numeric-char))
+
+;; name
+(define-peg-pattern crate-name all
+ (+ (or "-" alphabetic-char
+ "_" numeric-char)))
+
+;; version
+(define-peg-pattern non-negative-integer body
+ (+ numeric-char))
+
+(define-peg-pattern crate-version all
+ (and non-negative-integer "."
+ non-negative-integer "."
+ non-negative-integer
+ (? (+ (or "-" lowercase-char
+ "." uppercase-char
+ "+" numeric-char "_")))))
+
+;; source
+(define-peg-pattern crate-source all
+ (and (or "registry" "git")
+ "+https://"
+ (+ (or "/" "." "?" "=" "-" "#" "_"
+ alphanumeric-char))))
+
+;; checksum
+(define-peg-pattern crate-checksum all
+ (+ (or lowercase-char numeric-char)))
+
+;; dependency specification
+(define-peg-pattern dependency-specification all
+ (and crate-name (? (and (ignore " ") crate-version))))
+
+;; dependencies
+(define-peg-pattern crate-dependencies all
+ (and (ignore "[\n")
+ (+ (and (ignore " \"")
+ (capture dependency-specification)
+ (ignore "\",\n")))
+ (ignore "]")))
+
+;; crates
+(define-peg-pattern crate all
+ (and (ignore "[[package]]\n")
+ (ignore "name = \"") (capture crate-name) (ignore "\"\n")
+ (ignore "version = \"") (capture crate-version) (ignore "\"\n")
+ (? (and (ignore "source = \"") (capture crate-source) (ignore "\"\n")))
+ (? (and (ignore "checksum = \"") (capture crate-checksum) (ignore "\"\n")))
+ (? (ignore (and "dependencies = " crate-dependencies "\n")))))
+
+;; Cargo.lock
+(define-peg-pattern cargo-lock all
+ (+ (and (ignore "\n") crate)))
--
2.49.0
guix <at> cbaines.net, divya <at> subvertising.org, efraim <at> flashner.co.il, dev <at> jpoiret.xyz, ludo <at> gnu.org, othacehe <at> gnu.org, maxim.cournoyer <at> gmail.com, zimon.toutoune <at> gmail.com, steve <at> futurile.net, me <at> tobias.gr, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:30:28 GMT) Full text and rfc822 format available.Message #236 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 16/17] import: crate: Add ‘--lockfile’ option. Date: Sun, 23 Mar 2025 15:28:38 +0800
* guix/scripts/import/crate.scm (%options): Add ‘--lockfile’ option.
* guix/scripts/import/crate.scm (show-help): Add it.
(guix-import-crate): Use it.
* doc/guix.texi (Invoking guix import): Document it.
* tests/crate.scm (temp-file): New variable.
("crate-lockfile-import"): New test.
Change-Id: I291478e04adf9f2df0bf216425a5e8aeba0bedd9
---
doc/guix.texi | 14 ++++++
guix/scripts/import/crate.scm | 58 +++++++++++++++++++----
tests/crate.scm | 88 +++++++++++++++++++++++++++++++++++
3 files changed, 150 insertions(+), 10 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 7fa986b4b9..218c2ba630 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -14703,6 +14703,20 @@ Invoking guix import
If a crate dependency is not (yet) packaged, make the corresponding
input in @code{#:cargo-inputs} or @code{#:cargo-development-inputs} into
a comment.
+@item --lockfile=@var{file}
+@itemx -f @var{file}
+When @option{--lockfile} is specified, the importer will ignore other options
+and won't output package definitions, instead generating source definitions for
+all dependencies in @var{file}, a @file{Cargo.lock} file. For example:
+
+@example
+guix import crate --lockfile=/path/to/Cargo.lock my-package
+@end example
+
+generates sources from @file{/path/to/Cargo.lock} and a list
+@code{my-package-cargo-inputs} referencing these sources. The generated list is
+intended for the package's @code{inputs}, replacing @code{#:cargo-inputs} and
+@code{#:cargo-development-inputs}.
@end table
@item elm
diff --git a/guix/scripts/import/crate.scm b/guix/scripts/import/crate.scm
index 723cbb3665..3b536e135a 100644
--- a/guix/scripts/import/crate.scm
+++ b/guix/scripts/import/crate.scm
@@ -25,11 +25,13 @@
(define-module (guix scripts import crate)
#:use-module (guix ui)
#:use-module (guix utils)
+ #:use-module (guix read-print)
#:use-module (guix scripts)
#:use-module (guix import crate)
#:use-module (guix scripts import)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-11)
+ #:use-module (srfi srfi-26)
#:use-module (srfi srfi-37)
#:use-module (ice-9 match)
#:use-module (ice-9 format)
@@ -60,6 +62,9 @@ (define (show-help)
sufficient package exists for it"))
(newline)
(display (G_ "
+ -f, --lockfile=FILE import dependencies from FILE, a 'Cargo.lock' file"))
+ (newline)
+ (display (G_ "
-h, --help display this help and exit"))
(display (G_ "
-V, --version display version information and exit"))
@@ -87,6 +92,9 @@ (define %options
(option '("mark-missing") #f #f
(lambda (opt name arg result)
(alist-cons 'mark-missing #t result)))
+ (option '(#\f "lockfile") #f #t
+ (lambda (opt name arg result)
+ (alist-cons 'lockfile arg result)))
%standard-import-options))
@@ -101,6 +109,8 @@ (define (guix-import-crate . args)
#:build-options? #f))
(let* ((opts (parse-options))
+ (lockfile (assoc-ref opts 'lockfile))
+ (file-to-insert (assoc-ref opts 'file-to-insert))
(args (filter-map (match-lambda
(('argument . value)
value)
@@ -111,16 +121,44 @@ (define (guix-import-crate . args)
(define-values (name version)
(package-name->name+version spec))
- (match (if (assoc-ref opts 'recursive)
- (crate-recursive-import
- name #:version version
- #:recursive-dev-dependencies?
- (assoc-ref opts 'recursive-dev-dependencies)
- #:allow-yanked? (assoc-ref opts 'allow-yanked))
- (crate->guix-package
- name #:version version #:include-dev-deps? #t
- #:allow-yanked? (assoc-ref opts 'allow-yanked)
- #:mark-missing? (assoc-ref opts 'mark-missing)))
+ (match (cond
+ ((and=> lockfile
+ (lambda (file)
+ (or (file-exists? file)
+ (leave (G_ "file '~a' does not exist~%") file))))
+ (let-values (((crate-definitions input-list)
+ (cargo-lock->definitions lockfile name)))
+ (if file-to-insert
+ ;; Adjusted from ‘--insert’ option of guix-import.
+ (let ((term (second input-list)))
+ (begin
+ ;; Remove existing input list definition.
+ (and=> (find-definition-location file-to-insert term)
+ delete-expression)
+ ;; Insert input list alphabetically.
+ (or (and=> (find-definition-insertion-location
+ file-to-insert term)
+ (cut insert-expression <> input-list))
+ (let ((port (open-file file-to-insert "a")))
+ (newline port)
+ (pretty-print-with-comments port input-list)
+ (newline port)
+ (newline port)
+ (close-port port))))
+ crate-definitions)
+ `(,@crate-definitions
+ ,input-list))))
+ ((assoc-ref opts 'recursive)
+ (crate-recursive-import
+ name #:version version
+ #:recursive-dev-dependencies?
+ (assoc-ref opts 'recursive-dev-dependencies)
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)))
+ (else
+ (crate->guix-package
+ name #:version version #:include-dev-deps? #t
+ #:allow-yanked? (assoc-ref opts 'allow-yanked)
+ #:mark-missing? (assoc-ref opts 'mark-missing))))
((or #f '())
(leave (G_ "failed to download meta-data for package '~a'~%")
(if version
diff --git a/tests/crate.scm b/tests/crate.scm
index d0dc779cd2..9b7066c3b1 100644
--- a/tests/crate.scm
+++ b/tests/crate.scm
@@ -34,6 +34,7 @@ (define-module (test-crate)
#:use-module (gnu packages)
#:use-module (ice-9 iconv)
#:use-module (ice-9 match)
+ #:use-module (srfi srfi-11)
#:use-module (srfi srfi-64))
@@ -476,6 +477,9 @@ (define rust-leaf-bob-3.0.2-yanked
(description #f)
(license #f)))
+(define temp-file
+ (string-append "t-crate-" (number->string (getpid))))
+
(test-begin "crate")
@@ -1178,4 +1182,88 @@ (define rust-leaf-bob-3.0.2-yanked
(x
(pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))))
+
+(test-assert "crate-lockfile-import"
+ (begin
+ (call-with-output-file temp-file
+ (lambda (port)
+ (display "\
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 3
+
+[[package]]
+name = \"adler2\"
+version = \"2.0.0\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"512761e0bb2578dd7380c6baaa0f4ce03e84f95e960231d1dec8bf4d7d6e2627\"
+
+[[package]]
+name = \"aho-corasick\"
+version = \"1.1.3\"
+source = \"registry+https://github.com/rust-lang/crates.io-index\"
+checksum = \"8e60d3430d3a69478ad0993f19238d2df97c507009a52b3c10addcd7f6bcb916\"
+dependencies = [
+ \"memchr\",
+]
+
+[[package]]
+name = \"smithay\"
+version = \"0.4.0\"
+source = \"git+https://github.com/Smithay/smithay.git?rev=\
+0cd3345c59f7cb139521f267956a1a4e33248393#\
+0cd3345c59f7cb139521f267956a1a4e33248393\"
+dependencies = [
+ \"appendlist\",
+]
+
+[[package]]
+name = \"test\"
+version = \"25.2.0\"\n" port)))
+ (mock
+ ((guix scripts download) guix-download
+ (lambda _
+ (format #t "~a~%~a~%"
+ "/gnu/store/in056fyrz6nvy3jpxrxglgj30g0lwniv-smithay-0cd3345"
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))
+ (let-values
+ (((crates-definitions input-list)
+ (cargo-lock->definitions temp-file "test")))
+ (and
+ (match crates-definitions
+ (((define 'rust-adler2-2.0.0
+ (crate-source
+ "adler2" "2.0.0"
+ "09r6drylvgy8vv8k20lnbvwq8gp09h7smfn6h1rxsy15pgh629si"))
+ (define 'rust-aho-corasick-1.1.3
+ (crate-source
+ "aho-corasick" "1.1.3"
+ "05mrpkvdgp5d20y2p989f187ry9diliijgwrs254fs9s1m1x6q4f"))
+ (define 'rust-smithay-0.4.0.0cd3345
+ ($ <comment>
+ ";; TODO: Define standalone package if this is a workspace.\n"
+ #f)
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/Smithay/smithay.git")
+ (commit "0cd3345c59f7cb139521f267956a1a4e33248393")))
+ (file-name (git-file-name "rust-smithay" "0.4.0.0cd3345"))
+ (sha256
+ (base32
+ "191h87bpzg0l1ihfb4hmx00b86pfb5mwwc6s8i49al0vigc14l37")))))
+ #t)
+ (x
+ (pk 'fail (pretty-print-with-comments (current-output-port) x) #f)))
+ (match input-list
+ ((define-public 'test-cargo-inputs
+ (list rust-adler2-2.0.0
+ rust-aho-corasick-1.1.3
+ rust-smithay-0.4.0.0cd3345))
+ #t)
+ (x
+ (pk 'fail x #f))))))))
+
(test-end "crate")
+
+(false-if-exception (delete-file temp-file))
--
2.49.0
ludo <at> gnu.org, maxim.cournoyer <at> gmail.com, guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sun, 23 Mar 2025 07:33:02 GMT) Full text and rfc822 format available.Message #239 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Hilton Chain <hako <at> ultrarare.space> Subject: [PATCH rust-team v3 17/17] doc: Document lockfile importer based Rust packaging workflow. Date: Sun, 23 Mar 2025 15:28:39 +0800
* doc/contributing.texi (Packaging Guidelines)[Rust Crates]: Update
documentation.
* doc/guix-cookbook.texi (Packaging)[Packaging Workflow]: New section.
Change-Id: Ic0c6378cf5f5df97d6f8bdd040b486be62c7bddc
---
doc/contributing.texi | 91 ++++++++---
doc/guix-cookbook.texi | 352 +++++++++++++++++++++++++++++++++++++++++
2 files changed, 418 insertions(+), 25 deletions(-)
diff --git a/doc/contributing.texi b/doc/contributing.texi
index ab4f30d54b..17f22390d5 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1600,34 +1600,75 @@ Rust Crates
@subsection Rust Crates
@cindex rust
-Rust programs standing for themselves are named as any other package, using the
-lowercase upstream name.
+Rust programs (binary crates) and dependencies (library crates) are treated
+separately. We put our main efforts into programs and only package Rust
+dependencies as sources, utilizing automation with a manual focus on unbundling
+vendored dependencies. The following paragraphs will explain them and give
+several examples.
-To prevent namespace collisions we prefix all other Rust packages with the
-@code{rust-} prefix. The name should be changed to lowercase as appropriate and
-dashes should remain in place.
+Rust programs are treated like any other package and named using the lowercase
+upstream name. When using the Cargo build system (@pxref{Build Systems,
+@code{cargo-build-system}}), Rust programs should have @code{#:install-source?}
+argument set to @code{#f}, as this argument only makes sense for dependencies.
+When the package source is a Cargo workspace, @code{#:cargo-install-paths} must
+be set to enable relevant support.
+
+Rust dependencies are managed in two modules:
+
+@enumerate
+@item
+@code{(gnu packages rust-crates)}, storing source definitions imported from Rust
+programs' @file{Cargo.lock} via the lockfile importer (@pxref{Invoking guix
+import, @code{crate}, @code{--lockfile=@var{file}}}).
+
+Imported definitions must be checked and have vendored sources unbundled before
+being contributed to Guix. This module is managed by the Rust team
+(@pxref{Teams}).
+
+@item
+@code{(gnu packages rust-sources)}, storing more complex definitions that need
+to be full packages. This includes Rust dependencies requiring external inputs
+to unbundle and Cargo workspaces.
+
+These dependencies should have the @code{#:skip-build?} argument set to
+@code{#t}. For Cargo workspaces, @code{#:cargo-package-crates} must be set.
+
+Since they are added manually, the following naming convention applies:
+
+To prevent namespace collisions they are named with @code{rust-} prefix. The
+name should be changed to lowercase as appropriate and dashes should remain in
+place.
In the rust ecosystem it is common for multiple incompatible versions of a
-package to be used at any given time, so all package definitions should have a
-versioned suffix. The versioned suffix is the left-most non-zero digit (and
-any leading zeros, of course). This follows the ``caret'' version scheme
-intended by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
-
-Because of the difficulty in reusing rust packages as pre-compiled inputs for
-other packages the Cargo build system (@pxref{Build Systems,
-@code{cargo-build-system}}) presents the @code{#:cargo-inputs} and
-@code{cargo-development-inputs} keywords as build system arguments. It would be
-helpful to think of these as similar to @code{propagated-inputs} and
-@code{native-inputs}. Rust @code{dependencies} and @code{build-dependencies}
-should go in @code{#:cargo-inputs}, and @code{dev-dependencies} should go in
-@code{#:cargo-development-inputs}. If a Rust package links to other libraries
-then the standard placement in @code{inputs} and the like should be used.
-
-Care should be taken to ensure the correct version of dependencies are used; to
-this end we try to refrain from skipping the tests or using @code{#:skip-build?}
-when possible. Of course this is not always possible, as the package may be
-developed for a different Operating System, depend on features from the Nightly
-Rust compiler, or the test suite may have atrophied since it was released.
+package to be used at any given time, so all dependencies should have a
+versioned suffix. The versioned suffix is the left-most non-zero digit (and any
+leading zeros, of course). This follows the ``caret'' version scheme intended
+by Cargo. Examples@: @code{rust-clap-2}, @code{rust-rand-0.6}.
+
+In practice we are usually packaging development snapshots of Rust dependencies
+specifically for some Rust programs, and can't simply identity them by version.
+In this case we can use a @code{for-@var{program}} suffix, for example,
+@code{rust-pipewire-for-niri} and @code{rust-pubgrub-for-uv}.
+@end enumerate
+
+Rust dependencies are not referenced directly. @code{(guix build-sytem cargo)}
+provides a @code{cargo-inputs} procedure to create an input list, in combination
+with the lockfile importer.
+
+@cindex cargo-inputs
+@deffn {Procedure} cargo-inputs @var{name} @
+ [#:crates-module '(gnu packages rust-crates)] @
+ [#:sources-module '(gnu packages rust-sources)]
+Given symbol @code{'@var{name}}, resolve variable
+@code{@var{name}-cargo-inputs}, an input list, in @var{crates-module}, return
+its copy with @code{#f} removed and symbols resolved to variables defined in
+@var{sources-module} if the input list exists, otherwise return an empty list.
+
+@var{name} must be consistent with the one used in lockfile importer invocation.
+@end deffn
+
+For a more detailed packaging workflow, @pxref{Packaging Rust Crates,,,
+guix-cookbook, GNU Guix Cookbook}.
@node Elm Packages
diff --git a/doc/guix-cookbook.texi b/doc/guix-cookbook.texi
index d9b98a2ab3..1f0e5eee54 100644
--- a/doc/guix-cookbook.texi
+++ b/doc/guix-cookbook.texi
@@ -103,6 +103,7 @@ Top
Packaging
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
Packaging Tutorial
@@ -127,6 +128,16 @@ Top
* Automatic update::
* Inheritance::
+Packaging Workflows
+
+* Packaging Rust Crates::
+
+Packaging Rust Crates
+
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+
System Configuration
* Auto-Login to a Specific TTY:: Automatically Login a User to a Specific TTY
@@ -514,6 +525,7 @@ Packaging
@menu
* Packaging Tutorial:: A tutorial on how to add packages to Guix.
+* Packaging Workflows:: Real life examples on working with specific build systems.
@end menu
@node Packaging Tutorial
@@ -1593,6 +1605,346 @@ References
@uref{https://www.gnu.org/software/guix/guix-ghm-andreas-20130823.pdf, ``GNU Guix: Package without a scheme!''}, by Andreas Enge
@end itemize
+@node Packaging Workflows
+@section Packaging Workflows
+
+The following sections are real life examples on working with specific build
+systems, serving as extensions to the concise packaging guidelines
+(@pxref{Packaging Guidelines,,, guix, GNU Guix Reference Manual}).
+
+@menu
+* Packaging Rust Crates::
+@end menu
+
+@node Packaging Rust Crates
+@subsection Packaging Rust Crates
+
+In preparation, add the following packages to our environment:
+
+@example
+$ guix shell rust rust:cargo cargo-audit cargo-license
+@end example
+
+@menu
+* Common Workflow for Packaging Rust Crates::
+* Development Snapshots and Cargo Workspaces::
+* Rust Programs With Mixed Build Systems::
+@end menu
+
+@node Common Workflow for Packaging Rust Crates
+@subsubsection Common Workflow for Packaging Rust Crates
+
+In this example, we'll package @code{cargo-audit}, which is published on the
+@uref{https://crates.io, crates.io} Rust package repository. All its
+dependencies are on crates.io as well.
+
+@enumerate
+@item
+Since @code{cargo-audit} is available on crates.io, We can generate a draft
+definition via the crates.io importer (@pxref{Invoking guix import,,, guix, GNU
+Guix Reference Manual}). In the end we'll have the following definiton:
+
+@lisp
+(define-public cargo-audit
+ (package
+ (name "cargo-audit")
+ (version "0.21.2")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (crate-uri "cargo-audit" version))
+ (file-name (string-append name "-" version ".tar.gz"))
+ (sha256
+ (base32 "1a00yqpckkw86zh2hg7ra82c5fx0ird5766dyynimbvqiwg2ps0n"))))
+ (build-system cargo-build-system)
+ (arguments (list #:install-source? #f))
+ (inputs (cargo-inputs 'cargo-audit))
+ (home-page "https://rustsec.org/")
+ (synopsis "Audit Cargo.lock for crates with security vulnerabilities")
+ (description
+ "This package provides a Cargo subcommand, @@command@{cargo audit@}, to
+audit @@file@{Cargo.lock@} for crates with security vulnerabilities.")
+ (license (list license:asl2.0 license:expat))))
+@end lisp
+
+The symbol used in @code{cargo-inputs}, @code{'cargo-audit} here, must be a
+unique identifier, usually matching the variable name of the package.
+
+@item
+Unpack package source and navigate to the unpacked directory, then execute the
+following commands:
+
+@example
+$ cargo generate-lockfile
+$ cargo audit
+$ cargo license
+@end example
+
+@command{cargo generate-lockfile} updates dependencies to compatible versions,
+@command{cargo audit} checks known vulnerabilities and @command{cargo license}
+checks licenses of all dependencies.
+
+We must have an acceptable output of @command{cargo audit} and ensure all
+dependencies are licensed with our supported licenses (@pxref{Defining
+Packages,,, guix, GNU Guix Reference Manual}).
+
+@item
+Import dependencies from previously generated lockfile:
+
+@example
+$ guix import --insert=gnu/packages/rust-crates.scm \
+ crate --lockfile=/path/to/Cargo.lock cargo-audit
+@end example
+
+@code{cargo-audit} used here must be consistent with the identifier used for
+@code{cargo-inputs}.
+
+At this stage, package @code{cargo-audit} is buildable.
+
+@item
+Finally we'll unbundle vendored sources. The lockfile importer inserts
+@code{TODO:} comments to dependencies with high probability of bundled sources.
+@code{cargo-build-system} also performs additional check in its
+@code{check-for-pregenerated-files} phase:
+
+@example
+$ ./pre-inst-env guix build cargo-audit
+@dots{}
+starting phase `check-for-pregenerated-files'
+Searching for binary files...
+./guix-vendor/rust-addr2line-0.21.0.tar.gz/rustfmt.toml
+./guix-vendor/rust-arc-swap-1.7.1.tar.gz/rustfmt.toml
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/dictionary-rust-other
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/lib.rs.zst
+./guix-vendor/rust-async-compression-0.4.21.tar.gz/tests/artifacts/long-window-size-lib.rs.zst
+@dots{}
+@end example
+
+Although dependencies in @code{(gnu packages rust-crates)} are not exported, we
+can still select them via the Guix command-line interface through expression:
+
+@example
+$ guix build --expression='(@@@@ (gnu packages rust-crates) rust-ring-0.17.14)'
+@end example
+
+For most dependencies, a snippet is sufficient:
+
+@lisp
+(define rust-bzip2-sys-0.1.13+1.0.8
+ (crate-source "bzip2-sys" "0.1.13+1.0.8"
+ "056c39pgjh4272bdslv445f5ry64xvb0f7nph3z7860ln8rzynr2"
+ #:snippet
+ '(begin
+ (delete-file-recursively "bzip2-1.0.8")
+ (delete-file "build.rs")
+ ;; Inspired by Debian's patch.
+ (with-output-to-file "build.rs"
+ (lambda _
+ (format #t "fn main() @{~@@
+ println!(\"cargo:rustc-link-lib=bz2\");~@@
+ @}~%"))))))
+@end lisp
+
+In a more complex case, where unbundling one dependency requires other packages,
+we should package the dependency in @code{(gnu packages rust-sources)} first and
+reference it by a symbol in the imported definition.
+
+For example we have defined a @code{rust-ring-0.17} in @code{(gnu packages
+rust-sources)}, then the imported definition in @code{(gnu packages
+rust-crates)} should be modified to a matching symbol.
+
+@lisp
+(define rust-ring-0.17.14 'rust-ring-0.17)
+@end lisp
+
+When one dependency can be safely removed, modify it to @code{#f}.
+
+@lisp
+(define rust-openssl-src-300.4.2+3.4.1 #f)
+@end lisp
+@end enumerate
+
+@node Development Snapshots and Cargo Workspaces
+@subsubsection Development Snapshots and Cargo Workspaces
+
+In this example, we'll package @code{niri}, which depends on development
+snapshots (also Cargo workspaces here).
+
+As we can't ensure compatibility of a development snapshot, before executing
+@command{cargo generate-lockfile}, we should modify @file{Cargo.toml} to pin it
+to a known working revision.
+
+To use our packaged development snapshots, it's also necessary to modify
+@file{Cargo.toml} in a build phase, with a package-specific substitution
+pattern.
+
+@lisp
+(define-public niri
+ (package
+ (name "niri")
+ (version "25.02")
+ (source (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://github.com/YaLTeR/niri")
+ (commit (string-append "v" version))))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32
+ "0vzskaalcz6pcml687n54adjddzgf5r07gggc4fhfsa08h1wfd4r"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:install-source? #f
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("# version =.*")
+ "version = \"*\"")
+ (("git.*optional")
+ "version = \"*\", optional")
+ (("^git = .*")
+ "")))))))
+ (native-inputs
+ (list pkg-config))
+ (inputs
+ (cons* clang
+ libdisplay-info
+ libinput-minimal
+ libseat
+ libxkbcommon
+ mesa
+ pango
+ pipewire
+ wayland
+ (cargo-inputs 'niri)))
+ (home-page "https://github.com/YaLTeR/niri")
+ (synopsis "Scrollable-tiling Wayland compositor")
+ (description
+ "Niri is a scrollable-tiling Wayland compositor which arranges windows in a
+scrollable format. It is considered stable for daily use and performs most
+functions expected of a Wayland compositor.")
+ (license license:gpl3+)))
+@end lisp
+
+@code{niri} has Cargo workspace dependencies. When packaging a Cargo workspace,
+argument @code{#:cargo-package-crates} is required.
+
+@lisp
+(define-public rust-pipewire-for-niri
+ (let ((commit "fd3d8f7861a29c2eeaa4c393402e013578bb36d9")
+ (revision "0"))
+ (package
+ (name "rust-pipewire")
+ (version (git-version "0.8.0" revision commit))
+ (source
+ (origin
+ (method git-fetch)
+ (uri (git-reference
+ (url "https://gitlab.freedesktop.org/pipewire/pipewire-rs.git")
+ (commit commit)))
+ (file-name (git-file-name name version))
+ (sha256
+ (base32 "1hzyhz7xg0mz8a5y9j6yil513p1m610q3j9pzf6q55vdh5mcn79v"))))
+ (build-system cargo-build-system)
+ (arguments
+ (list #:skip-build? #t
+ #:cargo-package-crates
+ ''("libspa-sys" "libspa" "pipewire-sys" "pipewire")))
+ (inputs (cargo-inputs 'rust-pipewire-for-niri))
+ (home-page "https://pipewire.org/")
+ (synopsis "Rust bindings for PipeWire")
+ (description "This package provides Rust bindings for PipeWire.")
+ (license license:expat))))
+@end lisp
+
+Don't forget to modify all workspace members in @code{(gnu packages
+rust-crates)}:
+
+@lisp
+(define rust-pipewire-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-pipewire-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@dots{}
+(define rust-libspa-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+(define rust-libspa-sys-0.8.0.fd3d8f7 'rust-pipewire-for-niri)
+@end lisp
+
+@node Rust Programs With Mixed Build Systems
+@subsubsection Rust Programs With Mixed Build Systems
+
+In this example, we'll package @code{fish}, which combines two build systems.
+
+When building Rust packages in other build systems, we need to add @code{rust}
+and @code{rust:cargo} to @code{native-inputs}, import and use modules from both
+build systems, and apply necessary build phases from @code{cargo-build-system}.
+
+@lisp
+(define-public fish
+ (package
+ (name "fish")
+ (version "4.0.0")
+ (source
+ (origin
+ (method url-fetch)
+ (uri (string-append "https://github.com/fish-shell/fish-shell/"
+ "releases/download/" version "/"
+ "fish-" version ".tar.xz"))
+ (sha256
+ (base32 "1wv9kjwg6ax8m2f85i58l9f9cndshn1f15n8skc68w1mf3cmpnig"))))
+ (build-system cmake-build-system)
+ (inputs
+ (cons* fish-foreign-env ncurses pcre2
+ python ;for fish_config and manpage completions
+ (cargo-inputs 'fish)))
+ (native-inputs
+ (list doxygen groff ;for 'fish --help'
+ pkg-config
+ procps ;for the test suite
+ rust
+ `(,rust "cargo")))
+ (arguments
+ (list #:imported-modules
+ (append %cargo-build-system-modules
+ %cmake-build-system-modules)
+ #:modules
+ (((guix build cargo-build-system) #:prefix cargo:)
+ (guix build cmake-build-system)
+ (guix build utils))
+ #:phases
+ #~(modify-phases %standard-phases
+ (add-after 'unpack 'use-guix-vendored-dependencies
+ (lambda _
+ (substitute* "Cargo.toml"
+ (("git.*tag.*,")
+ "version = \"*\","))))
+ (add-after 'unpack 'prepare-cargo-build-system
+ (lambda args
+ (for-each
+ (lambda (phase)
+ (format #t "Running cargo phase: ~a~%" phase)
+ (apply (assoc-ref cargo:%standard-phases phase)
+ args))
+ '(unpack-rust-crates
+ configure
+ check-for-pregenerated-files
+ patch-cargo-checksums)))))))
+ (synopsis "The friendly interactive shell")
+ (description
+ "Fish (friendly interactive shell) is a shell focused on interactive use,
+discoverability, and friendliness. Fish has very user-friendly and powerful
+tab-completion, including descriptions of every completion, completion of
+strings with wildcards, and many completions for specific commands. It also
+has extensive and discoverable help. A special @@command@{help@} command gives
+access to all the fish documentation in your web browser. Other features
+include smart terminal handling based on terminfo, an easy to search history,
+and syntax highlighting.")
+ (home-page "https://fishshell.com/")
+ (license license:gpl2)))
+@end lisp
+
+
@c *********************************************************************
@node System Configuration
@chapter System Configuration
--
2.49.0
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Tue, 25 Mar 2025 09:24:02 GMT) Full text and rfc822 format available.Message #242 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Ludovic Courtès <ludo <at> gnu.org> To: Hilton Chain <hako <at> ultrarare.space> Cc: 77093 <at> debbugs.gnu.org, Maxim Cournoyer <maxim.cournoyer <at> gmail.com> Subject: Re: [bug#77093] [PATCH rust-team 00/18] New Rust packaging workflow based on lockfile importer. Date: Tue, 25 Mar 2025 10:22:51 +0100
Hi Hilton, Hilton Chain <hako <at> ultrarare.space> skribis: > This series: > * integrates cargo2guix (https://git.sr.ht/~look/cargo2guix) as a crate importer > option, ‘-f / --lockfile=’. > > * adds Cargo workspace support to cargo-build-system, via two new arguments: > #:cargo-package-crates and #:cargo-install-paths. > > * introduces a new Rust packaging workflow, as described in contributing.texi > change. I hadn’t noticed this patch series but I’m really happy you’re working on it, I think it can help make Rust packaging more sustainable. 👍 Ludo’.
guix-patches <at> gnu.org:bug#77093; Package guix-patches.
(Sat, 05 Apr 2025 02:58:01 GMT) Full text and rfc822 format available.Message #245 received at 77093 <at> debbugs.gnu.org (full text, mbox):
From: Hilton Chain <hako <at> ultrarare.space> To: 77093 <at> debbugs.gnu.org Cc: Ludovic Courtès <ludo <at> gnu.org>, Maxim Cournoyer <maxim.cournoyer <at> gmail.com>, Steve George <steve <at> futurile.net>, Efraim Flashner <efraim <at> flashner.co.il>, Divya Ranjan Pattanaik <divya <at> subvertising.org> Subject: Re: [PATCH rust-team v3 00/17] New Rust packaging workflow based on lockfile importer. Date: Sat, 05 Apr 2025 10:53:33 +0800
On Sun, 23 Mar 2025 15:23:04 +0800,
Hilton Chain wrote:
>
> V2 -> V3:
> * Add Efraim's etc/teams/rust/audit-rust-crates script.
> * Add (guix import crate cargo-lock) to Rust team's scope.
> * ‘definer’ -> ‘define-prefix’
> * Adjust documentation and move examples to Guix Cookbook.
> * Document ‘cargo-inputs’ and remove mentioning of files under etc/teams/rust.
>
> Diff inserted at end.
>
> Efraim Flashner (1):
> build/cargo: Don't try to unpack sanity-check.py.
>
> Hilton Chain (16):
> build/cargo: Pass ‘--offline’ to cargo.
> build/cargo: Print out all non-empty binary files.
> build-system: cargo: Support packaging Cargo workspace.
> build-system: cargo: Support installing Cargo workspace.
> build/cargo: Set default value of arguments for build phases.
> build/cargo: Support non-workspace directory source inputs.
> scripts: import: Document argument for ‘--insert’ option in help
> message.
> scripts: import: Add two newlines for ‘--insert’ option.
> scripts: import: Support expressions defined by 'define.
> scripts: import: Pass "--insert" to importers.
> scripts: import: Skip existing definition for ‘--insert’ option.
> import: crate: crate-name->package-name: Move to (guix build-system
> cargo).
> build-system: cargo: Add ‘cargo-inputs’.
> import: crate: Add Cargo.lock parser.
> import: crate: Add ‘--lockfile’ option.
> doc: Document lockfile importer based Rust packaging workflow.
>
> Makefile.am | 1 +
> doc/contributing.texi | 91 +++++---
> doc/guix-cookbook.texi | 352 ++++++++++++++++++++++++++++++
> doc/guix.texi | 24 ++
> etc/teams.scm | 1 +
> etc/teams/rust/audit-rust-crates | 70 ++++++
> etc/teams/rust/cleanup-crates.sh | 37 ++++
> etc/teams/rust/rust-crates.tmpl | 42 ++++
> gnu/local.mk | 2 +
> gnu/packages/rust-crates.scm | 42 ++++
> gnu/packages/rust-sources.scm | 29 +++
> guix/build-system/cargo.scm | 59 ++++-
> guix/build/cargo-build-system.scm | 150 +++++++++----
> guix/import/crate.scm | 81 ++++++-
> guix/import/crate/cargo-lock.scm | 105 +++++++++
> guix/scripts/import.scm | 54 +++--
> guix/scripts/import/crate.scm | 58 ++++-
> guix/utils.scm | 29 ++-
> tests/crate.scm | 88 ++++++++
> 19 files changed, 1209 insertions(+), 106 deletions(-)
> create mode 100755 etc/teams/rust/audit-rust-crates
> create mode 100755 etc/teams/rust/cleanup-crates.sh
> create mode 100644 etc/teams/rust/rust-crates.tmpl
> create mode 100644 gnu/packages/rust-crates.scm
> create mode 100644 gnu/packages/rust-sources.scm
> create mode 100644 guix/import/crate/cargo-lock.scm
I'll apply this series (except the documentation change, which will be tracked
here separately) to rust-team when I have checked an initial set of packages.
In case it's useful, I'm using this script to get a list of unstaged crates to
look through:
--8<---------------cut here---------------start------------->8---
for i in $(git diff gnu/packages/rust-crates.scm | grep -E '^\+\(define rust-' | cut -f2 -d' ')
do
guix build -e "(@@ (gnu packages rust-crates) $i)" -v0
done
--8<---------------cut here---------------end--------------->8---
Thanks
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.