Ludovic Courtès <ludo@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 74273) by debbugs.gnu.org; 24 Nov 2024 14:59:43 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Nov 24 09:59:43 2024
Received: from localhost ([127.0.0.1]:35604 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1tFE5O-0001mY-4A
for submit <at> debbugs.gnu.org; Sun, 24 Nov 2024 09:59:43 -0500
Received: from mail-pl1-f181.google.com ([209.85.214.181]:42304)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <maxim.cournoyer@HIDDEN>) id 1tFE5K-0001m0-Rp
for 74273 <at> debbugs.gnu.org; Sun, 24 Nov 2024 09:59:40 -0500
Received: by mail-pl1-f181.google.com with SMTP id
d9443c01a7336-2120f9ec28eso28044095ad.1
for <74273 <at> debbugs.gnu.org>; Sun, 24 Nov 2024 06:59:38 -0800 (PST)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1732460313; x=1733065113; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=sE/BWw91Yd2wGC6OpSCjsnNXkUUR5IO9YyebFR/0G4I=;
b=OlMkcruCG06snKEx0sOoWL7cWXMP0Ekb8Aq9e+qJj1YtursP/cSsd8hk2q4YYcBhep
ZvIgeSOy1TRt5vTkGMHrk2hTbsMpsgEzF1RK/M5HqzaOkDS2+UDNCPaIzzWnuoJ1K04g
c3s5PIYO1l1uViYg+3t+inPggAuHXJqFt18YNc7Dqr/lR5cal88Aw79h6UvwFHClD/bE
nMfvNuV16V2ajyrw7B4EEWnnMFEyibxndfDm6XGMhJIOQ7MobuSnDFzzRiwCSKGgXPr0
qm6Tlk8MfrVlmPxJVXuHWu3eBtrhp1QwaFVRDDNeSwEY7Sw81zNXavLQp2w6OCw/qyHN
NsJA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1732460313; x=1733065113;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:x-gm-message-state:from
:to:cc:subject:date:message-id:reply-to;
bh=sE/BWw91Yd2wGC6OpSCjsnNXkUUR5IO9YyebFR/0G4I=;
b=cJ8gx55az0QekvBwFhA/T5JDBhqp2uUo0lamFGKj2MkI1IHByT/NMrt3vCZ1gOFTAT
CEIPob7tT96OaAQvZQyDTOB3lTJr8fAACEVZ+bDuO0blkWkpuaY0YrA+EXiTXOJjlCrd
CJNQa2An8rYj1J85iwlqUgm9TxjojWAEFTxJlcssfMm/o7jTnCAGjLASv4cwVqeG26oN
zKwDkaNWADTSviI2ac8gPJM06xCi0jgjBfFb59r3Kr2H45yowg2tahYOnZ8DUveDSq8r
WDZVJj/Uh8nZ8aWk+q1+wUse1bS13RpE/wKa63LxOVj3A0K8kJf1ANnoQa9Dy76fFBwJ
MQEg==
X-Gm-Message-State: AOJu0YwwGu+9tVq8BxSzi3+pfI8DaZZDJATnXyYhpSe+XRNWx9TahLVh
RxD/T+j6dUBg+HRbOMgU1bYi9ATdnNg2guErApM+vO9/Q5ipO37w
X-Gm-Gg: ASbGnctT5tH03c68nVii9aUGhYp/CuKM5ccUvF08L5HuMdaAorETm6940Ko+McbPvFY
IOmTxSJXsFEks/MQTRn0eSzPSvtuU++MKwCvPYA9MTTmqtaWkRDdskD1qcIb+wFHv+jkm0wC48B
UXP7phT8SQ0iFPrV13luiMwYePjOQ5oaxP6I4rv/eLHrlERR37qCegHpxy5KaDGn7R308PVxghU
i4oKC2yNUsq0IxgAV/7VPBr2IZZe0bxnwOPoT1CihGUj6M=
X-Google-Smtp-Source: AGHT+IHvBhHrjJFvSQbNNn9OxtTF8zLjuPiwJY4c8crf0OvEjSl3nVpTOsu2EooQWXwNV+r1kOaz8A==
X-Received: by 2002:a17:903:41c2:b0:20c:5d5a:af6f with SMTP id
d9443c01a7336-2129fe288bcmr131288145ad.10.1732460312763;
Sun, 24 Nov 2024 06:58:32 -0800 (PST)
Received: from terra ([2405:6586:be0:0:c8ff:1707:9b9:af89])
by smtp.gmail.com with ESMTPSA id
d9443c01a7336-2129db8c859sm47632945ad.11.2024.11.24.06.58.29
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Sun, 24 Nov 2024 06:58:32 -0800 (PST)
From: Maxim Cournoyer <maxim.cournoyer@HIDDEN>
To: Massimo Zaniboni <mzan@HIDDEN>
Subject: Re: [bug#74273] [PATCH v2] Improve bcachefs support
In-Reply-To: <6c9b20a33f44c3412a094ba0f75b9a6cb4eecb02.1731611936.git.mzan@HIDDEN>
(Massimo Zaniboni's message of "Thu, 14 Nov 2024 20:18:56 +0100")
References: <2142f04036761f24a045a176098b1d0f958ce3bf.1731111823.git.mzan@HIDDEN>
<6c9b20a33f44c3412a094ba0f75b9a6cb4eecb02.1731611936.git.mzan@HIDDEN>
Date: Sun, 24 Nov 2024 23:58:21 +0900
Message-ID: <8734jgptki.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 74273
Cc: Josselin Poiret <dev@HIDDEN>,
Simon Tournier <zimon.toutoune@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>,
Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>,
Tobias Geerinckx-Rice <me@HIDDEN>, 74273 <at> debbugs.gnu.org,
Christopher Baines <guix@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi,
Massimo Zaniboni <mzan@HIDDEN> writes:
> Improve bcachefs support:=20
> - recognize multi-device setup;
> - mount degraded file-system with missing devices;=20
> - use the built-in kernel fscheck instead of user-space bcachefs-tools;
Sounds good, although you'll want to check how other commits are
formatted; the standard we follow is that of the GNU ChangeLog, as
explained here [0] (or 'info "(standards) Change Logs"' if you have
info-reader and autoconf installed).
[0] https://www.gnu.org/prep/standards/standards.html#Change-Logs
> Change-Id: Ic741b70a7bce930da02c821c83c0a060875f4771
> ---
>
> doc/guix.texi | 22 ++++++++
> gnu/build/file-systems.scm | 105 ++++++++++++++++++++++++++++++------
> gnu/build/linux-boot.scm | 3 +-
> gnu/machine/ssh.scm | 23 +++++++-
> gnu/system/file-systems.scm | 15 ++++++
> guix/scripts/system.scm | 25 ++++++++-
> 6 files changed, 175 insertions(+), 18 deletions(-)
>
> diff --git a/doc/guix.texi b/doc/guix.texi
> index 2ab78d6..d962536 100644
> --- a/doc/guix.texi
> +++ b/doc/guix.texi
> @@ -18152,6 +18152,28 @@ Btrfs file system
> compress-force=3Dzstd,space_cache=3Dv2"))
> @end lisp
>=20=20
> +@menu
> +* Bcachefs file system::
> +@end menu
> +
> +@node Bcachefs file system
> +@subsection Bcachefs file system
> +
> +Bcachefs supports RAID1/10-style redundancy, replicating data across mul=
tiple devices.
> +To mount a file system with potentially missing devices but all data int=
act,
> +the @code{degraded} option is required. This is an example of a multi-de=
vice setup:
Please use double-spaces between sentences; that's a Texinfo/GNU
convention that makes navigating between sentences unambiguous.
> +
> +@lisp
> +(file-system
> + (mount-point "/home")
> + (device "/dev/sdb:/dev/sdc:/dev/sdd")
> + (type "bcachefs")
> + (options "degraded")
> +@end lisp
> +
> +Currently, bcachefs cannot be used as the root file-system in Guix,
> +nor can it contain the Guix store.
> +
> @node Mapped Devices
> @section Mapped Devices
>=20=20
> diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm
> index 6fd9f95..89ef18c 100644
> --- a/gnu/build/file-systems.scm
> +++ b/gnu/build/file-systems.scm
> @@ -10,6 +10,7 @@
> ;;; Copyright =C2=A9 2024 Nicolas Graves <ngraves@HIDDEN>
> ;;; Copyright =C2=A9 2024 Richard Sent <richard@HIDDEN>
> ;;; Copyright =C2=A9 2024 Janneke Nieuwenhuizen <janneke@HIDDEN>
> +;;; Copyright =C2=A9 2024 Massimo Zaniboni <mzan@HIDDEN>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -348,6 +349,8 @@ (define-syntax %bcachefs-endianness
> ;; Endianness of bcachefs file systems.
> (identifier-syntax (endianness little)))
>=20=20
> +;; FIXME at least since Linux kernel 6.11, the superblock is not
> +;; recognized anymore.
> (define (bcachefs-superblock? sblock)
> "Return #t when SBLOCK is an bcachefs superblock."
> (bytevector=3D? (sub-bytevector sblock 24 16)
> @@ -1143,10 +1146,10 @@ (define find-partition-by-luks-uuid
> (find-partition luks-partition-uuid-predicate))
I think this should be investigated and fixed before this gets merged,
as that's a serious problem, right?
>
> -(define (canonicalize-device-spec spec)
> - "Return the device name corresponding to SPEC, which can be a <uuid>, a
> -<file-system-label>, the string 'none' or another string (typically a /d=
ev
> -file name or an nfs-root containing ':/')."
> +(define* (canonicalize-device-spec spec #:optional file-system-type)
> + "Return, usually at boot-time, the device name corresponding to SPEC,
> +which can be a <uuid>, a <file-system-label>, the string 'none'
> +or another string like a device, a multi-device, file name, nfs-root."
> (define max-trials
> ;; Number of times we retry partition label resolution, 1 second per
> ;; trial. Note: somebody reported a delay of 16 seconds (!) before =
their
> @@ -1154,6 +1157,11 @@ (define (canonicalize-device-spec spec)
> ;; this long.
> 20)
>=20=20
> + (define file-system-type-str
> + (if (string? file-system-type)
> + file-system-type
> + "unknown"))
> +
> (define (resolve find-partition spec fmt)
> (let loop ((count 0))
> (let ((device (find-partition spec)))
> @@ -1168,20 +1176,73 @@ (define (canonicalize-device-spec spec)
> (sleep 1)
> (loop (+ 1 count))))))))
>=20=20
> + (define (stat-device device)
> + (stat device #f))
> +
> + (define (check-bcachefs-superblock dev)
> + (=3D 0 (system*/tty "bcachefs" "show-super" "--field-only" "disk_gro=
ups" dev)))
nitpick: Please pay attention to the maximum 80 chars line width.
> +
> + (define (resolve-bcachefs-multi-device multi-device)
> + (let ((devices (string-split multi-device #\:)))
> + ;; Some devices take a bit of time to appear, most notably USB
> + ;; storage devices. Thus, wait for the device to appear.
> + ;; NOTE: it will wait MAX-TRIALS for all the devices,
> + ;; and not for any device.
> + (let loop
> + ((count 0))
> + (let ((missing-dev (find (lambda (d) (not (stat-device d))) devic=
es)))
> + (when (and missing-dev (<=3D count max-trials))
> + (format #t "waiting for device '~a' to appear...~%" missing-d=
ev)
> + (sleep 1)
> + (loop (+ 1 count)))))
> +
> + ;; bcachefs can work in degraded mode using only few of the device=
s.
> + ;; As of Linux kernel 6.11.6, it requires that the missing/fault
> + ;; devices are removed from the multi-device specification,
> + ;; and that it is mounted with the "degraded" option.
nitpick: [here and elsewhere, you'll want to ensure your sentences
ending use double spaces.
> + (let ((valid-specs
> + (filter
> + (lambda (d) (and (stat-device d)
> + (check-bcachefs-superblock d)))
> + devices)))
> + (if (null? valid-specs)
> + (error "failed to resolve multi-device " multi-device))
> + (string-join valid-specs ":"))))
> +
> (match spec
> ((? string?)
> - (if (or (string-contains spec ":/") ;nfs
> - (and (>=3D (string-length spec) 2)
> - (equal? (string-take spec 2) "//")) ;cifs
> - (string=3D? spec "none"))
> - spec ; do not resolve NFS / CIFS / tmpfs devic=
es
> - ;; Nothing to do, but wait until SPEC shows up.
> - (resolve identity spec identity)))
> + (cond
> + ((multi-device-spec? spec)
> + (cond
> + ((string=3D? file-system-type-str "bcachefs")
> + (resolve-bcachefs-multi-device spec))
> + (else (error
> + (string-append
> + "unsupported multi-device specification "
> + spec
> + " for file-system type "
> + file-system-type-str)))))
> + ((string-contains spec ":/")
> + ;NFS, something like 'server:/some/path'
nitpick: Prefix stand-alone comments like the above with two ';'
followed by a space (inline comments with single ; and no following
space are fine).
> + spec)
> + ((and (>=3D (string-length spec) 2)
> + (equal? (string-take spec 2) "//"))
> + ;CIFS
> + spec)
> + ((string=3D? spec "none")
> + ;tmpfs
>
I'd perhaps move the above comments inline, to the right of the first
line of the cond clause.
> + spec)
> + (else
> + ;; Nothing to do, but wait until SPEC shows up.
> + ; TODO it should use STAT on some devices instead of IDENTITY.
> + ; But using STAT on all DEVICES, the boot process will block.
> + ; At least, all other devices specified using labels and UUID are
> + ; processed using the more robust STAT function.
> + (resolve identity spec identity))))
> ((? file-system-label?)
> ;; Resolve the label.
> (resolve find-partition-by-label
> - (file-system-label->string spec)
> - identity))
> + (file-system-label->string spec) identity))
> ((? uuid?)
> (resolve find-partition-by-uuid
> (uuid-bytevector spec)
> @@ -1194,10 +1255,24 @@ (define (check-file-system device type force? rep=
air)
> found. Otherwise, fix only those considered safe to repair automaticall=
y. Not
> all TYPEs support all values or combinations of FORCE? and REPAIR. Don'=
t throw
> an exception in such cases but perform the nearest sane action."
> +
> + (define (built-in-file-system-check device force? repair)
> + 'pass)
> +
> (define check-procedure
> (cond
> ((string-prefix? "ext" type) check-ext2-file-system)
> - ((string-prefix? "bcachefs" type) check-bcachefs-file-system)
> + ((string-prefix? "bcachefs" type)
> + ;; According bcachefs manual: "No special handling is needed for =
recovering
> + ;; from unclean shutdown. Journal replay happens automatically,
> + ;; and diagnostic messages in the dmesg log will indicate whether=
recovery
> + ;; was from clean or unclean shutdown."
> + ;; Moreover, at least until Linux kernel 6.11, the bcachefs-tools=
package
> + ;; does not try to respect the bcachefs format supported by the k=
ernel.
> + ;; So, the fsck of bcachefs-tools is called only if explicitely s=
tated.
> + (if force?
> + check-bcachefs-file-system
> + built-in-file-system-check))
> ((string-prefix? "btrfs" type) check-btrfs-file-system)
> ((string-suffix? "exfat" type) check-exfat-file-system)
> ((string-suffix? "fat" type) check-fat-file-system)
> @@ -1385,7 +1460,7 @@ (define* (mount-file-system fs #:key (root "/root")
> "")))))
>=20=20
> (let* ((type (file-system-type fs))
> - (source (canonicalize-device-spec (file-system-device fs)))
> + (source (canonicalize-device-spec (file-system-device fs) type=
))
> (target (string-append root "/"
> (file-system-mount-point fs)))
> (flags (logior (mount-flags->bit-mask (file-system-flags fs))
> diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm
> index 548e28a..d184fcd 100644
> --- a/gnu/build/linux-boot.scm
> +++ b/gnu/build/linux-boot.scm
> @@ -635,7 +635,8 @@ (define* (boot-system #:key
>=20=20
> ;; Mount the root file system.
> (mount-root-file-system (canonicalize-device-spec
> - (file-system-device root-fs))
> + (file-system-device root-fs)
> + (file-system-type root-fs))
> (file-system-type root-fs)
> #:volatile-root? volatile-root?
> #:flags (mount-flags->bit-mask
> diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm
> index 3e10d98..0054adf 100644
> --- a/gnu/machine/ssh.scm
> +++ b/gnu/machine/ssh.scm
> @@ -2,6 +2,7 @@
> ;;; Copyright =C2=A9 2019 Jakob L. Kreuze <zerodaysfordays@HIDDEN>
> ;;; Copyright =C2=A9 2020-2023 Ludovic Court=C3=A8s <ludo@HIDDEN>
> ;;; Copyright =C2=A9 2024 Ricardo <rekado@HIDDEN>
> +;;; Copyright =C2=A9 2024 Massimo Zaniboni <mzan@HIDDEN>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -241,6 +242,22 @@ (define (machine-check-file-system-availability mach=
ine)
> (file-system-device fs)
> (strerror errno))))))
>=20=20
> + (define (check-multi-device-file-system fs)
> + (define multi-device (file-system-device fs))
> + (define devices (string-split multi-device #\:))
> + (define (check-device device)
> + (remote-let ((errno #~(catch 'system-error
> + (lambda ()
> + (stat #$device)
> + #t)
> + (lambda args
> + (system-error-errno args)))))
> + (when (number? errno)
> + (raise (formatted-message (G_ "device '~a' not found: ~a")
> + device
> + (strerror errno))))))
> + (map check-device devices))
> +
> (define (check-labeled-file-system fs)
> (define remote-exp
> (with-imported-modules (source-module-closure
> @@ -278,8 +295,12 @@ (define (machine-check-file-system-availability mach=
ine)
> (machine-configuration machine))
> (append (map check-literal-file-system
> (filter (lambda (fs)
> - (string? (file-system-device fs)))
> + (single-device-spec? (file-system-device fs=
)))
> file-systems))
> + (append-map check-multi-device-file-system
> + (filter (lambda (fs)
> + (multi-device-spec? (file-system-dev=
ice fs)))
> + file-systems))
> (map check-labeled-file-system
> (filter (lambda (fs)
> (file-system-label? (file-system-device fs)=
))
> diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm
> index 4ea8237..9f91bd7 100644
> --- a/gnu/system/file-systems.scm
> +++ b/gnu/system/file-systems.scm
> @@ -5,6 +5,7 @@
> ;;; Copyright =C2=A9 2020, 2021 Maxim Cournoyer <maxim.cournoyer@HIDDEN=
om>
> ;;; Copyright =C2=A9 2021 Tobias Geerinckx-Rice <me@HIDDEN>
> ;;; Copyright =C2=A9 2022 Oleg Pykhalov <go.wigust@HIDDEN>
> +;;; Copyright =C2=A9 2024 Massimo Zaniboni <mzan@HIDDEN>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -73,6 +74,9 @@ (define-module (gnu system file-systems)
> spec->file-system
> specification->file-system-mapping
>=20=20
> + multi-device-spec?
> + single-device-spec?
> +
> %pseudo-file-system-types
> %fuse-control-file-system
> %binary-format-file-system
> @@ -309,6 +313,17 @@ (define (file-system-needed-for-boot? fs)
> (and (file-prefix? (file-system-mount-point fs) (%store-prefix))
> (not (memq 'bind-mount (file-system-flags fs))))))
>=20=20
> +(define (multi-device-spec? spec)
> + "Return #t if the specification is like '/dev/sda:/dev/sdb'."
> + (and (string? spec)
> + (string-contains spec ":/")
> + (string-prefix? "/dev/" spec)))
> +
> +(define (single-device-spec? spec)
> + "Return #t if the specification is a string, but not a multi-device."
> + (and (string? spec)
> + (not (multi-device-spec? spec))))
At first, I wrote/thought:
--8<---------------cut here---------------start------------->8---
I'm not convinced we should check for the type (string?). We don't
typically do this, and it hurts functional composition here:
Without checking for string?, you have a single condition
(multi-device-spec? x), and you could easily and correctly simply negate
that to get the 'single-device-spec' API. The addition of the string?
check muddies that. I'd just document that spec is expected to be a
string, unless there's a good reason to guard against other potential
types currently in use.
--8<---------------cut here---------------end--------------->8---
But reviewing (gnu build file-systems), more specifically
canonicalize-device-spec, the spec indeed can be multiple things, such
as a <uuid> or <file-system-label> object or a string. So OK! I think
we can have single-device-spec? in the API.
> (define (file-system->spec fs)
> "Return a list corresponding to file-system FS that can be passed to t=
he
> initrd code."
> diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
> index 7989b18..4e9c581 100644
> --- a/guix/scripts/system.scm
> +++ b/guix/scripts/system.scm
> @@ -11,6 +11,7 @@
> ;;; Copyright =C2=A9 2021 Brice Waegeneire <brice@HIDDEN>
> ;;; Copyright =C2=A9 2021 Simon Tournier <zimon.toutoune@HIDDEN>
> ;;; Copyright =C2=A9 2022 Tobias Geerinckx-Rice <me@HIDDEN>
> +;;; Copyright =C2=A9 2024 Massimo Zaniboni <mzan@HIDDEN>
> ;;;
> ;;; This file is part of GNU Guix.
> ;;;
> @@ -605,9 +606,16 @@ (define (check-file-system-availability file-systems)
> (file-system-label? (file-system-device fs)))
> relevant))
>=20=20
> + (define multi-device
> + (filter (lambda (fs)
> + (and (string? (file-system-device fs))
> + (multi-device-spec? (file-system-device fs))))
> + relevant))
> +
> (define literal
> (filter (lambda (fs)
> - (string? (file-system-device fs)))
> + (and (string? (file-system-device fs))
> + (single-device-spec? (file-system-device fs))))
> relevant))
>=20=20
> (define uuid
> @@ -641,6 +649,21 @@ (define (check-file-system-availability file-systems)
> label, write @code{(file-system-label ~s)} in your @code{device} field.")
> device device))))))
> literal)
> + (for-each
> + (lambda (fs)
> + (let* ((devices-str (file-system-device fs))
> + (devices (string-split devices-str #\:)))
> + (for-each
> + (lambda (device)
> + (catch 'system-error
> + (lambda () (stat device))
> + (lambda args
> + (let ((errno (system-error-errno args)))
> + (error (file-system-location* fs)
> + (G_ " #8605 device '~a' not found in mu=
lti-device '~a': ~a~%")
> + device devices-str (strerror errno))))))
> + devices)))
> + multi-device)
> (for-each (lambda (fs)
> (let ((label (file-system-label->string
> (file-system-device fs))))
>
I've only read the diff, not applied nor run the code, but from this
simple review, it looks sane to me. I think in a v2 you could rewrite
the commit message to match the GNU Change-Log convention, add double
spacing to new comments/text, and ensure the line width stays in check
at max 80 columns.
Hopefully a Bcachefs afficionado (Tobias?) gets interested enough to
actually try it; it'd be nice to know if this works as there doesn't
seem to be any specific system test coverage for it (and given the
warning that it cannot be used as the root file system or /gnu/store yet
(why?), that would be a bit difficult or at least different that the
other tests we have for file systems).
Thank you for distilling this well crafted change.
--=20
Maxim
guix-patches@HIDDEN:bug#74273; Package guix-patches.
Full text available.
Received: (at 74273) by debbugs.gnu.org; 14 Nov 2024 19:23:42 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Nov 14 14:23:42 2024
Received: from localhost ([127.0.0.1]:47215 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1tBfRN-0007Rs-8e
for submit <at> debbugs.gnu.org; Thu, 14 Nov 2024 14:23:42 -0500
Received: from mail.asterisell.com ([193.30.121.134]:46140)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mzan@HIDDEN>) id 1tBfRI-0007RZ-Nc
for 74273 <at> debbugs.gnu.org; Thu, 14 Nov 2024 14:23:39 -0500
From: Massimo Zaniboni <mzan@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokmelody.org;
s=mail; t=1731612177;
bh=KBz+yjaD20EQUVPAvq6+ftL0d6s6YB5AeydVawdTx+A=;
h=From:To:Cc:Subject:Date;
b=GTHHujfdPcdduujH0fXyk1Noo+ZqedgidLPptSitU91Rv+ACFmJAoTuOgrPJ4/aPH
R7cu3+X3TeWugecezPxWZJFFRxqB/6paVg7agt87oL2C1Ha8dfeJ7cIoX7rAhtJLG7
9r/fOQPNV9iQQi6Z9CGW4/I4JUcekkmMZ4Vujt/g=
To: 74273 <at> debbugs.gnu.org
Subject: [PATCH v2] Improve bcachefs support
Date: Thu, 14 Nov 2024 20:18:56 +0100
Message-ID: <6c9b20a33f44c3412a094ba0f75b9a6cb4eecb02.1731611936.git.mzan@HIDDEN>
MIME-Version: 1.0
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Maxim Cournoyer <maxim.cournoyer@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 74273
Cc: Massimo Zaniboni <mzan@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Improve bcachefs support:
- recognize multi-device setup;
- mount degraded file-system with missing devices;
- use the built-in kernel fscheck instead of user-space bcachefs-tools;
Change-Id: Ic741b70a7bce930da02c821c83c0a060875f4771
---
doc/guix.texi | 22 ++++++++
gnu/build/file-systems.scm | 105 ++++++++++++++++++++++++++++++------
gnu/build/linux-boot.scm | 3 +-
gnu/machine/ssh.scm | 23 +++++++-
gnu/system/file-systems.scm | 15 ++++++
guix/scripts/system.scm | 25 ++++++++-
6 files changed, 175 insertions(+), 18 deletions(-)
diff --git a/doc/guix.texi b/doc/guix.texi
index 2ab78d6..d962536 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -18152,6 +18152,28 @@ Btrfs file system
compress-force=zstd,space_cache=v2"))
@end lisp
+@menu
+* Bcachefs file system::
+@end menu
+
+@node Bcachefs file system
+@subsection Bcachefs file system
+
+Bcachefs supports RAID1/10-style redundancy, replicating data across multiple devices.
+To mount a file system with potentially missing devices but all data intact,
+the @code{degraded} option is required. This is an example of a multi-device setup:
+
+@lisp
+(file-system
+ (mount-point "/home")
+ (device "/dev/sdb:/dev/sdc:/dev/sdd")
+ (type "bcachefs")
+ (options "degraded")
+@end lisp
+
+Currently, bcachefs cannot be used as the root file-system in Guix,
+nor can it contain the Guix store.
+
@node Mapped Devices
@section Mapped Devices
diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm
index 6fd9f95..89ef18c 100644
--- a/gnu/build/file-systems.scm
+++ b/gnu/build/file-systems.scm
@@ -10,6 +10,7 @@
;;; Copyright © 2024 Nicolas Graves <ngraves@HIDDEN>
;;; Copyright © 2024 Richard Sent <richard@HIDDEN>
;;; Copyright © 2024 Janneke Nieuwenhuizen <janneke@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -348,6 +349,8 @@ (define-syntax %bcachefs-endianness
;; Endianness of bcachefs file systems.
(identifier-syntax (endianness little)))
+;; FIXME at least since Linux kernel 6.11, the superblock is not
+;; recognized anymore.
(define (bcachefs-superblock? sblock)
"Return #t when SBLOCK is an bcachefs superblock."
(bytevector=? (sub-bytevector sblock 24 16)
@@ -1143,10 +1146,10 @@ (define find-partition-by-luks-uuid
(find-partition luks-partition-uuid-predicate))
-(define (canonicalize-device-spec spec)
- "Return the device name corresponding to SPEC, which can be a <uuid>, a
-<file-system-label>, the string 'none' or another string (typically a /dev
-file name or an nfs-root containing ':/')."
+(define* (canonicalize-device-spec spec #:optional file-system-type)
+ "Return, usually at boot-time, the device name corresponding to SPEC,
+which can be a <uuid>, a <file-system-label>, the string 'none'
+or another string like a device, a multi-device, file name, nfs-root."
(define max-trials
;; Number of times we retry partition label resolution, 1 second per
;; trial. Note: somebody reported a delay of 16 seconds (!) before their
@@ -1154,6 +1157,11 @@ (define (canonicalize-device-spec spec)
;; this long.
20)
+ (define file-system-type-str
+ (if (string? file-system-type)
+ file-system-type
+ "unknown"))
+
(define (resolve find-partition spec fmt)
(let loop ((count 0))
(let ((device (find-partition spec)))
@@ -1168,20 +1176,73 @@ (define (canonicalize-device-spec spec)
(sleep 1)
(loop (+ 1 count))))))))
+ (define (stat-device device)
+ (stat device #f))
+
+ (define (check-bcachefs-superblock dev)
+ (= 0 (system*/tty "bcachefs" "show-super" "--field-only" "disk_groups" dev)))
+
+ (define (resolve-bcachefs-multi-device multi-device)
+ (let ((devices (string-split multi-device #\:)))
+ ;; Some devices take a bit of time to appear, most notably USB
+ ;; storage devices. Thus, wait for the device to appear.
+ ;; NOTE: it will wait MAX-TRIALS for all the devices,
+ ;; and not for any device.
+ (let loop
+ ((count 0))
+ (let ((missing-dev (find (lambda (d) (not (stat-device d))) devices)))
+ (when (and missing-dev (<= count max-trials))
+ (format #t "waiting for device '~a' to appear...~%" missing-dev)
+ (sleep 1)
+ (loop (+ 1 count)))))
+
+ ;; bcachefs can work in degraded mode using only few of the devices.
+ ;; As of Linux kernel 6.11.6, it requires that the missing/fault
+ ;; devices are removed from the multi-device specification,
+ ;; and that it is mounted with the "degraded" option.
+ (let ((valid-specs
+ (filter
+ (lambda (d) (and (stat-device d)
+ (check-bcachefs-superblock d)))
+ devices)))
+ (if (null? valid-specs)
+ (error "failed to resolve multi-device " multi-device))
+ (string-join valid-specs ":"))))
+
(match spec
((? string?)
- (if (or (string-contains spec ":/") ;nfs
- (and (>= (string-length spec) 2)
- (equal? (string-take spec 2) "//")) ;cifs
- (string=? spec "none"))
- spec ; do not resolve NFS / CIFS / tmpfs devices
- ;; Nothing to do, but wait until SPEC shows up.
- (resolve identity spec identity)))
+ (cond
+ ((multi-device-spec? spec)
+ (cond
+ ((string=? file-system-type-str "bcachefs")
+ (resolve-bcachefs-multi-device spec))
+ (else (error
+ (string-append
+ "unsupported multi-device specification "
+ spec
+ " for file-system type "
+ file-system-type-str)))))
+ ((string-contains spec ":/")
+ ;NFS, something like 'server:/some/path'
+ spec)
+ ((and (>= (string-length spec) 2)
+ (equal? (string-take spec 2) "//"))
+ ;CIFS
+ spec)
+ ((string=? spec "none")
+ ;tmpfs
+ spec)
+ (else
+ ;; Nothing to do, but wait until SPEC shows up.
+ ; TODO it should use STAT on some devices instead of IDENTITY.
+ ; But using STAT on all DEVICES, the boot process will block.
+ ; At least, all other devices specified using labels and UUID are
+ ; processed using the more robust STAT function.
+ (resolve identity spec identity))))
((? file-system-label?)
;; Resolve the label.
(resolve find-partition-by-label
- (file-system-label->string spec)
- identity))
+ (file-system-label->string spec) identity))
((? uuid?)
(resolve find-partition-by-uuid
(uuid-bytevector spec)
@@ -1194,10 +1255,24 @@ (define (check-file-system device type force? repair)
found. Otherwise, fix only those considered safe to repair automatically. Not
all TYPEs support all values or combinations of FORCE? and REPAIR. Don't throw
an exception in such cases but perform the nearest sane action."
+
+ (define (built-in-file-system-check device force? repair)
+ 'pass)
+
(define check-procedure
(cond
((string-prefix? "ext" type) check-ext2-file-system)
- ((string-prefix? "bcachefs" type) check-bcachefs-file-system)
+ ((string-prefix? "bcachefs" type)
+ ;; According bcachefs manual: "No special handling is needed for recovering
+ ;; from unclean shutdown. Journal replay happens automatically,
+ ;; and diagnostic messages in the dmesg log will indicate whether recovery
+ ;; was from clean or unclean shutdown."
+ ;; Moreover, at least until Linux kernel 6.11, the bcachefs-tools package
+ ;; does not try to respect the bcachefs format supported by the kernel.
+ ;; So, the fsck of bcachefs-tools is called only if explicitely stated.
+ (if force?
+ check-bcachefs-file-system
+ built-in-file-system-check))
((string-prefix? "btrfs" type) check-btrfs-file-system)
((string-suffix? "exfat" type) check-exfat-file-system)
((string-suffix? "fat" type) check-fat-file-system)
@@ -1385,7 +1460,7 @@ (define* (mount-file-system fs #:key (root "/root")
"")))))
(let* ((type (file-system-type fs))
- (source (canonicalize-device-spec (file-system-device fs)))
+ (source (canonicalize-device-spec (file-system-device fs) type))
(target (string-append root "/"
(file-system-mount-point fs)))
(flags (logior (mount-flags->bit-mask (file-system-flags fs))
diff --git a/gnu/build/linux-boot.scm b/gnu/build/linux-boot.scm
index 548e28a..d184fcd 100644
--- a/gnu/build/linux-boot.scm
+++ b/gnu/build/linux-boot.scm
@@ -635,7 +635,8 @@ (define* (boot-system #:key
;; Mount the root file system.
(mount-root-file-system (canonicalize-device-spec
- (file-system-device root-fs))
+ (file-system-device root-fs)
+ (file-system-type root-fs))
(file-system-type root-fs)
#:volatile-root? volatile-root?
#:flags (mount-flags->bit-mask
diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm
index 3e10d98..0054adf 100644
--- a/gnu/machine/ssh.scm
+++ b/gnu/machine/ssh.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2019 Jakob L. Kreuze <zerodaysfordays@HIDDEN>
;;; Copyright © 2020-2023 Ludovic Courtès <ludo@HIDDEN>
;;; Copyright © 2024 Ricardo <rekado@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -241,6 +242,22 @@ (define (machine-check-file-system-availability machine)
(file-system-device fs)
(strerror errno))))))
+ (define (check-multi-device-file-system fs)
+ (define multi-device (file-system-device fs))
+ (define devices (string-split multi-device #\:))
+ (define (check-device device)
+ (remote-let ((errno #~(catch 'system-error
+ (lambda ()
+ (stat #$device)
+ #t)
+ (lambda args
+ (system-error-errno args)))))
+ (when (number? errno)
+ (raise (formatted-message (G_ "device '~a' not found: ~a")
+ device
+ (strerror errno))))))
+ (map check-device devices))
+
(define (check-labeled-file-system fs)
(define remote-exp
(with-imported-modules (source-module-closure
@@ -278,8 +295,12 @@ (define (machine-check-file-system-availability machine)
(machine-configuration machine))
(append (map check-literal-file-system
(filter (lambda (fs)
- (string? (file-system-device fs)))
+ (single-device-spec? (file-system-device fs)))
file-systems))
+ (append-map check-multi-device-file-system
+ (filter (lambda (fs)
+ (multi-device-spec? (file-system-device fs)))
+ file-systems))
(map check-labeled-file-system
(filter (lambda (fs)
(file-system-label? (file-system-device fs)))
diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm
index 4ea8237..9f91bd7 100644
--- a/gnu/system/file-systems.scm
+++ b/gnu/system/file-systems.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020, 2021 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
;;; Copyright © 2021 Tobias Geerinckx-Rice <me@HIDDEN>
;;; Copyright © 2022 Oleg Pykhalov <go.wigust@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -73,6 +74,9 @@ (define-module (gnu system file-systems)
spec->file-system
specification->file-system-mapping
+ multi-device-spec?
+ single-device-spec?
+
%pseudo-file-system-types
%fuse-control-file-system
%binary-format-file-system
@@ -309,6 +313,17 @@ (define (file-system-needed-for-boot? fs)
(and (file-prefix? (file-system-mount-point fs) (%store-prefix))
(not (memq 'bind-mount (file-system-flags fs))))))
+(define (multi-device-spec? spec)
+ "Return #t if the specification is like '/dev/sda:/dev/sdb'."
+ (and (string? spec)
+ (string-contains spec ":/")
+ (string-prefix? "/dev/" spec)))
+
+(define (single-device-spec? spec)
+ "Return #t if the specification is a string, but not a multi-device."
+ (and (string? spec)
+ (not (multi-device-spec? spec))))
+
(define (file-system->spec fs)
"Return a list corresponding to file-system FS that can be passed to the
initrd code."
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 7989b18..4e9c581 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -11,6 +11,7 @@
;;; Copyright © 2021 Brice Waegeneire <brice@HIDDEN>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@HIDDEN>
;;; Copyright © 2022 Tobias Geerinckx-Rice <me@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -605,9 +606,16 @@ (define (check-file-system-availability file-systems)
(file-system-label? (file-system-device fs)))
relevant))
+ (define multi-device
+ (filter (lambda (fs)
+ (and (string? (file-system-device fs))
+ (multi-device-spec? (file-system-device fs))))
+ relevant))
+
(define literal
(filter (lambda (fs)
- (string? (file-system-device fs)))
+ (and (string? (file-system-device fs))
+ (single-device-spec? (file-system-device fs))))
relevant))
(define uuid
@@ -641,6 +649,21 @@ (define (check-file-system-availability file-systems)
label, write @code{(file-system-label ~s)} in your @code{device} field.")
device device))))))
literal)
+ (for-each
+ (lambda (fs)
+ (let* ((devices-str (file-system-device fs))
+ (devices (string-split devices-str #\:)))
+ (for-each
+ (lambda (device)
+ (catch 'system-error
+ (lambda () (stat device))
+ (lambda args
+ (let ((errno (system-error-errno args)))
+ (error (file-system-location* fs)
+ (G_ " #8605 device '~a' not found in multi-device '~a': ~a~%")
+ device devices-str (strerror errno))))))
+ devices)))
+ multi-device)
(for-each (lambda (fs)
(let ((label (file-system-label->string
(file-system-device fs))))
base-commit: c1cb7f1031c5dde2a260d8d8ad7547d6c79cc532
prerequisite-patch-id: e3ec1271b30da286e1a2fdd1519a8c504e52d64a
prerequisite-patch-id: 25d78fbfbd3268c16c93cd5d222386a7f421979b
prerequisite-patch-id: 8ca774fc68440ec5233b5353f11886b1712e6b43
prerequisite-patch-id: 0000000000000000000000000000000000000000
prerequisite-patch-id: e22870a8d4b3ab67b12e05b6242b7f1bf5ac193b
--
2.46.0
guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, maxim.cournoyer@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:bug#74273; Package guix-patches.
Full text available.
Received: (at submit) by debbugs.gnu.org; 9 Nov 2024 00:34:34 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Nov 08 19:34:34 2024
Received: from localhost ([127.0.0.1]:52655 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1t9ZQv-0003LE-Lp
for submit <at> debbugs.gnu.org; Fri, 08 Nov 2024 19:34:34 -0500
Received: from lists.gnu.org ([209.51.188.17]:33444)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mzan@HIDDEN>) id 1t9ZQs-0003L5-3J
for submit <at> debbugs.gnu.org; Fri, 08 Nov 2024 19:34:32 -0500
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <mzan@HIDDEN>)
id 1t9ZQr-0002c2-MB
for guix-patches@HIDDEN; Fri, 08 Nov 2024 19:34:29 -0500
Received: from mail.asterisell.com ([193.30.121.134])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <mzan@HIDDEN>)
id 1t9ZQn-0007OW-MY
for guix-patches@HIDDEN; Fri, 08 Nov 2024 19:34:28 -0500
From: Massimo Zaniboni <mzan@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=dokmelody.org;
s=mail; t=1731112431;
bh=8D0oHQZ7lu02GmInNeEyMWnh/ELa6111/moiCw82ftU=;
h=From:To:Cc:Subject:Date;
b=lSZt/GQ+A9UF5UeNV7OSozHA1H2dfJVRYiX3Kj1B3h+FXHxZRY7P71tgirXkm/VUi
kau+nA58SqwoSnGEJDS0QJWx9isRLe3CjeoSQhBLUFzvXZeBJ4atIqwOT4PXzbFte9
yhP0dyTeniiXwZWvWu2ES1NA548sGoMc5LR7fAaQ=
To: guix-patches@HIDDEN
Subject: [PATCH] Support for bcachefs-like multi-device file-systems.
Date: Sat, 9 Nov 2024 01:23:43 +0100
Message-ID: <2142f04036761f24a045a176098b1d0f958ce3bf.1731111823.git.mzan@HIDDEN>
MIME-Version: 1.0
X-Debbugs-Cc: Christopher Baines <guix@HIDDEN>, Josselin Poiret <dev@HIDDEN>, Ludovic Courtès <ludo@HIDDEN>, Mathieu Othacehe <othacehe@HIDDEN>, Simon Tournier <zimon.toutoune@HIDDEN>, Tobias Geerinckx-Rice <me@HIDDEN>
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit
Received-SPF: pass client-ip=193.30.121.134; envelope-from=mzan@HIDDEN;
helo=mail.asterisell.com
X-Spam_score_int: -16
X-Spam_score: -1.7
X-Spam_bar: -
X-Spam_report: (-1.7 / 5.0 requ) BAYES_00=-1.9, DKIM_INVALID=0.1,
DKIM_SIGNED=0.1, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001,
RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=no autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
Cc: Massimo Zaniboni <mzan@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)
Support multi-device like "/dev/sda:/dev/sdb".
Change-Id: Iddd9c31f8c083a55e7a1fb193e7bbfb396e2def6
---
I'm using this patch on my system.
This is the first patch that I send using Stacked Git (`stg`).
I hope that the email format is correct.
gnu/build/file-systems.scm | 49 ++++++++++++++++++++++++++++---------
gnu/machine/ssh.scm | 23 ++++++++++++++++-
gnu/system/file-systems.scm | 15 ++++++++++++
guix/scripts/system.scm | 25 ++++++++++++++++++-
4 files changed, 98 insertions(+), 14 deletions(-)
diff --git a/gnu/build/file-systems.scm b/gnu/build/file-systems.scm
index 41e1c9e..7dba7e0 100644
--- a/gnu/build/file-systems.scm
+++ b/gnu/build/file-systems.scm
@@ -9,6 +9,7 @@
;;; Copyright © 2022 Oleg Pykhalov <go.wigust@HIDDEN>
;;; Copyright © 2024 Nicolas Graves <ngraves@HIDDEN>
;;; Copyright © 2024 Richard Sent <richard@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -1138,9 +1139,9 @@ (define find-partition-by-luks-uuid
(define (canonicalize-device-spec spec)
- "Return the device name corresponding to SPEC, which can be a <uuid>, a
-<file-system-label>, the string 'none' or another string (typically a /dev
-file name or an nfs-root containing ':/')."
+ "Return the device name corresponding to SPEC, which can be a <uuid>,
+a <file-system-label>, the string 'none' or another string like a device,
+a multi-device, file name, nfs-root."
(define max-trials
;; Number of times we retry partition label resolution, 1 second per
;; trial. Note: somebody reported a delay of 16 seconds (!) before their
@@ -1162,20 +1163,44 @@ (define (canonicalize-device-spec spec)
(sleep 1)
(loop (+ 1 count))))))))
+ (define (resolve-multi-device find-partition multi-device)
+ (let ((specs (string-split multi-device #\:)))
+ (let loop
+ ((count 0))
+ (let ((nfp (find (lambda (d) (not (find-partition d))) specs)))
+ (if nfp
+ ;; Some devices take a bit of time to appear, most notably USB
+ ;; storage devices. Thus, wait for the device to appear.
+ (if (> count max-trials)
+ (error "failed to resolve partition" nfp)
+ (begin
+ (format #t "waiting for partition '~a' to appear...~%" nfp)
+ (sleep 1)
+ (loop (+ 1 count))))
+ multi-device)))))
+
(match spec
((? string?)
- (if (or (string-contains spec ":/") ;nfs
- (and (>= (string-length spec) 2)
- (equal? (string-take spec 2) "//")) ;cifs
- (string=? spec "none"))
- spec ; do not resolve NFS / CIFS / tmpfs devices
- ;; Nothing to do, but wait until SPEC shows up.
- (resolve identity spec identity)))
+ (cond
+ ((multi-device-spec? spec)
+ (resolve-multi-device identity spec))
+ ((string-contains spec ":/")
+ ;NFS, something like 'server:/some/path'
+ spec)
+ ((and (>= (string-length spec) 2)
+ (equal? (string-take spec 2) "//"))
+ ;CIFS
+ spec)
+ ((string=? spec "none")
+ ;tmpfs
+ spec)
+ (else
+ ;; Nothing to do, but wait until SPEC shows up.
+ (resolve identity spec identity))))
((? file-system-label?)
;; Resolve the label.
(resolve find-partition-by-label
- (file-system-label->string spec)
- identity))
+ (file-system-label->string spec) identity))
((? uuid?)
(resolve find-partition-by-uuid
(uuid-bytevector spec)
diff --git a/gnu/machine/ssh.scm b/gnu/machine/ssh.scm
index 3e10d98..0054adf 100644
--- a/gnu/machine/ssh.scm
+++ b/gnu/machine/ssh.scm
@@ -2,6 +2,7 @@
;;; Copyright © 2019 Jakob L. Kreuze <zerodaysfordays@HIDDEN>
;;; Copyright © 2020-2023 Ludovic Courtès <ludo@HIDDEN>
;;; Copyright © 2024 Ricardo <rekado@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -241,6 +242,22 @@ (define (machine-check-file-system-availability machine)
(file-system-device fs)
(strerror errno))))))
+ (define (check-multi-device-file-system fs)
+ (define multi-device (file-system-device fs))
+ (define devices (string-split multi-device #\:))
+ (define (check-device device)
+ (remote-let ((errno #~(catch 'system-error
+ (lambda ()
+ (stat #$device)
+ #t)
+ (lambda args
+ (system-error-errno args)))))
+ (when (number? errno)
+ (raise (formatted-message (G_ "device '~a' not found: ~a")
+ device
+ (strerror errno))))))
+ (map check-device devices))
+
(define (check-labeled-file-system fs)
(define remote-exp
(with-imported-modules (source-module-closure
@@ -278,8 +295,12 @@ (define (machine-check-file-system-availability machine)
(machine-configuration machine))
(append (map check-literal-file-system
(filter (lambda (fs)
- (string? (file-system-device fs)))
+ (single-device-spec? (file-system-device fs)))
file-systems))
+ (append-map check-multi-device-file-system
+ (filter (lambda (fs)
+ (multi-device-spec? (file-system-device fs)))
+ file-systems))
(map check-labeled-file-system
(filter (lambda (fs)
(file-system-label? (file-system-device fs)))
diff --git a/gnu/system/file-systems.scm b/gnu/system/file-systems.scm
index 4ea8237..9f91bd7 100644
--- a/gnu/system/file-systems.scm
+++ b/gnu/system/file-systems.scm
@@ -5,6 +5,7 @@
;;; Copyright © 2020, 2021 Maxim Cournoyer <maxim.cournoyer@HIDDEN>
;;; Copyright © 2021 Tobias Geerinckx-Rice <me@HIDDEN>
;;; Copyright © 2022 Oleg Pykhalov <go.wigust@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -73,6 +74,9 @@ (define-module (gnu system file-systems)
spec->file-system
specification->file-system-mapping
+ multi-device-spec?
+ single-device-spec?
+
%pseudo-file-system-types
%fuse-control-file-system
%binary-format-file-system
@@ -309,6 +313,17 @@ (define (file-system-needed-for-boot? fs)
(and (file-prefix? (file-system-mount-point fs) (%store-prefix))
(not (memq 'bind-mount (file-system-flags fs))))))
+(define (multi-device-spec? spec)
+ "Return #t if the specification is like '/dev/sda:/dev/sdb'."
+ (and (string? spec)
+ (string-contains spec ":/")
+ (string-prefix? "/dev/" spec)))
+
+(define (single-device-spec? spec)
+ "Return #t if the specification is a string, but not a multi-device."
+ (and (string? spec)
+ (not (multi-device-spec? spec))))
+
(define (file-system->spec fs)
"Return a list corresponding to file-system FS that can be passed to the
initrd code."
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 99c58f3..3459891 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -11,6 +11,7 @@
;;; Copyright © 2021 Brice Waegeneire <brice@HIDDEN>
;;; Copyright © 2021 Simon Tournier <zimon.toutoune@HIDDEN>
;;; Copyright © 2022 Tobias Geerinckx-Rice <me@HIDDEN>
+;;; Copyright © 2024 Massimo Zaniboni <mzan@HIDDEN>
;;;
;;; This file is part of GNU Guix.
;;;
@@ -601,9 +602,16 @@ (define (check-file-system-availability file-systems)
(file-system-label? (file-system-device fs)))
relevant))
+ (define multi-device
+ (filter (lambda (fs)
+ (and (string? (file-system-device fs))
+ (multi-device-spec? (file-system-device fs))))
+ relevant))
+
(define literal
(filter (lambda (fs)
- (string? (file-system-device fs)))
+ (and (string? (file-system-device fs))
+ (single-device-spec? (file-system-device fs))))
relevant))
(define uuid
@@ -637,6 +645,21 @@ (define (check-file-system-availability file-systems)
label, write @code{(file-system-label ~s)} in your @code{device} field.")
device device))))))
literal)
+ (for-each
+ (lambda (fs)
+ (let* ((devices-str (file-system-device fs))
+ (devices (string-split devices-str #\:)))
+ (for-each
+ (lambda (device)
+ (catch 'system-error
+ (lambda () (stat device))
+ (lambda args
+ (let ((errno (system-error-errno args)))
+ (error (file-system-location* fs)
+ (G_ " #8605 device '~a' not found in multi-device '~a': ~a~%")
+ device devices-str (strerror errno))))))
+ devices)))
+ multi-device)
(for-each (lambda (fs)
(let ((label (file-system-label->string
(file-system-device fs))))
base-commit: 2a6d96425eea57dc6dd48a2bec16743046e32e06
prerequisite-patch-id: 25d78fbfbd3268c16c93cd5d222386a7f421979b
prerequisite-patch-id: 30bc9aa990c70c6c1c45c951a58cf9a532b388fb
prerequisite-patch-id: 0000000000000000000000000000000000000000
prerequisite-patch-id: e22870a8d4b3ab67b12e05b6242b7f1bf5ac193b
--
2.46.0
Massimo Zaniboni <mzan@HIDDEN>:guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN.
Full text available.guix@HIDDEN, dev@HIDDEN, ludo@HIDDEN, othacehe@HIDDEN, zimon.toutoune@HIDDEN, me@HIDDEN, guix-patches@HIDDEN:bug#74273; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.