X-Loop: help-debbugs@HIDDEN
Subject: bug#77606: =?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= on cached narinfo files is too expensive
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix@HIDDEN, bug-guix@HIDDEN
Resent-Date: Mon, 07 Apr 2025 10:21:02 +0000
Resent-Message-ID: <handler.77606.B.174402124517890 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: report 77606
X-GNU-PR-Package: guix
X-GNU-PR-Keywords:
To: 77606 <at> debbugs.gnu.org
Cc: Christopher Baines <guix@HIDDEN>
X-Debbugs-Original-To: bug-guix@HIDDEN
X-Debbugs-Original-Xcc: Christopher Baines <guix@HIDDEN>
Received: via spool by submit <at> debbugs.gnu.org id=B.174402124517890
(code B ref -1); Mon, 07 Apr 2025 10:21:02 +0000
Received: (at submit) by debbugs.gnu.org; 7 Apr 2025 10:20:45 +0000
Received: from localhost ([127.0.0.1]:52869 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1u1jau-0004eP-6D
for submit <at> debbugs.gnu.org; Mon, 07 Apr 2025 06:20:44 -0400
Received: from lists.gnu.org ([2001:470:142::17]:60896)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.84_2) (envelope-from <ludovic.courtes@HIDDEN>)
id 1u1jar-0004dm-Hy
for submit <at> debbugs.gnu.org; Mon, 07 Apr 2025 06:20:42 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10])
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludovic.courtes@HIDDEN>)
id 1u1jal-00043e-UK
for bug-guix@HIDDEN; Mon, 07 Apr 2025 06:20:35 -0400
Received: from mail2-relais-roc.national.inria.fr ([192.134.164.83])
by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <ludovic.courtes@HIDDEN>)
id 1u1jaj-0000ti-1x
for bug-guix@HIDDEN; Mon, 07 Apr 2025 06:20:35 -0400
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=inria.fr; s=dc;
h=from:to:subject:date:message-id:mime-version:
content-transfer-encoding;
bh=uwDl8Y3p9PYBSmwhEfkzjbSDGdahr46mdfxOK3pKDPQ=;
b=DQumi73DbqdWnhBfBwmX0bF7ArAKMHO82GYN3dOX7RuTp1DSti5IwP4g
SnOw955ebjKSIzCfOAmdo8djorkpUpkf76UdxuUFRkcpNEkmlsUSmW99j
idVXpDYEXoNrzTlQtw9Nt6/kCOnnji9KTlfVu3jhOuXHAFhp9T1cvRs3s g=;
Authentication-Results: mail2-relais-roc.national.inria.fr;
dkim=none (message not signed) header.i=none;
spf=SoftFail smtp.mailfrom=ludovic.courtes@HIDDEN;
dmarc=fail (p=none dis=none) d=inria.fr
X-IronPort-AV: E=Sophos;i="6.15,194,1739833200"; d="scan'208";a="216664827"
Received: from 91-160-117-201.subs.proxad.net (HELO ribbon) ([91.160.117.201])
by mail2-relais-roc.national.inria.fr with
ESMTP/TLS/ECDHE-RSA-AES256-GCM-SHA384; 07 Apr 2025 12:20:26 +0200
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: Octidi 18 Germinal an 233 de la =?UTF-8?Q?R=C3=A9volution,?= jour de la =?UTF-8?Q?Cigu=C3=AB?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Mon, 07 Apr 2025 12:20:26 +0200
Message-ID: <87frik9st1.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=192.134.164.83;
envelope-from=ludovic.courtes@HIDDEN;
helo=mail2-relais-roc.national.inria.fr
X-Spam_score_int: -43
X-Spam_score: -4.4
X-Spam_bar: ----
X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H2=-0.01,
RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001,
SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hello,
As Christopher Baines reported on IRC, calling =E2=80=98fdatasync=E2=80=99 =
on each
cached narinfo under /var/guix/substitute/cache can be unreasonably
expensive on spinning disks.
On bayfront, when tracing =E2=80=98guix substitute=E2=80=99 processes, we o=
bserve
=E2=80=98fdatasync=E2=80=99 occasionally taking more than a second, and typ=
ically taking
more than 0.2s:
--8<---------------cut here---------------start------------->8---
mkdir("/var", 0777) =3D -1 EEXIST (File exists) <0.0000=
48>
mkdir("/var/guix", 0777) =3D -1 EEXIST (File exists) <0.0000=
42>
mkdir("/var/guix/substitute", 0777) =3D -1 EEXIST (File exists) <0.0000=
40>
mkdir("/var/guix/substitute/cache", 0777) =3D -1 EEXIST (File exists) <0.00=
0042>
mkdir("/var/guix/substitute/cache/kzwjeblndsbkjzmjailrt4bnhguil7tqjmewzcyw2=
2hgajbhfy3q", 0777) =3D -1 EEXIST (File exists
) <0.000041>
clock_gettime(CLOCK_MONOTONIC, {tv_sec=3D153237, tv_nsec=3D595042938}) =3D =
0 <0.000037>
openat(AT_FDCWD, "/var/guix/substitute/cache/kzwjeblndsbkjzmjailrt4bnhguil7=
tqjmewzcyw22hgajbhfy3q/nfmn2nkfjz5kzklw165aj
6ilwphw9zrz.3kAGMM", O_RDWR|O_CREAT|O_EXCL, 0600) =3D 15 <0.000123>
fstat(15, {st_mode=3DS_IFREG|0600, st_size=3D0, ...}) =3D 0 <0.000037>
lseek(15, 0, SEEK_CUR) =3D 0 <0.000038>
write(15, "(narinfo (version 2) (cache-uri "..., 1066) =3D 1066 <0.000071>
fdatasync(15) =3D 0 <1.860604>
--8<---------------cut here---------------end--------------->8---
A longer sample:
--8<---------------cut here---------------start------------->8---
ludo@bayfront ~$ grep ^fdatasync log.strace=20
fdatasync(15) =3D 0 <1.860604>
fdatasync(15) =3D 0 <1.101868>
fdatasync(15) =3D 0 <0.464357>
fdatasync(15) =3D 0 <0.363363>
fdatasync(15) =3D 0 <0.307759>
fdatasync(15) =3D 0 <0.619979>
fdatasync(15) =3D 0 <0.269880>
fdatasync(15) =3D 0 <0.274093>
fdatasync(15) =3D 0 <0.247207>
fdatasync(15) =3D 0 <0.367507>
fdatasync(15) =3D 0 <0.212277>
fdatasync(15) =3D 0 <0.234929>
fdatasync(15) =3D 0 <0.330697>
fdatasync(15) =3D 0 <0.193925>
fdatasync(15) =3D 0 <0.251554>
fdatasync(15) =3D 0 <0.379571>
--8<---------------cut here---------------end--------------->8---
That call comes from =E2=80=98with-atomic-file-output=E2=80=99, commit
1752a17a1e6f7138892eeeb4806cd04ccb3ca1b0, which refers to this
discussion:
https://lists.gnu.org/archive/html/guix-devel/2016-06/msg00456.html
It probably makes sense in some uses of =E2=80=98with-atomic-file-output=E2=
=80=99, but
nor for a cache as is the case here.
Ludo=E2=80=99.
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludovic.courtes@HIDDEN> Subject: bug#77606: Acknowledgement (=?UTF-8?Q?=E2=80=98fdatasync=E2=80=99?= on cached narinfo files is too expensive) Message-ID: <handler.77606.B.174402124517890.ack <at> debbugs.gnu.org> References: <87frik9st1.fsf@HIDDEN> X-Gnu-PR-Message: ack 77606 X-Gnu-PR-Package: guix Reply-To: 77606 <at> debbugs.gnu.org Date: Mon, 07 Apr 2025 10:21:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. As you requested using X-Debbugs-CC, your message was also forwarded to Christopher Baines <guix@HIDDEN> (after having been given a bug report number, if it did not have one). Your message has been sent to the package maintainer(s): bug-guix@HIDDEN If you wish to submit further information on this problem, please send it to 77606 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 77606: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D77606 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.