Simon Tournier <zimon.toutoune@HIDDEN>
to control <at> debbugs.gnu.org.
Full text available.
Received: (at 41797) by debbugs.gnu.org; 11 Mar 2026 09:01:01 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Mar 11 05:01:00 2026
Received: from localhost ([127.0.0.1]:46739 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1w0FR3-0006Dj-9y
for submit <at> debbugs.gnu.org; Wed, 11 Mar 2026 05:01:00 -0400
Received: from mail-wm1-x331.google.com ([2a00:1450:4864:20::331]:53496)
by debbugs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.84_2) (envelope-from <zimon.toutoune@HIDDEN>)
id 1w0FQo-00068x-Og
for 41797 <at> debbugs.gnu.org; Wed, 11 Mar 2026 05:00:46 -0400
Received: by mail-wm1-x331.google.com with SMTP id
5b1f17b1804b1-48534b59cf3so31023665e9.2
for <41797 <at> debbugs.gnu.org>; Wed, 11 Mar 2026 02:00:42 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=gmail.com; s=20230601; t=1773219641; x=1773824441; darn=debbugs.gnu.org;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:from:to:cc:subject:date
:message-id:reply-to;
bh=1y3OVAlhks8TIEcr5bxWwRnepZ3lJNrDheMxEdaEjWY=;
b=kAlhxMXTIyO1V8MbwUwzsrYRQSCReRQ34ZwZU8TtVzTlsAcS363W/HuXX2fbaFhr86
bx9MZhoYkt57Ll8J3Sqxxj5zmzsYqHMjhP5m+954pcBluUzDgfmH7JKjldRZ11blY8aO
QFHKy6+pjlNjMjc/ek5NDb8p2Iubzt4iK6gLcAJJUDeKnA9Tk12h5wUpNigVVhKJrq4Z
KcBZ5x5rnhCtX9BFmrhRTGgcPaZveQJp4GszLAdN+40MxyxsiMxO3nU10FAY2rv9d0cy
OXGE9YpE34oew6AGv+1xfW4sWBF00RidXY2HpOpu/dJJgGnCZfR/174unTtQj57LEXY+
97sQ==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20230601; t=1773219641; x=1773824441;
h=content-transfer-encoding:mime-version:user-agent:message-id:date
:references:in-reply-to:subject:cc:to:from:x-gm-gg
:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to;
bh=1y3OVAlhks8TIEcr5bxWwRnepZ3lJNrDheMxEdaEjWY=;
b=iIbHFiwjkhsNBgyED9rHyp8BHjCHr3n16cSqqqMEgHDbSclsaL2rp8YV8CkSL4rLqV
6EiWKwtcYnX30YhEBPx5julNbwQssyk9kFFx3v0mcCJihE4j0WbeYCAFUJOOfQolWGAx
G0GsgLp4sxk7N3HW00T6XE8Ui6qLgPoNybWUggSNU9FhEWQxfmsVrfhrNiGc8hYQjX+7
g+ykg5lEmZaEJNmo6k22k9++ULpdeoJJ6PG3zUD/vBl5QJF7CpGgj28iRhzfJhGrrkjZ
DokMRfdNBIk0ifyUSNwZEGpe0vJTHKTzqS296TMRDUnIhbexsdMpgm0CExIeSXZtvH4g
vd9g==
X-Gm-Message-State: AOJu0YydsLP8M4lrVp6+M5c240zxgxISA/1p/ROqKX0GQmxnb+zjnNop
uyTmAYg14A+vKWdDaVzoNyjXrhjZB7iUt+J1OeEtJVt1T2izmC2JPMWc
X-Gm-Gg: ATEYQzy+H5mBkoNW8P/qkYqNumYIsyvUWf9lUP6nQ32sgDaAnvsmVyHvI+KVs8iVBXk
Cqt0+9y0Zk6Zx3QehjktNvx/RdxeS0ND7XjtMAAcrvxFwLKt5Rkf+PPyz/Twt7LLgakK0acIsCU
lqwVTxLFkmjzNlFO16gX3qqgsvnJdkiC9FssOZvHCdG7ZVZdljswTsUyofC5laSIFuiI4AROG3G
q33NYQvCuVhJsVahsD0T7QHtd3b625L2ACvqxbG6oFgN5cnFG7G7PYr+9mh5/UA8Cr4anjmAv/J
yN1HRPgRLid9rHg4QQCrNEdpXLzt0TFu4cz8uU5orTBRkAbPYfLiTBHC4dk8UiHT7OCt6zup1na
KDqsvqJESG0qseQJRH5vZ9V+QGe19VCDNV1okMtFy0UiBGypYuko+RY+uLqIQcnD0x9uOL/LB8T
CKmmAKcwo7GofmiVDERAmx
X-Received: by 2002:a05:600c:3b16:b0:485:4535:73d with SMTP id
5b1f17b1804b1-4854b0a4920mr28172645e9.2.1773219640604;
Wed, 11 Mar 2026 02:00:40 -0700 (PDT)
Received: from lili ([2a01:e0a:59b:9120:a734:9c0a:3494:d211])
by smtp.gmail.com with ESMTPSA id
5b1f17b1804b1-4854b0d4977sm14095465e9.19.2026.03.11.02.00.39
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Wed, 11 Mar 2026 02:00:40 -0700 (PDT)
From: Simon Tournier <zimon.toutoune@HIDDEN>
To: Caleb Ristvedt <caleb.ristvedt@HIDDEN>
Subject: Re: [bug#41797] [PATCH] nar: fix race conditions inherent to
with-temporary-store-file.
In-Reply-To: <87k10e5io6.fsf@HIDDEN> (Caleb Ristvedt's message of "Wed, 10
Jun 2020 23:29:45 -0500")
References: <87k10e5io6.fsf@HIDDEN>
Date: Wed, 11 Mar 2026 08:50:18 +0100
Message-ID: <87jyvieret.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: 1.0 (+)
X-Debbugs-Envelope-To: 41797
Cc: maxim@HIDDEN, zimon.toutoune@HIDDEN, morgan.arnold@HIDDEN,
ludo@HIDDEN, rutherther@HIDDEN, 41797 <at> debbugs.gnu.org, guix@HIDDEN,
ngraves@HIDDEN
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: 0.0 (/)
Hi,
For more context, see:
https://issues.guix.gnu.org/issue/41797
CC: core team
On Wed, 10 Jun 2020 at 23:29, Caleb Ristvedt <caleb.ristvedt@HIDDEN> wrot=
e:
> This addresses a problem that it's very likely nobody has run into yet,
> due to the sheer size of the tempfile namespace, but that could
> potentially cause serious issues, including store corruption.
It=E2=80=99s a long time that the patch is here and indeed nobody seems hav=
ing
run into yet. :-)
> with-temporary-store-file has a fundamental flaw: it assumes that if a
> temporary store file exists, is then added as a temporary root, and still
> exists, then it uniquely belongs to the current process. This is not alw=
ays
> the case, because the only criteria used for choosing temporary file name=
s is
> that they not be currently used and fit a certain template. This means i=
t's
> entirely possible for another process to choose the same temporary file n=
ame
> if it doesn't exist at the time it chooses.
>
> Suppose process A chooses F as its temporary file name in
> with-temporary-store-file. Suppose before it adds it as a temp root, F g=
ets
> garbage collected. Suppose process B then happens to, at that point, cho=
ose F
> as its temporary file name. This is completely valid, because F doesn't =
exist
> at that point. Then process A will check whether F exists, see that it d=
oes,
> and assume that it uniquely owns that file, when in reality two processes=
are
> now simultaneously trying to use it. Process A will then delete the file=
in
> question, causing errors in process A, B, or both.
>
> The same issue (two processes both believing they've exclusively allocate=
d a
> temp file) can occur any time between when process A deletes the temp file
> prior to running the body and when (if) its body actually creates the temp
> file.
>
> In short, allocating a temporary file *name* isn't possible with the
> mechanisms currently in place - we have no way of telling other users of
> with-temporary-store-file whether a file name is allocated or not. Alloc=
ating
> a temporary *file*, on the other hand, is possible. This removes
> with-temporary-store-file and replaces it with with-temporary-restored-fi=
le,
> and adjusts its only user, restore-one-item, accordingly.
>
> * guix/serialization.scm (restore-to-temp-store-file): new procedure.
> * guix/nar.scm (with-temporary-store-file): replaced with
> with-temporary-restored-file.
> (with-temporary-restored-file): new macro.
> (restore-one-item): use with-temporary-restored-file instead of
> with-temporary-store-file.
> (temporary-store-file): removed, as it is no longer used.
> (Local Variables footer): update indenting information for
> with-temporary-restored-file.
> ---
> guix/nar.scm | 38 +++++++-------------
> guix/serialization.scm | 79 +++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 90 insertions(+), 27 deletions(-)
Thanks for the explanation.
Well, the last activity for this submission had been 5 years, 37 weeks,
2 days ago. What is the next actionable step to move forward?
On my side, I=E2=80=99m not sure to understand the deep implication of this
=E2=80=9Cfundamental flaw=E2=80=9D. Does it have security implication?
Cheers,
simon
guix-patches@HIDDEN:bug#41797; Package guix-patches.
Full text available.
Received: (at 41797) by debbugs.gnu.org; 22 Jun 2020 18:03:36 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Jun 22 14:03:36 2020
Received: from localhost ([127.0.0.1]:34481 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jnQn9-0006AP-T8
for submit <at> debbugs.gnu.org; Mon, 22 Jun 2020 14:03:36 -0400
Received: from mail-io1-f67.google.com ([209.85.166.67]:39673)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <caleb.ristvedt@HIDDEN>) id 1jnQn6-0006A9-3B
for 41797 <at> debbugs.gnu.org; Mon, 22 Jun 2020 14:03:34 -0400
Received: by mail-io1-f67.google.com with SMTP id f23so15234916iof.6
for <41797 <at> debbugs.gnu.org>; Mon, 22 Jun 2020 11:03:32 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=cune-org.20150623.gappssmtp.com; s=20150623;
h=from:to:cc:subject:references:date:in-reply-to:message-id
:user-agent:mime-version;
bh=7qjhI9T9CmkCGl4VatFW+pWr447AAF4dSkNQEgKfMyA=;
b=xwufMRMfvVZ+lqiLqBS3dUdszLP+gMiWZ/u2LAYGrjzdceGpvK4XV9QIFTm3v3GBOK
d1ZDxLKUC8l8I3ppE/gFjUFkIoaLqDrMBbOnZDa7p/xckAFw6TDC0vDkRUCmf+8q9pEC
MglKup/gO0V2bdy9gKdRUn5jO0YKqMIre0I/OrBHWh4D295QAahOq1xSZQ+qYBxXOF2b
Wj/RQmsUqIjCu3vgMP322OrHeDJjpVEV4D4tJn+IoBs84UdHwMMuttASpovy+2Ywwcm8
BtWl6tSo2BVtJrQshZUaZXjJPS+m6ZdF6JrUrDdXgwKP8p5KK3rNqGiP4KuRxNkltS2T
I5tA==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to
:message-id:user-agent:mime-version;
bh=7qjhI9T9CmkCGl4VatFW+pWr447AAF4dSkNQEgKfMyA=;
b=F6X7up6DFvr0xjkklpi/vfxiiCuu4/ocy59IIWtkyhMLTIaFOrpqZRhJEJ2Vbuuwz0
t24ZCyH5fIz8fnLOOEzdlIqlDfTnAWXnB2vxhbY4NvmVOdRq/Mqte5Ao+WOuEQbLA6mk
fxmh1nRKmLhKm9xKDjJP8TLZRYJ+5VYsv1WEFiEJt1RvrJykgWP1gIX1apFldU7ngxZG
SsWr8/PeMB9aBwhR298Po01IlmdZXvLE8NVUJ9MF1IfsnljLGgoscAxSC22z0gUnEUwU
p/QmNgYduk2Rie6lwJSIijl+0JKnLJgVyXpHrqTlUh2ZA5SHmW4P0LuHAZSvSU2GZFoO
eQJw==
X-Gm-Message-State: AOAM533SsUSfCNwZ5F6HdAF7hp7M9FHTGbPAAkr39TPtpAxPsSFdDURO
hfBBUfHi9219Q7tNpzJAwxXduWe9vSgyaA==
X-Google-Smtp-Source: ABdhPJwemxmcR+S/WL5QDUfUAXlTSib0yjrsHGoVlVewGV5bRk3t6PuodcxLfKyzw7xDEtfYI6NNvA==
X-Received: by 2002:a02:694c:: with SMTP id e73mr12591049jac.17.1592849006078;
Mon, 22 Jun 2020 11:03:26 -0700 (PDT)
Received: from GuixPotato ([208.89.170.24])
by smtp.gmail.com with ESMTPSA id y13sm8612128iob.51.2020.06.22.11.03.24
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Mon, 22 Jun 2020 11:03:25 -0700 (PDT)
From: Caleb Ristvedt <caleb.ristvedt@HIDDEN>
To: Ludovic =?utf-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Subject: Re: [bug#41797] [PATCH] replace with-temporary-store-file
References: <87k10e5io6.fsf@HIDDEN> <877dwdbkkf.fsf@HIDDEN>
Date: Mon, 22 Jun 2020 13:03:12 -0500
In-Reply-To: <877dwdbkkf.fsf@HIDDEN> ("Ludovic
\=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
\=\?utf-8\?Q\?s\?\= message of "Thu, 11 Jun 2020 19:04:32 +0200")
Message-ID: <874kr35673.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
micalg=pgp-sha256; protocol="application/pgp-signature"
X-Spam-Score: -0.0 (/)
X-Debbugs-Envelope-To: 41797
Cc: 41797 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Apologies for the delay on this.
Ludovic Court=C3=A8s <ludo@HIDDEN> writes:
> Hi,
>
> Caleb Ristvedt <caleb.ristvedt@HIDDEN> skribis:
>
>> with-temporary-store-file has a fundamental flaw: it assumes that if a
>> temporary store file exists, is then added as a temporary root, and still
>> exists, then it uniquely belongs to the current process. This is not al=
ways
>> the case, because the only criteria used for choosing temporary file nam=
es is
>> that they not be currently used and fit a certain template. This means =
it's
>> entirely possible for another process to choose the same temporary file =
name
>> if it doesn't exist at the time it chooses.
>
> Then what about simply adding the PID to the file name template?
AFAIK that would work currently, though for the daemon we'd have to also
include a fiber identifier. What concerns me is that I can't find any
restrictions about the characters that mkstemp is allowed to use, so
we'd have to enforce a consistent tempfile naming scheme. Admittedly, I
spent 10 minutes trying to think of realistic ways in which collisions
could still occur and couldn't come up with anything - the fact that the
randomized portion is always the same length and at the end makes it
quite difficult.
It's worth noting that adding the PID (and fiber ID) to the template
will only make it thread-safe, not reentrant. So
with-temporary-store-file uses couldn't be nested - indeed, no temporary
store files that use the same template could be safely created within
the extent of with-temporary-store-file. We could add a parameter to
track all the "reserved" filenames for the current dynamic state, but it
seems like a lot of hassle, and there's still the risk that other
temp-file-creating code could simply ignore it.
I'd still prefer restore-to-temp-store-file for the stronger guarantees
it gives.
=2D reepca
--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEdNapMPRLm4SepVYGwWaqSV9/GJwFAl7w8mEACgkQwWaqSV9/
GJwvbwf+MCNm6d+ymsOi0/bbDvbCUv5IjeUsur34e4RnGg4Sy0MvY7zLcRPD1Y1C
aY5ucOYEwfkmWkkv7pk9Zk4ZFnxPutza5d/13CQKFTkoSjPM8UOcr8AiISmBiALM
rMmdCRCZ+qYX8NB56ylA0JEk6xL7tagM5FAyKSgDrkLlgJ46bTO9fQPRRM15sbAp
fjCdP8VmOHwj6BxLHItXCjkLxz7iNZ70C/JqwSzlU0UWmBRYF/+pZb60OR46MoBE
FSDsYGo0YAbMdoZsgxFDYx87pFbAVcAC9NOCyiWL2fahqHzNwjRK9DcQ6QtaWz6X
VJYBd5Ck5uvqHe4QAs5Wuq9ItLdZsQ==
=RZqc
-----END PGP SIGNATURE-----
--=-=-=--
guix-patches@HIDDEN:bug#41797; Package guix-patches.
Full text available.Received: (at 41797) by debbugs.gnu.org; 11 Jun 2020 17:04:43 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 11 13:04:43 2020 Received: from localhost ([127.0.0.1]:37915 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1jjQd9-000288-EW for submit <at> debbugs.gnu.org; Thu, 11 Jun 2020 13:04:43 -0400 Received: from eggs.gnu.org ([209.51.188.92]:50416) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1jjQd7-00027s-TO for 41797 <at> debbugs.gnu.org; Thu, 11 Jun 2020 13:04:42 -0400 Received: from fencepost.gnu.org ([2001:470:142:3::e]:37851) by eggs.gnu.org with esmtp (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1jjQd2-0007YE-49; Thu, 11 Jun 2020 13:04:36 -0400 Received: from [2a01:e0a:1d:7270:6a6c:dc17:fc02:cfda] (port=51998 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1jjQd1-0005qN-DL; Thu, 11 Jun 2020 13:04:35 -0400 From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN> To: Caleb Ristvedt <caleb.ristvedt@HIDDEN> Subject: Re: [bug#41797] [PATCH] replace with-temporary-store-file References: <87k10e5io6.fsf@HIDDEN> Date: Thu, 11 Jun 2020 19:04:32 +0200 In-Reply-To: <87k10e5io6.fsf@HIDDEN> (Caleb Ristvedt's message of "Wed, 10 Jun 2020 23:29:45 -0500") Message-ID: <877dwdbkkf.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 41797 Cc: 41797 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) Hi, Caleb Ristvedt <caleb.ristvedt@HIDDEN> skribis: > with-temporary-store-file has a fundamental flaw: it assumes that if a > temporary store file exists, is then added as a temporary root, and still > exists, then it uniquely belongs to the current process. This is not alw= ays > the case, because the only criteria used for choosing temporary file name= s is > that they not be currently used and fit a certain template. This means i= t's > entirely possible for another process to choose the same temporary file n= ame > if it doesn't exist at the time it chooses. Then what about simply adding the PID to the file name template? Trying to see if there=E2=80=99s a simpler solution that could address the problem. Thanks, Ludo=E2=80=99.
guix-patches@HIDDEN:bug#41797; Package guix-patches.
Full text available.
Received: (at submit) by debbugs.gnu.org; 11 Jun 2020 04:30:04 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Jun 11 00:30:04 2020
Received: from localhost ([127.0.0.1]:35777 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1jjEqp-0000MK-M3
for submit <at> debbugs.gnu.org; Thu, 11 Jun 2020 00:30:04 -0400
Received: from lists.gnu.org ([209.51.188.17]:41510)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <caleb.ristvedt@HIDDEN>) id 1jjEqn-0000LV-Nh
for submit <at> debbugs.gnu.org; Thu, 11 Jun 2020 00:30:02 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:60198)
by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
(Exim 4.90_1) (envelope-from <caleb.ristvedt@HIDDEN>)
id 1jjEqn-0006Zn-FE
for guix-patches@HIDDEN; Thu, 11 Jun 2020 00:30:01 -0400
Received: from mail-il1-x143.google.com ([2607:f8b0:4864:20::143]:42219)
by eggs.gnu.org with esmtps (TLS1.3:ECDHE_RSA_AES_128_GCM_SHA256:128)
(Exim 4.90_1) (envelope-from <caleb.ristvedt@HIDDEN>)
id 1jjEql-0000g5-8K
for guix-patches@HIDDEN; Thu, 11 Jun 2020 00:30:00 -0400
Received: by mail-il1-x143.google.com with SMTP id j19so35483ilk.9
for <guix-patches@HIDDEN>; Wed, 10 Jun 2020 21:29:53 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=cune-org.20150623.gappssmtp.com; s=20150623;
h=from:to:subject:date:message-id:user-agent:mime-version;
bh=W4S1oMK9uG4Mh+wpZd/HN2ELVd8LpEFmNPTe9SI7wSg=;
b=eC2nsM62o7tKUaPtoi57A6Tg4nve51kL7ima/kg2LWRfMeSQbe4u4RPtri1UT54CB7
zee0bH1D+95RwefgpG3ZPSfod4rZhVu7nVHI0zVyx+qz+QsBBoch751ki2ycUr0CViQN
yLUzue3NtdAua9pbZ81/h4nohEFc+vkrwPG1n3RglU7Mh+EpE+fG2YaffaCUg5S1NKuc
A+xNQP4YXUeAdzgF6btQ5PekdVkVCEuViJ7X1VUNX8yvomePdrKKz7feNdxVv+o1Q+a7
WLqM7OWcWBLo8nzg21jeTfb8oqX4L4UbIpcXbceve7rnoUqj1oIU5aaivz2aKScVAU+M
h8Sg==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
d=1e100.net; s=20161025;
h=x-gm-message-state:from:to:subject:date:message-id:user-agent
:mime-version;
bh=W4S1oMK9uG4Mh+wpZd/HN2ELVd8LpEFmNPTe9SI7wSg=;
b=DQbkDpW3EpZRa976nE/vmYtglVlLFbzbp35lCE3ARt6FroL70yiTxpFh1E3hOV7lA/
/BSeSFlvm/oSHQxycFJFo7+/Qk8OAAHHRZAWq7KW+NulNvZ6wwcibt7lAZ3y2fe5jTRJ
G1p9GUOgIcha4g20UYon2zHzreSUVj4TU5c5zrlDQxLaD7FafDF5xSE1UD7YM0h+NBxv
rTzo/N86ZgUCdoacYuUE1h57UDstZAesMt3HFFYjJ4TbKcVrw4kwBZ6Z172LFdEo5m08
bKhSE5C8Y7hT1dCWoNdnHrwkKXNaLDCqspcX9dmeauivelyT+9uP6zdKhTd3ZoHsOoQ1
I7GA==
X-Gm-Message-State: AOAM530FnC1J9TC2f3E5yi/5+BMAkbURHp5hoy9JBnWuseW3dj451owh
h6e87BOvElE+qPeZynBPrfyf9UJZqIk=
X-Google-Smtp-Source: ABdhPJyy7Zb9iNfladjcL8iqIVMtFawcWUD2XM0O6lmV0hl/xyDp8St4MlOJ/c2JyhheRLuCh7dQ4A==
X-Received: by 2002:a92:506:: with SMTP id q6mr6423193ile.107.1591849791775;
Wed, 10 Jun 2020 21:29:51 -0700 (PDT)
Received: from GuixPotato ([208.89.170.24])
by smtp.gmail.com with ESMTPSA id d13sm977891ilo.40.2020.06.10.21.29.50
for <guix-patches@HIDDEN>
(version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
Wed, 10 Jun 2020 21:29:50 -0700 (PDT)
From: Caleb Ristvedt <caleb.ristvedt@HIDDEN>
To: guix-patches@HIDDEN
Subject: [PATCH] replace with-temporary-store-file
Date: Wed, 10 Jun 2020 23:29:45 -0500
Message-ID: <87k10e5io6.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="==-=-=";
micalg=pgp-sha256; protocol="application/pgp-signature"
Received-SPF: pass client-ip=2607:f8b0:4864:20::143;
envelope-from=caleb.ristvedt@HIDDEN; helo=mail-il1-x143.google.com
X-detected-operating-system: by eggs.gnu.org: No matching host in p0f cache.
That's all we know.
X-Spam_score_int: -18
X-Spam_score: -1.9
X-Spam_bar: -
X-Spam_report: (-1.9 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
DKIM_VALID=-0.1, RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
SPF_PASS=-0.001 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)
--==-=-=
Content-Type: multipart/mixed; boundary="=-=-="
--=-=-=
Content-Type: text/plain
with-temporary-store-file suffers from race conditions - see attached
patch commit message for details.
This addresses a problem that it's very likely nobody has run into yet,
due to the sheer size of the tempfile namespace, but that could
potentially cause serious issues, including store corruption.
The new procedure used to resolve this (restore-to-temp-store-file) will
be of use in the guile-daemon anyway.
- reepca
--=-=-=
Content-Type: text/x-patch
Content-Disposition: attachment;
filename=0001-nar-fix-race-conditions-inherent-to-with-temporary-s.patch
Content-Transfer-Encoding: quoted-printable
From=208c26304951a2e21fe9b373b9cd543353d08fe003 Mon Sep 17 00:00:00 2001
From: Caleb Ristvedt <caleb.ristvedt@HIDDEN>
Date: Wed, 10 Jun 2020 22:24:27 -0500
Subject: [PATCH] nar: fix race conditions inherent to
with-temporary-store-file.
with-temporary-store-file has a fundamental flaw: it assumes that if a
temporary store file exists, is then added as a temporary root, and still
exists, then it uniquely belongs to the current process. This is not always
the case, because the only criteria used for choosing temporary file names =
is
that they not be currently used and fit a certain template. This means it's
entirely possible for another process to choose the same temporary file name
if it doesn't exist at the time it chooses.
Suppose process A chooses F as its temporary file name in
with-temporary-store-file. Suppose before it adds it as a temp root, F gets
garbage collected. Suppose process B then happens to, at that point, choos=
e F
as its temporary file name. This is completely valid, because F doesn't ex=
ist
at that point. Then process A will check whether F exists, see that it doe=
s,
and assume that it uniquely owns that file, when in reality two processes a=
re
now simultaneously trying to use it. Process A will then delete the file in
question, causing errors in process A, B, or both.
The same issue (two processes both believing they've exclusively allocated a
temp file) can occur any time between when process A deletes the temp file
prior to running the body and when (if) its body actually creates the temp
file.
In short, allocating a temporary file *name* isn't possible with the
mechanisms currently in place - we have no way of telling other users of
with-temporary-store-file whether a file name is allocated or not. Allocat=
ing
a temporary *file*, on the other hand, is possible. This removes
with-temporary-store-file and replaces it with with-temporary-restored-file,
and adjusts its only user, restore-one-item, accordingly.
* guix/serialization.scm (restore-to-temp-store-file): new procedure.
* guix/nar.scm (with-temporary-store-file): replaced with
with-temporary-restored-file.
(with-temporary-restored-file): new macro.
(restore-one-item): use with-temporary-restored-file instead of
with-temporary-store-file.
(temporary-store-file): removed, as it is no longer used.
(Local Variables footer): update indenting information for
with-temporary-restored-file.
=2D--
guix/nar.scm | 38 +++++++-------------
guix/serialization.scm | 79 +++++++++++++++++++++++++++++++++++++++++-
2 files changed, 90 insertions(+), 27 deletions(-)
diff --git a/guix/nar.scm b/guix/nar.scm
index eff4becbce..2666d32169 100644
=2D-- a/guix/nar.scm
+++ b/guix/nar.scm
@@ -128,30 +128,18 @@ held."
(force-output lock)
(unlock-file lock))))))
=20
=2D(define (temporary-store-file)
=2D "Return the file name of a temporary file created in the store."
=2D (let* ((template (string-append (%store-prefix) "/guix-XXXXXX"))
=2D (port (mkstemp! template)))
=2D (close-port port)
=2D template))
=2D
=2D(define-syntax-rule (with-temporary-store-file name body ...)
+(define-syntax-rule (with-temporary-restored-file port name body ...)
"Evaluate BODY with NAME bound to the file name of a temporary store item
=2Dprotected from GC."
+restored from PORT and protected from GC. Note that the temporary store i=
tem
+won't be automatically deleted."
(with-store store
=2D (let loop ((name (temporary-store-file)))
=2D ;; Add NAME to the current process' roots. (Opening this connecti=
on to
=2D ;; the daemon allows us to reuse its code that deals with the
=2D ;; per-process roots file.)
=2D (add-temp-root store name)
=2D
=2D ;; There's a window during which GC could delete NAME. Try again =
when
=2D ;; that happens.
=2D (if (file-exists? name)
=2D (begin
=2D (delete-file name)
=2D body ...)
=2D (loop (temporary-store-file))))))
+ ;; Add NAME to the current process' roots. (Opening this connection to
+ ;; the daemon allows us to reuse its code that deals with the per-proc=
ess
+ ;; roots file.)
+ (let ((name (restore-to-temp-store-file port
+ (lambda (root)
+ (add-temp-root store root)))=
))
+ body ...)))
=20
(define* (restore-one-item port
#:key acl (verify-signature? #t) (lock? #t)
@@ -208,9 +196,7 @@ s-expression"))
=20
(let-values (((port get-hash)
(open-sha256-input-port port)))
=2D (with-temporary-store-file temp
=2D (restore-file port temp)
=2D
+ (with-temporary-restored-file port temp
(let ((magic (read-int port)))
(unless (=3D magic %export-magic)
(raise (condition
@@ -286,7 +272,7 @@ while the locks are held."
(port port) (file #f) (token #f))))))))
=20
;;; Local Variables:
=2D;;; eval: (put 'with-temporary-store-file 'scheme-indent-function 1)
+;;; eval: (put 'with-temporary-restored-file 'scheme-indent-function 2)
;;; End:
=20
;;; nar.scm ends here
diff --git a/guix/serialization.scm b/guix/serialization.scm
index 836ad06caf..c6b9674104 100644
=2D-- a/guix/serialization.scm
+++ b/guix/serialization.scm
@@ -18,6 +18,7 @@
=20
(define-module (guix serialization)
#:use-module (guix combinators)
+ #:use-module (guix config)
#:use-module (rnrs bytevectors)
#:use-module (srfi srfi-1)
#:use-module (srfi srfi-26)
@@ -28,6 +29,7 @@
#:use-module (ice-9 match)
#:use-module (ice-9 ftw)
#:use-module (system foreign)
+ #:use-module (rnrs io ports)
#:export (write-int read-int
write-long-long read-long-long
write-padding
@@ -51,7 +53,8 @@
write-file
write-file-tree
fold-archive
=2D restore-file))
+ restore-file
+ restore-to-temp-store-file))
=20
;;; Comment:
;;;
@@ -477,6 +480,80 @@ Restore it as FILE."
port
file))
=20
+(define* (restore-to-temp-store-file port add-temp-root #:optional name)
+ "Read a file (possibly a directory structure) in Nar format from PORT.
+Restore it as a temporary file and return the temporary file name.
+ADD-TEMP-ROOT must be a procedure of one argument (the file name) that
+protects that file name from garbage collection. Note that ADD-TEMP-ROOT =
may
+be called multiple times."
+ (define (tempname)
+ (let ((template (string-append %store-directory "/guix-temp-file"
+ (if name
+ (string-append "-" name)
+ "")
+ ".XXXXXX")))
+ (close (mkstemp! template))
+ (delete-file template)
+ template))
+
+ (define (create/top-level create)
+ (let ((name (tempname)))
+ (add-temp-root name)
+ (catch 'system-error
+ (lambda ()
+ (create name)
+ name)
+ (lambda args
+ (if (=3D (system-error-errno args) EEXIST)
+ (mkdir/top-level)
+ (apply throw args))))))
+
+ (define (create-output-file name input size type)
+ (call-with-port (open name (logior O_WRONLY
+ O_EXCL
+ O_CREAT))
+ (lambda (output)
+ (dump input output size)
+ (when (eq? type 'executable)
+ (chmod output #o755)))))
+
+ (define (create-output-file/top-level input size type)
+ (create/top-level (cut create-output-file <> input size type)))
+
+ (define mkdir/top-level
+ (cut create/top-level mkdir))
+
+ (define (symlink/top-level target)
+ (create/top-level (cut symlink target <>)))
+
+ (fold-archive (lambda (file type content top-name)
+ (define-syntax-rule (if-top-level exp1 exp2)
+ (if top-name
+ (begin
+ exp2
+ top-name)
+ exp1))
+
+ (match type
+ ('directory
+ (if-top-level (mkdir/top-level)
+ (mkdir (string-append top-name file))))
+ ('symlink
+ (if-top-level
+ (symlink/top-level content)
+ (symlink content (string-append top-name file))))
+ ((or 'regular 'executable)
+ (match content
+ ((input . size)
+ (if-top-level
+ (create-output-file/top-level input size type)
+ (create-output-file (string-append top-name
+ file)
+ input size type)))))))
+ #f
+ port
+ ""))
+
;;; Local Variables:
;;; eval: (put 'call-with-binary-input-file 'scheme-indent-function 1)
;;; End:
=2D-=20
2.26.2
--=-=-=--
--==-=-=
Content-Type: application/pgp-signature; name="signature.asc"
-----BEGIN PGP SIGNATURE-----
iQEzBAEBCAAdFiEEdNapMPRLm4SepVYGwWaqSV9/GJwFAl7hszoACgkQwWaqSV9/
GJxEMQgAibod0c4JnSgJ3aSvFMaqZRQ+uZIB8RAWDAHq+TH7GFXul1YTilFdFFaC
gwSeEjjQvmpMmH0OK4tUHJNRiu/pmEtg1PjDXZgez9bTkYMYuZAEQd27ciTQCLhx
O0Z1nl1oP9O+CCMcCcOdxygHJOBe/ARekE9EJdXpVqAFtW006QHRn5YnEGRw0dP1
LPl9QqZD1snm2mAd48oX39RJAmIxptg+9CPr+MD9haXzy17Cue6P/jdPgTkVKPGK
l4kqojing7ySTF93gNEXPUBTzfO+/Cv+9C6nGRf8JoHewTml3FuypGlOgeURC+In
S9S5AFT1GNyfThbYfUN2W1CEIAfE7A==
=vRI3
-----END PGP SIGNATURE-----
--==-=-=--
Caleb Ristvedt <caleb.ristvedt@HIDDEN>:guix-patches@HIDDEN.
Full text available.guix-patches@HIDDEN:bug#41797; Package guix-patches.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.