GNU logs - #74203, boring messages

Message sent to bug-guix@HIDDEN:

Subject: bug#74203: coreutils fails to build
Resent-From: "Collin J. Doering" <collin@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Mon, 04 Nov 2024 15:39:01 +0000
Resent-Message-ID: <handler.74203.B.173073469227709 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 74203 <at> debbugs.gnu.org
From: "Collin J. Doering" <collin@HIDDEN>
Date: Mon, 04 Nov 2024 10:37:53 -0500
Message-ID: <874j4nt3fy.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
Received-SPF: pass client-ip=95.215.58.188; envelope-from=collin@HIDDEN;
 helo=out-188.mta1.migadu.com
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1,
 RCVD_IN_DNSWL_LOW=-0.7, RCVD_IN_MSPIKE_H2=-0.001,
 RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001,
 SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.4 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.4 (--)

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Hi lovely maintainers of Guix!

Some time ago I announced the availability of a guix build farm running out=
 of the University of Tennessee[1]. Some time ago, builds started failing d=
ue to a failure to build coreutils[2]; investigation showed a unexpected fa=
iling test:

=2D-8<---------------cut here---------------start------------->8---
FAIL tests/cp/reflink-auto.sh (exit status: 1)
=2D-8<---------------cut here---------------end--------------->8---

I found that on other guix systems, this is not occurring. After some onlin=
e sleuthing, it appears that the nix folks have seen this before[3]. They o=
pted to disable the test 'tests/cp/reflink-auto.sh' as it can fail when usi=
ng btrfs. On the guix system impacted, disabling coreutils tests makes the =
package build.

For reference, coreutils was building on cuirass.genenetwork.org on guix co=
mmit `0c908518375aea50be6dec703367c01944c8c721` and stopped building on `66=
611696975409a52478b95a862a464daeaefe2a`.

I suggest we follow what the nix folks did (disable `tests/cp/reflink-auto.=
sh`). In a following email you will find a patch that does so, however, bec=
ause it changes coreutils, this will cause many packages to be rebuilt, so =
I'm unsure whats the best way to correct this without having to wait for co=
re-updates to be merged.

Any advise or insight appreciated.

[1]: https://lists.gnu.org/archive/html/guix-devel/2024-07/msg00033.html
[2]: https://cuirass.genenetwork.org/eval/157119/log/raw
[3]: https://github.com/NixOS/nixpkgs/pull/190211

=2D-=20
Collin J. Doering

http://rekahsoft.ca
http://blog.rekahsoft.ca
http://git.rekahsoft.ca

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iIoEARYKADIWIQSg4F3ACfM0j/GRGeP3fjGTl82nFgUCZyjqURQcY29sbGluQHJl
a2Foc29mdC5jYQAKCRD3fjGTl82nFvPBAQC3DfQP5Dp4VH+4J7MXVjU3l2VPG9bd
9tAe7TWGfOXiNAEAjNk8MWoEhTqDvNlfHIRV6X5z3eef1LzQFIJkVxp3GAQ=
=N/kV
-----END PGP SIGNATURE-----
--=-=-=--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: "Collin J. Doering" <collin@HIDDEN>
Subject: bug#74203: Acknowledgement (coreutils fails to build)
Message-ID: <handler.74203.B.173073469227709.ack <at> debbugs.gnu.org>
References: <874j4nt3fy.fsf@HIDDEN>
X-Gnu-PR-Message: ack 74203
X-Gnu-PR-Package: guix
Reply-To: 74203 <at> debbugs.gnu.org
Date: Mon, 04 Nov 2024 15:39:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-guix@HIDDEN

If you wish to submit further information on this problem, please
send it to 74203 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
74203: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D74203
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to andreas@HIDDEN, ludo@HIDDEN, bug-guix@HIDDEN:

Subject: bug#74203: [PATCH] gnu: coreutils: Disable cp/reflink-auto.sh as it can fail on btrfs
References: <874j4nt3fy.fsf@HIDDEN>
In-Reply-To: <874j4nt3fy.fsf@HIDDEN>
Resent-From: "Collin J. Doering" <collin@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: andreas@HIDDEN, ludo@HIDDEN, bug-guix@HIDDEN
Resent-Date: Mon, 04 Nov 2024 15:44:02 +0000
Resent-Message-ID: <handler.74203.B74203.173073499228353 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 74203 <at> debbugs.gnu.org
Cc: "Collin J. Doering" <collin@HIDDEN>, Andreas Enge <andreas@HIDDEN>, Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
From: "Collin J. Doering" <collin@HIDDEN>
Date: Mon,  4 Nov 2024 10:42:21 -0500
Message-ID: <3f7ec312925bdf1b89b0fc4630b4c73f820f4af7.1730734380.git.collin@HIDDEN>
MIME-Version: 1.0
Content-Transfer-Encoding: 8bit
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

* gnu/packages/base.scm: Similarly to
nix (https://github.com/NixOS/nixpkgs/pull/190211), disable
tests/cp/reflink-auto.sh test as it can fail on btrfs. This was discovered by
the cuirass.genenetwork.org build farm.

Change-Id: If1cc3d516c5807e580ec64ab93670e30090581a7
---
 gnu/packages/base.scm | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/gnu/packages/base.scm b/gnu/packages/base.scm
index 4e8121ae2c..bed708fc27 100644
--- a/gnu/packages/base.scm
+++ b/gnu/packages/base.scm
@@ -506,6 +506,8 @@ (define-public coreutils
                                    "tests/split/fail.sh"
                                    ;; These tests error
                                    "tests/dd/nocache.sh"
+                                   ;; These tests can intermitently fail on btrfs
+                                   "tests/cp/reflink-auto.sh"
                                    ;; These tests fail
                                    "tests/cp/sparse.sh"
                                    "tests/cp/special-f.sh"

base-commit: 915f807ce61c48c34141f0300ea7623170f4148a
-- 
2.46.0

Message sent to bug-guix@HIDDEN:

Subject: bug#74203: Further investigation and workaround
References: <874j4nt3fy.fsf@HIDDEN>
In-Reply-To: <874j4nt3fy.fsf@HIDDEN>
Resent-From: "Collin J. Doering" <collin@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Thu, 14 Nov 2024 02:52:02 +0000
Resent-Message-ID: <handler.74203.B74203.17315526695427 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 74203 <at> debbugs.gnu.org
From: "Collin J. Doering" <collin@HIDDEN>
Date: Wed, 13 Nov 2024 21:50:47 -0500
Message-ID: <87ldxmsf3s.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/signed; boundary="=-=-=";
 micalg=pgp-sha512; protocol="application/pgp-signature"
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

--=-=-=
Content-Type: text/plain
Content-Transfer-Encoding: quoted-printable

Hi again,

I wanted to follow up on my previous report and patch. I still think its us=
eful to consider disabling the coreutils test I previously suggested, howev=
er I found a way to work around the issue and wanted to make note of it, as=
 well as provide some details of my investigation.

To work around the coreutils test `tests/cp/reflink-auto.sh` failing on gui=
x commit `66611696975409a52478b95a862a464daeaefe2a`, I temporarily mounted =
a tmpfs to replace /tmp (which was on btrfs).

=2D-8<---------------cut here---------------start------------->8---
mv /tmp /tmp.old
mkdir /tmp
mount -t tmpfs tmpfs /tmp
chmod 1777 /tmp
mv /tmp.old/{.*,*} /tmp/
=2D-8<---------------cut here---------------end--------------->8---

Now, what made me do this? Well let me explain!

In `tests/cp/reflink-auto.sh` (https://github.com/coreutils/coreutils/blob/=
v9.1/tests/cp/reflink-auto.sh), the failing part of the test:

=2D-8<---------------cut here---------------start------------->8---
# we shouldn't be able to reflink() files on separate partitions
. "$abs_srcdir/tests/other-fs-tmpdir"
a_other=3D"$other_partition_tmpdir/a"
<..>
returns_ 1 cp --reflink "$a_other" b || fail=3D1
=2D-8<---------------cut here---------------end--------------->8---

'$other_partition_tmpdir' is defined in 'tests/other-fs-tmpdir' (https://gi=
thub.com/coreutils/coreutils/blob/v9.1/tests/other-fs-tmpdir) by looking th=
rough a list of candidate directories, comparing the current working direct=
ory to each candidate to see if they have different device ids (as given by=
 'stat -c %d <path>') and that the current user can create directories ther=
e. Once it finds a candidate, it sets '$other_partition_tmpdir' to the temp=
orary directory it created. The candidate directories that are considered a=
re as follows:

=2D-8<---------------cut here---------------start------------->8---
test "${CANDIDATE_TMP_DIRS+set}" =3D set \
  || CANDIDATE_TMP_DIRS=3D"$TMPDIR /tmp /dev/shm /var/tmp /usr/tmp $HOME"
=2D-8<---------------cut here---------------end--------------->8---

Looking at a remaining failed build of coreutils (left over by building wit=
h `--keep-failed`), I see that in 'top/environment-variables', 'TMPDIR' is =
set to '/tmp/guix-build-guix-1.4.0-26.5ab3c4c.drv-0'. This directory is the=
 same place the build is taking place, so I would expect it to 'be on the s=
ame partition'. So, next would be /tmp, where the same premise applies; nex=
t is /dev/shm. From my tests simulating the coreutils guix shell build envi=
ronment, this would meet the conditions and be selected. However, if this w=
ere the case, I wouldn't expect the coreutils reflink test to fail.

My suspicion is that for some reason, 'stat -c %d <path>' to check whether =
two files, a and b are on the same partition doesn't play well with btrfs s=
ubvolumes in some instances with guix-daemon sandboxed builds. However, whe=
n trying to test this in a simulated coreutils guix shell build environment=
, I found that paths outside of the environment on different subvolumes (th=
at do indeed show different device ids (as per 'stat -c %d <path>' outside =
of the guix shell container)), show the same id's within it. I suspect this=
 is related to why the coreutils test fails, but does not when I use a tmpf=
s for /tmp. Its worth noting that on the system impacted, /gnu/store is a b=
trfs subvolume.

I am not yet satisfied with my with my partial explanation, and am very cur=
ious if anyone spots something I'm missing (eg. has a better understanding =
of the guix build environment and why the reflink coreutils test could be f=
ailing like this).

Thanks for your time and attention.

=2D-=20
Collin J. Doering

http://rekahsoft.ca
http://blog.rekahsoft.ca
http://git.rekahsoft.ca

--=-=-=
Content-Type: application/pgp-signature; name="signature.asc"

-----BEGIN PGP SIGNATURE-----

iIoEARYKADIWIQSg4F3ACfM0j/GRGeP3fjGTl82nFgUCZzVlhxQcY29sbGluQHJl
a2Foc29mdC5jYQAKCRD3fjGTl82nFgzbAQD2b+IJGAyaN0KjUxKIJ47eZD+nWgMu
VsLKjo6ZosMcTgEApfsAskaxs/lfHil7AalJdkFNiR32ZoBmiRvMyqYFGgo=
=ofWN
-----END PGP SIGNATURE-----
--=-=-=--

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 20 Nov 2024 21:49:15 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed Nov 20 16:49:15 2024
Received: from localhost ([127.0.0.1]:48034 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1tDsZW-0001w2-Oe
	for submit <at> debbugs.gnu.org; Wed, 20 Nov 2024 16:49:15 -0500
Received: from eggs.gnu.org ([209.51.188.92]:38062)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1tDsZU-0001vl-OX
 for control <at> debbugs.gnu.org; Wed, 20 Nov 2024 16:49:13 -0500
Received: from fencepost.gnu.org ([2001:470:142:3::e])
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>) id 1tDsZP-00056z-F3
 for control <at> debbugs.gnu.org; Wed, 20 Nov 2024 16:49:07 -0500
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=gnu.org;
 s=fencepost-gnu-org; h=MIME-version:Subject:From:To:Date:in-reply-to:
 references; bh=Z4zT1MFIZxrXzb/9PaG+t24X490ll6pXEWEan5Vk2C4=; b=IyLFvm8JnRT7T4
 kiTYvAWbdIoqdxarGA6V0bI4fEM7nyjOPn4hHJCfo66fMgB+biisNxgyBkLstErsCsx2C/xst1uq8
 AWekgSwfjLejmtsbOzaNst66E3KEIk8/6JbhxcMeV31holH3m0QTqDw0BRW8BILQGtOAwcEMimp+r
 7NeW3KpZp+pMp/K8uEXLCSo6vzHRZIqWs+t2bXN7mvCzVZxjHIUQgOgKb+FveN7dJ/oAH8lLd7d5Z
 HNZamJKOMG0nKS5HXODiDlEcJyCYgoEx1RJnL+lL3QxIDI6Ka1Pqsyx7zIlnaajTkTxg/iTwL16gc
 mF/fts1q9oD/hm3c3JDA==;
Date: Wed, 20 Nov 2024 22:49:05 +0100
Message-Id: <87y11dvani.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
Subject: control message for bug #74203
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

retitle 74203 Coreutils test failure on Btrfs
quit

Last modified: Sun, 12 Jan 2025 05:45:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.