GNU bug report logs - #29363
GC test failure on Btrfs

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29363 in the body.
You can then email your comments to 29363 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Rutger Helling <rhelling <at> mykolab.com>
To: bug-guix <at> gnu.org
Subject: Single test failure building Guix
Date: Mon, 20 Nov 2017 11:08:59 +0100

[Message part 1 (text/plain, inline)]

Hi,

when building Guix with 'guix build guix' I keep running into a single 
test failure. I've attached the test-suite.log.

My kernel:

$ uname -a
Linux guixsd 4.14.0-gnu #1 SMP 1 x86_64 GNU/Linux

This is the guix-daemon I'm building with:

$ guix gc --references 
/gnu/store/mwvmdfksm9iwj1symfiinlikiz56s2nl-guix-0.13.0-9.ff23b47/bin/guix-daemon
/gnu/store/2b22079yrvs59j256z4scccq506csy7c-guile-git-0.0-4.951a32c
/gnu/store/3h31zsqxjjg52da5gp3qmhkh4x8klhah-glibc-2.25
/gnu/store/5jgpy83fga8jjj02m0ncyvgggmbwsfdy-gnutls-3.5.13
/gnu/store/5rk1b2riamhzk8s1hs83mb1i8vdif0vd-gzip-1.8
/gnu/store/6d4ihp7xbdh3a0ffbpm5n45q4v3w0l35-sqlite-3.19.3
/gnu/store/6wyjls0q2c9gjskkplsr1ad09p3d8gzg-gcc-5.4.0-lib
/gnu/store/b1dg82khbvr2abaa346vv7r93ryqrb3j-xz-5.2.2
/gnu/store/bcmf06k2n1pfwqkzpclvvc3w9jdfi71a-guile-json-0.6.0
/gnu/store/f8k940vy9gck66m9r4id5m098w3hxgka-bash-minimal-4.4.12
/gnu/store/mwvmdfksm9iwj1symfiinlikiz56s2nl-guix-0.13.0-9.ff23b47
/gnu/store/navpkpm1jf6zf8zmi54wl5w3b2ddv1sw-zlib-1.2.11
/gnu/store/qfzl5frp52wdz1vbdj958sz35yfl94xi-libgcrypt-1.8.1
/gnu/store/swyipr8smrd5bc72n92sdfxzx0p4cjpi-guile-2.2.2
/gnu/store/wd01nlc7apixl4kyvhpzasxprqymw32s-bzip2-1.0.6
/gnu/store/xfaqdvk060yz7ddc9isk3wkybqmcfj3w-guile-ssh-0.11.2

Does anyone have any idea about what's going wrong?

[test-suite.log (text/plain, attachment)]

Message #8 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Rutger Helling <rhelling <at> mykolab.com>
Cc: 29363 <at> debbugs.gnu.org
Subject: Re: bug#29363: Single test failure building Guix
Date: Mon, 20 Nov 2017 16:55:23 +0100

Hi Rutger,

Rutger Helling <rhelling <at> mykolab.com> skribis:

> test-name: dead path can be explicitly collected
> location: /tmp/guix-build-guix-0.13.0-10.0b4c385.drv-0/source/tests/store.scm:178
> source:
> + (test-assert
> +   "dead path can be explicitly collected"
> +   (let ((p (add-text-to-store
> +              %store
> +              "random-text"
> +              (random-text)
> +              '())))
> +     (let-values
> +       (((paths freed) (delete-paths %store (list p))))
> +       (and (equal? paths (list p))
> +            (> freed 0)
> +            (not (file-exists? p))))))
> actual-value: #f
> result: FAIL

I didn’t experience this on my laptop.  Is it reproducible if you run
“guix build guix” a second time?

Thanks,
Ludo’.

Message #11 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Rutger Helling <rhelling <at> mykolab.com>
To: ludo <at> gnu.org
Subject: Re: bug#29363: Single test failure building Guix
Date: Mon, 20 Nov 2017 18:49:08 +0100

[Message part 1 (text/plain, inline)]

Hi Ludo, 

it is indeed reproducible. No matter how many times it always keeps
failing on this one test. 

I've had this problem for a long time, which is a little bit annoying
since it means I have to wait until a substitute is available every
time. 

On 2017-11-20 16:55, ludo <at> gnu.org wrote:

> Hi Rutger,
> 
> Rutger Helling <rhelling <at> mykolab.com> skribis:
> 
>> test-name: dead path can be explicitly collected
>> location: /tmp/guix-build-guix-0.13.0-10.0b4c385.drv-0/source/tests/store.scm:178
>> source:
>> + (test-assert
>> +   "dead path can be explicitly collected"
>> +   (let ((p (add-text-to-store
>> +              %store
>> +              "random-text"
>> +              (random-text)
>> +              '())))
>> +     (let-values
>> +       (((paths freed) (delete-paths %store (list p))))
>> +       (and (equal? paths (list p))
>> +            (> freed 0)
>> +            (not (file-exists? p))))))
>> actual-value: #f
>> result: FAIL
> 
> I didn't experience this on my laptop.  Is it reproducible if you run
> "guix build guix" a second time?
> 
> Thanks,
> Ludo'.

[Message part 2 (text/html, inline)]

Message #14 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Rutger Helling <rhelling <at> mykolab.com>, 29363 <at> debbugs.gnu.org
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 01:31:04 +0100

[Message part 1 (text/plain, inline)]

Hi Rutger,

Rutger Helling <rhelling <at> mykolab.com> writes:

> when building Guix with 'guix build guix' I keep running into a single 
> test failure. I've attached the test-suite.log.

Is this a Btrfs system by any chance, possibly on an SSD?

> test-name: dead path can be explicitly collected
> location: /tmp/guix-build-guix-0.13.0-10.0b4c385.drv-0/source/tests/store.scm:178
> source:
> + (test-assert
> +   "dead path can be explicitly collected"
> +   (let ((p (add-text-to-store
> +              %store
> +              "random-text"
> +              (random-text)
> +              '())))
> +     (let-values
> +       (((paths freed) (delete-paths %store (list p))))
> +       (and (equal? paths (list p))
> +            (> freed 0)
> +            (not (file-exists? p))))))
> actual-value: #f
> result: FAIL

I can reproduce this error on two different systems that have
Btrfs+LUKS+SSD, and the problem is that freed == 0.

I suspect it's related to Btrfs' "lazy" reporting of disk space, but
haven't dug very far.

Until we figure out what's going on, I suggest applying the patch
below.  Can you confirm that it works on your system?

[0001-gnu-guix-Disable-test-that-fails-on-Btrfs.patch (text/x-patch, inline)]

From bdc7b5310111e21801529ea57e290f6eb72ac6ed Mon Sep 17 00:00:00 2001
From: Marius Bakke <mbakke <at> fastmail.com>
Date: Tue, 21 Nov 2017 00:27:08 +0100
Subject: [PATCH] gnu: guix: Disable test that fails on Btrfs.

Works around <https://bugs.gnu.org/29363>.
Reported by Rutger Helling <rhelling <at> mykolab.com>.

* gnu/packages/package-management.scm (guix)[arguments]: Rename
'disable-container-tests' phase to 'disable-failing-tests' and add substitution
to disable "dead path can be explicitly collected" test.
---
 gnu/packages/package-management.scm | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)

diff --git a/gnu/packages/package-management.scm b/gnu/packages/package-management.scm
index 4f1f7f577..3321ab1eb 100644
--- a/gnu/packages/package-management.scm
+++ b/gnu/packages/package-management.scm
@@ -169,8 +169,7 @@
                         (copy "armhf")
                         (copy "aarch64")
                         #t))
-                    (add-after
-                        'unpack 'disable-container-tests
+                    (add-after 'unpack 'disable-failing-tests
                       ;; XXX FIXME: These tests fail within the build container.
                       (lambda _
                         (substitute* "tests/syscalls.scm"
@@ -183,6 +182,11 @@
                           (substitute* "tests/guix-environment-container.sh"
                             (("guix environment --version")
                              "exit 77\n")))
+                        ;; XXX: This test may fail on some file systems.
+                        ;; See <https://bugs.gnu.org/29363>.
+                        (substitute* "tests/store.scm"
+                          (("^(.*dead path can be explicitly collected\")" all)
+                           (string-append "(test-skip 1)\n" all)))
                         #t))
                     (add-before 'check 'set-SHELL
                       (lambda _
-- 
2.15.0

[Message part 3 (text/plain, inline)]

Ludo, WDYT?

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Rutger Helling <rhelling <at> mykolab.com>
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 29363 <at> debbugs.gnu.org, ludo <at> gnu.org
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 08:31:59 +0100

[Message part 1 (text/plain, inline)]

Hi Marius, 

your patch did the trick, thanks! 

I'm indeed on Btrfs (with LUKS), no SSD though. 

On 2017-11-21 01:31, Marius Bakke wrote:

> Hi Rutger,
> 
> Rutger Helling <rhelling <at> mykolab.com> writes:
> 
>> when building Guix with 'guix build guix' I keep running into a single 
>> test failure. I've attached the test-suite.log.
> 
> Is this a Btrfs system by any chance, possibly on an SSD?
> 
>> test-name: dead path can be explicitly collected
>> location: /tmp/guix-build-guix-0.13.0-10.0b4c385.drv-0/source/tests/store.scm:178
>> source:
>> + (test-assert
>> +   "dead path can be explicitly collected"
>> +   (let ((p (add-text-to-store
>> +              %store
>> +              "random-text"
>> +              (random-text)
>> +              '())))
>> +     (let-values
>> +       (((paths freed) (delete-paths %store (list p))))
>> +       (and (equal? paths (list p))
>> +            (> freed 0)
>> +            (not (file-exists? p))))))
>> actual-value: #f
>> result: FAIL
> 
> I can reproduce this error on two different systems that have
> Btrfs+LUKS+SSD, and the problem is that freed == 0.
> 
> I suspect it's related to Btrfs' "lazy" reporting of disk space, but
> haven't dug very far.
> 
> Until we figure out what's going on, I suggest applying the patch
> below.  Can you confirm that it works on your system?
> 
> Ludo, WDYT?

[Message part 2 (text/html, inline)]

Message #20 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 29363 <at> debbugs.gnu.org, Rutger Helling <rhelling <at> mykolab.com>
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 08:47:52 +0100

Hello,

Marius Bakke <mbakke <at> fastmail.com> skribis:

> Rutger Helling <rhelling <at> mykolab.com> writes:
>
>> when building Guix with 'guix build guix' I keep running into a single 
>> test failure. I've attached the test-suite.log.
>
> Is this a Btrfs system by any chance, possibly on an SSD?
>
>> test-name: dead path can be explicitly collected
>> location: /tmp/guix-build-guix-0.13.0-10.0b4c385.drv-0/source/tests/store.scm:178
>> source:
>> + (test-assert
>> +   "dead path can be explicitly collected"
>> +   (let ((p (add-text-to-store
>> +              %store
>> +              "random-text"
>> +              (random-text)
>> +              '())))
>> +     (let-values
>> +       (((paths freed) (delete-paths %store (list p))))
>> +       (and (equal? paths (list p))
>> +            (> freed 0)
>> +            (not (file-exists? p))))))
>> actual-value: #f
>> result: FAIL
>
> I can reproduce this error on two different systems that have
> Btrfs+LUKS+SSD, and the problem is that freed == 0.

If you comment out (> freed 0), does the test pass?

> I suspect it's related to Btrfs' "lazy" reporting of disk space, but
> haven't dug very far.
>
> Until we figure out what's going on, I suggest applying the patch
> below.  Can you confirm that it works on your system?
>
> From bdc7b5310111e21801529ea57e290f6eb72ac6ed Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke <at> fastmail.com>
> Date: Tue, 21 Nov 2017 00:27:08 +0100
> Subject: [PATCH] gnu: guix: Disable test that fails on Btrfs.
>
> Works around <https://bugs.gnu.org/29363>.
> Reported by Rutger Helling <rhelling <at> mykolab.com>.
>
> * gnu/packages/package-management.scm (guix)[arguments]: Rename
> 'disable-container-tests' phase to 'disable-failing-tests' and add substitution
> to disable "dead path can be explicitly collected" test.

Alternately, we could comment out (> freed 0) if that’s enough, with a
comment explaining why, and do “make update-guix-package”.  That way
we’d avoid the extra build phase.

WDYT?

Thanks for finding out the root cause!

Ludo’.

Message #25 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Rutger Helling <rhelling <at> mykolab.com>
To: ludo <at> gnu.org
Cc: 29363 <at> debbugs.gnu.org, Marius Bakke <mbakke <at> fastmail.com>
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 10:10:12 +0100

[Message part 1 (text/plain, inline)]

Commenting out that line still made the test fail for me. 

On 2017-11-21 08:47, ludo <at> gnu.org wrote:

> Hello,
> 
> Marius Bakke <mbakke <at> fastmail.com> skribis:
> 
> Rutger Helling <rhelling <at> mykolab.com> writes:
> 
> when building Guix with 'guix build guix' I keep running into a single 
> test failure. I've attached the test-suite.log. 
> Is this a Btrfs system by any chance, possibly on an SSD?
> 
> test-name: dead path can be explicitly collected
> location: /tmp/guix-build-guix-0.13.0-10.0b4c385.drv-0/source/tests/store.scm:178
> source:
> + (test-assert
> +   "dead path can be explicitly collected"
> +   (let ((p (add-text-to-store
> +              %store
> +              "random-text"
> +              (random-text)
> +              '())))
> +     (let-values
> +       (((paths freed) (delete-paths %store (list p))))
> +       (and (equal? paths (list p))
> +            (> freed 0)
> +            (not (file-exists? p))))))
> actual-value: #f
> result: FAIL 
> I can reproduce this error on two different systems that have
> Btrfs+LUKS+SSD, and the problem is that freed == 0.

If you comment out (> freed 0), does the test pass?

> I suspect it's related to Btrfs' "lazy" reporting of disk space, but
> haven't dug very far.
> 
> Until we figure out what's going on, I suggest applying the patch
> below.  Can you confirm that it works on your system?
> 
> From bdc7b5310111e21801529ea57e290f6eb72ac6ed Mon Sep 17 00:00:00 2001
> From: Marius Bakke <mbakke <at> fastmail.com>
> Date: Tue, 21 Nov 2017 00:27:08 +0100
> Subject: [PATCH] gnu: guix: Disable test that fails on Btrfs.
> 
> Works around <https://bugs.gnu.org/29363>.
> Reported by Rutger Helling <rhelling <at> mykolab.com>.
> 
> * gnu/packages/package-management.scm (guix)[arguments]: Rename
> 'disable-container-tests' phase to 'disable-failing-tests' and add substitution
> to disable "dead path can be explicitly collected" test.

Alternately, we could comment out (> freed 0) if that's enough, with a
comment explaining why, and do "make update-guix-package".  That way
we'd avoid the extra build phase.

WDYT?

Thanks for finding out the root cause!

Ludo'.

[Message part 2 (text/html, inline)]

Message #28 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Rutger Helling <rhelling <at> mykolab.com>
Cc: 29363 <at> debbugs.gnu.org, Marius Bakke <mbakke <at> fastmail.com>
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 13:53:35 +0100

Rutger Helling <rhelling <at> mykolab.com> skribis:

> Commenting out that line still made the test fail for me. 

Could you figure out which of the conditions in the ‘and’ is failing?

> +       (and (equal? paths (list p))
> +            (> freed 0)
> +            (not (file-exists? p))))))

TIA,
Ludo’.

Message #31 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Rutger Helling <rhelling <at> mykolab.com>
To: ludo <at> gnu.org
Cc: 29363 <at> debbugs.gnu.org, Marius Bakke <mbakke <at> fastmail.com>
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 15:50:24 +0100

[Message part 1 (text/plain, inline)]

I think commenting out (> freed 0) does work after all. When I ran 'guix
environment --pure guix' and 'make check TESTS=tests/store.scm' against
my git checkout the test passed. 

On 2017-11-21 13:53, ludo <at> gnu.org wrote:

> Rutger Helling <rhelling <at> mykolab.com> skribis:
> 
>> Commenting out that line still made the test fail for me.
> 
> Could you figure out which of the conditions in the 'and' is failing?
> 
>> +       (and (equal? paths (list p))
>> +            (> freed 0)
>> +            (not (file-exists? p))))))
> 
> TIA,
> Ludo'.

[Message part 2 (text/html, inline)]

Message #34 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Rutger Helling <rhelling <at> mykolab.com>, ludo <at> gnu.org
Cc: 29363 <at> debbugs.gnu.org
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 22:11:39 +0100

[Message part 1 (text/plain, inline)]

Rutger Helling <rhelling <at> mykolab.com> writes:

> I think commenting out (> freed 0) does work after all. When I ran 'guix
> environment --pure guix' and 'make check TESTS=tests/store.scm' against
> my git checkout the test passed. 

Thanks for confirming.  I pushed a revised patch that only removes the
test for freed space as per Ludos suggestion.

[signature.asc (application/pgp-signature, inline)]

Message #37 received at 29363 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <mbakke <at> fastmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 29363 <at> debbugs.gnu.org, Rutger Helling <rhelling <at> mykolab.com>
Subject: Re: bug#29363: Single test failure building Guix
Date: Tue, 21 Nov 2017 22:39:03 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:
> Alternately, we could comment out (> freed 0) if that’s enough, with a
> comment explaining why, and do “make update-guix-package”.  That way
> we’d avoid the extra build phase.
>
> WDYT?

Oh, I just re-read this message and realized you mentioned doing this in
tests/store.scm rather than working around it in the "guix" package.

Sound good to me.  I will do this and remove the workaround.  Sorry for
the confusion.

[signature.asc (application/pgp-signature, inline)]

Message #42 received at 29363-done <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Marius Bakke <mbakke <at> fastmail.com>
Cc: 29363-done <at> debbugs.gnu.org, Rutger Helling <rhelling <at> mykolab.com>
Subject: Re: bug#29363: Single test failure building Guix
Date: Wed, 22 Nov 2017 16:55:50 +0100

Heya!

Marius Bakke <mbakke <at> fastmail.com> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>> Alternately, we could comment out (> freed 0) if that’s enough, with a
>> comment explaining why, and do “make update-guix-package”.  That way
>> we’d avoid the extra build phase.
>>
>> WDYT?
>
> Oh, I just re-read this message and realized you mentioned doing this in
> tests/store.scm rather than working around it in the "guix" package.
>
> Sound good to me.  I will do this and remove the workaround.  Sorry for
> the confusion.

I see you’ve done this now, so I’m closing the bug.

Thank you gentlefolks!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.