GNU bug report logs - #77716
Corrupted store after Guix deploy

Previous Next

To reply to this bug, email your comments to 77716 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tomas Volf <~@wolfsden.cz>
To: bug-guix <at> gnu.org
Subject: Corrupted store after Guix deploy
Date: Thu, 10 Apr 2025 23:32:53 +0200

Hello,

I just finished Guix deploy.  The command has finished successfully,
without any warnings or errors.  However when I checked the store, I can
see that some paths are corrupted:

--8<---------------cut here---------------start------------->8---
$ guix gc --verify=contents
reading the store...
checking path existence...
checking hashes...
path `/gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden' was modified! expected hash `b40d36eb683b0143cb192e37ba18f1bb31a437016e8a2f82a3623942b39c93b8', got `b3ecf9df08d6bd70dc2482ff68d606802aa8d1623b8efd8bcd0485bbe670cd4c'
path `/gnu/store/my2g20c7spa8ixpnr92s302wj97cxl86-nonguix' was modified! expected hash `4e478e5e311d2a1205b01681de5d4fd2aaba7e9367bdf83ac2c74b3a18322bc5', got `9b8c244c6a1592c1a067bb0f1682b087322258c52dd91a95da7f2d7fbb03a7c3'
--8<---------------cut here---------------end--------------->8---

I know there already are some issues in the similar spirit, but
consensus there seems to be that missing/wrong sync is to blame.  So I
am making a new bug, because this one is not affected by (possible) sync
issues.  I did not even reboot (yet), I have verified the store *before*
rebooting.

Indeed, the guix hashes do differ:

Laptop:

--8<---------------cut here---------------start------------->8---
$ guix hash -r /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
1f4kkjrl4fb2lf12z2kf04vs8cdvy4cbldrf375l609vd3mkc3dl
--8<---------------cut here---------------end--------------->8---

Remote:

--8<---------------cut here---------------start------------->8---
$ guix hash -r /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
0k6df3kbp184rn5zv3ivcb8shal00vb6izw24kf71gfn13gzkv5k
--8<---------------cut here---------------end--------------->8---

I have checked the files themselves (sha256sum), and they are the same
on both machines.  However I have noticed couple of differences listed
below:

The permissions on the very top level directory differ (which one are
correct?):

Laptop:

--8<---------------cut here---------------start------------->8---
$ ls -al /gnu/store/ | grep 1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
drwxr-xr-x   1 root          root              16 Jan  1  1970 1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
--8<---------------cut here---------------end--------------->8---

Remote:

--8<---------------cut here---------------start------------->8---
$ ls -al /gnu/store/ | grep 1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
dr-xr-xr-x  1 root root           16 Jan  1  1970 1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
--8<---------------cut here---------------end--------------->8---



The target of symbolic link in the store item differs.

Laptop:

--8<---------------cut here---------------start------------->8---
$ ls -l /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile/site/
total 4
lrwxrwxrwx 1 root root 61 Jan  1  1970 3.0 -> /gnu/store/rmwi813mcsg8d5ilzq0rxwqd8dp558r5-wolfsden-989d91d/
--8<---------------cut here---------------end--------------->8---

Remote:

--8<---------------cut here---------------start------------->8---
$ ls -l /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile/site/
total 4
lrwxrwxrwx 1 root root 60 Jan  1  1970 3.0 -> /gnu/store/rmwi813mcsg8d5ilzq0rxwqd8dp558r5-wolfsden-989d91d
--8<---------------cut here---------------end--------------->8---

Notice that for the Laptop, the target ends in `/', for the Remote it
does not.



Additionally it is interesting that in addition I have exactly on more
corrupted path, and that is another channel:

--8<---------------cut here---------------start------------->8---
$ guix gc --verify=contents
reading the store...
checking path existence...
checking hashes...
path `/gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden' was modified! expected hash `b40d36eb683b0143cb192e37ba18f1bb31a437016e8a2f82a3623942b39c93b8', got `b3ecf9df08d6bd70dc2482ff68d606802aa8d1623b8efd8bcd0485bbe670cd4c'
path `/gnu/store/my2g20c7spa8ixpnr92s302wj97cxl86-nonguix' was modified! expected hash `4e478e5e311d2a1205b01681de5d4fd2aaba7e9367bdf83ac2c74b3a18322bc5', got `9b8c244c6a1592c1a067bb0f1682b087322258c52dd91a95da7f2d7fbb03a7c3'
--8<---------------cut here---------------end--------------->8---



I have copied the store item back using `rsync -a ...', and this is what
diffoscope has to say (the same as above):

--8<---------------cut here---------------start------------->8---
2025-04-10 21:06:56 W: diffoscope.comparators.xml: Vulnerable version of pyexpat detected; disabling comparison of XML documents. Install defusedxml or upgrade your pyexpat.
--- /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
+++ /tmp/xxxx/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden
├── /gnu/store/lm25j7769nfzgfmx8gh28zarrlczgh8r-coreutils-9.1/bin/stat {}
│ @@ -1,8 +1,8 @@
│
│    Size: 16        	Blocks: 0          IO Block: 4096   directory
│  Device: 0,25	Links: 1
│ -Access: (0755/drwxr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
│ +Access: (0555/dr-xr-xr-x)  Uid: (    0/    root)   Gid: (    0/    root)
│
│  Modify: 1970-01-01 00:00:01.000000000 +0000
│   --- /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share
├── +++ /tmp/xxxx/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share
│ │   --- /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile
│ ├── +++ /tmp/xxxx/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile
│ │ │   --- /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile/site
│ │ ├── +++ /tmp/xxxx/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile/site
│ │ │ │   --- /gnu/store/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile/site/3.0
│ │ │ ├── +++ /tmp/xxxx/1bjf1lza4i7fc1jh8qzb1n9xnv3rfi17-wolfsden/share/guile/site/3.0
│ │ │ │┄ symlink
│ │ │ │ @@ -1 +1 @@
│ │ │ │ +destination: /gnu/store/rmwi813mcsg8d5ilzq0rxwqd8dp558r5-wolfsden-989d91d
│ │ │ │ -destination: /gnu/store/rmwi813mcsg8d5ilzq0rxwqd8dp558r5-wolfsden-989d91d/
│ │ │ │ ├── /gnu/store/lm25j7769nfzgfmx8gh28zarrlczgh8r-coreutils-9.1/bin/stat {}
│ │ │ │ │ @@ -1,8 +1,8 @@
│ │ │ │ │
│ │ │ │ │ +  Size: 60        	Blocks: 8          IO Block: 4096   symbolic link
│ │ │ │ │ -  Size: 61        	Blocks: 8          IO Block: 4096   symbolic link
│ │ │ │ │  Device: 0,25	Links: 1
│ │ │ │ │  Access: (0777/lrwxrwxrwx)  Uid: (    0/    root)   Gid: (    0/    root)
│ │ │ │ │
│ │ │ │ │  Modify: 1970-01-01 00:00:01.000000000 +0000
--8<---------------cut here---------------end--------------->8---

Any idea what might have happen?

I cannot really reproduce it now, not sure it the issue went away with
newer Guix or whether it was random flap.  I just wonder whether the
other end should not checksum the entries before putting them into the
store and reject them if the hash is wrong.

Tomas

-- 
There are only two hard things in Computer Science:
cache invalidation, naming things and off-by-one errors.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.