GNU bug report logs - #63817
[PATCH] substitute: Delete cached narinfos more than two-month old.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 63817 in the body.
You can then email your comments to 63817 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: [PATCH] substitute: Delete cached narinfos more than two-month old.
Date: Wed, 31 May 2023 23:22:01 +0200

This allows 'guix substitute' to shrink the cache a bit more, which
saves space and improves performance of cache-cleanup phases since fewer
entries need to be traversed.

* guix/scripts/substitute.scm (cached-narinfo-expiration-time): Define
'max-ttl' and use it as an upper bound.
---
 guix/scripts/substitute.scm | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)

Hi!

You might have noticed that /var/guix/substitute/cache tends to grow.
On my laptop I have:

--8<---------------cut here---------------start------------->8---
$ du -hs /var/guix/substitute/cache/
152M    /var/guix/substitute/cache/
$ du /var/guix/substitute/cache/ |sort -k1 -n

[...]

16196   /var/guix/substitute/cache/kzwjeblndsbkjzmjailrt4bnhguil7tqjmewzcyw22hgajbhfy3q
136216  /var/guix/substitute/cache/4refhwxbjmeua2kwg2nmzhv4dg4d3dorpjefq7kiciw2pfhaf26a
155364  /var/guix/substitute/cache/
--8<---------------cut here---------------end--------------->8---

This full of old entries that I’m unlikely to need:

--8<---------------cut here---------------start------------->8---
$ ls -lt /var/guix/substitute/cache/4refhwxbjmeua2kwg2nmzhv4dg4d3dorpjefq7kiciw2pfhaf26a |tail -10
-rw------- 1 root root  1187 Nov 30 18:49 xlxgm0lzg25fsz99qql8wp8y4l0ijvy6
-rw------- 1 root root  3182 Nov 30 18:49 yqdklfzzxx2zm2vvx11pm5my2jzlzwmy
-rw------- 1 root root  1180 Nov 30 18:49 18pzv1gjw9mi3bzkvb2ms9am37kd960i
-rw------- 1 root root  1484 Nov 30 18:49 qqpc6n1q5jz29zzwf1jn8srq3a5ysy86
-rw------- 1 root root  1060 Nov 30 18:27 9r6w8hrw2bpm2sjp03ynl1aifjkcn9wi
-rw------- 1 root root  1056 Nov 30 18:27 ghdwy0xa2h9xi5qh2picb19fhhk0fzhp
-rw------- 1 root root  2108 Nov 30 18:27 ixp9xd4cbs774yh5qkvywmjk1606s8i8
-rw------- 1 root root  1052 Nov 30 18:27 laawrywlbx1y76sdribiw8aygcvadili
-rw------- 1 root root  1092 Nov 30 18:27 m72pfbhqhbhi7gj8kvy66lr99v7hvk7z
-rw------- 1 root root  1070 Nov 30 18:27 ym69k8mzmxbw1ar76nzghl38h4g7g8v4
--8<---------------cut here---------------end--------------->8---

That six-month old because that’s the TTL and ci.guix.gnu.org has been
advertising for some time now.

Long story short: we’d rather trim the cache more aggressively.

Thoughts?

Ludo’.

diff --git a/guix/scripts/substitute.scm b/guix/scripts/substitute.scm
index 3626832dda..92935268b3 100755
--- a/guix/scripts/substitute.scm
+++ b/guix/scripts/substitute.scm
@@ -167,6 +167,12 @@ (define (lookup-narinfo caches path authorized?)
 
 (define (cached-narinfo-expiration-time file)
   "Return the expiration time for FILE, which is a cached narinfo."
+  (define max-ttl
+    ;; Higher bound on the TTL used to avoid keeping around cached narinfos
+    ;; for too long, which makes the cache bigger and more expensive to
+    ;; traverse.
+    (* 2 30 24 60 60))                            ;2 months
+
   (catch 'system-error
     (lambda ()
       (call-with-input-file file
@@ -174,10 +180,10 @@ (define (cached-narinfo-expiration-time file)
           (match (read port)
             (('narinfo ('version 2) ('cache-uri uri)
                        ('date date) ('ttl ttl) ('value #f))
-             (+ date ttl))
+             (+ date (min ttl max-ttl)))
             (('narinfo ('version 2) ('cache-uri uri)
                        ('date date) ('ttl ttl) ('value value))
-             (+ date ttl))
+             (+ date (min ttl max-ttl)))
             (x
              0)))))
     (lambda args

base-commit: 77f52db416a13e195d090cad4e9e7658feb2e86b
-- 
2.40.1

Message #10 received at 63817-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 63817-done <at> debbugs.gnu.org
Cc: Josselin Poiret <dev <at> jpoiret.xyz>, Tobias Geerinckx-Rice <me <at> tobias.gr>,
 Simon Tournier <zimon.toutoune <at> gmail.com>, Mathieu Othacehe <othacehe <at> gnu.org>,
 Christopher Baines <mail <at> cbaines.net>, Ricardo Wurmus <rekado <at> elephly.net>
Subject: Re: bug#63817: [PATCH] substitute: Delete cached narinfos more than
 two-month old.
Date: Thu, 08 Jun 2023 23:49:50 +0200

Ludovic Courtès <ludo <at> gnu.org> skribis:

> This allows 'guix substitute' to shrink the cache a bit more, which
> saves space and improves performance of cache-cleanup phases since fewer
> entries need to be traversed.
>
> * guix/scripts/substitute.scm (cached-narinfo-expiration-time): Define
> 'max-ttl' and use it as an upper bound.

Pushed as 3f5e14182931f123c10513a3e1e2abaebfb52279.

Ludo'.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.