GNU bug report logs - #52919
Hidden "disk-image-rw" files aren't deleted after use, filling $tmpdir

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 52919 in the body.
You can then email your comments to 52919 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: bug-guix <at> gnu.org
Subject: Hidden "disk-image-rw" files aren't deleted after use, filling $tmpdir
Date: Fri, 31 Dec 2021 13:18:21 -0500

I noticed that files such as
".gk8hqaqb6z4q1gmk678sixjbc9r7kd9g-disk-image-rw" are filling up my
$TMPDIR when building ISO images with Guix.

This is new. I think the issue was introduced with recent work on (gnu
system image), perhaps from this commit:

https://git.savannah.gnu.org/cgit/guix.git/commit/?id=da924796744bbb7b035a986fe5d28d8d613ff6af

Message #8 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: 52919 <at> debbugs.gnu.org
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Mon, 03 Jan 2022 15:56:54 +0100

Hello Leo,

> This is new. I think the issue was introduced with recent work on (gnu
> system image), perhaps from this commit:
>
> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=da924796744bbb7b035a986fe5d28d8d613ff6af

Before that commit "guix system vm" was always producing volatile
images, regardless of the "--volatile" argument. The new behaviour is to
only produce volatile images if "--volatile" is passed.

This means that by default, "disk-image-rw" are created in /tmp so that
QEMU can operate on RW images. The side effect is that those big images
can fill /tmp pretty quickly as you noticed.

We could use the ~/.cache/guix directory instead to store those
images. We could also make "--volatile" the default and introduce a
"--persistent" argument instead. That would restore the previous "guix
system vm" behaviour.

WDYT?

Thanks,

Mathieu

Message #11 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 52919 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Tue, 04 Jan 2022 09:17:53 +0100

Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>> This is new. I think the issue was introduced with recent work on (gnu
>> system image), perhaps from this commit:
>>
>> https://git.savannah.gnu.org/cgit/guix.git/commit/?id=da924796744bbb7b035a986fe5d28d8d613ff6af
>
> Before that commit "guix system vm" was always producing volatile
> images, regardless of the "--volatile" argument. The new behaviour is to
> only produce volatile images if "--volatile" is passed.

I had overlooked the change of default.

> This means that by default, "disk-image-rw" are created in /tmp so that
> QEMU can operate on RW images. The side effect is that those big images
> can fill /tmp pretty quickly as you noticed.
>
> We could use the ~/.cache/guix directory instead to store those
> images. We could also make "--volatile" the default and introduce a
> "--persistent" argument instead. That would restore the previous "guix
> system vm" behaviour.

Yes, reverting to the old behavior for ‘guix system vm’ and adding
‘--persistent’ sounds like a nice option to me.

As for the image backing storage, using ~/.cache/guix/images (with non
dot-prefixed file names) sounds cleaner and safer to me (no risk of /tmp
race or whatever).

Thanks,
Ludo’.

Message #14 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 52919 <at> debbugs.gnu.org
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Tue, 4 Jan 2022 12:26:48 -0500

On Tue, Jan 04, 2022 at 09:17:53AM +0100, Ludovic Courtès wrote:
> As for the image backing storage, using ~/.cache/guix/images (with non
> dot-prefixed file names) sounds cleaner and safer to me (no risk of /tmp
> race or whatever).

I wonder if there are any other tools in Guix that tend to create
multi-gigabyte files in "$HOME"? It might be a bit surprising, but maybe
not a big deal.

Message #17 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Leo Famulari <leo <at> famulari.name>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 52919 <at> debbugs.gnu.org
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Tue, 11 Jan 2022 13:33:30 +0100

[Message part 1 (text/plain, inline)]

Hey,

Here is a patch to store the rw images in the cache directory. Regarding
the volatile flag, things are a little more complex than what I thought
initially.

The "guix system" help menu states that:

--8<---------------cut here---------------start------------->8---
      --volatile         for 'image', make the root file system volatile
--8<---------------cut here---------------end--------------->8---

because for an image, it often makes sense to have the root file system
persistent. For the 'vm' command however, it's probably the other way
around as we would prefer not to copy the huge image files if possible.

I feel like creating a '--volatile-image' and a '--persistent-vm' option
is going to make things too much complex.

WDYT?

Thanks,

Mathieu

[0001-system-vm-Do-not-store-rw-image-in-the-tmp-directory.patch (text/x-patch, inline)]

From e4efaf2ffc6d7c699f9d1c41744cfff89be65c12 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Tue, 11 Jan 2022 13:27:35 +0100
Subject: [PATCH 1/1] system: vm: Do not store rw image in the /tmp directory.

* gnu/system/vm.scm (system-qemu-image/shared-store-script): Store them in the
cache directory instead.
---
 gnu/system/vm.scm | 17 ++++++++++++-----
 1 file changed, 12 insertions(+), 5 deletions(-)

diff --git a/gnu/system/vm.scm b/gnu/system/vm.scm
index 0fc9fb57f4..bd21f9416c 100644
--- a/gnu/system/vm.scm
+++ b/gnu/system/vm.scm
@@ -283,7 +283,9 @@ (define kernel-arguments
               #+@(operating-system-kernel-arguments os "/dev/vda1")))
 
     (define rw-image
-      #~(format #f "/tmp/.~a-rw" (basename #$base-image)))
+      #~(format #f "~a/~a-rw"
+                #$(string-append (cache-directory) "/images")
+                (basename #$base-image)))
 
     (define qemu-exec
       #~(list #+(file-append qemu "/bin/"
@@ -310,10 +312,15 @@ (define builder
                     #+(file-append bash "/bin/sh"))
             (when (not #$volatile?)
               (format port "~a~%"
-                      #$(program-file "copy-image"
-                                      #~(unless (file-exists? #$rw-image)
-                                          (copy-file #$base-image #$rw-image)
-                                          (chmod #$rw-image #o640)))))
+                      #$(program-file
+                         "copy-image"
+                         (with-imported-modules '((guix build utils))
+                           #~(begin
+                               (use-modules (guix build utils))
+                               (unless (file-exists? #$rw-image)
+                                 (mkdir-p (dirname #$rw-image))
+                                 (copy-file #$base-image #$rw-image)
+                                 (chmod #$rw-image #o640)))))))
             (format port "exec ~a \"$@\"~%"
                     (string-join #$qemu-exec " "))
             (chmod port #o555))))
-- 
2.34.0

Message #20 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 52919 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Wed, 12 Jan 2022 21:54:29 +0100

Hi,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

> Here is a patch to store the rw images in the cache directory. Regarding
> the volatile flag, things are a little more complex than what I thought
> initially.
>
> The "guix system" help menu states that:
>
>       --volatile         for 'image', make the root file system volatile
>
> because for an image, it often makes sense to have the root file system
> persistent. For the 'vm' command however, it's probably the other way
> around as we would prefer not to copy the huge image files if possible.
>
> I feel like creating a '--volatile-image' and a '--persistent-vm' option
> is going to make things too much complex.
>
> WDYT?

Hmm.  Can we keep “image” persistent by default, and make ‘vm’ volatile
by default?  That way, ‘--volatile’ would only make sense for ‘image’,
and ‘--persistent’ would only make sense for ‘vm’.  (So we’d be adding
just one option: ‘--persistent’.)

WDYT?

> From e4efaf2ffc6d7c699f9d1c41744cfff89be65c12 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe <at> gnu.org>
> Date: Tue, 11 Jan 2022 13:27:35 +0100
> Subject: [PATCH 1/1] system: vm: Do not store rw image in the /tmp directory.
>
> * gnu/system/vm.scm (system-qemu-image/shared-store-script): Store them in the
> cache directory instead.

Leo had a point on the size of those files, and the fact that if they’re
in ~/.cache they’re not going to be removed automatically, which could
lead to user disks filling up.

So after all, it might be best to keep those images in ${TMPDIR:-/tmp}.

I would still ensure they have a name like “guix-image-$USER-XXX”, where
XXX is the store file basename.

Thoughts?

Apologies for the hesitations!

Ludo’.

Message #23 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 52919 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Thu, 13 Jan 2022 11:55:21 +0100

[Message part 1 (text/plain, inline)]

Hey,

> Hmm.  Can we keep “image” persistent by default, and make ‘vm’ volatile
> by default?  That way, ‘--volatile’ would only make sense for ‘image’,
> and ‘--persistent’ would only make sense for ‘vm’.  (So we’d be adding
> just one option: ‘--persistent’.)
>
> WDYT?

I'm not fan of adding antithetic options: --x and --no-x. There's an
attached patch introducing --volatile-image and --persistent-vm options,
and documenting them. It's maybe not that bad after all.

> I would still ensure they have a name like “guix-image-$USER-XXX”, where
> XXX is the store file basename.

Sure.

Thanks,

Mathieu

[0001-scripts-system-Rationalize-persistency.patch (text/x-patch, inline)]

From b0c84a411f9f23f4f1a4155ba5efa68cac9004a2 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Thu, 13 Jan 2022 11:35:40 +0100
Subject: [PATCH 1/2] scripts: system: Rationalize persistency.

Make sure that the images are created with a non volatile root by default and
the vm are created with a volatile root by default. Break the --volatile
option into --volatile-image and --persistent-vm options.

* guix/scripts/system.scm (perform-action): Turn volatile? argument into
volatile-vm-root?.
(show-help): Introduce --volatile-image and --persistent-vm options instead of
--volatile.
(%default-options): Adapt it.
(%options): Handle those options.
(process-action): Honor them.
* doc/guix.texi (Invoking guix system): Adapt it accordingly.
---
 doc/guix.texi           | 15 ++++++++++-----
 guix/scripts/system.scm | 25 +++++++++++++++++--------
 2 files changed, 27 insertions(+), 13 deletions(-)

diff --git a/doc/guix.texi b/doc/guix.texi
index bc289bad7b..9f763bcfa7 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -35152,6 +35152,11 @@ $ $(guix system vm my-config.scm) -m 1024 -smp 2 -nic user,model=virtio-net-pci
 
 The VM shares its store with the host system.
 
+By default, the root file system of the VM is mounted volatile; the
+@option{--persistent-vm} option can be provided to make it persistent
+instead.  In that case, the VM disk-image file will be copied from the
+store to the @env{TMPDIR} directory to make it writable.
+
 Additional file systems can be shared between the host and the VM using
 the @option{--share} and @option{--expose} command-line options: the former
 specifies a directory to be shared with write access, while the latter
@@ -35189,14 +35194,14 @@ QEMU monitor and the VM.
 @cindex Creating system images in various formats
 @item image
 @cindex image, creating disk images
-The @code{image} command can produce various image types.  The
-image type can be selected using the @option{--image-type} option.  It
+The @code{image} command can produce various image types.  The image
+type can be selected using the @option{--image-type} option.  It
 defaults to @code{efi-raw}.  When its value is @code{iso9660}, the
 @option{--label} option can be used to specify a volume ID with
 @code{image}.  By default, the root file system of a disk image is
-mounted non-volatile; the @option{--volatile} option can be provided to
-make it volatile instead.  When using @code{image}, the bootloader
-installed on the generated image is taken from the provided
+mounted non-volatile; the @option{--volatile-image} option can be
+provided to make it volatile instead.  When using @code{image}, the
+bootloader installed on the generated image is taken from the provided
 @code{operating-system} definition.  The following example demonstrates
 how to generate an image that uses the @code{grub-efi-bootloader}
 bootloader and boot it with QEMU:
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 98e788c657..3ca5592e34 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -772,7 +772,7 @@ (define* (perform-action action image
                          dry-run? derivations-only?
                          use-substitutes? target
                          full-boot?
-                         volatile?
+                         volatile-vm-root?
                          (graphic? #t)
                          container-shared-network?
                          (mappings '())
@@ -827,7 +827,8 @@ (define bootcfg
   (mlet* %store-monad
       ((sys       (system-derivation-for-action image action
                                                 #:full-boot? full-boot?
-                                                #:volatile? volatile?
+                                                #:volatile?
+                                                volatile-vm-root?
                                                 #:graphic? graphic?
                                                 #:container-shared-network? container-shared-network?
                                                 #:mappings mappings))
@@ -997,7 +998,9 @@ (define (show-help)
   (display (G_ "
       --no-bootloader    for 'init', do not install a bootloader"))
   (display (G_ "
-      --volatile         for 'image', make the root file system volatile"))
+      --volatile-image   for 'image', make the root file system volatile"))
+  (display (G_ "
+      --persistent-vm    for 'vm', make the root file system persistent"))
   (display (G_ "
       --label=LABEL      for 'image', label disk image with LABEL"))
   (display (G_ "
@@ -1078,9 +1081,12 @@ (define %options
          (option '("no-bootloader" "no-grub") #f #f
                  (lambda (opt name arg result)
                    (alist-cons 'install-bootloader? #f result)))
-         (option '("volatile") #f #f
+         (option '("volatile-image") #f #f
+                 (lambda (opt name arg result)
+                   (alist-cons 'volatile-image-root? #t result)))
+         (option '("persistent-vm") #f #f
                  (lambda (opt name arg result)
-                   (alist-cons 'volatile-root? #t result)))
+                   (alist-cons 'volatile-vm-root? #f result)))
          (option '("label") #t #f
                  (lambda (opt name arg result)
                    (alist-cons 'label arg result)))
@@ -1149,7 +1155,8 @@ (define %default-options
     (image-size . guess)
     (install-bootloader? . #t)
     (label . #f)
-    (volatile-root? . #f)
+    (volatile-image-root? . #f)
+    (volatile-vm-root? . #t)
     (graph-backend . "graphviz")))
 
 (define (verbosity-level opts)
@@ -1219,7 +1226,8 @@ (define save-provenance?
                                            ((docker-image) docker-image-type)
                                            (else image-type)))
                             (image-size (assoc-ref opts 'image-size))
-                            (volatile?  (assoc-ref opts 'volatile-root?))
+                            (volatile?
+                             (assoc-ref opts 'volatile-image-root?))
                             (shared-network?
                                (assoc-ref opts 'container-shared-network?))
                             (base-image (if (operating-system? obj)
@@ -1279,7 +1287,8 @@ (define (graph-backend)
                                #:validate-reconfigure
                                (assoc-ref opts 'validate-reconfigure)
                                #:full-boot? (assoc-ref opts 'full-boot?)
-                               #:volatile? (assoc-ref opts 'volatile-root?)
+                               #:volatile-vm-root?
+                               (assoc-ref opts 'volatile-vm-root?)
                                #:graphic? (not (assoc-ref opts 'no-graphic?))
                                #:container-shared-network?
                                (assoc-ref opts 'container-shared-network?)
-- 
2.34.0

Message #26 received at 52919 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: 52919 <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Thu, 13 Jan 2022 15:49:54 +0100

Hello,

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>> Hmm.  Can we keep “image” persistent by default, and make ‘vm’ volatile
>> by default?  That way, ‘--volatile’ would only make sense for ‘image’,
>> and ‘--persistent’ would only make sense for ‘vm’.  (So we’d be adding
>> just one option: ‘--persistent’.)
>>
>> WDYT?
>
> I'm not fan of adding antithetic options: --x and --no-x. There's an
> attached patch introducing --volatile-image and --persistent-vm options,
> and documenting them. It's maybe not that bad after all.

[...]

> From b0c84a411f9f23f4f1a4155ba5efa68cac9004a2 Mon Sep 17 00:00:00 2001
> From: Mathieu Othacehe <othacehe <at> gnu.org>
> Date: Thu, 13 Jan 2022 11:35:40 +0100
> Subject: [PATCH 1/2] scripts: system: Rationalize persistency.
>
> Make sure that the images are created with a non volatile root by default and
> the vm are created with a volatile root by default. Break the --volatile
> option into --volatile-image and --persistent-vm options.
>
> * guix/scripts/system.scm (perform-action): Turn volatile? argument into
> volatile-vm-root?.
> (show-help): Introduce --volatile-image and --persistent-vm options instead of
> --volatile.
> (%default-options): Adapt it.
> (%options): Handle those options.
> (process-action): Honor them.
> * doc/guix.texi (Invoking guix system): Adapt it accordingly.

It’s maybe not that important but I’m not convinced about the extra
“-image” and “-vm” suffixes; I don’t think it makes things clearer.


[...]

> -         (option '("volatile") #f #f
> +         (option '("volatile-image") #f #f
> +                 (lambda (opt name arg result)
> +                   (alist-cons 'volatile-image-root? #t result)))

As a rule of thumb, we should not remove an option without going through
a deprecation period.

So if we take that route, “volatile” should still be accepted, only with
deprecation warning emitted.  We can remove it entirely in 1.5.0 or so.

Thanks!

Ludo’.

Message #31 received at 52919-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 52919-done <at> debbugs.gnu.org, Leo Famulari <leo <at> famulari.name>
Subject: Re: bug#52919: Hidden "disk-image-rw" files aren't deleted after
 use, filling $tmpdir
Date: Fri, 14 Jan 2022 12:27:33 +0100

Hey Ludo!

> It’s maybe not that important but I’m not convinced about the extra
> “-image” and “-vm” suffixes; I don’t think it makes things clearer.

Pushed without those suffixes!

Thanks for reviewing,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.