GNU bug report logs - #62253
Fakechroot execution engine doesn’t outlive ‘exec’ calls

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 62253 in the body.
You can then email your comments to 62253 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: bug-guix <at> gnu.org
Subject: Fakechroot execution engine doesn’t outlive
 ‘exec’ calls
Date: Sat, 18 Mar 2023 12:48:32 +0100

For ‘guix pack -RR’ packs, unlike the “userns” and “proot” execution
engines, the “fakechroot” execution engine doesn’t survive ‘exec’ calls:

--8<---------------cut here---------------start------------->8---
$ mkdir -p /tmp/fakechroot-test && cd /tmp/fakechroot-test/ && tar xf $(guix pack -RR openmpi intel-mpi-benchmarks bash-minimal -S /bin=bin)
$ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/bash'
bash-5.1# echo /gnu/store/*coreutils*/bin/ls
/gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/ls /gnu/store/vqdsrvs9jbn0ix2a58s99jwkh74124y5-coreutils-minimal-8.32/bin/ls
bash-5.1# test -f /gnu/store/*coreutils-8*/bin/ls
bash-5.1# echo $?
0
bash-5.1# /gnu/store/*coreutils-8*/bin/ls
bash: /gnu/store/d251rfgc9nm2clzffzhgiipdvfvzkvwi-coreutils-8.32/bin/ls: No such file or directory
--8<---------------cut here---------------end--------------->8---

This is because the ELF interpreter of the unwrapped ‘ls’ binary remains
/gnu/store/…-glibc-2.33/lib/ld-linux-x86-64-so.2 and no LD_PRELOAD
interposition can address that.

In this case, adding ‘coreutils’ to the profile (on the ‘guix pack’
command line) would give us wrapped binaries, and the problem is solved.
But in other cases, it’s not that simple.  For instance, libmpi.so from
Open MPI tries to exec one its programs, using its absolute file name:

--8<---------------cut here---------------start------------->8---
$ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/IMB-MPI1'
--------------------------------------------------------------------------
The singleton application was not able to find the executable "orted" in
your PATH or in the directory where Open MPI/OpenRTE was initially installed,
and therefore cannot continue.

For reference, we tried the following command:

  /gnu/store/c7g9qalmbz4a94hwzk1v1cbq7n5m8plq-openmpi-4.1.4/bin/orted

and got the error No such file or directory.

[…]
--8<---------------cut here---------------end--------------->8---

I can think of several ways to address that:

  1. Change the exec* wrappers in libfakechroot such that, on ENOENT,
     they try a direct ld.so invocation to run program, like
     ‘run-in-namespace.c’ does.

     Problem is that for this to work correctly, it would need to
     compute the ‘--library-path’ argument at run time, by computing the
     equivalent of (map dirname (file-needed/recursive program)).
     Impractical at best.

  2. Wrap/graft every package in the closure (as opposed to generating
     wrappers for just those packages that appear in the profile, which
     is what ‘guix pack’ currently does).

     The downside is that the “userns” and “proot” execution engines
     don’t need something this heavyweight: they just need a leaf
     package to be wrapped.

  3. Ignore the problem.  After all, we’re talking about a corner case
     of the “fakechroot” engine, which is a niche within a niche.

Food for thought…

Ludo’.

Message #8 received at 62253 <at> debbugs.gnu.org (full text, mbox):

From: Josselin Poiret <dev <at> jpoiret.xyz>
To: Ludovic Courtès <ludovic.courtes <at> inria.fr>,
 62253 <at> debbugs.gnu.org
Subject: Re: bug#62253: Fakechroot execution engine doesn’t outlive ‘exec’ calls
Date: Sat, 18 Mar 2023 22:47:52 +0100

[Message part 1 (text/plain, inline)]

Hi Ludo,

Ludovic Courtès <ludovic.courtes <at> inria.fr> writes:

> I can think of several ways to address that:
>
>   1. Change the exec* wrappers in libfakechroot such that, on ENOENT,
>      they try a direct ld.so invocation to run program, like
>      ‘run-in-namespace.c’ does.
>
>      Problem is that for this to work correctly, it would need to
>      compute the ‘--library-path’ argument at run time, by computing the
>      equivalent of (map dirname (file-needed/recursive program)).
>      Impractical at best.
>
>   2. Wrap/graft every package in the closure (as opposed to generating
>      wrappers for just those packages that appear in the profile, which
>      is what ‘guix pack’ currently does).
>
>      The downside is that the “userns” and “proot” execution engines
>      don’t need something this heavyweight: they just need a leaf
>      package to be wrapped.
>
>   3. Ignore the problem.  After all, we’re talking about a corner case
>      of the “fakechroot” engine, which is a niche within a niche.
>
> Food for thought…

I would like to be proven wrong, but I don't think anyone has run into
this, and there are other possible engines (that do require more
privileges, sure). It seems quite non-trivial to fix, so this can
probably go on the back-burner until someone actually complains (please
do so).

Best,
-- 
Josselin Poiret

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 62253 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: Josselin Poiret <dev <at> jpoiret.xyz>
Cc: 62253 <at> debbugs.gnu.org
Subject: Re: bug#62253: Fakechroot execution engine doesn’t outlive ‘exec’ calls
Date: Mon, 20 Mar 2023 10:17:16 +0100

Hi,

Josselin Poiret <dev <at> jpoiret.xyz> skribis:

> I would like to be proven wrong, but I don't think anyone has run into
> this, and there are other possible engines (that do require more
> privileges, sure). It seems quite non-trivial to fix, so this can
> probably go on the back-burner until someone actually complains (please
> do so).

The Open MPI example I gave earlier is one that I’m interested in for
work (running packs on HPC clusters that have neither unprivileged user
namespaces nor Singularity¹).  It might be that we can work around the
problem though, we’ll see…

Ludo’.

¹ https://hpc.guix.info/blog/2020/05/faster-relocatable-packs-with-fakechroot/

Message #14 received at 62253 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: 62253 <at> debbugs.gnu.org
Subject: Re: bug#62253: Fakechroot execution engine doesn’t outlive ‘exec’ calls
Date: Tue, 18 Apr 2023 14:09:27 +0200

Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:

> In this case, adding ‘coreutils’ to the profile (on the ‘guix pack’
> command line) would give us wrapped binaries, and the problem is solved.
> But in other cases, it’s not that simple.  For instance, libmpi.so from
> Open MPI tries to exec one its programs, using its absolute file name:
>
> $ unshare -m -U -r -f sh -c 'mount -t tmpfs none /gnu; GUIX_EXECUTION_ENGINE=fakechroot /tmp/fakechroot-test/bin/IMB-MPI1'
> --------------------------------------------------------------------------
> The singleton application was not able to find the executable "orted" in
> your PATH or in the directory where Open MPI/OpenRTE was initially installed,
> and therefore cannot continue.
>
> For reference, we tried the following command:
>
>   /gnu/store/c7g9qalmbz4a94hwzk1v1cbq7n5m8plq-openmpi-4.1.4/bin/orted
>
> and got the error No such file or directory.

In the case of Open MPI as shown above, there’s actually an easy fix:
telling Open MPI where to look for ‘orted’.

Assuming you created a pack with:

  guix pack -RR openmpi intel-mpi-benchmarks bash-minimal -S /bin=bin

You can run code extracted from the tarball like this:

--8<---------------cut here---------------start------------->8---
export GUIX_EXECUTION_ENGINE=performance
salloc -N2 ./bin/mpirun -np 2 --launch-agent ./bin/orted \
  --map-by node -x GUIX_EXECUTION_ENGINE=performance -- \
  ./bin/IMB-MPI1 PingPong
--8<---------------cut here---------------end--------------->8---

The ‘--launch-agent’ trick allows us to work around the fact that exec’d
code escapes the fakechroot environment.

I’m closing this bug as “wontfix”.

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.