GNU bug report logs - #42688
Running a script with `guix repl` doesn't "see" additional channels using (%package-module-path)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 42688 in the body.
You can then email your comments to 42688 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: pkill9 <pkill9 <at> runbox.com>
To: bug-guix <at> gnu.org
Subject: Running a script with `guix repl` doesn't "see" additional channels
 using (%package-module-path)
Date: Mon, 3 Aug 2020 04:33:31 +0100

Running the following in `guix repl` returns additional channels:
```
itsme <at> antelope ~> guix repl
GNU Guile 3.0.4
Copyright (C) 1995-2020 Free Software Foundation, Inc.

Guile comes with ABSOLUTELY NO WARRANTY; for details type `,show w'.
This program is free software, and you are welcome to redistribute it
under certain conditions; type `,show c' for details.

Enter `,help' for help.
scheme@(guix-user)> (use-modules (gnu packages))
scheme@(guix-user)> (%package-module-path)
$1 =
(("/gnu/store/kds0mq06qpin125gkikwzdm6mjfwjffc-guix-module-union/share/guile/site/3.0"
. "gnu/packages")
"/gnu/store/96pa4rc57zgqf36y2kv8z20p2jvlgypq-pkill9-free-channel-dependency/share/guile/site/3.0"
"/gnu/store/3ilx18ywdm6xk9f5l1mznrn45vcbncsq-pkill9-free/share/guile/site/3.0")
scheme@(guix-user)>
```

But running the following in "test.scm" with `guix repl /tmp/test.scm
doesn't return additional channels:
```
(use-modules (gnu packages))
(display (%package-module-path))
```

```
((/gnu/store/kds0mq06qpin125gkikwzdm6mjfwjffc-guix-module-union/share/guile/site/3.0
. gnu/packages))
```

fold-available-packages uses this to search for packages, which I am
using for a script. As a result, the script doesn't know about packages
from the additional channels.

Message #8 received at 42688 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: pkill9 <pkill9 <at> runbox.com>
Cc: 42688 <at> debbugs.gnu.org
Subject: Re: bug#42688: Running a script with `guix repl` doesn't "see"
 additional channels using (%package-module-path)
Date: Wed, 16 Sep 2020 16:15:54 +0200

Dear,

Thank you for the report.  Well, it seems similar to #37399
<https://issues.guix.gnu.org/issue/37399>.


On Mon, 03 Aug 2020 at 04:33, pkill9 <pkill9 <at> runbox.com> wrote:

> fold-available-packages uses this to search for packages, which I am
> using for a script. As a result, the script doesn't know about packages
> from the additional channels.

What happens if you try the “trick” using GUILE_LOAD_PATH?
Or the option ’-L’?

(Well, maybe a solution for the mean time if you do not have too much
channels).


All the best,
simon

Message #11 received at 42688 <at> debbugs.gnu.org (full text, mbox):

From: Leo Prikler <leo.prikler <at> edu.uni-graz.at>
To: <42688 <at> debbugs.gnu.org>
Subject: Running a script with `guix repl` doesn't "see" additional channels
 using (%package-module-path)
Date: Wed, 16 Sep 2020 17:16:52 +0200

Hi Guix,

I've finally figured out, what causes this issue.

Guix repl uses the following code to call scripts:
```
    (unless (null? script)
      ;; Run script
      (save-module-excursion
       (lambda ()
         (set-program-arguments script)
         (set-user-module)
         (load-in-vicinity "." (car script)))))
```

But `guix describe` (which is used to initialize %package-module-path)
has the following:

```
(define current-profile
  (mlambda ()
    "Return the profile (created by 'guix pull') the calling process
lives in,
or #f if this is not applicable."
    (match (command-line)
      ((program . _)
       (and (string-suffix? "/bin/guix" program)
           [...])))))

(define current-profile-entries [...])
(define current-channel-entries [...])
(define package-path-entries [...])
```

Each of these procedures depends on the previous, building up a chain
that fails exactly in the case where we (set-program-arguments [...])
with a script other than the current channel's guix (which is probably
the way you'd want to use `guix repl`).  

There are some ways of resolving this.  One would be to access earlier
versions of "command-line" – it does resolve to a fluid, but that fluid
itself is not exposed to Guile.  Perhaps there might be some FFI magic
to access it.

You could also set up your script to fake being a Guix command by
setting the command line to be (cons*
"$HOME/.config/guix/current/bin/guix" "repl" (command-line)), i.e.
reconstructing the way your script has been invoked.  This would
obviously break if you were to call it with a different Guix, also
you'd have to resolve $HOME instead of writing it like that, but you'd
have access to your channels.

On the other hand, we could patch `guix repl` to initialize %package-
module-path earlier (still leaving `guix describe` broken) or somehow
try to work around that issue in `guix describe`.

Regards,
Leo

Message #14 received at 42688 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Leo Prikler <leo.prikler <at> edu.uni-graz.at>
Cc: 42688 <at> debbugs.gnu.org
Subject: Re: bug#42688: Running a script with `guix repl` doesn't "see"
 additional channels using (%package-module-path)
Date: Thu, 17 Sep 2020 17:31:05 +0200

[Message part 1 (text/plain, inline)]

Hi Leo,

Leo Prikler <leo.prikler <at> edu.uni-graz.at> skribis:

> I've finally figured out, what causes this issue.
>
> Guix repl uses the following code to call scripts:
> ```
>     (unless (null? script)
>       ;; Run script
>       (save-module-excursion
>        (lambda ()
>          (set-program-arguments script)
>          (set-user-module)
>          (load-in-vicinity "." (car script)))))
> ```
>
> But `guix describe` (which is used to initialize %package-module-path)
> has the following:
>
> ```
> (define current-profile
>   (mlambda ()
>     "Return the profile (created by 'guix pull') the calling process
> lives in,
> or #f if this is not applicable."
>     (match (command-line)
>       ((program . _)
>        (and (string-suffix? "/bin/guix" program)
>            [...])))))
>
> (define current-profile-entries [...])
> (define current-channel-entries [...])
> (define package-path-entries [...])
> ```
>
> Each of these procedures depends on the previous, building up a chain
> that fails exactly in the case where we (set-program-arguments [...])
> with a script other than the current channel's guix (which is probably
> the way you'd want to use `guix repl`).  

Good catch!

> There are some ways of resolving this.  One would be to access earlier
> versions of "command-line" – it does resolve to a fluid, but that fluid
> itself is not exposed to Guile.  Perhaps there might be some FFI magic
> to access it.

‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s really
inaccessible.

However, perhaps we could save the initial value of (program-arguments)
in (guix ui) and use that in (guix describe)?

> On the other hand, we could patch `guix repl` to initialize %package-
> module-path earlier (still leaving `guix describe` broken) or somehow
> try to work around that issue in `guix describe`.

Initializing (%package-module-path) earlier sounds like a good idea too,
maybe like this:

[Message part 2 (text/x-patch, inline)]

diff --git a/guix/scripts/repl.scm b/guix/scripts/repl.scm
index 7d4e474e92..b672489ed6 100644
--- a/guix/scripts/repl.scm
+++ b/guix/scripts/repl.scm
@@ -22,6 +22,7 @@
   #:use-module (guix ui)
   #:use-module (guix scripts)
   #:use-module (guix repl)
+  #:autoload   (gnu packages) (%package-module-path)
   #:use-module (srfi srfi-1)
   #:use-module (srfi srfi-26)
   #:use-module (srfi srfi-37)
@@ -173,6 +174,10 @@ call THUNK."
   (with-error-handling
 
     (unless (null? script)
+      ;; Before running SCRIPT, initialize %PACKAGE-MODULE-PATH so that it
+      ;; contains the user's channels (the statement triggers an autoload).
+      (%package-module-path)
+
       ;; Run script
       (save-module-excursion
        (lambda ()

[Message part 3 (text/plain, inline)]

?

Thanks!

Ludo’.

Message #17 received at 42688 <at> debbugs.gnu.org (full text, mbox):

From: Leo Prikler <leo.prikler <at> student.tugraz.at>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 42688 <at> debbugs.gnu.org
Subject: Re: bug#42688: Running a script with `guix repl` doesn't "see"
 additional channels using (%package-module-path)
Date: Thu, 17 Sep 2020 18:15:23 +0200

Hi Ludo,
Am Donnerstag, den 17.09.2020, 17:31 +0200 schrieb Ludovic Courtès:
> Hi Leo,
> 
> [...]
> 
> ‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s
> really
> inaccessible.
Thought so.

> However, perhaps we could save the initial value of (program-
> arguments)
> in (guix ui) and use that in (guix describe)?
I'd personally put it in (guix describe) and use the same autoload
trick, that you've now used for %package-module-path (or a dedicated
save-...-excursion).  (guix ui) has a heavy closure for (guix describe)
to pull.

> > On the other hand, we could patch `guix repl` to initialize
> > %package-
> > module-path earlier (still leaving `guix describe` broken) or
> > somehow
> > try to work around that issue in `guix describe`.
> 
> Initializing (%package-module-path) earlier sounds like a good idea
> too,
> maybe like this:
> 
> [...] 
> 
I haven't tested that yet (pre-inst-env makes it so Guix doesn't have
any channels anyway), but yeah, something like that would have been my
idea.

Regards,
Leo

Message #20 received at 42688 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Leo Prikler <leo.prikler <at> student.tugraz.at>
Cc: 42688 <at> debbugs.gnu.org
Subject: Re: bug#42688: Running a script with `guix repl` doesn't "see"
 additional channels using (%package-module-path)
Date: Thu, 17 Sep 2020 21:10:53 +0200

Hi,

Leo Prikler <leo.prikler <at> student.tugraz.at> skribis:

> Am Donnerstag, den 17.09.2020, 17:31 +0200 schrieb Ludovic Courtès:
>> Hi Leo,
>> 
>> [...]
>> 
>> ‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s
>> really
>> inaccessible.
> Thought so.
>
>> However, perhaps we could save the initial value of (program-
>> arguments)
>> in (guix ui) and use that in (guix describe)?
> I'd personally put it in (guix describe) and use the same autoload
> trick, that you've now used for %package-module-path (or a dedicated
> save-...-excursion).

In general, (guix …) module should not depend on (gnu …) modules, which
rules out this option.

> (guix ui) has a heavy closure for (guix describe) to pull.

Every (guix scripts …) module depends on (guix ui) via the ‘guix’
command.  (Probably something we could improve, but that’s the way it
is.)

Now, I realize my proposal was misguided because (guix describe) should
remain “UI-free” so to speak.  Hmm…

>> > On the other hand, we could patch `guix repl` to initialize
>> > %package-
>> > module-path earlier (still leaving `guix describe` broken) or
>> > somehow
>> > try to work around that issue in `guix describe`.
>> 
>> Initializing (%package-module-path) earlier sounds like a good idea
>> too,
>> maybe like this:
>> 
>> [...] 
>> 
> I haven't tested that yet (pre-inst-env makes it so Guix doesn't have
> any channels anyway), but yeah, something like that would have been my
> idea.

Alright, I’ll give it a spin.

Thank you!

Ludo’.

Message #23 received at 42688 <at> debbugs.gnu.org (full text, mbox):

From: Leo Prikler <leo.prikler <at> student.tugraz.at>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 42688 <at> debbugs.gnu.org
Subject: Re: bug#42688: Running a script with `guix repl` doesn't "see"
 additional channels using (%package-module-path)
Date: Thu, 17 Sep 2020 21:56:52 +0200

Hi,
Am Donnerstag, den 17.09.2020, 21:10 +0200 schrieb Ludovic Courtès:
> Hi,
> 
> Leo Prikler <leo.prikler <at> student.tugraz.at> skribis:
> 
> > Am Donnerstag, den 17.09.2020, 17:31 +0200 schrieb Ludovic Courtès:
> > > Hi Leo,
> > > 
> > > [...]
> > > 
> > > ‘scm_program_arguments_fluid’ is marked as SCM_INTERNAL, so it’s
> > > really
> > > inaccessible.
> > Thought so.
> > 
> > > However, perhaps we could save the initial value of (program-
> > > arguments)
> > > in (guix ui) and use that in (guix describe)?
> > I'd personally put it in (guix describe) and use the same autoload
> > trick, that you've now used for %package-module-path (or a
> > dedicated
> > save-...-excursion).
> 
> In general, (guix …) module should not depend on (gnu …) modules,
> which
> rules out this option.
Sure, but program-arguments are not defined in (gnu …) and it is a
(guix scripts …) that eventually pulls in %package-module-path. 
Therefore defining %guix-initial-program-arguments (or whatever it will
be called in the end) in (guix describe) still seems like an option to
me. 

> > (guix ui) has a heavy closure for (guix describe) to pull.
> 
> Every (guix scripts …) module depends on (guix ui) via the ‘guix’
> command.  (Probably something we could improve, but that’s the way it
> is.)
> 
> Now, I realize my proposal was misguided because (guix describe)
> should
> remain “UI-free” so to speak.  Hmm…
With that however, I am no longer so sure.  The initial program
arguments are part of the UI, but at the same time, that would make it
not UI-free to begin with.  Kinda strengthens the argument, that it
should be made a fluid/parameter/what have you, that gets initialized
with program-arguments at some point.

Regards,
Leo

Message #28 received at 42688-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Leo Prikler <leo.prikler <at> student.tugraz.at>
Cc: 42688-done <at> debbugs.gnu.org
Subject: Re: bug#42688: Running a script with `guix repl` doesn't "see"
 additional channels using (%package-module-path)
Date: Sat, 19 Sep 2020 23:03:07 +0200

Hello,

Leo Prikler <leo.prikler <at> student.tugraz.at> skribis:

> With that however, I am no longer so sure.  The initial program
> arguments are part of the UI, but at the same time, that would make it
> not UI-free to begin with.  Kinda strengthens the argument, that it
> should be made a fluid/parameter/what have you, that gets initialized
> with program-arguments at some point.

Alright.  I went with something along these lines in commit
1b179d7876f19f04009a2f9e248ac10711f4c660.

I tested that it works as intended with ‘guix pull --url=$PWD’ and
running a script from there that accesses modules of a secondary
channel.

Thank you!

Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.