GNU bug report logs - #42164
Combining file-append with gexps results in incomprehensible errors

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 42164 in the body.
You can then email your comments to 42164 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: bug-guix <at> gnu.org
Subject: Combining file-append with gexps results in incomprehensible errors
Date: Thu, 2 Jul 2020 14:00:29 +0200

[Message part 1 (text/plain, inline)]

Consider this minimal operating-system definition:

(use-modules (gnu))
(use-package-modules gcc)

(operating-system
  (host-name "test")
  (timezone "UTC")
  (bootloader (bootloader-configuration
                (bootloader grub-efi-bootloader)
                (target "/boot/efi")))
  (file-systems (cons*
                  (file-system
                    (device (file-system-label "root"))
                    (mount-point "/")
                    (type "ext4"))
                  %base-file-systems))
  (packages '())
  (services (cons*
              (service special-files-service-type
                       `(("/lib64" ,(directory-union "rustup-libs"
                                      (list
                                        (file-append glibc "/lib")
                                        (file-append #~#$gcc:lib "/lib"))))))
              %base-services)))

I would expect this way of specifying a specific output of a package to
work, but it results in the following error:

ice-9/boot-9.scm:1515:18: object is not an exception of the right type #<&gexp-input-error input: #<gexp #<gexp-input #<package gcc <at> 7.5.0 gnu/packages/gcc.scm:520 7f06c996c960>:lib> 7f06c6b06990>> #<record-type &package-input-error>

This also happens when I omit the directory-union:

              (service special-files-service-type
                       `(("/lib64" ,(file-append #~#$gcc:lib "/lib"))))

What *does* work, is this variant with string-append:

              (service special-files-service-type
                       `(("/lib64" ,#~(string-append #$gcc:lib "/lib"))))

However, using it in the directory-union breaks again:

              (service special-files-service-type
                       `(("/lib64" ,(directory-union "rustup-libs"
                                      (list
                                        (file-append glibc "/lib")
                                        #~(string-append #$gcc:lib "/lib"))))))

ERROR: In procedure opendir:
Wrong type (expecting string): (string-append "/gnu/store/mdxmdhrlkgdik6ay9pzmmy8mjcbibpwb-gcc-7.5.0-lib" "/lib")
builder for `/gnu/store/p5hf7hqxn35fgsb75s5i7326vyzb8jkr-rustup-libs.drv' failed with exit code 1

I have figured out that the following does work:

              (service special-files-service-type
                       `(("/lib64" ,#~(directory-union "rustup-libs"
                                        (list
                                          (string-append #$glibc "/lib")
                                          (string-append #$gcc:lib "/lib"))))))

However, I would expect the other variants to work as well. The
documentation and error messages are lacking in this regard.

Regards,
Jakub Kądziołka

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Jakub Kądziołka <kuba <at> kadziolka.net>
Cc: 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Mon, 12 Apr 2021 21:58:37 -0400

Hi Jakub!

Jakub Kądziołka <kuba <at> kadziolka.net> writes:

> Consider this minimal operating-system definition:
>
> (use-modules (gnu))
> (use-package-modules gcc)
>
> (operating-system
>   (host-name "test")
>   (timezone "UTC")
>   (bootloader (bootloader-configuration
>                 (bootloader grub-efi-bootloader)
>                 (target "/boot/efi")))
>   (file-systems (cons*
>                   (file-system
>                     (device (file-system-label "root"))
>                     (mount-point "/")
>                     (type "ext4"))
>                   %base-file-systems))
>   (packages '())
>   (services (cons*
>               (service special-files-service-type
>                        `(("/lib64" ,(directory-union "rustup-libs"
>                                       (list
>                                         (file-append glibc "/lib")
>                                         (file-append #~#$gcc:lib "/lib"))))))
>               %base-services)))
>
> I would expect this way of specifying a specific output of a package to
> work, but it results in the following error:
>
> ice-9/boot-9.scm:1515:18: object is not an exception of the right type #<&gexp-input-error input: #<gexp #<gexp-input #<package gcc <at> 7.5.0 gnu/packages/gcc.scm:520 7f06c996c960>:lib> 7f06c6b06990>> #<record-type &package-input-error>

[...]

> I have figured out that the following does work:
>
>               (service special-files-service-type
>                        `(("/lib64" ,#~(directory-union "rustup-libs"
>                                         (list
>                                           (string-append #$glibc "/lib")
>                                           (string-append #$gcc:lib "/lib"))))))
>
> However, I would expect the other variants to work as well. The
> documentation and error messages are lacking in this regard.

This seems to come up every time I have to specify a non-default package
output; I've investigated the issue a bit and it seems to be caused by
the lack of a gexp-compiler for the gexp object resulting from
#$package:output, which is of type gexp-input as represented by, for
example:

#<gexp-input #<package opendht <at> 2.2.0rc4 /home/maxim/src/guix/gnu/packages/networking.scm:3426 7f2dbcae1aa0>:tools>

From the REPL

(lower-object (gexp (ungexp opendht "tools")))
$17 = #<procedure 7f2db56ed100 at guix/gexp.scm:246:2 (state)>
scheme@(gnu services jami)> ,run-in-store $17
While executing meta-command:
ERROR:
  1. &gexp-input-error: #<gexp #<gexp-input #<package opendht <at> 2.2.0rc4
  /home/maxim/src/guix/gnu/packages/networking.scm:3426
  7f2dbcae1aa0>:tools> 7f2db7427690>
  
So if we were to define/register a gexp-compiler for this kind of
gexp-input, we could make it work, but I haven't yet investigated how
easy/difficult that would be.

Seems worthwhile though, this issue is quite annoying when attempting to
work with specific outputs!

Thanks for the detailed report.

Maxim

Message #11 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Brian Cully <bjc <at> spork.org>
To: 42164 <at> debbugs.gnu.org
Subject: Combining file-append with gexps results in incomprehensible errors
Date: Sat, 23 Apr 2022 12:03:21 -0400

	I was having a similar issue, so I dug into this issue a bit,
and it turns out that you *can* select an output from file-append, but
the syntax is a bit wonky:

---[snip]---
(let ((fa (file-append pkg "/path")))
  $~$#fa:output)
---[snip]---

	I’ve tried to remove the let:

---[snip]---
$~$#(file-append pkg "/path"):output
---[snip]---

	But this confuses the reader, an it things ‘:output’ is a
variable. Not too surprising, but it does make the Guix configuration
files more awkward, since now you have to ‘define’ the file-append
operations at the top level so you can refer to them with the syntax
above.

	I would prefer to have something like any of the following (in
order of preference):

---[snip]---
(file-append pkg:output "/path")
(file-append (list pkg "output") "/path")
(file-append $~$#pkg:output "/path")
---[snip]---

	The reason being that it’s awkward to select the output after
the file-append due to syntax, and it’s less intuitive than trying to
select the output directly from the package inside the file-append (and
it seems I’m not alone in this). I’ve been through the code, so to some
degree I realize why things are being done the way they are, but that
doesn’t change my current preferences.

	I prefer the list over the gexp/ungexp because we already use
lists to select outputs in other parts of the config declarations, and
the gexp/ungexp stuff is harder to understand from the point-of-view of
a user, rather than package author. It’s also not present in the
majority of configs, I’d wager.

	Before I found the “correct” way to do things, I did cobble
together some code that allowed the (list pkg "output") form to work,
and would be happy to submit it upstream. It’s not perfect, since it
ignores the ‘output’ argument in the gexp expander in favor of the
stored output in the file-append, but I don’t think that’s an actual
issue in practice.

-bjc

Message #14 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Brian Cully <bjc <at> spork.org>, 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Sat, 23 Apr 2022 19:06:34 +0200

[Message part 1 (text/plain, inline)]

Brian Cully schreef op za 23-04-2022 om 12:03 [-0400]:
> 	I was having a similar issue, so I dug into this issue a bit,
> and it turns out that you *can* select an output from file-append, but
> the syntax is a bit wonky:
> 
> ---[snip]---
> (let ((fa (file-append pkg "/path")))
>   $~$#fa:output)
> ---[snip]---
> 
> 	I’ve tried to remove the let:
> 
> ---[snip]---
> $~$#(file-append pkg "/path"):output
> ---[snip]---

Do you mean #~#$fa:output here?

> 
> 	But this confuses the reader, an it things ‘:output’ is a
> variable. Not too surprising, but it does make the Guix configuration
> files more awkward, since now you have to ‘define’ the file-append
> operations at the top level so you can refer to them with the syntax
> above.
> 
> 	I would prefer to have something like any of the following (in
> order of preference):
> 
> ---[snip]---
> (file-append pkg:output "/path")

This one is only possible if file-append becomes syntax instead of a
procedure.

> (file-append (list pkg "output") "/path")

Possible, but seems semantically wrong to me -- what's a list doing in
a file-manipulating procedure?

> (file-append $~$#pkg:output "/path")

I assume this would be #~#$pkg:output?

My bikeshed proposal:

  (file-append (output glibc "static") "/lib/...")

Here, 'output' is a procedure constructing some kind of object.
This would also be useful outside 'file-append', as it allows
programmatically constructing equivalents #~#$pkg:output for multiple
'output' (there was some package definition some while back where this
would have been useful).

This 'output' could perhaps also be used in 'inputs' / 'native-inputs'
fields.  YMMV.

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Brian Cully <bjc <at> spork.org>
Cc: 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Sat, 23 Apr 2022 20:38:12 +0200

[Message part 1 (text/plain, inline)]

Brian Cully schreef op za 23-04-2022 om 13:29 [-0400]:
> 	I like this, and would prefer it over the ‘list’ proposal. I’m
> not sure how to implement it given my limited knowledge of Guix
> internals, though. Special-casing a list inside of file-append was
> fairly straightforward and solved the immediate issue.

Proposed implementation:

(define (output thing output)
  (gexp-input package output))

Seems like this thing already exists, it's just not well-known and not
documented in the manual.

Greetings,
Maxime

[signature.asc (application/pgp-signature, inline)]

Message #20 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Brian Cully <bjc <at> spork.org>
Cc: 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Sat, 23 Apr 2022 23:07:08 +0200

[Message part 1 (text/plain, inline)]

Brian Cully schreef op za 23-04-2022 om 16:40 [-0400]:
> 	Where is this in the code?

Nowhere.  I tried implementing 'output', and noticed I just ended up
with something equivalent to a simple call to 'gexp-input' (hence,
‘this thing already exists’).  'gexp-input' can be found in (guix
gexp).

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Brian Cully <bjc <at> spork.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Sat, 23 Apr 2022 13:29:18 -0400

Maxime Devos <maximedevos <at> telenet.be> writes:

> Do you mean #~#$fa:output here?

	Yes, I did. Sorry, my fingers have minds of their own, and the
gexp syntax, in particular, really runs counter to their memory.

>> (file-append pkg:output "/path")
>
> This one is only possible if file-append becomes syntax instead of a
> procedure.

	I did not expect this to be particularly possible, but I find it
aesthetically and intuitively pleasing enough to propose it anyway.

>> (file-append (list pkg "output") "/path")
>
> Possible, but seems semantically wrong to me -- what's a list doing in
> a file-manipulating procedure?

	It need not be a list, but I do think it makes sense inside the
file-append, rather than a modification to file-append’s results, as is
currently the case. I picked list because it was being used elsewhere to
explicitly select a derivation’s output.

> My bikeshed proposal:
>
>   (file-append (output glibc "static") "/lib/...")
>
> Here, 'output' is a procedure constructing some kind of object.
> This would also be useful outside 'file-append', as it allows
> programmatically constructing equivalents #~#$pkg:output for multiple
> 'output' (there was some package definition some while back where this
> would have been useful).
>
> This 'output' could perhaps also be used in 'inputs' / 'native-inputs'
> fields.  YMMV.

	I like this, and would prefer it over the ‘list’ proposal. I’m
not sure how to implement it given my limited knowledge of Guix
internals, though. Special-casing a list inside of file-append was
fairly straightforward and solved the immediate issue.

	I’d be happy to take a look at implementing this, but I’m a bit
clueless where to begin, as it’s much more general than my previous
attempt. If you (or others) have the time, I’d appreciate any pointers
that’d make it easier to begin.

	For instance: I would expect the ‘output’ form to be lowerable,
which would be done in the ‘compile’ branch of a gexp-compiler, but then
it would have to be expanded later in the ‘expand’ branch, which
presents an issue: the third argument to the expander procedure is
already an output. Would we do away with that argument? Somehow fill it
in based on the ‘output’ form (either expressed or implicit)? Let the
‘output’ form, if present, override it?

-bjc

Message #26 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Brian Cully <bjc <at> spork.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Sat, 23 Apr 2022 16:40:08 -0400

Maxime Devos <maximedevos <at> telenet.be> writes:

> Proposed implementation:
>
> (define (output thing output)
>   (gexp-input package output))
>
> Seems like this thing already exists, it's just not well-known and not
> documented in the manual.

	Where is this in the code? I couldn’t find it myself (grepping
for these keywords turns up a lot of false positives). On the off chance
it might work, I did try ‘(file-append (output pkg "lib") "/path")’ and
was told that ‘output’ is an unbound variable.

-bjc

Message #29 received at 42164 <at> debbugs.gnu.org (full text, mbox):

From: Brian Cully <bjc <at> spork.org>
To: Maxime Devos <maximedevos <at> telenet.be>
Cc: 42164 <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Sun, 24 Apr 2022 19:27:11 -0400

Maxime Devos <maximedevos <at> telenet.be> writes:

> Nowhere.  I tried implementing 'output', and noticed I just 
> ended up
> with something equivalent to a simple call to 'gexp-input' 
> (hence,
> ‘this thing already exists’).  'gexp-input' can be found in 
> (guix
> gexp).

	I finally had a chance to mess around with this, and it 
	does, indeed, do the job. Since it wasn’t clear to me how 
	it worked, I’d like to document it here.

	To accomplish the goal at the begging of this thread, 
	rather than using:
--8<---------------cut here---------------start------------->8---
(file-append #~#$gcc:lib “/lib”)
--8<---------------cut here---------------end--------------->8---

	The “correct” way is this:
--8<---------------cut here---------------start------------->8---
(gexp-input (file-append gcc "/lib") "lib")
--8<---------------cut here---------------end--------------->8---

	The name seems a little weird, since we’re selecting an 
	output, but it makes sense from the other end: we’re 
	creating an input for the higher-level expression (in this 
	case, the special-file-services-type).

	Anyway, this works for me, so this ticket can probably be 
	closed.

-bjc

Message #34 received at 42164-done <at> debbugs.gnu.org (full text, mbox):

From: Maxime Devos <maximedevos <at> telenet.be>
To: Brian Cully <bjc <at> spork.org>
Cc: 42164-done <at> debbugs.gnu.org
Subject: Re: bug#42164: Combining file-append with gexps results in
 incomprehensible errors
Date: Mon, 25 Apr 2022 08:20:57 +0200

[Message part 1 (text/plain, inline)]

Brian Cully schreef op zo 24-04-2022 om 19:27 [-0400]:
> --8<---------------cut here---------------start------------->8---
> (file-append #~#$gcc:lib “/lib”)
> --8<---------------cut here---------------end--------------->8---
> 
> 	The “correct” way is this:
> --8<---------------cut here---------------start------------->8---
> (gexp-input (file-append gcc "/lib") "lib")
> --8<---------------cut here---------------end--------------->8---

FWIW, I was actually thinking of

  (file-append (gexp-input gcc "lib") "/lib")

(First select the output with 'gexp-input', then append a suffix.)

Greetings,
Maxime.

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.