GNU bug report logs - #45017
asdf-build-system packages have priority over user ones

Previous Next

Package: guix;

Reported by: Pierre Neidhardt <mail <at> ambrevar.xyz>

Date: Thu, 3 Dec 2020 09:43:02 UTC

Severity: normal

Done: Guillaume Le Vaillant <glv <at> posteo.net>

Bug is archived. No further changes may be made.

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45017 in the body.
You can then email your comments to 45017 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

View this report as an mbox folder, status mbox, maintainer mbox


Report forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Thu, 03 Dec 2020 09:43:02 GMT) Full text and rfc822 format available.

Acknowledgement sent to Pierre Neidhardt <mail <at> ambrevar.xyz>:
New bug report received and forwarded. Copy sent to bug-guix <at> gnu.org. (Thu, 03 Dec 2020 09:43:02 GMT) Full text and rfc822 format available.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: bug-guix <at> gnu.org
Cc: Guillaume Le Vaillant <glv <at> posteo.net>
Subject: asdf-build-system packages have priority over user ones
Date: Thu, 03 Dec 2020 10:42:29 +0100
[Message part 1 (text/plain, inline)]
Since staging was merged a few days ago, I've noticed an undesirable
side-effect of our revamped asdf-build-system: the systems packages have
priority over the user-local ones.

Here is the default list of registries:

--8<---------------cut here---------------start------------->8---
(defparameter* *default-source-registries*
    '(environment-source-registry
      user-source-registry
      user-source-registry-directory
      default-user-source-registry
      system-source-registry
      system-source-registry-directory
      default-system-source-registry)
    "List of default source registries" "3.1.0.102")
--8<---------------cut here---------------end--------------->8---

`user-source-registry' points to what is read in
~/.config/common-lisp/source-registry.conf(.d/)?, so nothing by default.

user-source-registry-directory reads what's in XDG_CONFIG_DIRS, and it
finds the Guix packages there.

The default-user-source-registry expands to

--8<---------------cut here---------------start------------->8---
(:SOURCE-REGISTRY (:TREE (:HOME "common-lisp/"))
                (:DIRECTORY (:HOME ".sbcl/systems/"))
                (:DIRECTORY
                 #P"/home/ambrevar/.local/share/common-lisp/systems/")
                (:TREE #P"/home/ambrevar/.local/share/common-lisp/source/")
                :INHERIT-CONFIGURATION)
--8<---------------cut here---------------end--------------->8---

which is where I would drop my user-local packages.

Finally, default-system-source-registry reads what's in XDG_DATA_DIRS,
and it finds the packages there again, but it does not read the
Guix-generated conf which collects the dependencies of each package.



I believe the issue is that `user-source-registry-directory' should read
from XDG_CONFIG_HOME, and not XDG_CONFIG_DIRS (which would make it
redundant with `user-source-registry').  This would make it an upstream
issue, but I wanted to double-check I properly understood the problem
with the Guix community first.

Guillaume, thoughts on this?

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Thu, 03 Dec 2020 12:05:02 GMT) Full text and rfc822 format available.

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: bug-guix <at> gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Thu, 03 Dec 2020 13:04:25 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Since staging was merged a few days ago, I've noticed an undesirable
> side-effect of our revamped asdf-build-system: the systems packages have
> priority over the user-local ones.
>
> Here is the default list of registries:
>
> --8<---------------cut here---------------start------------->8---
> (defparameter* *default-source-registries*
>     '(environment-source-registry
>       user-source-registry
>       user-source-registry-directory
>       default-user-source-registry
>       system-source-registry
>       system-source-registry-directory
>       default-system-source-registry)
>     "List of default source registries" "3.1.0.102")
> --8<---------------cut here---------------end--------------->8---
>
> `user-source-registry' points to what is read in
> ~/.config/common-lisp/source-registry.conf(.d/)?, so nothing by default.
>
> user-source-registry-directory reads what's in XDG_CONFIG_DIRS, and it
> finds the Guix packages there.
>
> The default-user-source-registry expands to
>
> --8<---------------cut here---------------start------------->8---
> (:SOURCE-REGISTRY (:TREE (:HOME "common-lisp/"))
>                 (:DIRECTORY (:HOME ".sbcl/systems/"))
>                 (:DIRECTORY
>                  #P"/home/ambrevar/.local/share/common-lisp/systems/")
>                 (:TREE #P"/home/ambrevar/.local/share/common-lisp/source/")
>                 :INHERIT-CONFIGURATION)
> --8<---------------cut here---------------end--------------->8---
>
> which is where I would drop my user-local packages.
>
> Finally, default-system-source-registry reads what's in XDG_DATA_DIRS,
> and it finds the packages there again, but it does not read the
> Guix-generated conf which collects the dependencies of each package.
>
>
>
> I believe the issue is that `user-source-registry-directory' should read
> from XDG_CONFIG_HOME, and not XDG_CONFIG_DIRS (which would make it
> redundant with `user-source-registry').  This would make it an upstream
> issue, but I wanted to double-check I properly understood the problem
> with the Guix community first.
>
> Guillaume, thoughts on this?

Our cl-asdf package definition has little patches for
'user-source-registry-directory' and
'user-output-translations-directory-pathname' to make ASDF read
configuration files in all the directories listed in 'XDG_CONFIG_DIRS'
(IIRC regular ASDF was only reading from the first valid one in
'XDG_CONFIG_HOME' or 'XDG_CONFIG_DIRS'):

--8<---------------cut here---------------start------------->8---
(substitute* dst-asdf
  (("\\(xdg-config-pathname \\*source-registry-directory\\* direction\\)")
   "`(:source-registry
      ,@(loop
           for dir in (xdg-config-dirs
                       \"common-lisp/source-registry.conf.d/\")
           collect `(:include ,dir))
      :inherit-configuration)")
  (("\\(xdg-config-pathname \\*output-translations-directory\\* direction\\)")
   "`(:output-translations
      ,@(loop
           for dir in (xdg-config-dirs
                       \"common-lisp/asdf-output-translations.conf.d/\")
           collect `(:include ,dir))
      :inherit-configuration)")))
--8<---------------cut here---------------end--------------->8---

I guess 'XDG_CONFIG_HOME' is missing here and we could add it in the
list before the 'XDG_CONFIG_DIRS' directories. Something like:

--8<---------------cut here---------------start------------->8---
(substitute* dst-asdf
  (("\\(xdg-config-pathname \\*source-registry-directory\\* direction\\)")
   "`(:source-registry
      (:include (xdg-config-home \"common-lisp/source-registry.conf.d/\"))
      ,@(loop
           for dir in (xdg-config-dirs
                       \"common-lisp/source-registry.conf.d/\")
           collect `(:include ,dir))
      :inherit-configuration)")
  (("\\(xdg-config-pathname \\*output-translations-directory\\* direction\\)")
   "`(:output-translations
      (:include (xdg-config-home \"common-lisp/asdf-output-translations.conf.d/\"))
      ,@(loop
           for dir in (xdg-config-dirs
                       \"common-lisp/asdf-output-translations.conf.d/\")
           collect `(:include ,dir))
      :inherit-configuration)")))
--8<---------------cut here---------------end--------------->8---

Or maybe we could make a better patch not changing
'user-source-registry-directory', but adding a new
'guix-source-registry' between 'default-user-source-registry' and
'system-source-registry' in the list of registries (and likewise for
output-translations).
I think this way the systems in '$HOME/.config/...', '$HOME/.local/...'
or '$HOME/common-lisp/...' will have priority on the ones in Guix
profiles.

What do you think?
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Thu, 03 Dec 2020 12:41:02 GMT) Full text and rfc822 format available.

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: bug-guix <at> gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Thu, 03 Dec 2020 13:40:02 +0100
[Message part 1 (text/plain, inline)]
Oh, I had missed that!  Good thing I asked you first then ;)

> --8<---------------cut here---------------start------------->8---
> (substitute* dst-asdf
>   (("\\(xdg-config-pathname \\*source-registry-directory\\* direction\\)")
>    "`(:source-registry
>       (:include (xdg-config-home \"common-lisp/source-registry.conf.d/\"))
>       ,@(loop
>            for dir in (xdg-config-dirs
>                        \"common-lisp/source-registry.conf.d/\")
>            collect `(:include ,dir))
>       :inherit-configuration)")
>   (("\\(xdg-config-pathname \\*output-translations-directory\\* direction\\)")
>    "`(:output-translations
>       (:include (xdg-config-home \"common-lisp/asdf-output-translations.conf.d/\"))
>       ,@(loop
>            for dir in (xdg-config-dirs
>                        \"common-lisp/asdf-output-translations.conf.d/\")
>            collect `(:include ,dir))
>       :inherit-configuration)")))
> --8<---------------cut here---------------end--------------->8---

Hmm, I don't think this would work when the user has no
~/.config/common-lisp/source-registry.conf.
Would it?

> Or maybe we could make a better patch not changing
> 'user-source-registry-directory', but adding a new
> 'guix-source-registry' between 'default-user-source-registry' and
> 'system-source-registry' in the list of registries (and likewise for
> output-translations).
> I think this way the systems in '$HOME/.config/...', '$HOME/.local/...'
> or '$HOME/common-lisp/...' will have priority on the ones in Guix
> profiles.

I like it!

Possible pitfall: if the user set *default-source-registries* in their
Lisp RC file (e.g. ~/.sbclrc) this will prevent ASDF from seeing this
new Guix source registry.

Alternately, we could simply move the patch from
user-source-registry-directory to system-source-registry-directory; same
for the output translation.

Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Thu, 03 Dec 2020 14:30:02 GMT) Full text and rfc822 format available.

Message #14 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Thu, 03 Dec 2020 15:29:10 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Alternately, we could simply move the patch from
> user-source-registry-directory to system-source-registry-directory; same
> for the output translation.

What do you think of the attached patch? Does it work for you?
[0001-gnu-cl-asdf-Improve-priorities-of-configuration-file.patch (text/x-patch, attachment)]
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Fri, 04 Dec 2020 08:52:02 GMT) Full text and rfc822 format available.

Message #17 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Fri, 04 Dec 2020 09:51:33 +0100
[Message part 1 (text/plain, inline)]
Thanks for taking a shot at this, looks great!

I'll test later, but for now one question: we patch sbcl to use our
cl-asdf, but what about the other compilers?  Seems to me that the other
compilers are going to have the same problem, aren't they?

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Fri, 04 Dec 2020 09:13:01 GMT) Full text and rfc822 format available.

Message #20 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Fri, 04 Dec 2020 10:12:06 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Thanks for taking a shot at this, looks great!
>
> I'll test later, but for now one question: we patch sbcl to use our
> cl-asdf, but what about the other compilers?  Seems to me that the other
> compilers are going to have the same problem, aren't they?

SBCL and ECL are patched to use our cl-asdf because it is necessary to
build the sbcl-* and ecl-* packages. Also patching ABCL, CCL, and Clisp
sounds like a good idea. At least all the compilers would use the same
version of ASDF, and hopefully behave in the same way when looking for
configuration files.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Fri, 04 Dec 2020 12:47:02 GMT) Full text and rfc822 format available.

Message #23 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Fri, 04 Dec 2020 13:46:28 +0100
[Message part 1 (text/plain, inline)]
Guillaume Le Vaillant <glv <at> posteo.net> writes:

> SBCL and ECL are patched to use our cl-asdf because it is necessary to
> build the sbcl-* and ecl-* packages. Also patching ABCL, CCL, and Clisp
> sounds like a good idea. At least all the compilers would use the same
> version of ASDF, and hopefully behave in the same way when looking for
> configuration files.

Yup, that'd be neat :)

Related to this, I've noticed another issue with SBCL packages: when an
SBCL package has a dependency that's updated in ~/common-lisp,
(asdf:load-system ...) tries to recompile it in its folder, which fails
since /gnu/store is read-only.

Example:

- Apply the above patch or edit
  ~/.config/common-lisp/source-registry.conf
  to make sure ~/common-lisp is loaded before system packages.

- Install sbcl-cl-cookie.

- Check out https://github.com/fukamachi/quri:
  git clone https://github.com/fukamachi/quri ~/common-lisp/quri

- Now run

--8<---------------cut here---------------start------------->8---
$ sbcl
* (asdf:load-system :cl-cookie)
WARNING: System definition file #P"/gnu/store/81cwmspx3366vdjs6v20rnd8a0xyr6in-sbcl-cl-fad-0.7.6/share/common-lisp/sbcl/cl-fad/cl-fad.asd" contains definition for system "cl-fad-test". Please only define "cl-fad" and secondary systems with a name starting with "cl-fad/" (e.g. "cl-fad/test") in that file.

debugger invoked on a SB-INT:SIMPLE-FILE-ERROR in thread
#<THREAD "main thread" RUNNING {100B790203}>:
  Error opening #P"/gnu/store/22q4ydm0pagi4irz0clssgkhkyh115j8-sbcl-cl-cookie-0.9.10-1.cea55ae/lib/common-lisp/sbcl/cl-cookie/src/cl-cookie-tmpGHU3ALSV.fasl":

    Read-only file system
--8<---------------cut here---------------end--------------->8---

Not sure what to do about this.

I'm guessing that Common Lisp development is not practical with Guix'
SBCL packages and I should just stick to the CL packages.

Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Fri, 04 Dec 2020 13:54:01 GMT) Full text and rfc822 format available.

Message #26 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Fri, 04 Dec 2020 14:53:08 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Guillaume Le Vaillant <glv <at> posteo.net> writes:
>
>> SBCL and ECL are patched to use our cl-asdf because it is necessary to
>> build the sbcl-* and ecl-* packages. Also patching ABCL, CCL, and Clisp
>> sounds like a good idea. At least all the compilers would use the same
>> version of ASDF, and hopefully behave in the same way when looking for
>> configuration files.
>
> Yup, that'd be neat :)

Patches for CCL and Clisp attached. ABCL fails to build with ASDF 3.3.4
and I have not seen an obvious error message in the logs, so this one
will need more investigation...


> Related to this, I've noticed another issue with SBCL packages: when an
> SBCL package has a dependency that's updated in ~/common-lisp,
> (asdf:load-system ...) tries to recompile it in its folder, which fails
> since /gnu/store is read-only.
>
> Example:
>
> - Apply the above patch or edit
>   ~/.config/common-lisp/source-registry.conf
>   to make sure ~/common-lisp is loaded before system packages.
>
> - Install sbcl-cl-cookie.
>
> - Check out https://github.com/fukamachi/quri:
>   git clone https://github.com/fukamachi/quri ~/common-lisp/quri
>
> - Now run
>
> --8<---------------cut here---------------start------------->8---
> $ sbcl
> * (asdf:load-system :cl-cookie)
> WARNING: System definition file #P"/gnu/store/81cwmspx3366vdjs6v20rnd8a0xyr6in-sbcl-cl-fad-0.7.6/share/common-lisp/sbcl/cl-fad/cl-fad.asd" contains definition for system "cl-fad-test". Please only define "cl-fad" and secondary systems with a name starting with "cl-fad/" (e.g. "cl-fad/test") in that file.
>
> debugger invoked on a SB-INT:SIMPLE-FILE-ERROR in thread
> #<THREAD "main thread" RUNNING {100B790203}>:
>   Error opening #P"/gnu/store/22q4ydm0pagi4irz0clssgkhkyh115j8-sbcl-cl-cookie-0.9.10-1.cea55ae/lib/common-lisp/sbcl/cl-cookie/src/cl-cookie-tmpGHU3ALSV.fasl":
>
>     Read-only file system
> --8<---------------cut here---------------end--------------->8---
>
> Not sure what to do about this.
>
> I'm guessing that Common Lisp development is not practical with Guix'
> SBCL packages and I should just stick to the CL packages.
>
> Thoughts?

The Lisp systems of sbcl-* and ecl-* packages installed in a profile are
supposed to be already compiled and immutable, so recompiling them
anyway would require tweaking the ASDF configuration from inside SBCL to
remove the output translations for these systems.
When doing development, it might be more convenient to just use the
sources or source packages of every system that has to be recompiled.
[0002-gnu-ccl-Use-system-ASDF-instead-of-bundled-one.patch (text/x-patch, attachment)]
[0003-gnu-clisp-Use-system-ASDF-instead-of-bundled-one.patch (text/x-patch, attachment)]
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Fri, 04 Dec 2020 17:31:01 GMT) Full text and rfc822 format available.

Message #29 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Fri, 04 Dec 2020 18:30:14 +0100
[Message part 1 (text/plain, inline)]
Guillaume Le Vaillant <glv <at> posteo.net> skribis:

> The Lisp systems of sbcl-* and ecl-* packages installed in a profile are
> supposed to be already compiled and immutable, so recompiling them
> anyway would require tweaking the ASDF configuration from inside SBCL to
> remove the output translations for these systems.
> When doing development, it might be more convenient to just use the
> sources or source packages of every system that has to be recompiled.

I think entering the following expressions just after starting SBCL will
allow development mixing libraries from user directories and from Guix
profiles:

--8<---------------cut here---------------start------------->8---
(require :asdf)
(asdf:clear-output-translations)
(asdf:initialize-output-translations
 '(:output-translations
   :enable-user-cache
   :ignore-inherited-configuration))
--8<---------------cut here---------------end--------------->8---

However with this configuration, the pre-compiled files in
"/gnu/store/..." will not be used at all; everything will be compiled
again if necessary and put in the user cache in ".cache/common-lisp/...".
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 08:20:02 GMT) Full text and rfc822 format available.

Message #32 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 09:18:50 +0100
[Message part 1 (text/plain, inline)]
A few comments:

> @@ -603,7 +603,8 @@ statistical profiler, a code coverage tool, and many other extensions.")
>                  "0x4bjx6cxsjvxyagijhlvmc7jkyxifdvz5q5zvz37028va65243c")
>                 (_ "15l7cfa4a7jkfwdzsfm4q3n22jnb57imxahpql3h77xin57v1gbz"))))))))
>      (native-inputs
> -     `(("m4" ,m4)))
> +     `(("cl-asdf" ,cl-asdf)
> +       ("m4" ,m4)))
>      (arguments
>       `(#:tests? #f                      ;no 'check' target
>         #:modules ((ice-9 match)
> @@ -615,6 +616,17 @@ statistical profiler, a code coverage tool, and many other extensions.")
>           (add-after 'unpack 'unpack-image
>             (lambda* (#:key inputs #:allow-other-keys)
>               (invoke "tar" "xzvf" (assoc-ref inputs "ccl-bootstrap"))))
> +         (add-after 'unpack 'replace-asdf
> +           ;; Use system ASDF instead of bundled one.
> +           (lambda* (#:key inputs outputs #:allow-other-keys)
> +             (let* ((cl-asdf (assoc-ref inputs "cl-asdf"))
> +                    (guix-asdf (string-append
> +                                cl-asdf
> +                                "/share/common-lisp/source/asdf/asdf.lisp"))
> +                    (out (string-append (assoc-ref outputs "out")))

"out" is unused.  By the way, the same "out" is unused in SBCL.

> +                    (contrib-asdf "tools/asdf.lisp"))
> +               (delete-file contrib-asdf)

I think you don't need to delete the file, copy-file should overwrite.
That's what SBCL does.

Same with CLISP.

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 09:18:02 GMT) Full text and rfc822 format available.

Message #35 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 10:17:41 +0100
[Message part 1 (text/plain, inline)]
Tested!

I've installed sbcl to my "common-lisp" profile along quri.
I've also cloned quri to ~/common-lisp.

Then:

--8<---------------cut here---------------start------------->8---
$ sbcl
* (asdf:locate-system :quri)
T
NIL
#P"/gnu/store/5gj1inwiqpn2fm9w384zd0grpyadx96m-sbcl-quri-0.1.0-2.b53231c/share/common-lisp/sbcl/quri/quri.asd"
NIL
NIL
NIL

;; Indeed:
* (asdf:user-source-registry-directory)
#P"/gnu/store/75qppl3a062b138fkrn324qq8f912zqh-profile/etc/common-lisp/source-registry.conf.d/"
--8<---------------cut here---------------end--------------->8---

It does not work because the sbcl package definition sets the
XDG_CONFIG_DIRS native search path which is picked by
user-source-registry-directory, which has higher priority than
default-user-source-registry.

I think the flaw is ASDF's this time:

--8<---------------cut here---------------start------------->8---
  (defun user-source-registry-directory (&key (direction :input))
    (xdg-config-pathname *source-registry-directory* direction))

;...

  (defun xdg-config-pathnames (&rest more)
    "Return a list of pathnames for application configuration.
MORE may contain specifications for a subpath relative to these directories: a
subpathname specification and keyword arguments as per RESOLVE-LOCATION \(see
also \"Configuration DSL\"\) in the ASDF manual."
    (filter-pathname-set
     `(,(xdg-config-home more)
       ,@(xdg-config-dirs more))))
--8<---------------cut here---------------end--------------->8---

So the user registry looks into XDG_CONFIG_DIRS, while really it should
just look at XDG_CONFIG_HOME.

So this patch should do (untested):

--8<---------------cut here---------------start------------->8---
- (defun user-source-registry-directory (&key (direction :input))
-   (xdg-config-pathname *source-registry-directory* direction))
+ (defun user-source-registry-directory (&key (direction :input))
+   (find-preferred-file (list (xdg-config-home *source-registry-directory*)) :direction direction))
--8<---------------cut here---------------end--------------->8---

If you confirm, then we should also report upstream I think.

Cheers!

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 09:44:02 GMT) Full text and rfc822 format available.

Message #38 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 10:42:59 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> A few comments:
>
>> @@ -603,7 +603,8 @@ statistical profiler, a code coverage tool, and many other extensions.")
>>                  "0x4bjx6cxsjvxyagijhlvmc7jkyxifdvz5q5zvz37028va65243c")
>>                 (_ "15l7cfa4a7jkfwdzsfm4q3n22jnb57imxahpql3h77xin57v1gbz"))))))))
>>      (native-inputs
>> -     `(("m4" ,m4)))
>> +     `(("cl-asdf" ,cl-asdf)
>> +       ("m4" ,m4)))
>>      (arguments
>>       `(#:tests? #f                      ;no 'check' target
>>         #:modules ((ice-9 match)
>> @@ -615,6 +616,17 @@ statistical profiler, a code coverage tool, and many other extensions.")
>>           (add-after 'unpack 'unpack-image
>>             (lambda* (#:key inputs #:allow-other-keys)
>>               (invoke "tar" "xzvf" (assoc-ref inputs "ccl-bootstrap"))))
>> +         (add-after 'unpack 'replace-asdf
>> +           ;; Use system ASDF instead of bundled one.
>> +           (lambda* (#:key inputs outputs #:allow-other-keys)
>> +             (let* ((cl-asdf (assoc-ref inputs "cl-asdf"))
>> +                    (guix-asdf (string-append
>> +                                cl-asdf
>> +                                "/share/common-lisp/source/asdf/asdf.lisp"))
>> +                    (out (string-append (assoc-ref outputs "out")))
>
> "out" is unused.  By the way, the same "out" is unused in SBCL.
>

Indeed, I'll fix that.

>> +                    (contrib-asdf "tools/asdf.lisp"))
>> +               (delete-file contrib-asdf)
>
> I think you don't need to delete the file, copy-file should overwrite.
> That's what SBCL does.
>
> Same with CLISP.

Actually, for CCL and Clisp, I get a "In procedure copy-file: Permission
denied" error if I don't delete the current file before copying the new
one.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 10:04:02 GMT) Full text and rfc822 format available.

Message #41 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 11:03:21 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Tested!
>
> I've installed sbcl to my "common-lisp" profile along quri.
> I've also cloned quri to ~/common-lisp.
>
> Then:
>
> --8<---------------cut here---------------start------------->8---
> $ sbcl
> * (asdf:locate-system :quri)
> T
> NIL
> #P"/gnu/store/5gj1inwiqpn2fm9w384zd0grpyadx96m-sbcl-quri-0.1.0-2.b53231c/share/common-lisp/sbcl/quri/quri.asd"
> NIL
> NIL
> NIL
>
> ;; Indeed:
> * (asdf:user-source-registry-directory)
> #P"/gnu/store/75qppl3a062b138fkrn324qq8f912zqh-profile/etc/common-lisp/source-registry.conf.d/"
> --8<---------------cut here---------------end--------------->8---
>
> It does not work because the sbcl package definition sets the
> XDG_CONFIG_DIRS native search path which is picked by
> user-source-registry-directory, which has higher priority than
> default-user-source-registry.
>
> I think the flaw is ASDF's this time:
>
> --8<---------------cut here---------------start------------->8---
>   (defun user-source-registry-directory (&key (direction :input))
>     (xdg-config-pathname *source-registry-directory* direction))
>
> ;...
>
>   (defun xdg-config-pathnames (&rest more)
>     "Return a list of pathnames for application configuration.
> MORE may contain specifications for a subpath relative to these directories: a
> subpathname specification and keyword arguments as per RESOLVE-LOCATION \(see
> also \"Configuration DSL\"\) in the ASDF manual."
>     (filter-pathname-set
>      `(,(xdg-config-home more)
>        ,@(xdg-config-dirs more))))
> --8<---------------cut here---------------end--------------->8---
>
> So the user registry looks into XDG_CONFIG_DIRS, while really it should
> just look at XDG_CONFIG_HOME.
>

The same test works for me. I think it's because I have
a ".config/common-lisp/source-registry.conf.d" directory (even if it's
empty), and '(user-source-registry-directory)' returns it instead of the
first directory in XDG_CONFIG_DIRS.

> So this patch should do (untested):
>
> --8<---------------cut here---------------start------------->8---
> - (defun user-source-registry-directory (&key (direction :input))
> -   (xdg-config-pathname *source-registry-directory* direction))
> + (defun user-source-registry-directory (&key (direction :input))
> +   (find-preferred-file (list (xdg-config-home *source-registry-directory*)) :direction direction))
> --8<---------------cut here---------------end--------------->8---
>
> If you confirm, then we should also report upstream I think.
>
> Cheers!

I don't know if the fact that ASDF can give priority to XDG_CONFIG_DIRS
over "~/common-lisp" is a feature or a bug, so you could ask upstream.
Anyway in our case, your patch makes sense since we always want to give
priority to "~/common-lisp" and other usual user directories. I'll add
it to our patch for cl-asdf.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 10:54:02 GMT) Full text and rfc822 format available.

Message #44 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 11:52:43 +0100
[Message part 1 (text/plain, inline)]
Updated patches in attachment.
Do you see something else to fix or improve?
[0001-gnu-cl-asdf-Improve-priorities-of-configuration-file.patch (text/x-patch, attachment)]
[0002-gnu-ccl-Use-system-ASDF-instead-of-bundled-one.patch (text/x-patch, attachment)]
[0003-gnu-clisp-Use-system-ASDF-instead-of-bundled-one.patch (text/x-patch, attachment)]
[0004-gnu-ecl-Simplify-replace-asdf-phase.patch (text/x-patch, attachment)]
[0005-gnu-sbcl-Simplify-replace-asdf-phase.patch (text/x-patch, attachment)]
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 14:43:01 GMT) Full text and rfc822 format available.

Message #47 received at 45017 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 45017 <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 15:42:28 +0100
[Message part 1 (text/plain, inline)]
Guillaume Le Vaillant <glv <at> posteo.net> writes:

> Updated patches in attachment.
> Do you see something else to fix or improve?

Tested and approved!

I suggest we merge on master since this is not very disruptive and it
fixes a regression introduced with the build system overhaul.

Thoughts?

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

Reply sent to Guillaume Le Vaillant <glv <at> posteo.net>:
You have taken responsibility. (Sat, 05 Dec 2020 15:18:02 GMT) Full text and rfc822 format available.

Notification sent to Pierre Neidhardt <mail <at> ambrevar.xyz>:
bug acknowledged by developer. (Sat, 05 Dec 2020 15:18:02 GMT) Full text and rfc822 format available.

Message #52 received at 45017-done <at> debbugs.gnu.org (full text, mbox):

From: Guillaume Le Vaillant <glv <at> posteo.net>
To: Pierre Neidhardt <mail <at> ambrevar.xyz>
Cc: 45017-done <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 16:17:39 +0100
[Message part 1 (text/plain, inline)]
Pierre Neidhardt <mail <at> ambrevar.xyz> skribis:

> Guillaume Le Vaillant <glv <at> posteo.net> writes:
>
>> Updated patches in attachment.
>> Do you see something else to fix or improve?
>
> Tested and approved!
>
> I suggest we merge on master since this is not very disruptive and it
> fixes a regression introduced with the build system overhaul.
>
> Thoughts?

Thanks for the review. Pushed as
031fbebafef81244a72e584a46b3fcc219256fcb and following.
[signature.asc (application/pgp-signature, inline)]

Information forwarded to bug-guix <at> gnu.org:
bug#45017; Package guix. (Sat, 05 Dec 2020 15:20:01 GMT) Full text and rfc822 format available.

Message #55 received at 45017-done <at> debbugs.gnu.org (full text, mbox):

From: Pierre Neidhardt <mail <at> ambrevar.xyz>
To: Guillaume Le Vaillant <glv <at> posteo.net>
Cc: 45017-done <at> debbugs.gnu.org
Subject: Re: asdf-build-system packages have priority over user ones
Date: Sat, 05 Dec 2020 16:19:23 +0100
[Message part 1 (text/plain, inline)]
Thanks a lot!

-- 
Pierre Neidhardt
https://ambrevar.xyz/
[signature.asc (application/pgp-signature, inline)]

bug archived. Request was from Debbugs Internal Request <help-debbugs <at> gnu.org> to internal_control <at> debbugs.gnu.org. (Sun, 03 Jan 2021 12:24:04 GMT) Full text and rfc822 format available.

This bug report was last modified 3 years and 85 days ago.

Previous Next


GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.