Received: (at 22138) by debbugs.gnu.org; 24 Aug 2019 13:52:59 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sat Aug 24 09:52:59 2019
Received: from localhost ([127.0.0.1]:41724 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1i1WTS-0003Sb-Rx
for submit <at> debbugs.gnu.org; Sat, 24 Aug 2019 09:52:59 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39227)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1i1WTR-0003SP-Ja
for 22138 <at> debbugs.gnu.org; Sat, 24 Aug 2019 09:52:57 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:34097)
by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
id 1i1WTI-0002gY-Po; Sat, 24 Aug 2019 09:52:48 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56162 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1i1WTI-0005yt-BF; Sat, 24 Aug 2019 09:52:48 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Mark H Weaver <mhw@HIDDEN>
Subject: Re: bug#22138: Search paths of dependencies are not honored
References: <87bn9yk5mf.fsf@HIDDEN>
<20190801221206.17965136@HIDDEN>
<87ftmfa8cp.fsf@HIDDEN> <87d0gw54fz.fsf@HIDDEN>
Date: Sat, 24 Aug 2019 15:52:46 +0200
In-Reply-To: <87d0gw54fz.fsf@HIDDEN> ("Ludovic
\=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
\=\?utf-8\?Q\?s\?\= message of "Fri, 23 Aug 2019 16:42:08 +0200")
Message-ID: <87pnkuwtzl.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 22138
Cc: Julien Lepiller <julien@HIDDEN>, 22138 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:
> The attached procedure computes the search paths of a package based on
> its run-time dependencies. =E2=80=98profile-derivation=E2=80=99 could us=
e this to
> compute its =E2=80=98etc/profile=E2=80=99 file and its =E2=80=98manifest=
=E2=80=99 file.
One difficulty with this approach is that search paths are currently a
=E2=80=9Chost-side=E2=80=9D feature: there=E2=80=99s a =E2=80=98search-path=
s=E2=80=99 field in <manifest-entry>,
which is populated before the profile is actually built.
What I suggest above would mean turning search paths into a =E2=80=9Cbuild-=
side=E2=80=9D
feature, which has some implications.
Needs more thought=E2=80=A6
Ludo=E2=80=99.
bug-guix@HIDDEN:bug#22138; Package guix.
Full text available.
Received: (at 22138) by debbugs.gnu.org; 23 Aug 2019 14:42:23 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Fri Aug 23 10:42:23 2019
Received: from localhost ([127.0.0.1]:40975 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1i1Alj-0002yw-8H
for submit <at> debbugs.gnu.org; Fri, 23 Aug 2019 10:42:23 -0400
Received: from eggs.gnu.org ([209.51.188.92]:43912)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <ludo@HIDDEN>) id 1i1Alf-0002yf-16
for 22138 <at> debbugs.gnu.org; Fri, 23 Aug 2019 10:42:20 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43923)
by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
id 1i1AlX-0000sl-5h; Fri, 23 Aug 2019 10:42:11 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=48682 helo=ribbon)
by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
(Exim 4.82) (envelope-from <ludo@HIDDEN>)
id 1i1AlW-0003tq-Ls; Fri, 23 Aug 2019 10:42:10 -0400
From: =?utf-8?Q?Ludovic_Court=C3=A8s?= <ludo@HIDDEN>
To: Mark H Weaver <mhw@HIDDEN>
Subject: Re: bug#22138: Search paths of dependencies are not honored
References: <87bn9yk5mf.fsf@HIDDEN>
<20190801221206.17965136@HIDDEN>
<87ftmfa8cp.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 6 Fructidor an 227 de la =?utf-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Fri, 23 Aug 2019 16:42:08 +0200
In-Reply-To: <87ftmfa8cp.fsf@HIDDEN> (Mark H. Weaver's message of "Mon, 05
Aug 2019 12:23:55 -0400")
Message-ID: <87d0gw54fz.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Spam-Score: -2.3 (--)
X-Debbugs-Envelope-To: 22138
Cc: Julien Lepiller <julien@HIDDEN>, 22138 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)
--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Hi Mark,
Mark H Weaver <mhw@HIDDEN> skribis:
> Julien Lepiller <julien@HIDDEN> writes:
>
>> Hi, I've been looking at our current code and would like to propose the
>> attached patch for that issue.
>>
>> From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 2001
>> From: Julien Lepiller <julien@HIDDEN>
>> Date: Thu, 1 Aug 2019 22:09:38 +0200
>> Subject: [PATCH] guix: Recursively honor search paths of dependencies.
>>
>> * guix/packages.scm (all-transitive-inputs)
>> (package-all-transitive-inputs)
>> (package-all-transitive-native-search-paths): New procedures.
>> * guix/profiles.scm (package->manifest-entry): Use
>> package-all-transitive-native-search-paths to generate manifest search
>> paths.
>
> As I recall this kind of solution has been proposed in the past and
> rejected. It's a reasonable suggestion, but I personally think that it
> goes too far, because it would include a great many packages whose code
> is nowhere to be found in the resulting profile.
I agree.
We need to take into account run-time dependencies (references), not
build-time dependencies, and that=E2=80=99s what makes things more difficul=
t.
The attached procedure computes the search paths of a package based on
its run-time dependencies. =E2=80=98profile-derivation=E2=80=99 could use =
this to
compute its =E2=80=98etc/profile=E2=80=99 file and its =E2=80=98manifest=E2=
=80=99 file. Here=E2=80=99s what it
gives for =E2=80=98w3m=E2=80=99:
--8<---------------cut here---------------start------------->8---
(("BASH_LOADABLES_PATH"
("lib/bash")
":"
directory
#f)
("GUIX_LOCPATH" ("lib/locale") ":" directory #f)
("SSL_CERT_DIR"
("etc/ssl/certs")
#f
directory
#f)
("SSL_CERT_FILE"
("etc/ssl/certs/ca-certificates.crt")
#f
regular
#f)
("TERMINFO_DIRS"
("share/terminfo")
":"
directory
#f)
("XDG_DATA_DIRS" ("share") ":" directory #f)
("GIO_EXTRA_MODULES"
("lib/gio/modules")
":"
directory
#f)
("PERL5LIB"
("lib/perl5/site_perl")
":"
directory
#f))
--8<---------------cut here---------------end--------------->8---
We get GUIX_LOCPATH and SSL_CERT_DIR, which is exactly what we want.
However, the other search paths that we get look less desirable:
intuitively, it seems that we may not need them, and in the case of
XDG_DATA_DIRS, setting it may be detrimental.
For =E2=80=98icecat=E2=80=99 this gives additional things like PYTHONPATH,
GUILE_LOAD_PATH, XML_CATALOG_FILES, and so on.
In practice that=E2=80=99s probably OK though: if you only have =E2=80=98ic=
ecat=E2=80=99 and
other GUIs in your profile, Guix won=E2=80=99t suggest setting GUILE_LOAD_P=
ATH
or PYTHONPATH.
So I think we can come up with a solution based on the patch below, but
we=E2=80=99ll have to test it on Guix System and on foreign distros to see =
if
the extra search paths that it brings are warranted.
Thoughts?
Ludo=E2=80=99.
--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline
diff --git a/guix/packages.scm b/guix/packages.scm
index c94a651f27..7d5c8198ac 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -26,6 +26,7 @@
#:use-module (guix store)
#:use-module (guix monads)
#:use-module (guix gexp)
+ #:use-module (guix modules)
#:use-module (guix base32)
#:use-module (guix grafts)
#:use-module (guix derivations)
@@ -749,6 +750,64 @@ recursively."
'()))
(package-transitive-propagated-inputs package))))
+(define (package-run-time-search-paths package)
+ "Return a file that contains a list of search path specifications
+corresponding to the run-time references of PACKAGE."
+ (define search-paths
+ (map (lambda (package)
+ (cons (package-full-name package "-")
+ (map search-path-specification->sexp
+ (package-native-search-paths package))))
+ (package-closure (list package))))
+
+ (define build
+ (with-imported-modules (source-module-closure
+ '((guix build utils)
+ (guix search-paths)
+ (guix build store-copy)))
+ #~(begin
+ (use-modules (guix build utils)
+ (guix build store-copy)
+ (guix search-paths)
+ (srfi srfi-1)
+ (ice-9 match)
+ (ice-9 pretty-print))
+
+ (define search-paths
+ ;; Map items like "coreutils-8.20" to their search path specs.
+ (map (match-lambda
+ ((item . paths)
+ (cons item (map sexp->search-path-specification paths))))
+ '#$search-paths))
+
+ (define (store-item-search-paths item)
+ "Return the search paths that apply to ITEM, a store item."
+ (or (any (match-lambda
+ ((entry . paths)
+ (and (string-suffix? entry item)
+ paths)))
+ search-paths)
+ '()))
+
+ (let* ((references (map store-info-item
+ (call-with-input-file "graph"
+ read-reference-graph)))
+ (paths (delete-duplicates
+ (append-map store-item-search-paths
+ references))))
+ (call-with-output-file #$output
+ (lambda (port)
+ (pretty-print (map search-path-specification->sexp
+ paths)
+ port)))))))
+
+ (computed-file (string-append (package-full-name package "-")
+ "-search-paths")
+ build
+ #:options
+ `(#:references-graphs (("graph" ,package))
+ #:properties '((type . search-paths)))))
+
(define (transitive-input-references alist inputs)
"Return a list of (assoc-ref ALIST <label>) for each (<label> <package> . _)
in INPUTS and their transitive propagated inputs."
--=-=-=--
bug-guix@HIDDEN:bug#22138; Package guix.
Full text available.Received: (at 22138) by debbugs.gnu.org; 5 Aug 2019 16:31:32 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 05 12:31:32 2019 Received: from localhost ([127.0.0.1]:35763 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1huftT-0008Gu-QW for submit <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:31:32 -0400 Received: from lepiller.eu ([89.234.186.109]:35866) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <julien@HIDDEN>) id 1huftR-0008Gk-Oi for 22138 <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:31:30 -0400 Received: from lepiller.eu (localhost [127.0.0.1]) by lepiller.eu (OpenSMTPD) with ESMTP id 465d2675; Mon, 5 Aug 2019 16:31:27 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:cc:from:message-id; s= dkim; bh=nKKFvTSdnC6gFrIoREMKO/Cs4+A=; b=O0EbkcABURiTS//ivBP3Kph 6/tlka5HiQZaPCHDWwfiiMnmzyH09LnzkQjlsX3do766/tWhs0JDGcIXb8Vx7gAf KjjNIbJjsM4tkLP4mW1GDPyxk5azBaNXq1kABioUY3vyxD2BI1gG9cbnFAxcflVE 6mhNX+NjrIq073Pkckx49DEcxs3m3RDCwSGSkPfMiToqljW8RDn2m6lNzs2KCNdv MmviY0Uq5I4N7CBGjKNhQcdsGEcId51oECrJsqFoddnuSfrUxGBKym/45SNHVbCj CFUSgnbipSXZW+QUxdr81zQ7xf+ngH7lF0uMB6EkepfiQm1VKtkStdDbynfXFhA= = DomainKey-Signature: a=rsa-sha1; c=nofws; d=lepiller.eu; h=date :in-reply-to:references:mime-version:content-type :content-transfer-encoding:subject:to:cc:from:message-id; q=dns; s=dkim; b=ZCXLU837tA7CKPEx8+BVF/zjr0cDPhjfo4O9k5OsCKSMKcu1ePcYs OQBMnigSBfHS/QROItD9vU9T52p2KBx2wHk7qkxlkbuBYIhOQcsIuJrbIsES7/rv XMmgdMASBKdF7/ET9AvhXnUby7bH509TNf2PzqWRQ7dEKIwL/UMw1P7JiACKQ+IX mUAHmk1UD8wpR+R3Nro3vEpg7RD1BJmLlC1fLSkb4a+iHxfiP6+i1PB21cthtwsP foL58pH+TtMp5dGQDnQomJPeWASHq2iediSujqYQvybagFr13OzR1t+6GLYP64aL iHrTss85GKvWHj8uVJ3xpMxm5GQjoFhsg== Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id c2bf4a32 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); Mon, 5 Aug 2019 16:31:26 +0000 (UTC) Date: Mon, 05 Aug 2019 18:31:15 +0200 User-Agent: K-9 Mail for Android In-Reply-To: <87ftmfa8cp.fsf@HIDDEN> References: <87bn9yk5mf.fsf@HIDDEN> <20190801221206.17965136@HIDDEN> <87ftmfa8cp.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable Subject: Re: bug#22138: Search paths of dependencies are not honored To: Mark H Weaver <mhw@HIDDEN> From: Julien Lepiller <julien@HIDDEN> Message-ID: <DA98A3D3-98B0-444C-B53A-3A6D37366908@HIDDEN> X-Spam-Score: 0.0 (/) X-Debbugs-Envelope-To: 22138 Cc: 22138 <at> debbugs.gnu.org X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Le 5 ao=C3=BBt 2019 18:23:55 GMT+02:00, Mark H Weaver <mhw@netris=2Eorg> a = =C3=A9crit : >Hi Julien, > >Julien Lepiller <julien@lepiller=2Eeu> writes: > >> Hi, I've been looking at our current code and would like to propose >the >> attached patch for that issue=2E >> >> From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 >2001 >> From: Julien Lepiller <julien@lepiller=2Eeu> >> Date: Thu, 1 Aug 2019 22:09:38 +0200 >> Subject: [PATCH] guix: Recursively honor search paths of >dependencies=2E >> >> * guix/packages=2Escm (all-transitive-inputs) >> (package-all-transitive-inputs) >> (package-all-transitive-native-search-paths): New procedures=2E >> * guix/profiles=2Escm (package->manifest-entry): Use >> package-all-transitive-native-search-paths to generate manifest >search >> paths=2E > >As I recall this kind of solution has been proposed in the past and >rejected=2E It's a reasonable suggestion, but I personally think that it >goes too far, because it would include a great many packages whose code >is nowhere to be found in the resulting profile=2E For example, it would >include documentation generators used to build man pages, and the >compilers that were used to build those documentation generators, etc, >all the way back to the early bootstrap binaries=2E > >Having said this, I agree that there is a longstanding problem in Guix >with search-paths not including enough packages in its calculation=2E >We've known about this problem for a long time, but as far as I know we >have not yet found a satisfactory solution=2E > >Our current hacky workaround for problems like this has been to set >certain environment variables unconditionally in /etc/profile=2E For >example, you'll see that MANPATH, INFOPATH, XDG_DATA_DIRS, >XDG_CONFIG_DIRS, XCURSOR_PATH, DICPATH, and GST_PLUGIN_PATH are all set >there=2E See 'operating-system-etc-service' in (gnu system) for the >relevant code=2E > >At the very least, I think we should wait for input from Ludovic before >applying anything along these lines=2E > >Anyway, thanks for looking into it and making the proposal=2E > > Best, > Mark Tgis patch doesn't add any dependency to any package, since it doesn't rec= ord any more stone paths than without=2E It indeed defines more variables t= han needed, but isn't that ok?
bug-guix@HIDDEN:bug#22138; Package guix.
Full text available.
Received: (at 22138) by debbugs.gnu.org; 5 Aug 2019 16:24:18 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Mon Aug 05 12:24:18 2019
Received: from localhost ([127.0.0.1]:35754 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1hufmS-00084Y-AC
for submit <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:24:16 -0400
Received: from world.peace.net ([64.112.178.59]:44712)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <mhw@HIDDEN>) id 1hufmQ-00084J-Jr
for 22138 <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:24:15 -0400
Received: from mhw by world.peace.net with esmtpsa
(TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89)
(envelope-from <mhw@HIDDEN>)
id 1hufmK-0007lI-FC; Mon, 05 Aug 2019 12:24:08 -0400
From: Mark H Weaver <mhw@HIDDEN>
To: Julien Lepiller <julien@HIDDEN>
Subject: Re: bug#22138: Search paths of dependencies are not honored
References: <87bn9yk5mf.fsf@HIDDEN>
<20190801221206.17965136@HIDDEN>
Date: Mon, 05 Aug 2019 12:23:55 -0400
In-Reply-To: <20190801221206.17965136@HIDDEN> (Julien Lepiller's
message of "Thu, 1 Aug 2019 22:12:06 +0200")
Message-ID: <87ftmfa8cp.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 22138
Cc: 22138 <at> debbugs.gnu.org
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
Hi Julien,
Julien Lepiller <julien@HIDDEN> writes:
> Hi, I've been looking at our current code and would like to propose the
> attached patch for that issue.
>
> From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 2001
> From: Julien Lepiller <julien@HIDDEN>
> Date: Thu, 1 Aug 2019 22:09:38 +0200
> Subject: [PATCH] guix: Recursively honor search paths of dependencies.
>
> * guix/packages.scm (all-transitive-inputs)
> (package-all-transitive-inputs)
> (package-all-transitive-native-search-paths): New procedures.
> * guix/profiles.scm (package->manifest-entry): Use
> package-all-transitive-native-search-paths to generate manifest search
> paths.
As I recall this kind of solution has been proposed in the past and
rejected. It's a reasonable suggestion, but I personally think that it
goes too far, because it would include a great many packages whose code
is nowhere to be found in the resulting profile. For example, it would
include documentation generators used to build man pages, and the
compilers that were used to build those documentation generators, etc,
all the way back to the early bootstrap binaries.
Having said this, I agree that there is a longstanding problem in Guix
with search-paths not including enough packages in its calculation.
We've known about this problem for a long time, but as far as I know we
have not yet found a satisfactory solution.
Our current hacky workaround for problems like this has been to set
certain environment variables unconditionally in /etc/profile. For
example, you'll see that MANPATH, INFOPATH, XDG_DATA_DIRS,
XDG_CONFIG_DIRS, XCURSOR_PATH, DICPATH, and GST_PLUGIN_PATH are all set
there. See 'operating-system-etc-service' in (gnu system) for the
relevant code.
At the very least, I think we should wait for input from Ludovic before
applying anything along these lines.
Anyway, thanks for looking into it and making the proposal.
Best,
Mark
bug-guix@HIDDEN:bug#22138; Package guix.
Full text available.
Received: (at 22138) by debbugs.gnu.org; 1 Aug 2019 20:12:24 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Thu Aug 01 16:12:24 2019
Received: from localhost ([127.0.0.1]:55406 helo=debbugs.gnu.org)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
id 1htHR2-0007wr-0o
for submit <at> debbugs.gnu.org; Thu, 01 Aug 2019 16:12:24 -0400
Received: from lepiller.eu ([89.234.186.109]:35742)
by debbugs.gnu.org with esmtp (Exim 4.84_2)
(envelope-from <julien@HIDDEN>) id 1htHQz-0007we-JJ
for 22138 <at> debbugs.gnu.org; Thu, 01 Aug 2019 16:12:22 -0400
Received: from lepiller.eu (localhost [127.0.0.1])
by lepiller.eu (OpenSMTPD) with ESMTP id 29eb82a8
for <22138 <at> debbugs.gnu.org>; Thu, 1 Aug 2019 20:12:16 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=lepiller.eu; h=date:from
:to:subject:message-id:mime-version:content-type; s=dkim; bh=pnd
zzYMka2zr8cQ4jlNihuwiRrA=; b=kiL/y7FsCMQrqFEl4Jgx8Vuy+J5r3InjKzO
e3+dZIrMPIgkZG9zRLSDfh0CMU42o6HWue2MiK8vZaF3mJL/71xIohnqwXJNY3+l
36CSPkzA4yXxU2/l3cSYDy7VBu4a41royL+ag8g1fjP+mxNwB8mPyqzXT3CqoJk1
4lKq0VLupf9o7IfXGQ671E8Qx+BraPN0litchZGtVeJ9vpsIy81qiRqEq70w4aB1
Co815W2GQg/Np/IMpsM37XJvfgYE37QwjrP/qffUC16GL9YDLmTYqyqQXP6uyFUi
JRr255h6KqyTpG9P4swk22JU/U2OsWi/2ZHDnV8aDqGaf/eW0Kg==
DomainKey-Signature: a=rsa-sha1; c=nofws; d=lepiller.eu; h=date:from:to
:subject:message-id:mime-version:content-type; q=dns; s=dkim; b=
kUrtrHA+Tlp1M1NxnQauMoh1ofEW+k+etr0bAUAFeUSyfq1hCJC5+h53Dk2PAiik
gJFxME9MkMCqa8De1iONyZ5zlB7lXoHWEUv1pds91O3GlHkddEvnE6JG/yqkRcc4
lLos34QHJv5lCBlgKPKr+QiH/xBGsmrVV2eYv5NozGhCCJ9sMLfz9P9b9BkcaPoX
Z9UNTMCnqOQ8mYa3Vh9bhMP4iuIXbZmLqvECgTS/M/1Rqy4zMfITKYG+LuYqiyf2
zM6steVksE6L3u3XehNyJidiO5CaOtBMvfWGvFjkuBzzv+78OavGmig7q9XFMREn
pdlOXLyceQzkMf/z4SPcnQ==
Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id a64bdbcb
(TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO)
for <22138 <at> debbugs.gnu.org>; Thu, 1 Aug 2019 20:12:16 +0000 (UTC)
Date: Thu, 1 Aug 2019 22:12:06 +0200
From: Julien Lepiller <julien@HIDDEN>
To: 22138 <at> debbugs.gnu.org
Subject: Search paths of dependencies are not honored
Message-ID: <20190801221206.17965136@HIDDEN>
X-Mailer: Claws Mail 3.17.3 (GTK+ 2.24.32; x86_64-unknown-linux-gnu)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="MP_/P7b3w4QD0RSr.lNrEvJblE1"
X-Spam-Score: 0.0 (/)
X-Debbugs-Envelope-To: 22138
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>,
<mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)
--MP_/P7b3w4QD0RSr.lNrEvJblE1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Disposition: inline
Hi, I've been looking at our current code and would like to propose the
attached patch for that issue.
--MP_/P7b3w4QD0RSr.lNrEvJblE1
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
filename=0001-guix-Recursively-honor-search-paths-of-dependencies.patch
From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@HIDDEN>
Date: Thu, 1 Aug 2019 22:09:38 +0200
Subject: [PATCH] guix: Recursively honor search paths of dependencies.
* guix/packages.scm (all-transitive-inputs)
(package-all-transitive-inputs)
(package-all-transitive-native-search-paths): New procedures.
* guix/profiles.scm (package->manifest-entry): Use
package-all-transitive-native-search-paths to generate manifest search
paths.
---
guix/packages.scm | 53 +++++++++++++++++++++++++++++++++++++++++++++++
guix/profiles.scm | 2 +-
2 files changed, 54 insertions(+), 1 deletion(-)
diff --git a/guix/packages.scm b/guix/packages.scm
index c94a651f27..f9095759f1 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -101,6 +101,7 @@
package-transitive-propagated-inputs
package-transitive-native-search-paths
package-transitive-supported-systems
+ package-all-transitive-native-search-paths
package-mapping
package-input-rewriting
package-input-rewriting/spec
@@ -686,6 +687,42 @@ preserved, and only duplicate propagated inputs are removed."
((input rest ...)
(loop rest (cons input result) propagated first? seen)))))
+(define (all-transitive-inputs inputs)
+ "Return the closure of INPUTS when considering the 'propagated-inputs',
+'inputs' and 'native-inputs' edges. Omit duplicate inputs, except for
+those already present in INPUTS itself.
+
+This is implemented as a breadth-first traversal such that INPUTS is
+preserved, and only duplicate propagated inputs are removed."
+ (define (seen? seen item outputs)
+ ;; FIXME: We're using pointer identity here, which is extremely sensitive
+ ;; to memoization in package-producing procedures; see
+ ;; <https://bugs.gnu.org/30155>.
+ (match (vhash-assq item seen)
+ ((_ . o) (equal? o outputs))
+ (_ #f)))
+
+ (let loop ((inputs inputs)
+ (result '())
+ (transitive '())
+ (first? #t)
+ (seen vlist-null))
+ (match inputs
+ (()
+ (if (null? transitive)
+ (reverse result)
+ (loop (reverse (concatenate transitive)) result '() #f seen)))
+ (((and input (label (? package? package) outputs ...)) rest ...)
+ (if (and (not first?) (seen? seen package outputs))
+ (loop rest result transitive first? seen)
+ (loop rest
+ (cons input result)
+ (cons (package-direct-inputs package) transitive)
+ first?
+ (vhash-consq package outputs seen))))
+ ((input rest ...)
+ (loop rest (cons input result) transitive first? seen)))))
+
(define (package-direct-sources package)
"Return all source origins associated with PACKAGE; including origins in
PACKAGE's inputs."
@@ -720,6 +757,11 @@ with their propagated inputs."
with their propagated inputs, recursively."
(transitive-inputs (package-direct-inputs package)))
+(define (package-all-transitive-inputs package)
+ "Return the transitive inputs of PACKAGE---i.e., its direct inputs along
+with their propagated inputs, recursively."
+ (all-transitive-inputs (package-direct-inputs package)))
+
(define (package-transitive-target-inputs package)
"Return the transitive target inputs of PACKAGE---i.e., its direct inputs
along with their propagated inputs, recursively. This only includes inputs
@@ -749,6 +791,17 @@ recursively."
'()))
(package-transitive-propagated-inputs package))))
+(define (package-all-transitive-native-search-paths package)
+ "Return the list of search paths for PACKAGE and its propagated inputs,
+recursively."
+ (append (package-native-search-paths package)
+ (append-map (match-lambda
+ ((label (? package? p) _ ...)
+ (package-native-search-paths p))
+ (_
+ '()))
+ (package-all-transitive-inputs package))))
+
(define (transitive-input-references alist inputs)
"Return a list of (assoc-ref ALIST <label>) for each (<label> <package> . _)
in INPUTS and their transitive propagated inputs."
diff --git a/guix/profiles.scm b/guix/profiles.scm
index f5c863945c..dd6a31562f 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -318,7 +318,7 @@ file name."
(item package)
(dependencies (delete-duplicates deps))
(search-paths
- (package-transitive-native-search-paths package))
+ (package-all-transitive-native-search-paths package))
(parent parent)
(properties properties))))
entry))
--
2.22.0
--MP_/P7b3w4QD0RSr.lNrEvJblE1--
bug-guix@HIDDEN:bug#22138; Package guix.
Full text available.ludo@HIDDEN (Ludovic Courtès)
to control <at> debbugs.gnu.org.
Full text available.Received: (at submit) by debbugs.gnu.org; 10 Dec 2015 09:36:23 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Dec 10 04:36:23 2015 Received: from localhost ([127.0.0.1]:38609 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1a6xeA-0004Wb-PK for submit <at> debbugs.gnu.org; Thu, 10 Dec 2015 04:36:22 -0500 Received: from eggs.gnu.org ([208.118.235.92]:49949) by debbugs.gnu.org with esmtp (Exim 4.84) (envelope-from <ludo@HIDDEN>) id 1a6xe9-0004WO-12 for submit <at> debbugs.gnu.org; Thu, 10 Dec 2015 04:36:21 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1a6xe3-0002e4-6h for submit <at> debbugs.gnu.org; Thu, 10 Dec 2015 04:36:15 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,T_RP_MATCHES_RCVD autolearn=disabled version=3.3.2 Received: from lists.gnu.org ([2001:4830:134:3::11]:60937) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1a6xe3-0002e0-42 for submit <at> debbugs.gnu.org; Thu, 10 Dec 2015 04:36:15 -0500 Received: from eggs.gnu.org ([2001:4830:134:3::10]:59955) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1a6xe2-0003Jd-Bd for bug-guix@HIDDEN; Thu, 10 Dec 2015 04:36:15 -0500 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1a6xdy-0002cb-ED for bug-guix@HIDDEN; Thu, 10 Dec 2015 04:36:14 -0500 Received: from fencepost.gnu.org ([2001:4830:134:3::e]:47226) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1a6xdy-0002cW-B7 for bug-guix@HIDDEN; Thu, 10 Dec 2015 04:36:10 -0500 Received: from pluto.bordeaux.inria.fr ([193.50.110.57]:36476 helo=pluto) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_128_CBC_SHA1:128) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1a6xdx-0002pi-Pv for bug-guix@HIDDEN; Thu, 10 Dec 2015 04:36:10 -0500 From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=) To: bug-guix@HIDDEN Subject: Search paths of dependencies are not honored X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 20 Frimaire an 224 de la =?utf-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x3D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-unknown-linux-gnu Date: Thu, 10 Dec 2015 10:36:08 +0100 Message-ID: <87bn9yk5mf.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Received-From: 2001:4830:134:3::11 X-Spam-Score: -5.6 (-----) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -5.6 (-----) As of 0.9.0+, only the search paths of packages explicitly in the profile are shown by =E2=80=98--search-paths=E2=80=99 and similar. This is= a problem for libraries that have associated environment variables. For instance, if one installs an application linked against OpenSSL, they will not know about =E2=80=98SSL_CERT_DIR=E2=80=99 and =E2=80=98SSL_CE= RT_FILE=E2=80=99. Similarly for GStreamer and =E2=80=98GST_PLUGIN_PATH=E2=80=99, libc and =E2=80=98GUIX= _LOCPATH=E2=80=99, and so on. Fixing it seems tricky. We could pass the profile builder the closure=E2= =80=99s graph (via #:references-graph) and the the set of search paths of all the direct and indirect dependencies of the profile. The builder would then need to somehow determine the subset of these search paths that is actually useful, and use it to build the search path spec in the manifest as well as etc/profile. Ludo=E2=80=99.
ludo@HIDDEN (Ludovic Courtès):bug-guix@HIDDEN.
Full text available.bug-guix@HIDDEN:bug#22138; Package guix.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.