GNU logs - #22138, boring messages

Message sent to bug-guix@HIDDEN:

Subject: bug#22138: Search paths of dependencies are not honored
Resent-From: ludo@HIDDEN (Ludovic =?UTF-8?Q?Court=C3=A8s?=)
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Thu, 10 Dec 2015 09:37:02 +0000
Resent-Message-ID: <handler.22138.B.144974018317402 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 22138 <at> debbugs.gnu.org
From: ludo@HIDDEN (Ludovic =?UTF-8?Q?Court=C3=A8s?=)
Date: Thu, 10 Dec 2015 10:36:08 +0100
Message-ID: <87bn9yk5mf.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/24.5 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

As of 0.9.0+, only the search paths of packages explicitly in the
profile are shown by =E2=80=98--search-paths=E2=80=99 and similar.  This is=
 a problem
for libraries that have associated environment variables.

For instance, if one installs an application linked against OpenSSL,
they will not know about =E2=80=98SSL_CERT_DIR=E2=80=99 and =E2=80=98SSL_CE=
RT_FILE=E2=80=99.  Similarly
for GStreamer and =E2=80=98GST_PLUGIN_PATH=E2=80=99, libc and =E2=80=98GUIX=
_LOCPATH=E2=80=99, and so on.

Fixing it seems tricky.  We could pass the profile builder the closure=E2=
=80=99s
graph (via #:references-graph) and the the set of search paths of all
the direct and indirect dependencies of the profile.  The builder would
then need to somehow determine the subset of these search paths that is
actually useful, and use it to build the search path spec in the
manifest as well as etc/profile.

Ludo=E2=80=99.

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: ludo@HIDDEN (Ludovic =?UTF-8?Q?Court=C3=A8s?=)
Subject: bug#22138: Acknowledgement (Search paths of dependencies are not
 honored)
Message-ID: <handler.22138.B.144974018317402.ack <at> debbugs.gnu.org>
References: <87bn9yk5mf.fsf@HIDDEN>
X-Gnu-PR-Message: ack 22138
X-Gnu-PR-Package: guix
Reply-To: 22138 <at> debbugs.gnu.org
Date: Thu, 10 Dec 2015 09:37:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-guix@HIDDEN

If you wish to submit further information on this problem, please
send it to 22138 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
22138: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D22138
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message received at control <at> debbugs.gnu.org:

Received: (at control) by debbugs.gnu.org; 26 Mar 2017 13:18:39 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Sun Mar 26 09:18:39 2017
Received: from localhost ([127.0.0.1]:45120 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1cs847-0006b0-C2
	for submit <at> debbugs.gnu.org; Sun, 26 Mar 2017 09:18:39 -0400
Received: from eggs.gnu.org ([208.118.235.92]:48628)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1cs845-0006am-4S
 for control <at> debbugs.gnu.org; Sun, 26 Mar 2017 09:18:37 -0400
Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71)
 (envelope-from <ludo@HIDDEN>) id 1cs83u-0000us-Rc
 for control <at> debbugs.gnu.org; Sun, 26 Mar 2017 09:18:31 -0400
X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org
X-Spam-Level: 
X-Spam-Status: No, score=-1.9 required=5.0 tests=BAYES_00,RP_MATCHES_RCVD
 autolearn=disabled version=3.3.2
Received: from fencepost.gnu.org ([2001:4830:134:3::e]:38316)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1cs83u-0000uo-OT
 for control <at> debbugs.gnu.org; Sun, 26 Mar 2017 09:18:26 -0400
Received: from reverse-83.fdn.fr ([80.67.176.83]:35044 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1cs83u-0001cB-55
 for control <at> debbugs.gnu.org; Sun, 26 Mar 2017 09:18:26 -0400
Date: Sun, 26 Mar 2017 15:18:24 +0200
Message-Id: <87shm0rxb3.fsf@HIDDEN>
To: control <at> debbugs.gnu.org
From: ludo@HIDDEN (Ludovic =?utf-8?Q?Court=C3=A8s?=)
Subject: control message for bug #22138
MIME-version: 1.0
Content-type: text/plain; charset=utf-8
Content-Transfer-Encoding: 8bit
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Received-From: 2001:4830:134:3::e
X-Spam-Score: -5.0 (-----)
X-Debbugs-Envelope-To: control
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -5.0 (-----)

severity 22138 important

Message sent to bug-guix@HIDDEN:

Subject: bug#22138: Search paths of dependencies are not honored
References: <87bn9yk5mf.fsf@HIDDEN>
In-Reply-To: <87bn9yk5mf.fsf@HIDDEN>
Resent-From: Julien Lepiller <julien@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Thu, 01 Aug 2019 20:13:01 +0000
Resent-Message-ID: <handler.22138.B22138.156469034430562 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 22138 <at> debbugs.gnu.org
DomainKey-Signature: a=rsa-sha1; c=nofws; d=lepiller.eu; h=date:from:to
 :subject:message-id:mime-version:content-type; q=dns; s=dkim; b=
 kUrtrHA+Tlp1M1NxnQauMoh1ofEW+k+etr0bAUAFeUSyfq1hCJC5+h53Dk2PAiik
 gJFxME9MkMCqa8De1iONyZ5zlB7lXoHWEUv1pds91O3GlHkddEvnE6JG/yqkRcc4
 lLos34QHJv5lCBlgKPKr+QiH/xBGsmrVV2eYv5NozGhCCJ9sMLfz9P9b9BkcaPoX
 Z9UNTMCnqOQ8mYa3Vh9bhMP4iuIXbZmLqvECgTS/M/1Rqy4zMfITKYG+LuYqiyf2
 zM6steVksE6L3u3XehNyJidiO5CaOtBMvfWGvFjkuBzzv+78OavGmig7q9XFMREn
 pdlOXLyceQzkMf/z4SPcnQ==
Date: Thu, 1 Aug 2019 22:12:06 +0200
From: Julien Lepiller <julien@HIDDEN>
Message-ID: <20190801221206.17965136@HIDDEN>
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="MP_/P7b3w4QD0RSr.lNrEvJblE1"
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

--MP_/P7b3w4QD0RSr.lNrEvJblE1
Content-Type: text/plain; charset=US-ASCII
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

Hi, I've been looking at our current code and would like to propose the
attached patch for that issue.

--MP_/P7b3w4QD0RSr.lNrEvJblE1
Content-Type: text/x-patch
Content-Transfer-Encoding: 7bit
Content-Disposition: attachment;
 filename=0001-guix-Recursively-honor-search-paths-of-dependencies.patch

From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 2001
From: Julien Lepiller <julien@HIDDEN>
Date: Thu, 1 Aug 2019 22:09:38 +0200
Subject: [PATCH] guix: Recursively honor search paths of dependencies.

* guix/packages.scm (all-transitive-inputs)
(package-all-transitive-inputs)
(package-all-transitive-native-search-paths): New procedures.
* guix/profiles.scm (package->manifest-entry): Use
package-all-transitive-native-search-paths to generate manifest search
paths.
---
 guix/packages.scm | 53 +++++++++++++++++++++++++++++++++++++++++++++++
 guix/profiles.scm |  2 +-
 2 files changed, 54 insertions(+), 1 deletion(-)

diff --git a/guix/packages.scm b/guix/packages.scm
index c94a651f27..f9095759f1 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -101,6 +101,7 @@
             package-transitive-propagated-inputs
             package-transitive-native-search-paths
             package-transitive-supported-systems
+	    package-all-transitive-native-search-paths
             package-mapping
             package-input-rewriting
             package-input-rewriting/spec
@@ -686,6 +687,42 @@ preserved, and only duplicate propagated inputs are removed."
       ((input rest ...)
        (loop rest (cons input result) propagated first? seen)))))
 
+(define (all-transitive-inputs inputs)
+  "Return the closure of INPUTS when considering the 'propagated-inputs',
+'inputs' and 'native-inputs' edges.  Omit duplicate inputs, except for
+those already present in INPUTS itself.
+
+This is implemented as a breadth-first traversal such that INPUTS is
+preserved, and only duplicate propagated inputs are removed."
+  (define (seen? seen item outputs)
+    ;; FIXME: We're using pointer identity here, which is extremely sensitive
+    ;; to memoization in package-producing procedures; see
+    ;; <https://bugs.gnu.org/30155>.
+    (match (vhash-assq item seen)
+      ((_ . o) (equal? o outputs))
+      (_       #f)))
+
+  (let loop ((inputs     inputs)
+             (result     '())
+             (transitive '())
+             (first?     #t)
+             (seen       vlist-null))
+    (match inputs
+      (()
+       (if (null? transitive)
+           (reverse result)
+           (loop (reverse (concatenate transitive)) result '() #f seen)))
+      (((and input (label (? package? package) outputs ...)) rest ...)
+       (if (and (not first?) (seen? seen package outputs))
+           (loop rest result transitive first? seen)
+           (loop rest
+                 (cons input result)
+                 (cons (package-direct-inputs package) transitive)
+                 first?
+                 (vhash-consq package outputs seen))))
+      ((input rest ...)
+       (loop rest (cons input result) transitive first? seen)))))
+
 (define (package-direct-sources package)
   "Return all source origins associated with PACKAGE; including origins in
 PACKAGE's inputs."
@@ -720,6 +757,11 @@ with their propagated inputs."
 with their propagated inputs, recursively."
   (transitive-inputs (package-direct-inputs package)))
 
+(define (package-all-transitive-inputs package)
+  "Return the transitive inputs of PACKAGE---i.e., its direct inputs along
+with their propagated inputs, recursively."
+  (all-transitive-inputs (package-direct-inputs package)))
+
 (define (package-transitive-target-inputs package)
   "Return the transitive target inputs of PACKAGE---i.e., its direct inputs
 along with their propagated inputs, recursively.  This only includes inputs
@@ -749,6 +791,17 @@ recursively."
                          '()))
                       (package-transitive-propagated-inputs package))))
 
+(define (package-all-transitive-native-search-paths package)
+  "Return the list of search paths for PACKAGE and its propagated inputs,
+recursively."
+  (append (package-native-search-paths package)
+          (append-map (match-lambda
+                        ((label (? package? p) _ ...)
+                         (package-native-search-paths p))
+                        (_
+                         '()))
+                      (package-all-transitive-inputs package))))
+
 (define (transitive-input-references alist inputs)
   "Return a list of (assoc-ref ALIST <label>) for each (<label> <package> . _)
 in INPUTS and their transitive propagated inputs."
diff --git a/guix/profiles.scm b/guix/profiles.scm
index f5c863945c..dd6a31562f 100644
--- a/guix/profiles.scm
+++ b/guix/profiles.scm
@@ -318,7 +318,7 @@ file name."
                      (item package)
                      (dependencies (delete-duplicates deps))
                      (search-paths
-                      (package-transitive-native-search-paths package))
+                      (package-all-transitive-native-search-paths package))
                      (parent parent)
                      (properties properties))))
     entry))
-- 
2.22.0


--MP_/P7b3w4QD0RSr.lNrEvJblE1--

Message sent to bug-guix@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#22138: Search paths of dependencies are not honored
Resent-From: Mark H Weaver <mhw@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Mon, 05 Aug 2019 16:25:02 +0000
Resent-Message-ID: <handler.22138.B22138.156502225831040 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 22138
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: Julien Lepiller <julien@HIDDEN>
Cc: 22138 <at> debbugs.gnu.org
Received: via spool by 22138-submit <at> debbugs.gnu.org id=B22138.156502225831040
          (code B ref 22138); Mon, 05 Aug 2019 16:25:02 +0000
Received: (at 22138) by debbugs.gnu.org; 5 Aug 2019 16:24:18 +0000
Received: from localhost ([127.0.0.1]:35754 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1hufmS-00084Y-AC
	for submit <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:24:16 -0400
Received: from world.peace.net ([64.112.178.59]:44712)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <mhw@HIDDEN>) id 1hufmQ-00084J-Jr
 for 22138 <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:24:15 -0400
Received: from mhw by world.peace.net with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.89)
 (envelope-from <mhw@HIDDEN>)
 id 1hufmK-0007lI-FC; Mon, 05 Aug 2019 12:24:08 -0400
From: Mark H Weaver <mhw@HIDDEN>
References: <87bn9yk5mf.fsf@HIDDEN>
 <20190801221206.17965136@HIDDEN>
Date: Mon, 05 Aug 2019 12:23:55 -0400
In-Reply-To: <20190801221206.17965136@HIDDEN> (Julien Lepiller's
 message of "Thu, 1 Aug 2019 22:12:06 +0200")
Message-ID: <87ftmfa8cp.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Hi Julien,

Julien Lepiller <julien@HIDDEN> writes:

> Hi, I've been looking at our current code and would like to propose the
> attached patch for that issue.
>
> From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 2001
> From: Julien Lepiller <julien@HIDDEN>
> Date: Thu, 1 Aug 2019 22:09:38 +0200
> Subject: [PATCH] guix: Recursively honor search paths of dependencies.
>
> * guix/packages.scm (all-transitive-inputs)
> (package-all-transitive-inputs)
> (package-all-transitive-native-search-paths): New procedures.
> * guix/profiles.scm (package->manifest-entry): Use
> package-all-transitive-native-search-paths to generate manifest search
> paths.

As I recall this kind of solution has been proposed in the past and
rejected.  It's a reasonable suggestion, but I personally think that it
goes too far, because it would include a great many packages whose code
is nowhere to be found in the resulting profile.  For example, it would
include documentation generators used to build man pages, and the
compilers that were used to build those documentation generators, etc,
all the way back to the early bootstrap binaries.

Having said this, I agree that there is a longstanding problem in Guix
with search-paths not including enough packages in its calculation.
We've known about this problem for a long time, but as far as I know we
have not yet found a satisfactory solution.

Our current hacky workaround for problems like this has been to set
certain environment variables unconditionally in /etc/profile.  For
example, you'll see that MANPATH, INFOPATH, XDG_DATA_DIRS,
XDG_CONFIG_DIRS, XCURSOR_PATH, DICPATH, and GST_PLUGIN_PATH are all set
there.  See 'operating-system-etc-service' in (gnu system) for the
relevant code.

At the very least, I think we should wait for input from Ludovic before
applying anything along these lines.

Anyway, thanks for looking into it and making the proposal.

      Best,
       Mark

Message sent to bug-guix@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#22138: Search paths of dependencies are not honored
Resent-From: Julien Lepiller <julien@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Mon, 05 Aug 2019 16:32:02 +0000
Resent-Message-ID: <handler.22138.B22138.156502269231804 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 22138
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: Mark H Weaver <mhw@HIDDEN>
Cc: 22138 <at> debbugs.gnu.org
Received: via spool by 22138-submit <at> debbugs.gnu.org id=B22138.156502269231804
          (code B ref 22138); Mon, 05 Aug 2019 16:32:02 +0000
Received: (at 22138) by debbugs.gnu.org; 5 Aug 2019 16:31:32 +0000
Received: from localhost ([127.0.0.1]:35763 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1huftT-0008Gu-QW
	for submit <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:31:32 -0400
Received: from lepiller.eu ([89.234.186.109]:35866)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <julien@HIDDEN>) id 1huftR-0008Gk-Oi
 for 22138 <at> debbugs.gnu.org; Mon, 05 Aug 2019 12:31:30 -0400
Received: from lepiller.eu (localhost [127.0.0.1])
 by lepiller.eu (OpenSMTPD) with ESMTP id 465d2675;
 Mon, 5 Aug 2019 16:31:27 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha1; c=relaxed; d=lepiller.eu; h=date
 :in-reply-to:references:mime-version:content-type
 :content-transfer-encoding:subject:to:cc:from:message-id; s=
 dkim; bh=nKKFvTSdnC6gFrIoREMKO/Cs4+A=; b=O0EbkcABURiTS//ivBP3Kph
 6/tlka5HiQZaPCHDWwfiiMnmzyH09LnzkQjlsX3do766/tWhs0JDGcIXb8Vx7gAf
 KjjNIbJjsM4tkLP4mW1GDPyxk5azBaNXq1kABioUY3vyxD2BI1gG9cbnFAxcflVE
 6mhNX+NjrIq073Pkckx49DEcxs3m3RDCwSGSkPfMiToqljW8RDn2m6lNzs2KCNdv
 MmviY0Uq5I4N7CBGjKNhQcdsGEcId51oECrJsqFoddnuSfrUxGBKym/45SNHVbCj
 CFUSgnbipSXZW+QUxdr81zQ7xf+ngH7lF0uMB6EkepfiQm1VKtkStdDbynfXFhA=
 =
DomainKey-Signature: a=rsa-sha1; c=nofws; d=lepiller.eu; h=date
 :in-reply-to:references:mime-version:content-type
 :content-transfer-encoding:subject:to:cc:from:message-id; q=dns;
 s=dkim; b=ZCXLU837tA7CKPEx8+BVF/zjr0cDPhjfo4O9k5OsCKSMKcu1ePcYs
 OQBMnigSBfHS/QROItD9vU9T52p2KBx2wHk7qkxlkbuBYIhOQcsIuJrbIsES7/rv
 XMmgdMASBKdF7/ET9AvhXnUby7bH509TNf2PzqWRQ7dEKIwL/UMw1P7JiACKQ+IX
 mUAHmk1UD8wpR+R3Nro3vEpg7RD1BJmLlC1fLSkb4a+iHxfiP6+i1PB21cthtwsP
 foL58pH+TtMp5dGQDnQomJPeWASHq2iediSujqYQvybagFr13OzR1t+6GLYP64aL
 iHrTss85GKvWHj8uVJ3xpMxm5GQjoFhsg==
Received: by lepiller.eu (OpenSMTPD) with ESMTPSA id c2bf4a32
 (TLSv1.2:ECDHE-RSA-AES256-GCM-SHA384:256:NO); 
 Mon, 5 Aug 2019 16:31:26 +0000 (UTC)
Date: Mon, 05 Aug 2019 18:31:15 +0200
User-Agent: K-9 Mail for Android
In-Reply-To: <87ftmfa8cp.fsf@HIDDEN>
References: <87bn9yk5mf.fsf@HIDDEN>
 <20190801221206.17965136@HIDDEN> <87ftmfa8cp.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain;
 charset=utf-8
Content-Transfer-Encoding: quoted-printable
From: Julien Lepiller <julien@HIDDEN>
Message-ID: <DA98A3D3-98B0-444C-B53A-3A6D37366908@HIDDEN>
X-Spam-Score: 0.0 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.0 (-)

Le 5 ao=C3=BBt 2019 18:23:55 GMT+02:00, Mark H Weaver <mhw@netris=2Eorg> a =
=C3=A9crit :
>Hi Julien,
>
>Julien Lepiller <julien@lepiller=2Eeu> writes:
>
>> Hi, I've been looking at our current code and would like to propose
>the
>> attached patch for that issue=2E
>>
>> From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00
>2001
>> From: Julien Lepiller <julien@lepiller=2Eeu>
>> Date: Thu, 1 Aug 2019 22:09:38 +0200
>> Subject: [PATCH] guix: Recursively honor search paths of
>dependencies=2E
>>
>> * guix/packages=2Escm (all-transitive-inputs)
>> (package-all-transitive-inputs)
>> (package-all-transitive-native-search-paths): New procedures=2E
>> * guix/profiles=2Escm (package->manifest-entry): Use
>> package-all-transitive-native-search-paths to generate manifest
>search
>> paths=2E
>
>As I recall this kind of solution has been proposed in the past and
>rejected=2E  It's a reasonable suggestion, but I personally think that it
>goes too far, because it would include a great many packages whose code
>is nowhere to be found in the resulting profile=2E  For example, it would
>include documentation generators used to build man pages, and the
>compilers that were used to build those documentation generators, etc,
>all the way back to the early bootstrap binaries=2E
>
>Having said this, I agree that there is a longstanding problem in Guix
>with search-paths not including enough packages in its calculation=2E
>We've known about this problem for a long time, but as far as I know we
>have not yet found a satisfactory solution=2E
>
>Our current hacky workaround for problems like this has been to set
>certain environment variables unconditionally in /etc/profile=2E  For
>example, you'll see that MANPATH, INFOPATH, XDG_DATA_DIRS,
>XDG_CONFIG_DIRS, XCURSOR_PATH, DICPATH, and GST_PLUGIN_PATH are all set
>there=2E  See 'operating-system-etc-service' in (gnu system) for the
>relevant code=2E
>
>At the very least, I think we should wait for input from Ludovic before
>applying anything along these lines=2E
>
>Anyway, thanks for looking into it and making the proposal=2E
>
>      Best,
>       Mark

Tgis patch doesn't add any dependency to any package, since it doesn't rec=
ord any more stone paths than without=2E It indeed defines more variables t=
han needed, but isn't that ok?

Message sent to bug-guix@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#22138: Search paths of dependencies are not honored
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Fri, 23 Aug 2019 14:43:02 +0000
Resent-Message-ID: <handler.22138.B22138.156657134311471 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 22138
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: Mark H Weaver <mhw@HIDDEN>
Cc: Julien Lepiller <julien@HIDDEN>, 22138 <at> debbugs.gnu.org
Received: via spool by 22138-submit <at> debbugs.gnu.org id=B22138.156657134311471
          (code B ref 22138); Fri, 23 Aug 2019 14:43:02 +0000
Received: (at 22138) by debbugs.gnu.org; 23 Aug 2019 14:42:23 +0000
Received: from localhost ([127.0.0.1]:40975 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1i1Alj-0002yw-8H
	for submit <at> debbugs.gnu.org; Fri, 23 Aug 2019 10:42:23 -0400
Received: from eggs.gnu.org ([209.51.188.92]:43912)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1i1Alf-0002yf-16
 for 22138 <at> debbugs.gnu.org; Fri, 23 Aug 2019 10:42:20 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:43923)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1i1AlX-0000sl-5h; Fri, 23 Aug 2019 10:42:11 -0400
Received: from [2001:660:6102:320:e120:2c8f:8909:cdfe] (port=48682 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1i1AlW-0003tq-Ls; Fri, 23 Aug 2019 10:42:10 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <87bn9yk5mf.fsf@HIDDEN>
 <20190801221206.17965136@HIDDEN>
 <87ftmfa8cp.fsf@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 6 Fructidor an 227 de la =?UTF-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Fri, 23 Aug 2019 16:42:08 +0200
In-Reply-To: <87ftmfa8cp.fsf@HIDDEN> (Mark H. Weaver's message of "Mon, 05
 Aug 2019 12:23:55 -0400")
Message-ID: <87d0gw54fz.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: multipart/mixed; boundary="=-=-="
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

--=-=-=
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable

Hi Mark,

Mark H Weaver <mhw@HIDDEN> skribis:

> Julien Lepiller <julien@HIDDEN> writes:
>
>> Hi, I've been looking at our current code and would like to propose the
>> attached patch for that issue.
>>
>> From cfd2c229087166ab4cc0a9e2bdb72c8b393bcdd5 Mon Sep 17 00:00:00 2001
>> From: Julien Lepiller <julien@HIDDEN>
>> Date: Thu, 1 Aug 2019 22:09:38 +0200
>> Subject: [PATCH] guix: Recursively honor search paths of dependencies.
>>
>> * guix/packages.scm (all-transitive-inputs)
>> (package-all-transitive-inputs)
>> (package-all-transitive-native-search-paths): New procedures.
>> * guix/profiles.scm (package->manifest-entry): Use
>> package-all-transitive-native-search-paths to generate manifest search
>> paths.
>
> As I recall this kind of solution has been proposed in the past and
> rejected.  It's a reasonable suggestion, but I personally think that it
> goes too far, because it would include a great many packages whose code
> is nowhere to be found in the resulting profile.

I agree.

We need to take into account run-time dependencies (references), not
build-time dependencies, and that=E2=80=99s what makes things more difficul=
t.

The attached procedure computes the search paths of a package based on
its run-time dependencies.  =E2=80=98profile-derivation=E2=80=99 could use =
this to
compute its =E2=80=98etc/profile=E2=80=99 file and its =E2=80=98manifest=E2=
=80=99 file.  Here=E2=80=99s what it
gives for =E2=80=98w3m=E2=80=99:

--8<---------------cut here---------------start------------->8---
(("BASH_LOADABLES_PATH"
  ("lib/bash")
  ":"
  directory
  #f)
 ("GUIX_LOCPATH" ("lib/locale") ":" directory #f)
 ("SSL_CERT_DIR"
  ("etc/ssl/certs")
  #f
  directory
  #f)
 ("SSL_CERT_FILE"
  ("etc/ssl/certs/ca-certificates.crt")
  #f
  regular
  #f)
 ("TERMINFO_DIRS"
  ("share/terminfo")
  ":"
  directory
  #f)
 ("XDG_DATA_DIRS" ("share") ":" directory #f)
 ("GIO_EXTRA_MODULES"
  ("lib/gio/modules")
  ":"
  directory
  #f)
 ("PERL5LIB"
  ("lib/perl5/site_perl")
  ":"
  directory
  #f))
--8<---------------cut here---------------end--------------->8---

We get GUIX_LOCPATH and SSL_CERT_DIR, which is exactly what we want.
However, the other search paths that we get look less desirable:
intuitively, it seems that we may not need them, and in the case of
XDG_DATA_DIRS, setting it may be detrimental.

For =E2=80=98icecat=E2=80=99 this gives additional things like PYTHONPATH,
GUILE_LOAD_PATH, XML_CATALOG_FILES, and so on.

In practice that=E2=80=99s probably OK though: if you only have =E2=80=98ic=
ecat=E2=80=99 and
other GUIs in your profile, Guix won=E2=80=99t suggest setting GUILE_LOAD_P=
ATH
or PYTHONPATH.

So I think we can come up with a solution based on the patch below, but
we=E2=80=99ll have to test it on Guix System and on foreign distros to see =
if
the extra search paths that it brings are warranted.

Thoughts?

Ludo=E2=80=99.


--=-=-=
Content-Type: text/x-patch
Content-Disposition: inline

diff --git a/guix/packages.scm b/guix/packages.scm
index c94a651f27..7d5c8198ac 100644
--- a/guix/packages.scm
+++ b/guix/packages.scm
@@ -26,6 +26,7 @@
   #:use-module (guix store)
   #:use-module (guix monads)
   #:use-module (guix gexp)
+  #:use-module (guix modules)
   #:use-module (guix base32)
   #:use-module (guix grafts)
   #:use-module (guix derivations)
@@ -749,6 +750,64 @@ recursively."
                          '()))
                       (package-transitive-propagated-inputs package))))
 
+(define (package-run-time-search-paths package)
+  "Return a file that contains a list of search path specifications
+corresponding to the run-time references of PACKAGE."
+  (define search-paths
+    (map (lambda (package)
+           (cons (package-full-name package "-")
+                 (map search-path-specification->sexp
+                      (package-native-search-paths package))))
+         (package-closure (list package))))
+
+  (define build
+    (with-imported-modules (source-module-closure
+                            '((guix build utils)
+                              (guix search-paths)
+                              (guix build store-copy)))
+      #~(begin
+          (use-modules (guix build utils)
+                       (guix build store-copy)
+                       (guix search-paths)
+                       (srfi srfi-1)
+                       (ice-9 match)
+                       (ice-9 pretty-print))
+
+          (define search-paths
+            ;; Map items like "coreutils-8.20" to their search path specs.
+            (map (match-lambda
+                   ((item . paths)
+                    (cons item (map sexp->search-path-specification paths))))
+                 '#$search-paths))
+
+          (define (store-item-search-paths item)
+            "Return the search paths that apply to ITEM, a store item."
+            (or (any (match-lambda
+                       ((entry . paths)
+                        (and (string-suffix? entry item)
+                             paths)))
+                     search-paths)
+                '()))
+
+          (let* ((references (map store-info-item
+                                  (call-with-input-file "graph"
+                                    read-reference-graph)))
+                 (paths      (delete-duplicates
+                              (append-map store-item-search-paths
+                                          references))))
+            (call-with-output-file #$output
+              (lambda (port)
+                (pretty-print (map search-path-specification->sexp
+                                   paths)
+                              port)))))))
+
+  (computed-file (string-append (package-full-name package "-")
+                                "-search-paths")
+                 build
+                 #:options
+                 `(#:references-graphs (("graph" ,package))
+                   #:properties '((type . search-paths)))))
+
 (define (transitive-input-references alist inputs)
   "Return a list of (assoc-ref ALIST <label>) for each (<label> <package> . _)
 in INPUTS and their transitive propagated inputs."

--=-=-=--

Message sent to bug-guix@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: bug#22138: Search paths of dependencies are not honored
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-guix@HIDDEN
Resent-Date: Sat, 24 Aug 2019 13:53:01 +0000
Resent-Message-ID: <handler.22138.B22138.156665477913310 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 22138
X-GNU-PR-Package: guix
X-GNU-PR-Keywords: 
To: Mark H Weaver <mhw@HIDDEN>
Cc: Julien Lepiller <julien@HIDDEN>, 22138 <at> debbugs.gnu.org
Received: via spool by 22138-submit <at> debbugs.gnu.org id=B22138.156665477913310
          (code B ref 22138); Sat, 24 Aug 2019 13:53:01 +0000
Received: (at 22138) by debbugs.gnu.org; 24 Aug 2019 13:52:59 +0000
Received: from localhost ([127.0.0.1]:41724 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1i1WTS-0003Sb-Rx
	for submit <at> debbugs.gnu.org; Sat, 24 Aug 2019 09:52:59 -0400
Received: from eggs.gnu.org ([209.51.188.92]:39227)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1i1WTR-0003SP-Ja
 for 22138 <at> debbugs.gnu.org; Sat, 24 Aug 2019 09:52:57 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:34097)
 by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>)
 id 1i1WTI-0002gY-Po; Sat, 24 Aug 2019 09:52:48 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=56162 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256)
 (Exim 4.82) (envelope-from <ludo@HIDDEN>)
 id 1i1WTI-0005yt-BF; Sat, 24 Aug 2019 09:52:48 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <87bn9yk5mf.fsf@HIDDEN>
 <20190801221206.17965136@HIDDEN>
 <87ftmfa8cp.fsf@HIDDEN> <87d0gw54fz.fsf@HIDDEN>
Date: Sat, 24 Aug 2019 15:52:46 +0200
In-Reply-To: <87d0gw54fz.fsf@HIDDEN> ("Ludovic
 \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\=
 \=\?utf-8\?Q\?s\?\= message of "Fri, 23 Aug 2019 16:42:08 +0200")
Message-ID: <87pnkuwtzl.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic]
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Ludovic Court=C3=A8s <ludo@HIDDEN> skribis:

> The attached procedure computes the search paths of a package based on
> its run-time dependencies.  =E2=80=98profile-derivation=E2=80=99 could us=
e this to
> compute its =E2=80=98etc/profile=E2=80=99 file and its =E2=80=98manifest=
=E2=80=99 file.

One difficulty with this approach is that search paths are currently a
=E2=80=9Chost-side=E2=80=9D feature: there=E2=80=99s a =E2=80=98search-path=
s=E2=80=99 field in <manifest-entry>,
which is populated before the profile is actually built.

What I suggest above would mean turning search paths into a =E2=80=9Cbuild-=
side=E2=80=9D
feature, which has some implications.

Needs more thought=E2=80=A6

Ludo=E2=80=99.

Last modified: Mon, 25 Nov 2019 12:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.