GNU bug report logs - #48238
Julia contains references to its build directory

Message received at submit <at> debbugs.gnu.org:

Received: (at submit) by debbugs.gnu.org; 5 May 2021 13:27:11 +0000
From debbugs-submit-bounces <at> debbugs.gnu.org Wed May 05 09:27:11 2021
Received: from localhost ([127.0.0.1]:60214 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1leHYU-0007pS-Hr
	for submit <at> debbugs.gnu.org; Wed, 05 May 2021 09:27:11 -0400
Received: from lists.gnu.org ([209.51.188.17]:40600)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <zimon.toutoune@HIDDEN>) id 1leHYR-0007pL-Sa
 for submit <at> debbugs.gnu.org; Wed, 05 May 2021 09:27:08 -0400
Received: from eggs.gnu.org ([2001:470:142:3::10]:54052)
 by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <zimon.toutoune@HIDDEN>)
 id 1leHYR-0004Lw-N6
 for bug-guix@HIDDEN; Wed, 05 May 2021 09:27:07 -0400
Received: from mail-wr1-x432.google.com ([2a00:1450:4864:20::432]:38894)
 by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_128_GCM_SHA256:128)
 (Exim 4.90_1) (envelope-from <zimon.toutoune@HIDDEN>)
 id 1leHYJ-0003bl-IL
 for bug-guix@HIDDEN; Wed, 05 May 2021 09:27:07 -0400
Received: by mail-wr1-x432.google.com with SMTP id l14so1841480wrx.5
 for <bug-guix@HIDDEN>; Wed, 05 May 2021 06:26:58 -0700 (PDT)
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025;
 h=from:to:cc:subject:date:message-id:mime-version
 :content-transfer-encoding;
 bh=0HXsIVYO/DQ1xaSl3ecUI/0EPbr/Nc0DGTZbAYWTob0=;
 b=ixvyVxTsfud0uvZKKzAnvAR2z18Ccxm8FOv1RNLxkSFT2qYQj1NV5yDKNB6egc1Vu1
 vUwNikqpsjUShIr2pf4DpBErmfzIe773AbItZXBMSz7tDBNaVS6zw3wnRqRo1e3zwXcY
 AhSEbHlXn+CFUjwWgeh9AFnE0N11UbLU9JBSaqxgo9E8RznoYUrrg1M/vNmltwm0/5jJ
 km0i9DSn7y0AgOWRJm1ehVcbKAa6gRZEDbwblDuZyYADFsE3A9f61oxMlq9llR3YUzsL
 WxhGHWVavk2QAFPz+mIDbb03486VwsMSGzVzHMmB56w3ejcoiWApZUW45sLBeOPJEz1Z
 o2Jw==
X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed;
 d=1e100.net; s=20161025;
 h=x-gm-message-state:from:to:cc:subject:date:message-id:mime-version
 :content-transfer-encoding;
 bh=0HXsIVYO/DQ1xaSl3ecUI/0EPbr/Nc0DGTZbAYWTob0=;
 b=dzlOYdiuicu0ct15pYrMxcnVnOGC3tawaFuLxiiH0WTBtaQiVW6FffYeIcYT3JreWq
 Y5PhOG1CeYn/A1UAk+2CP8f2DfQ524UyXggjJ1eTIvwrwSEw4mkHQIgi7j2aPZo9W9U6
 +uGm7DqwLAwWWbAm4AU/QMj7CMNRKrt4F7tni1Y0aAvkhvN2K2ASQNZHv5mwglj4XFso
 BhxIiHdbRl9SaUOYvfCDsOWlqrsTNUPDuBqETMCJBZtWYw/0aCut6lqNvzDb0/eag5A5
 Hv0pOsQ2tfRuKHrIthEmN2T++VGuEYVSYRK+ZXx4yglPUoDb5DRrLCdcNzXS5M11WfW8
 cw3w==
X-Gm-Message-State: AOAM533BpSOfKViU6cNVdX+wM2145e2nccUCH0IrYQx7MMr4BZ4fe8P2
 jOHJPqf7LCZvHMmIg80l347I/LLfVBw=
X-Google-Smtp-Source: ABdhPJzni/NiY0JtDBu3my1C9vWCLhhzRmJOh7i3BDIeDE/q00Gk8SroT0EbsiKhxd86vEkepbKN3g==
X-Received: by 2002:a5d:5741:: with SMTP id q1mr38092365wrw.212.1620221217518; 
 Wed, 05 May 2021 06:26:57 -0700 (PDT)
Received: from lili ([2a01:e0a:59b:9120:65d2:2476:f637:db1e])
 by smtp.gmail.com with ESMTPSA id y5sm19704705wrm.61.2021.05.05.06.26.56
 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256);
 Wed, 05 May 2021 06:26:57 -0700 (PDT)
From: zimoun <zimon.toutoune@HIDDEN>
To: Bug Guix <bug-guix@HIDDEN>
Subject: Julia keeps build directory in the output
Date: Wed, 05 May 2021 15:22:58 +0200
Message-ID: <864kfh48nh.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Received-SPF: pass client-ip=2a00:1450:4864:20::432;
 envelope-from=zimon.toutoune@HIDDEN; helo=mail-wr1-x432.google.com
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
Cc: =?utf-8?Q?Nicol=C3=B2?= Balzarotti <anothersms@HIDDEN>
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

Hi,

Packaging Julia stuff, I note something as a bug, I guess.  Julia seems
to keep references to the build directory in the output.

--8<---------------cut here---------------start------------->8---
$ find $(guix build julia --no-grafts) -type f \
       -exec grep '/tmp/guix-build-julia-1.5.3.drv-0' {} \;
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/lib=
julia.so.1.5 matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/jul=
ia/sys.so matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/jul=
ia/libccalltest.so.debug matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/lib/jul=
ia/libllvmcalltest.so matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/bin/.ju=
lia-real matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/j=
ulia/base.cache matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/j=
ulia/test/depot/compiled/v1.5/Bar/HXSAn_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/j=
ulia/test/depot/compiled/v1.5/Foo/MYb1d_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/j=
ulia/test/depot/compiled/v1.5/Foo/TeeT6_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/j=
ulia/test/depot/compiled/v1.5/Qux/YFfiR_w3IH9.ji matches
Binary file /gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vslq50y-julia-1.5.3/share/j=
ulia/test/depot/compiled/v1.5/Baz/rONVA_w3IH9.ji matches
--8<---------------cut here---------------end--------------->8---

The consequence is =E2=80=99dlopen=E2=80=99 is broken and some Julia packag=
es cannot be
pre-compiled.  Here, an example using the Julia package manager:

--8<---------------cut here---------------start------------->8---
$ rm -fr ~/.julia # for the sake of the illustration ;-)
$ cat /tmp/foo.jl
using Pkg
Pkg.add("GZip")
println("# =3D> Now let pre-compile")
using GZip
exit()

$ guix environment --ad-hoc julia -- julia --load /tmp/foo.jl
 Installing known registries into `~/.julia`
######################################################################### 1=
00.0%
      Added registry `General` to `~/.julia/registries/General`
  Resolving package versions...
  Installed GZip =E2=94=80 v0.5.1
Updating `~/.julia/environments/v1.5/Project.toml`
  [92fee26a] + GZip v0.5.1
Updating `~/.julia/environments/v1.5/Manifest.toml`
  [92fee26a] + GZip v0.5.1
  [8f399da3] + Libdl
# =3D> Now let pre-compile
ERROR: LoadError: LoadError: could not load library "libz"
libz.so: cannot open shared object file: No such file or directory
Stacktrace:

[...]

in expression starting at /home/simon/.julia/packages/GZip/JNmGn/src/zlib_h=
.jl:13
in expression starting at /home/simon/.julia/packages/GZip/JNmGn/src/GZip.j=
l:73
ERROR: LoadError: Failed to precompile GZip [92fee26a-97fe-5a0c-ad85-20a5f3=
185b63] to /home/simon/.julia/compiled/v1.5/GZip/s2LKY_jQTtL.ji.
=E2=80=A6
--8<---------------cut here---------------end--------------->8---

Let tweak the GZip =E2=80=99ccall=E2=80=99 and pre-compile again to show th=
at the
build-directory is contained in the output, and the Julia internal trace
raises it.

--8<---------------cut here---------------start------------->8---
$ diff ~/.julia/packages/GZip/JNmGn/src/GZip.jl{,.orig}=20
75c75
< const GZLIB_VERSION =3D unsafe_string(ccall(:zlibVersion, Ptr{UInt8}, ()))
---
> const GZLIB_VERSION =3D unsafe_string(ccall((:zlibVersion, GZip._zlib), P=
tr{UInt8}, ()))

$ diff ~/.julia/packages/GZip/JNmGn/src/zlib_h.jl{,.orig}=20
13c13
< zlib_version =3D  unsafe_string(ccall(:zlibVersion, Ptr{UInt8}, ()))
---
> zlib_version =3D  unsafe_string(ccall((:zlibVersion, _zlib), Ptr{UInt8}, =
()))
84c84
< const zlib_compile_flags =3D ccall(:zlibCompileFlags, UInt, ())
---
> const zlib_compile_flags =3D ccall((:zlibCompileFlags, _zlib), UInt, ())

$ guix environment --ad-hoc julia -- julia -e 'using GZip'
ERROR: LoadError: could not load library "libz"
libz.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] dlopen(::String, ::UInt32; throw_error::Bool) at /tmp/guix-build-julia=
-1.5.3.drv-0/julia-1.5.3/usr/share/julia/stdlib/v1.5/Libdl/src/Libdl.jl:109
 [2] dlopen at /tmp/guix-build-julia-1.5.3.drv-0/julia-1.5.3/usr/share/juli=
a/stdlib/v1.5/Libdl/src/Libdl.jl:109 [inlined] (repeats 2 times)

[...]

in expression starting at /home/simon/.julia/packages/GZip/JNmGn/src/GZip.j=
l:206
=E2=80=A6
--8<---------------cut here---------------end--------------->8---


I have tried to tweak LD_LIBRARY_PATH in the julia definition and
rebuild it.  Still the same error.  Note building the julia package
takes ~2h on my machine, so the trial-error takes ages. ;-)

Another illustration:

--8<---------------cut here---------------start------------->8---
$ cat /tmp/bar/pkg.scm
(define-module (pkg)
  #:use-module (guix packages)
  #:use-module (guix git-download)
  #:use-module (guix build-system julia)
  #:use-module ((guix licenses) #:prefix license:))

(define-public julia-gzip
  (package
    (name "julia-gzip")
    (version "0.5.1")
    (source
     (origin
       (method git-fetch)
       (uri (git-reference
             (url "https://github.com/JuliaIO/GZip.jl")
             (commit (string-append "v" version))))
       (file-name (git-file-name name version))
       (sha256
        (base32 "06bx3l1nsvdswgl344347cjh6ffd2r1hw3lz4h0aicayc566717f"))))
    (build-system julia-build-system)
    (home-page "https://github.com/JuliaIO/GZip.jl")
    (synopsis "TODO")
    (description "TODO")
    (license license:expat)))

$ guix build -L /tmp/bar/ julia-gzip
The following derivation will be built:
   /gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv
building /gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv...
starting phase `set-SOURCE-DATE-EPOCH'
phase `set-SOURCE-DATE-EPOCH' succeeded after 0.0 seconds
starting phase `set-paths'
environment variable `PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc33lz17vsl=
q50y-julia-1.5.3/bin:/gnu/store/v6f44zccwh9z5zk3pjlywjybbi8n2hjh-tar-1.32/b=
in:/gnu/store/ncydgq2znms5n1d2k5yqshhf58nsixwv-gzip-1.10/bin:/gnu/store/i8h=
2pcxqdq07ijm3ibkka8f4smn1w48v-bzip2-1.0.8/bin:/gnu/store/9860f1abqj8wjjnwl8=
a9v54pdcc3bhgf-xz-5.2.4/bin:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-fil=
e-5.38/bin:/gnu/store/n4n560pfvvw50a9369axw5vj5rrqfj1n-diffutils-3.7/bin:/g=
nu/store/cd5qf3kcnlq35p9k392pjdpdzpsnds70-patch-2.7.6/bin:/gnu/store/hic7sn=
hayfl7m6cpfqqr73nmm19bpqkg-findutils-4.7.0/bin:/gnu/store/swqdvwri9dbv6zssg=
6v0by7l05hd6wxp-gawk-5.0.1/bin:/gnu/store/ishk7fswcs4gkwcp8mh788z4mvvl9bxh-=
sed-4.8/bin:/gnu/store/bhs4rj58v8j1narb2454raan2ps38xd8-grep-3.4/bin:/gnu/s=
tore/57xj5gcy1jbl9ai2lnrqnpr0dald9i65-coreutils-8.32/bin:/gnu/store/hm40bxn=
v8jxmbc1lpb7zfimii4xm9m81-make-4.3/bin:/gnu/store/pwcp239kjf7lnj5i4lkdzcfcx=
wcfyk72-bash-minimal-5.0.16/bin:/gnu/store/mpa04aq8lblbcviyxywxcsb1zbi0mf39=
-ld-wrapper-0/bin:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7gkcg3m4-binutils-2.34=
/bin:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk-gcc-7.5.0/bin:/gnu/store/f=
a6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/bin:/gnu/store/fa6wj5bxkj5ll1d72=
92a70knmyl7a0cr-glibc-2.31/sbin'
environment variable `JULIA_LOAD_PATH' unset
environment variable `JULIA_DEPOT_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhp=
cwc33lz17vslq50y-julia-1.5.3/share/julia/'
environment variable `BASH_LOADABLES_PATH' unset
environment variable `C_INCLUDE_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcw=
c33lz17vslq50y-julia-1.5.3/include:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w=
48v-bzip2-1.0.8/include:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.=
4/include:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/include:/gn=
u/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/include:/gnu/store/hm40=
bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/include:/gnu/store/m1z7cdbqsqyp9xnjw5=
cvlb4a7gkcg3m4-binutils-2.34/include:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1=
y17lk-gcc-7.5.0/include:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2=
.31/include:/gnu/store/gfapkk5c6hvl1d94m4sqnhn7f9l5gqyh-linux-libre-headers=
-5.4.20/include'
environment variable `CPLUS_INCLUDE_PATH' set to `/gnu/store/i1dgzqfjvkkjlf=
hpcwc33lz17vslq50y-julia-1.5.3/include:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4s=
mn1w48v-bzip2-1.0.8/include:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-=
5.2.4/include:/gnu/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/include=
:/gnu/store/swqdvwri9dbv6zssg6v0by7l05hd6wxp-gawk-5.0.1/include:/gnu/store/=
hm40bxnv8jxmbc1lpb7zfimii4xm9m81-make-4.3/include:/gnu/store/m1z7cdbqsqyp9x=
njw5cvlb4a7gkcg3m4-binutils-2.34/include:/gnu/store/rn75fm7adgx3pw5j8pg3bcz=
fqq1y17lk-gcc-7.5.0/include/c++:/gnu/store/rn75fm7adgx3pw5j8pg3bczfqq1y17lk=
-gcc-7.5.0/include:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-2.31/i=
nclude:/gnu/store/gfapkk5c6hvl1d94m4sqnhn7f9l5gqyh-linux-libre-headers-5.4.=
20/include'
environment variable `LIBRARY_PATH' set to `/gnu/store/i1dgzqfjvkkjlfhpcwc3=
3lz17vslq50y-julia-1.5.3/lib:/gnu/store/i8h2pcxqdq07ijm3ibkka8f4smn1w48v-bz=
ip2-1.0.8/lib:/gnu/store/9860f1abqj8wjjnwl8a9v54pdcc3bhgf-xz-5.2.4/lib:/gnu=
/store/60g7r3l01fd7c58yjbm6krgcwj1jkpwg-file-5.38/lib:/gnu/store/swqdvwri9d=
bv6zssg6v0by7l05hd6wxp-gawk-5.0.1/lib:/gnu/store/m1z7cdbqsqyp9xnjw5cvlb4a7g=
kcg3m4-binutils-2.34/lib:/gnu/store/fa6wj5bxkj5ll1d7292a70knmyl7a0cr-glibc-=
2.31/lib:/gnu/store/s3dcqzwqaakv1yx37by9chksdbkgih17-glibc-2.31-static/lib:=
/gnu/store/hwcky7446s952w0mwchhmm211ll07zrq-glibc-utf8-locales-2.31/lib'
environment variable `GUIX_LOCPATH' set to `/gnu/store/hwcky7446s952w0mwchh=
mm211ll07zrq-glibc-utf8-locales-2.31/lib/locale'
phase `set-paths' succeeded after 0.0 seconds
starting phase `install-locale'
using 'en_US.utf8' locale for category "LC_ALL"
phase `install-locale' succeeded after 0.0 seconds
starting phase `unpack'

[...]

phase `unpack' succeeded after 0.0 seconds
starting phase `patch-source-shebangs'
phase `patch-source-shebangs' succeeded after 0.0 seconds
starting phase `patch-generated-file-shebangs'
phase `patch-generated-file-shebangs' succeeded after 0.0 seconds
starting phase `install'

[...]

phase `install' succeeded after 0.0 seconds
starting phase `check'
ERROR: LoadError: LoadError: could not load library "libz"
libz.so: cannot open shared object file: No such file or directory
Stacktrace:

[...]

in expression starting at /gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia=
-gzip-0.5.1/share/julia/packages/GZip/src/zlib_h.jl:13
in expression starting at /gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia=
-gzip-0.5.1/share/julia/packages/GZip/src/GZip.jl:73
ERROR: LoadError: Failed to precompile GZip [92fee26a-97fe-5a0c-ad85-20a5f3=
185b63] to /tmp/.julia/compiled/v1.5/GZip/s2LKY_y94HW.ji.
Stacktrace:

[...]

in expression starting at /gnu/store/dak0gnj5kqdma55xy279345qzsal9zdq-julia=
-gzip-0.5.1/share/julia/packages/GZip/test/runtests.jl:1
command "julia" "--depwarn=3Dyes" "/gnu/store/dak0gnj5kqdma55xy279345qzsal9=
zdq-julia-gzip-0.5.1/share/julia/packages/GZip/test/runtests.jl" failed wit=
h status 1
builder for `/gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.d=
rv' failed with exit code 1
build of /gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-julia-gzip-0.5.1.drv f=
ailed
View build log at '/var/log/guix/drvs/2w/iyhvqrhq8bbiw34gyxclv6zzf98vy3-jul=
ia-gzip-0.5.1.drv.bz2'.
guix build: error: build of `/gnu/store/2wiyhvqrhq8bbiw34gyxclv6zzf98vy3-ju=
lia-gzip-0.5.1.drv' failed
--8<---------------cut here---------------end--------------->8---


Cheers,
simon