GNU bug report logs - #61121
Cannot import IJulia in Julia

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 61121 in the body.
You can then email your comments to 61121 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com>
To: bug-guix <at> gnu.org
Subject: Cannot import IJulia in Julia
Date: Sat, 28 Jan 2023 13:45:00 +0000

[Message part 1 (text/plain, inline)]

Hi Guix,

I would like to run a Jupyter notebook using Julia, so I need to install
the IJulia backend:

guix install julia
julia # Enter julia REPL
] # To go into the julia pkg REPL
add IJulia
# Now type backspace to go to julia REPL
using IJulia

This produces the error:

[ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
ERROR: LoadError: InitError: SystemError: opening file
"/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
No such file or directory

The full error message is here: https://pastebin.com/qC8yyHXT

I saw a very similar bug on Gentoo:

Without this file (which can be a symbolic link to
`/etc/ssl/certs/ca-certificates.crt`) many Julia 1.8.3 packages, e.g.
`HTTP`, do not work.
This is what happens:

julia> import HTTP
[ Info: Precompiling HTTP [cd3eb016-35fb-5094-929b-558a96fad6f3]
ERROR: LoadError: InitError: SystemError: opening file
"/usr/share/julia/cert.pem":

(https://bugs.gentoo.org/888978)

Any help would be greatly appreciated.

Best regards,
Theodore Ehrenborg

[Message part 2 (text/html, inline)]

Message #8 received at 61121 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com>, 61121 <at> debbugs.gnu.org
Cc: Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: bug#61121: Cannot import IJulia in Julia
Date: Mon, 30 Jan 2023 13:27:50 +0100

Hi,

I confirm this bug.

On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com> wrote:

> [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> ERROR: LoadError: InitError: SystemError: opening file
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> No such file or directory

[...]

> I saw a very similar bug on Gentoo:

[...]

> (https://bugs.gentoo.org/888978)

Well, that’s because Julia upstream does not take care about packagers;
as explicitly mentioned in this comment:

    https://github.com/JuliaLang/MbedTLS.jl/pull/261#issuecomment-1346886879

The Guixer Cayetano Santos fixed upstream the issue for one package.
But as you are noticing it is not done for all.

I do not know what is the best solution because the issue is coming from
Julia itself.

Efraim, any suggestion?

Cheers,
simon

Message #11 received at 61121 <at> debbugs.gnu.org (full text, mbox):

From: Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: 61121 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: bug#61121: Cannot import IJulia in Julia
Date: Mon, 30 Jan 2023 21:55:32 +0000

[Message part 1 (text/plain, inline)]

Hi,

Thanks for getting back with me.

Gentoo appears to have fixed this bug by linking julia/cert.pem to the
system's ca-certificates.crt.
https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168

Is there a way I could rebuild my own slightly modified Julia with a link
like that?

I understand that there's probably a good reason that Guix's Julia doesn't
by default have cert.pem, but I would be pleased with a hacky custom
solution if it made Jupyter notebooks work.

Thanks,
Theodore

Den mån 30 jan. 2023 kl 12:47 skrev Simon Tournier <zimon.toutoune <at> gmail.com
>:

> Hi,
>
> I confirm this bug.
>
> On sam., 28 janv. 2023 at 13:45, Theodore Ehrenborg <
> theodore.ehrenborg <at> gmail.com> wrote:
>
> > [ Info: Precompiling IJulia [7073ff75-c697-5162-941a-fcdaad2a7d2a]
> > ERROR: LoadError: InitError: SystemError: opening file
> >
> "/gnu/store/npj8z0g9nx14wl22yphqfs2c5w4qk5jk-julia-1.8.3/share/julia/cert.pem":
> > No such file or directory
>
> [...]
>
> > I saw a very similar bug on Gentoo:
>
> [...]
>
> > (https://bugs.gentoo.org/888978)
>
> Well, that’s because Julia upstream does not take care about packagers;
> as explicitly mentioned in this comment:
>
>
> https://github.com/JuliaLang/MbedTLS.jl/pull/261#issuecomment-1346886879
>
> The Guixer Cayetano Santos fixed upstream the issue for one package.
> But as you are noticing it is not done for all.
>
> I do not know what is the best solution because the issue is coming from
> Julia itself.
>
> Efraim, any suggestion?
>
> Cheers,
> simon
>

[Message part 2 (text/html, inline)]

Message #14 received at 61121 <at> debbugs.gnu.org (full text, mbox):

From: Simon Tournier <zimon.toutoune <at> gmail.com>
To: Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com>
Cc: 61121 <at> debbugs.gnu.org, Efraim Flashner <efraim <at> flashner.co.il>
Subject: Re: bug#61121: Cannot import IJulia in Julia
Date: Tue, 31 Jan 2023 12:34:16 +0100

Hi,

On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com> wrote:

> Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> system's ca-certificates.crt.
> https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168

This trick is not possible, IIUC.

> Is there a way I could rebuild my own slightly modified Julia with a link
> like that?

Maybe, by adding the package nss-certs as propagated-inputs in the
definition of julia.

> I understand that there's probably a good reason that Guix's Julia doesn't
> by default have cert.pem, but I would be pleased with a hacky custom
> solution if it made Jupyter notebooks work.

The reason is security. ;-)  It’s Julia that does poorly here.

As pointed with the upstream package MbedTLS.jl, the fix should come
from Julia itself; therefore, it could be worth to open an issue, if it
is not already the case. ;-)

From my understanding, the culprit is this [1]:

--8<---------------cut here---------------start------------->8---
function __init__()
    global artifact_dir = dirname(Sys.BINDIR)
    global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
end
--8<---------------cut here---------------end--------------->8---

And it is not clear for me if NetworkOptions.jl [2] provides the option
of not, and I am missing why Julia itself does not depend on it.

1: https://github.com/JuliaLang/julia/blob/master/stdlib/MozillaCACerts_jll/src/MozillaCACerts_jll.jl#L20
2: https://github.com/JuliaLang/NetworkOptions.jl


Efraim, do you think it would be possible to patch Julia to point to
some certificates via bundled_ca_roots or ca_roots_path?

Well, somehow turn back these tests:

--8<---------------cut here---------------start------------->8---
             ;; julia embeds a certificate, we are not doing that
             (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
               (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
                "@test_broken isfile(MozillaCACerts_jll.cacert)"))
             ;; since certificate is not present some tests are failing in network option
             (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
               (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
                "@test_broken isfile(bundled_ca_roots())")
               (("@test ispath\\(ca_roots_path\\(\\)\\)")
                "@test_broken ispath(ca_roots_path())")
               (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
                "@test_broken ca_roots_path() != bundled_ca_roots()"))
--8<---------------cut here---------------end--------------->8---


Cheers,
simon

Message #17 received at 61121 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: 61121 <at> debbugs.gnu.org, Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com>
Subject: Re: bug#61121: Cannot import IJulia in Julia
Date: Thu, 9 Feb 2023 11:29:53 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
> Hi,
> 
> On Mon, 30 Jan 2023 at 21:55, Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com> wrote:
> 
> > Gentoo appears to have fixed this bug by linking julia/cert.pem to the
> > system's ca-certificates.crt.
> > https://gitweb.gentoo.org/repo/gentoo.git/commit/?id=26b59330b5222996defa4536237e62404bf21168
> 
> This trick is not possible, IIUC.
> 
> > Is there a way I could rebuild my own slightly modified Julia with a link
> > like that?
> 
> Maybe, by adding the package nss-certs as propagated-inputs in the
> definition of julia.

By itself I don't think this would do anything.

> > I understand that there's probably a good reason that Guix's Julia doesn't
> > by default have cert.pem, but I would be pleased with a hacky custom
> > solution if it made Jupyter notebooks work.
> 
> The reason is security. ;-)  It’s Julia that does poorly here.
> 
> As pointed with the upstream package MbedTLS.jl, the fix should come
> from Julia itself; therefore, it could be worth to open an issue, if it
> is not already the case. ;-)
> 
> From my understanding, the culprit is this [1]:
> 
> --8<---------------cut here---------------start------------->8---
> function __init__()
>     global artifact_dir = dirname(Sys.BINDIR)
>     global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---
> 
> And it is not clear for me if NetworkOptions.jl [2] provides the option
> of not, and I am missing why Julia itself does not depend on it.
> 
> 1: https://github.com/JuliaLang/julia/blob/master/stdlib/MozillaCACerts_jll/src/MozillaCACerts_jll.jl#L20
> 2: https://github.com/JuliaLang/NetworkOptions.jl
> 
> 
> Efraim, do you think it would be possible to patch Julia to point to
> some certificates via bundled_ca_roots or ca_roots_path?

In the initial patch for julia-1.8.1 I think there was a substitution to
hardcode /etc/ssl/something instead for 'global cacert' but I took that
out since we don't like hardcoding that.

GIT_SSL_CAINFO=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_DIR=/run/current-system/profile/etc/ssl/certs
CURL_CA_BUNDLE=/home/efraim/.guix-home/profile/etc/ssl/certs/ca-certificates.crt
SSL_CERT_FILE=/run/current-system/profile/etc/ssl/certs/ca-certificates.crt

I think it would be fine to tell Julia to look at SSL_CERT_FILE as the
cacert so it can be overridden as desired, and then we can add a
(native-?)search-path to Julia for SSL_CERT_FILE.

Does anyone know offhand how to get the environment variable? If not
I'll grep the sources and then look online.

> Well, somehow turn back these tests:
> 
> --8<---------------cut here---------------start------------->8---
>              ;; julia embeds a certificate, we are not doing that
>              (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
>                (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
>                 "@test_broken isfile(MozillaCACerts_jll.cacert)"))
>              ;; since certificate is not present some tests are failing in network option
>              (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
>                (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
>                 "@test_broken isfile(bundled_ca_roots())")
>                (("@test ispath\\(ca_roots_path\\(\\)\\)")
>                 "@test_broken ispath(ca_roots_path())")
>                (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
>                 "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

That one might be a little harder, I'd rather not add nss-certs to the
build just for the test suite, but I'll see how it goes. Or at least
update the comment afterward.

> 
> Cheers,
> simon

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #22 received at 61121-done <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Simon Tournier <zimon.toutoune <at> gmail.com>
Cc: 61121-done <at> debbugs.gnu.org,
 Theodore Ehrenborg <theodore.ehrenborg <at> gmail.com>
Subject: Re: bug#61121: Cannot import IJulia in Julia
Date: Thu, 9 Feb 2023 16:53:01 +0200

[Message part 1 (text/plain, inline)]

On Tue, Jan 31, 2023 at 12:34:16PM +0100, Simon Tournier wrote:
> 
> --8<---------------cut here---------------start------------->8---
> function __init__()
>     global artifact_dir = dirname(Sys.BINDIR)
>     global cacert = normpath(Sys.BINDIR, Base.DATAROOTDIR, "julia", "cert.pem")
> end
> --8<---------------cut here---------------end--------------->8---

I've changed this line to:

global cacert = get(ENV, \"SSL_CERT_FILE\", "\"/etc/ssl/certs/ca-certificates.crt\")

and then tested it with the example at the beginning of the bug report.

> Well, somehow turn back these tests:
> 
> --8<---------------cut here---------------start------------->8---
>              ;; julia embeds a certificate, we are not doing that
>              (substitute* "stdlib/MozillaCACerts_jll/test/runtests.jl"
>                (("@test isfile\\(MozillaCACerts_jll.cacert\\)")
>                 "@test_broken isfile(MozillaCACerts_jll.cacert)"))
>              ;; since certificate is not present some tests are failing in network option
>              (substitute* "usr/share/julia/stdlib/v1.8/NetworkOptions/test/runtests.jl"
>                (("@test isfile\\(bundled_ca_roots\\(\\)\\)")
>                 "@test_broken isfile(bundled_ca_roots())")
>                (("@test ispath\\(ca_roots_path\\(\\)\\)")
>                 "@test_broken ispath(ca_roots_path())")
>                (("@test ca_roots_path\\(\\) \\!= bundled_ca_roots\\(\\)")
>                 "@test_broken ca_roots_path() != bundled_ca_roots()"))
> --8<---------------cut here---------------end--------------->8---

I wasn't able to turn these tests back on though.

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.