GNU bug report logs - #42888
[PATCH] gnu: rust: Fix install phase for non-x86_64 platforms.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 42888 in the body.
You can then email your comments to 42888 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: Guix-patches <guix-patches <at> gnu.org>
Subject: [PATCH] gnu: rust: Fix install phase for non-x86_64 platforms.
Date: Sun, 16 Aug 2020 15:12:08 +0100

[Message part 1 (text/plain, inline)]

Hello Guix!

In an optimistic attempt to eventually have Icecat on a Pinebook Pro, I
thought I'd try and get rust building on aarch64. Here's a fix for the
post-install phase that had an x86 triplet hardcoded. With this we're
able to start off the bootstrap chain!

That being said, each step takes ~5 hours on this machine so this is
going to take a while :-), it's currently working on 1.23.

So, I suppose this should go into either core-updates or staging? WDYT?

Thanks!
Pierre

[0001-gnu-rust-Fix-install-phase-for-non-x86_64-platforms.patch (text/x-patch, attachment)]

Message #8 received at 42888 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Pierre Langlois <pierre.langlois <at> gmx.com>
Cc: 42888 <at> debbugs.gnu.org
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Sun, 16 Aug 2020 17:19:40 +0300

[Message part 1 (text/plain, inline)]

On Sun, Aug 16, 2020 at 03:12:08PM +0100, Pierre Langlois wrote:
> Hello Guix!
> 
> In an optimistic attempt to eventually have Icecat on a Pinebook Pro, I
> thought I'd try and get rust building on aarch64. Here's a fix for the
> post-install phase that had an x86 triplet hardcoded. With this we're
> able to start off the bootstrap chain!

I forgot about that! It takes so long to build I never bothered with it
on my boards.

> That being said, each step takes ~5 hours on this machine so this is
> going to take a while :-), it's currently working on 1.23.
> 
> So, I suppose this should go into either core-updates or staging? WDYT?

I'd go with staging. No one wants to rebuild the rust bootstrap, but
they're basically all leaf packages so it's fine.

> 
> Thanks!
> Pierre
> 



-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #11 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: Pierre Langlois <pierre.langlois <at> gmx.com>
Cc: 42888 <at> debbugs.gnu.org, guix-patches <at> gnu.org
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Sun, 16 Aug 2020 15:42:38 +0100

[Message part 1 (text/plain, inline)]

Pierre Langlois writes:

> Hello Guix!
>
> In an optimistic attempt to eventually have Icecat on a Pinebook Pro, I
> thought I'd try and get rust building on aarch64. Here's a fix for the
> post-install phase that had an x86 triplet hardcoded. With this we're
> able to start off the bootstrap chain!
>
> That being said, each step takes ~5 hours on this machine so this is
> going to take a while :-), it's currently working on 1.23.
>
> So, I suppose this should go into either core-updates or staging? WDYT?

Whoops, I forgot the copyright line on that file.

[0001-gnu-rust-Fix-install-phase-for-non-x86_64-platforms.patch (text/x-patch, attachment)]

Message #17 received at 42888 <at> debbugs.gnu.org (full text, mbox):

From: Jakub Kądziołka <kuba <at> kadziolka.net>
To: Pierre Langlois <pierre.langlois <at> gmx.com>
Cc: 42888 <at> debbugs.gnu.org, guix-patches <at> gnu.org
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Wed, 19 Aug 2020 23:55:52 +0200

[Message part 1 (text/plain, inline)]

On Sun, Aug 16, 2020 at 03:42:38PM +0100, Pierre Langlois wrote:
> 
> Pierre Langlois writes:
> 
> > Hello Guix!
> >
> > In an optimistic attempt to eventually have Icecat on a Pinebook Pro, I
> > thought I'd try and get rust building on aarch64. Here's a fix for the
> > post-install phase that had an x86 triplet hardcoded. With this we're
> > able to start off the bootstrap chain!
> >
> > That being said, each step takes ~5 hours on this machine so this is
> > going to take a while :-), it's currently working on 1.23.
> >
> > So, I suppose this should go into either core-updates or staging? WDYT?
> 
> Whoops, I forgot the copyright line on that file.
> 

Pierre,

thanks for your patch! I was working on a similar change before, but
when I tried it, it failed even earlier in the bootstrap chain. It
might've been QEMU weirdness, though, a la #42448.

> @@ -612,9 +613,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
>                         (cargo-out (assoc-ref outputs "cargo")))
>                     (for-each
>                       (lambda (file) (delete-manifest-file out file))
> -                     '("install.log"
> +                     `("install.log"
>                         "manifest-rust-docs"
> -                       "manifest-rust-std-x86_64-unknown-linux-gnu"
> +                       ,,(string-append "manifest-rust-std-"
> +                                        (nix-system->gnu-triplet-for-rust))
>                         "manifest-rustc"))
>                     (for-each
>                       (lambda (file) (delete-manifest-file cargo-out file))

If I understand the code correctly, this quasiquote is unnecessary, as
the host-side code will evaluate to a string that can be inserted as-is,
without another unquote on the build side.

Fixing this would mean that the patch can go on master, since it would
now only trigger rebuilds on architectures that are already broken.

I wish you best of luck on your quest for Rust on ARM boards. This has
been a long-standing issue, and it'd be nice to have it fixed. Let me
know if you need any help - I packaged the last few versions, so I got
quite familiar with the various failure modes of the build process. I'm
NieDzejkob on IRC, if you prefer.

Regards,
Jakub Kądziołka

[signature.asc (application/pgp-signature, inline)]

Message #23 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: Jakub Kądziołka <kuba <at> kadziolka.net>
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>, 42888 <at> debbugs.gnu.org,
 guix-patches <at> gnu.org
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Thu, 20 Aug 2020 21:55:49 +0100

[Message part 1 (text/plain, inline)]

Hi Jakub,

Jakub Kądziołka writes:

> On Sun, Aug 16, 2020 at 03:42:38PM +0100, Pierre Langlois wrote:
>> 
>> Pierre Langlois writes:
>> 
>> > Hello Guix!
>> >
>> > In an optimistic attempt to eventually have Icecat on a Pinebook Pro, I
>> > thought I'd try and get rust building on aarch64. Here's a fix for the
>> > post-install phase that had an x86 triplet hardcoded. With this we're
>> > able to start off the bootstrap chain!
>> >
>> > That being said, each step takes ~5 hours on this machine so this is
>> > going to take a while :-), it's currently working on 1.23.
>> >
>> > So, I suppose this should go into either core-updates or staging? WDYT?
>> 
>> Whoops, I forgot the copyright line on that file.
>> 
>
> Pierre,
>
> thanks for your patch! I was working on a similar change before, but
> when I tried it, it failed even earlier in the bootstrap chain. It
> might've been QEMU weirdness, though, a la #42448.
>
>> @@ -612,9 +613,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
>>                         (cargo-out (assoc-ref outputs "cargo")))
>>                     (for-each
>>                       (lambda (file) (delete-manifest-file out file))
>> -                     '("install.log"
>> +                     `("install.log"
>>                         "manifest-rust-docs"
>> -                       "manifest-rust-std-x86_64-unknown-linux-gnu"
>> +                       ,,(string-append "manifest-rust-std-"
>> +                                        (nix-system->gnu-triplet-for-rust))
>>                         "manifest-rustc"))
>>                     (for-each
>>                       (lambda (file) (delete-manifest-file cargo-out file))
>
> If I understand the code correctly, this quasiquote is unnecessary, as
> the host-side code will evaluate to a string that can be inserted as-is,
> without another unquote on the build side.
>
> Fixing this would mean that the patch can go on master, since it would
> now only trigger rebuilds on architectures that are already broken.

Ooooh yeah, that'll be much better, I've attached a patch that does just
that. I can confirm it doesn't trigger a full rebuild, nice!

> I wish you best of luck on your quest for Rust on ARM boards. This has
> been a long-standing issue, and it'd be nice to have it fixed. Let me
> know if you need any help - I packaged the last few versions, so I got
> quite familiar with the various failure modes of the build process. I'm
> NieDzejkob on IRC, if you prefer.

haha, thanks :-), I haven't got very far with it yet, the build failed
at 1.23 with a test case failure. Something to do with receiving the
wrong signal on an expected crash. I'm optimistic though!

I don't have the build log around anymore, but we can see if the CI has
the same problem. I haven't spent much time investigating, instead
thinking I'm better off trying to get the patches that allow
bootstrapping from 1.29 working. I was planning on looking into that.

Otherwise we can skip any tests that don't pass and carry on, I suspect
it's normal behaviour on that platform and the tests need updating.  I
think aarch64 is officially supported now but it's probably quite
recent.

Thanks,
Pierre

[0001-gnu-rust-Fix-install-phase-for-non-x86_64-platforms.patch (text/x-patch, attachment)]

Message #29 received at 42888 <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: 42888 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Sat, 19 Sep 2020 14:36:04 +0100

[Message part 1 (text/plain, inline)]

Hello Guix!

Pierre Langlois writes:

> Hi Jakub,
>
> Jakub Kądziołka writes:
>

(snip)

>> Pierre,
>>
>> thanks for your patch! I was working on a similar change before, but
>> when I tried it, it failed even earlier in the bootstrap chain. It
>> might've been QEMU weirdness, though, a la #42448.
>>
>>> @@ -612,9 +613,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
>>>                         (cargo-out (assoc-ref outputs "cargo")))
>>>                     (for-each
>>>                       (lambda (file) (delete-manifest-file out file))
>>> -                     '("install.log"
>>> +                     `("install.log"
>>>                         "manifest-rust-docs"
>>> -                       "manifest-rust-std-x86_64-unknown-linux-gnu"
>>> +                       ,,(string-append "manifest-rust-std-"
>>> +                                        (nix-system->gnu-triplet-for-rust))
>>>                         "manifest-rustc"))
>>>                     (for-each
>>>                       (lambda (file) (delete-manifest-file cargo-out file))
>>
>> If I understand the code correctly, this quasiquote is unnecessary, as
>> the host-side code will evaluate to a string that can be inserted as-is,
>> without another unquote on the build side.
>>
>> Fixing this would mean that the patch can go on master, since it would
>> now only trigger rebuilds on architectures that are already broken.
>
> Ooooh yeah, that'll be much better, I've attached a patch that does just
> that. I can confirm it doesn't trigger a full rebuild, nice!
>

I've just rebased this patch and fixed a little formatting issue. I've
made sure, it doesn't trigger any rust rebuilds on x86.

Is this OK to commit?

Thanks,
Pierre

[signature.asc (application/pgp-signature, inline)]

[0001-gnu-rust-Fix-install-phase-for-non-x86_64-platforms.patch (text/x-patch, inline)]

From 207071445a0a637075138fb678bdbcdcfbaa038d Mon Sep 17 00:00:00 2001
From: Pierre Langlois <pierre.langlois <at> gmx.com>
Date: Sun, 16 Aug 2020 12:17:18 +0100
Subject: [PATCH] gnu: rust: Fix install phase for non-x86_64 platforms.

* gnu/packages/rust.scm (rust-1.20)[arguments]: Replace hardcoded x86_64
triplet with nix-system->gnu-triplet-for-rust in 'delete-install-logs phase.
---
 gnu/packages/rust.scm | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

diff --git a/gnu/packages/rust.scm b/gnu/packages/rust.scm
index d5914f012b..1690ad96c5 100644
--- a/gnu/packages/rust.scm
+++ b/gnu/packages/rust.scm
@@ -9,6 +9,7 @@
 ;;; Copyright © 2018 Danny Milosavljevic <dannym+a <at> scratchpost.org>
 ;;; Copyright © 2019 Ivan Petkov <ivanppetkov <at> gmail.com>
 ;;; Copyright © 2020 Jakub Kądziołka <kuba <at> kadziolka.net>
+;;; Copyright © 2020 Pierre Langlois <pierre.langlois <at> gmx.com>
 ;;;
 ;;; This file is part of GNU Guix.
 ;;;
@@ -614,7 +615,8 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
                      (lambda (file) (delete-manifest-file out file))
                      '("install.log"
                        "manifest-rust-docs"
-                       "manifest-rust-std-x86_64-unknown-linux-gnu"
+                       ,(string-append "manifest-rust-std-"
+                                       (nix-system->gnu-triplet-for-rust))
                        "manifest-rustc"))
                    (for-each
                      (lambda (file) (delete-manifest-file cargo-out file))
-- 
2.28.0

Message #32 received at 42888 <at> debbugs.gnu.org (full text, mbox):

From: Marius Bakke <marius <at> gnu.org>
To: Pierre Langlois <pierre.langlois <at> gmx.com>, 42888 <at> debbugs.gnu.org
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Fri, 25 Sep 2020 20:57:27 +0200

[Message part 1 (text/plain, inline)]

Pierre Langlois <pierre.langlois <at> gmx.com> writes:

> Hello Guix!
>
> Pierre Langlois writes:
>
>> Hi Jakub,
>>
>> Jakub Kądziołka writes:
>>
>
> (snip)
>
>>> Pierre,
>>>
>>> thanks for your patch! I was working on a similar change before, but
>>> when I tried it, it failed even earlier in the bootstrap chain. It
>>> might've been QEMU weirdness, though, a la #42448.
>>>
>>>> @@ -612,9 +613,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
>>>>                         (cargo-out (assoc-ref outputs "cargo")))
>>>>                     (for-each
>>>>                       (lambda (file) (delete-manifest-file out file))
>>>> -                     '("install.log"
>>>> +                     `("install.log"
>>>>                         "manifest-rust-docs"
>>>> -                       "manifest-rust-std-x86_64-unknown-linux-gnu"
>>>> +                       ,,(string-append "manifest-rust-std-"
>>>> +                                        (nix-system->gnu-triplet-for-rust))
>>>>                         "manifest-rustc"))
>>>>                     (for-each
>>>>                       (lambda (file) (delete-manifest-file cargo-out file))
>>>
>>> If I understand the code correctly, this quasiquote is unnecessary, as
>>> the host-side code will evaluate to a string that can be inserted as-is,
>>> without another unquote on the build side.
>>>
>>> Fixing this would mean that the patch can go on master, since it would
>>> now only trigger rebuilds on architectures that are already broken.
>>
>> Ooooh yeah, that'll be much better, I've attached a patch that does just
>> that. I can confirm it doesn't trigger a full rebuild, nice!
>>
>
> I've just rebased this patch and fixed a little formatting issue. I've
> made sure, it doesn't trigger any rust rebuilds on x86.
>
> Is this OK to commit?

LGTM!  Thanks a lot for tackling this long-standing issue.

Were you able to build all the way to the latest Rust?

[signature.asc (application/pgp-signature, inline)]

Message #37 received at 42888-done <at> debbugs.gnu.org (full text, mbox):

From: Pierre Langlois <pierre.langlois <at> gmx.com>
To: Marius Bakke <marius <at> gnu.org>
Cc: Pierre Langlois <pierre.langlois <at> gmx.com>, 42888-done <at> debbugs.gnu.org
Subject: Re: [bug#42888] [PATCH] gnu: rust: Fix install phase for non-x86_64
 platforms.
Date: Fri, 25 Sep 2020 21:03:23 +0100

[Message part 1 (text/plain, inline)]

Marius Bakke writes:

> Pierre Langlois <pierre.langlois <at> gmx.com> writes:
>
>> Hello Guix!
>>
>> Pierre Langlois writes:
>>
>>> Hi Jakub,
>>>
>>> Jakub Kądziołka writes:
>>>
>>
>> (snip)
>>
>>>> Pierre,
>>>>
>>>> thanks for your patch! I was working on a similar change before, but
>>>> when I tried it, it failed even earlier in the bootstrap chain. It
>>>> might've been QEMU weirdness, though, a la #42448.
>>>>
>>>>> @@ -612,9 +613,10 @@ jemalloc = \"" jemalloc "/lib/libjemalloc_pic.a" "\"
>>>>>                         (cargo-out (assoc-ref outputs "cargo")))
>>>>>                     (for-each
>>>>>                       (lambda (file) (delete-manifest-file out file))
>>>>> -                     '("install.log"
>>>>> +                     `("install.log"
>>>>>                         "manifest-rust-docs"
>>>>> -                       "manifest-rust-std-x86_64-unknown-linux-gnu"
>>>>> +                       ,,(string-append "manifest-rust-std-"
>>>>> +                                        (nix-system->gnu-triplet-for-rust))
>>>>>                         "manifest-rustc"))
>>>>>                     (for-each
>>>>>                       (lambda (file) (delete-manifest-file cargo-out file))
>>>>
>>>> If I understand the code correctly, this quasiquote is unnecessary, as
>>>> the host-side code will evaluate to a string that can be inserted as-is,
>>>> without another unquote on the build side.
>>>>
>>>> Fixing this would mean that the patch can go on master, since it would
>>>> now only trigger rebuilds on architectures that are already broken.
>>>
>>> Ooooh yeah, that'll be much better, I've attached a patch that does just
>>> that. I can confirm it doesn't trigger a full rebuild, nice!
>>>
>>
>> I've just rebased this patch and fixed a little formatting issue. I've
>> made sure, it doesn't trigger any rust rebuilds on x86.
>>
>> Is this OK to commit?
>
> LGTM!  Thanks a lot for tackling this long-standing issue.

Thanks! Pushed as 2bab532fdfaa54a085abc0ac7fc2c859ee31f640.

>
> Were you able to build all the way to the latest Rust?

No not yet :-/, I got it building up to 1.26, having to fix most steps
with tests failing here and there. In the end I ran out of steam trying
to build 1.27 and thought we'd be better off waiting until we can
bootstrap from 1.29. I don't remember exactly what issues I had with
1.27, I've attacked my WIP patch in case somebody is interested
actually!

After that, I gave the patches on https://issues.guix.gnu.org/38110 a
go, however as Danny mentions on the thread, linking consumes too much
memory. I'm using a RockPro64 and it only has 4G of RAM.

Somebody on github https://github.com/thepowersgang/mrustc/issues/138
mentioned we could try building with LLD so I tried that as well (patch
attached) but memory consumption was still too much. That being said, I
might not have had any swap file at the time, I'll try again :-).

Soooo, yeah I don't know what the best approach is. We can try and get
the current chain building and start upstreaming bits of my WIP patch or
try and find a solution for the mrustc memory issue.

Anyway, I haven't given up! :-) I'd really like to see IceCat on aarch64
eventually for the PineBook Pro. Although there are quite a few other
things to fix on that platform first.

Thanks,
Pierre

[signature.asc (application/pgp-signature, inline)]

[0001-wip.patch (text/x-patch, attachment)]

[0001-mrustc-Use-lld.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.