GNU bug report logs - #45020
[PATCH 0/2] image: Add system field.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 45020 in the body.
You can then email your comments to 45020 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: guix-patches <at> gnu.org
Cc: dannym <at> scratchpost.org, Mathieu Othacehe <othacehe <at> gnu.org>
Subject: [PATCH 0/2] image: Add system field.
Date: Thu,  3 Dec 2020 11:53:51 +0100

Hello,

Here's a small patchset to improve the creation of disk-images on non-Intel
systems. Currently, when selecting "arm32-raw" or "arm64-raw" image types,
"guix system" will try to cross-compile to the relevant architectures,
regardless of the current system architecture.

This adds a "system" field to the image definition that indicates the
appropriate system. Then, if we are already running on this system,
"system-image" will build the image natively instead of using
cross-compilation. The image type "raw" is also renamed to "efi-raw" which is
more accurate.

Finally, as discussed with Danny on IRC, it could make sense to change the
default image type depending on the current system: efi-raw on x86_64-linux
and i686-linux, arm32-raw on armhf-linux and so on.

WDYT?

Thanks,

Mathieu

Mathieu Othacehe (2):
  image: Add system field.
  image: Rename "raw" image-type to "efi-raw".

 doc/guix.texi           | 10 +++++-----
 gnu/image.scm           |  3 +++
 gnu/system/image.scm    | 18 ++++++++++++++----
 guix/scripts/system.scm |  2 +-
 4 files changed, 23 insertions(+), 10 deletions(-)

-- 
2.29.2

Message #12 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>, 45020 <at> debbugs.gnu.org
Cc: dannym <at> scratchpost.org, Mathieu Othacehe <othacehe <at> gnu.org>
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Thu, 03 Dec 2020 14:11:17 +0100

Hi Mathieu,

On Thu, 03 Dec 2020 at 11:53, Mathieu Othacehe <othacehe <at> gnu.org> wrote:

> Finally, as discussed with Danny on IRC, it could make sense to change the
> default image type depending on the current system: efi-raw on x86_64-linux
> and i686-linux, arm32-raw on armhf-linux and so on.

The “less astonishment” default sounds good to me. :-)


Cheers,
simon

Message #15 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Wed, 9 Dec 2020 10:25:56 +0200

[Message part 1 (text/plain, inline)]

On Thu, Dec 03, 2020 at 11:53:51AM +0100, Mathieu Othacehe wrote:
> Hello,
> 
> Here's a small patchset to improve the creation of disk-images on non-Intel
> systems. Currently, when selecting "arm32-raw" or "arm64-raw" image types,
> "guix system" will try to cross-compile to the relevant architectures,
> regardless of the current system architecture.
> 
> This adds a "system" field to the image definition that indicates the
> appropriate system. Then, if we are already running on this system,
> "system-image" will build the image natively instead of using
> cross-compilation. The image type "raw" is also renamed to "efi-raw" which is
> more accurate.
> 
> Finally, as discussed with Danny on IRC, it could make sense to change the
> default image type depending on the current system: efi-raw on x86_64-linux
> and i686-linux, arm32-raw on armhf-linux and so on.
> 
> WDYT?

I'm not sure about i686-linux being an efi-raw type. I always assumed
they were all not EFI.

> Thanks,
> 
> Mathieu
> 
> Mathieu Othacehe (2):
>   image: Add system field.
>   image: Rename "raw" image-type to "efi-raw".
> 
>  doc/guix.texi           | 10 +++++-----
>  gnu/image.scm           |  3 +++
>  gnu/system/image.scm    | 18 ++++++++++++++----
>  guix/scripts/system.scm |  2 +-
>  4 files changed, 23 insertions(+), 10 deletions(-)
> 
> -- 
> 2.29.2
> 
> 
> 
> 

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #18 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Efraim Flashner <efraim <at> flashner.co.il>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Wed, 09 Dec 2020 11:15:32 +0100

Hello Efraim,

> I'm not sure about i686-linux being an efi-raw type. I always assumed
> they were all not EFI.

The "efi-raw" image type is actually an hybrid image type supporting EFI
and BIOS legacy boot methods.

Maybe we should then rename "raw" to "bios-efi-raw" or
"bios-efi-hybrid-raw", what do you think?

Thanks,

Mathieu

Message #21 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Efraim Flashner <efraim <at> flashner.co.il>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Wed, 9 Dec 2020 12:27:41 +0200

[Message part 1 (text/plain, inline)]

On Wed, Dec 09, 2020 at 11:15:32AM +0100, Mathieu Othacehe wrote:
> 
> Hello Efraim,
> 
> > I'm not sure about i686-linux being an efi-raw type. I always assumed
> > they were all not EFI.
> 
> The "efi-raw" image type is actually an hybrid image type supporting EFI
> and BIOS legacy boot methods.
> 
> Maybe we should then rename "raw" to "bios-efi-raw" or
> "bios-efi-hybrid-raw", what do you think?

My hope is to eventually produce an image for mips64el again, which
would need an ext2 formatted /boot partition. I'm hoping to make it not
too hard to hack that in in the future :)

Perhaps it makes sense to name them according to the architecture more?
we have arm32-raw and aarch64-raw, how about i686-raw and x86_64-raw?
Make it clear exactly what architecures they target. Then we can
continue with the hybrid bios/efi images anyway.

> Thanks,
> 
> Mathieu

-- 
Efraim Flashner   <efraim <at> flashner.co.il>   אפרים פלשנר
GPG key = A28B F40C 3E55 1372 662D  14F7 41AA E7DC CA3D 8351
Confidentiality cannot be guaranteed on emails sent or received unencrypted

[signature.asc (application/pgp-signature, inline)]

Message #24 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Fri, 11 Dec 2020 17:50:48 +0100

Hi!

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

> Here's a small patchset to improve the creation of disk-images on non-Intel
> systems. Currently, when selecting "arm32-raw" or "arm64-raw" image types,
> "guix system" will try to cross-compile to the relevant architectures,
> regardless of the current system architecture.
>
> This adds a "system" field to the image definition that indicates the
> appropriate system. Then, if we are already running on this system,
> "system-image" will build the image natively instead of using
> cross-compilation. The image type "raw" is also renamed to "efi-raw" which is
> more accurate.
>
> Finally, as discussed with Danny on IRC, it could make sense to change the
> default image type depending on the current system: efi-raw on x86_64-linux
> and i686-linux, arm32-raw on armhf-linux and so on.

I understand the need for an easier way to create images.  However, I
feel like <image> is the wrong place for ‘system’ and ‘target’: the
image format, conceptually, has nothing to do with whether we’re
cross-compiling, compiling for a specific system, etc.

It also seems wrong to me that ‘--image-type’ would, in some cases (but
not all?), override ‘-s’ and ‘--target’.

I feel like we’re missing an abstraction that would build on top of
images, but I’m not sure what that would look like.

Thoughts?

Ludo’.

Message #27 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Sat, 12 Dec 2020 09:30:54 +0100

Hey,

> I understand the need for an easier way to create images.  However, I
> feel like <image> is the wrong place for ‘system’ and ‘target’: the
> image format, conceptually, has nothing to do with whether we’re
> cross-compiling, compiling for a specific system, etc.

On the one hand, I agree that adding "system" and "target" to <image>,
so that they can override the corresponding arguments doesn't feel
nice. On the other hand, I think that dealing with system/target is too
low level for most users.

When using Yocto, Buildroot or even OpenWrt, you say "build me an image
for that board/machine" and not, "build me an image for that board by
cross-compiling to this mysterious triplet".

If the user selects the image type "pine64" or "novena", it's obvious
that the image has to be built for ARM, so I think it makes sense to
hardcode it somewhere. The <image> record might not be the right
location for that information but I cannot think of another one.

Maybe someone else?

Thanks,

Mathieu

Message #30 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>, Ludovic Courtès
 <ludo <at> gnu.org>
Cc: 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Sat, 12 Dec 2020 13:34:14 +0100

Hi,

On Sat, 12 Dec 2020 at 09:30, Mathieu Othacehe <othacehe <at> gnu.org> wrote:

> When using Yocto, Buildroot or even OpenWrt, you say "build me an image
> for that board/machine" and not, "build me an image for that board by
> cross-compiling to this mysterious triplet".

I confirm that the triplet is still mysterious to me.  Since I do not do
that often, each time I am trying, I need to browse the doc, when I am
not asking again and again on IRC.

Therefore, I do not know if the record is the correct abstraction but
somehow a mapping helper is welcome. :-) Maybe via a command-line option
displaying the “board“ and the “triplet”.

All the best,
simon

Message #33 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Sat, 12 Dec 2020 18:51:21 +0100

Hi!

Mathieu Othacehe <othacehe <at> gnu.org> skribis:

>> I understand the need for an easier way to create images.  However, I
>> feel like <image> is the wrong place for ‘system’ and ‘target’: the
>> image format, conceptually, has nothing to do with whether we’re
>> cross-compiling, compiling for a specific system, etc.
>
> On the one hand, I agree that adding "system" and "target" to <image>,
> so that they can override the corresponding arguments doesn't feel
> nice. On the other hand, I think that dealing with system/target is too
> low level for most users.
>
> When using Yocto, Buildroot or even OpenWrt, you say "build me an image
> for that board/machine" and not, "build me an image for that board by
> cross-compiling to this mysterious triplet".

Agreed; I understand that this is a desirable level of abstraction.

> If the user selects the image type "pine64" or "novena", it's obvious
> that the image has to be built for ARM, so I think it makes sense to
> hardcode it somewhere. The <image> record might not be the right
> location for that information but I cannot think of another one.

OTOH, I might want to cross-build a Novena image from x86_64, or I might
want to build it natively.  Perhaps what could be said is that a Novena
image can either be built natively on armhf-linux, or cross-built to
arm-linux-gnueabihf.  Perhaps we should encode this constraint rather
than a specific ‘system’ or ‘target’?  (I’m thinking out loud…)

Regarding ARM boards, do you think some additional abstraction is needed
to encode cross-cutting concerns that affect not just the partition
layout and choice of a bootloader, but also kernel build options, and
maybe options for some userland packages (are there examples of that,
though?)?

Maybe the best course of action is to add all this info to <image> until
we have a better idea, after all.

I guess I’m contributing more questions that answers.  :-)

Thanks,
Ludo’.

Message #36 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Sun, 13 Dec 2020 15:15:49 +0100

Hi!

zimoun <zimon.toutoune <at> gmail.com> skribis:

> On Sat, 12 Dec 2020 at 09:30, Mathieu Othacehe <othacehe <at> gnu.org> wrote:
>
>> When using Yocto, Buildroot or even OpenWrt, you say "build me an image
>> for that board/machine" and not, "build me an image for that board by
>> cross-compiling to this mysterious triplet".
>
> I confirm that the triplet is still mysterious to me.  Since I do not do
> that often, each time I am trying, I need to browse the doc, when I am
> not asking again and again on IRC.

The triplet is a stringy DSL to designate a CPU architecture, hardware
vendor, and operating system; nowadays people often piggy-back
information to distinguish “OS” from “kernel”, to specify the ABI, etc.
(info "(autoconf) Specifying Target Triplets").

It was designed at a time where things were quite different (nowadays
the “vendor” part is almost always useless), and it’s primarily for
userland software.  It’s well-documented though, no mystery.  ;-)

Now, a triplet does not capture all the things we’re interested in, like
details of the boot-up sequence of the ARM board, preferred bootloader,
Binutils tweaks, etc.

From a Guix System viewpoint, we could define an abstract
architecture/platform/target as something that embodies the info
contained in triplets and more, say:

--8<---------------cut here---------------start------------->8---
;; Description of a platform supported by the GNU system.
(define-record-type* <platform> platform make-platform
  platform?
  (triplet            platform-triplet)            ;"x86_64-linux-gnu"
  (system-type        platform-system-type)        ;"x86_64-linux"
  (linux-architecture platform-linux-architecture) ;"amd64"
  (kernel             platform-kernel)             ;<package>
  (ld.so              platform-ld.so)              ;"ld-linux-x86-64.so.2"
  (gcc                platform-gcc)                ;<package>
  (binutils           platform-binutils)           ;<package>
  (libc               platform-transform-libc))    ;<package>
--8<---------------cut here---------------end--------------->8---

Currently that info is scattered in various pieces in Guix: in base.scm,
cross-base.scm, linux.scm, bootstrap.scm, etc.  Having all that in a
single place would be an improvement.

Of course this is going beyond what was originally discussed in this
thread and I’m not claiming this is the solution to work on right now.
It might be a general direction to follow longer-term, though.

Thoughts?

Ludo’.

Message #39 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Sun, 13 Dec 2020 15:59:13 +0100

[Message part 1 (text/plain, inline)]

Hi,

I want to note that there's a difference between cross-compiling things
and natively compiling things (even if only qemu transparent emulation).

The resulting images between cross vs non-cross could be (and probably are)
different.

So I think there should be a command line parameter to the image script that
specifies whether to cross-compile or not--there's no way around it that I can
see.

Also, this image generation thing is mostly for bootstrapping Guix, so it is
fine if that only supports configurations we actually tested.

Ludo said:

> > However, I
> > feel like <image> is the wrong place for ‘system’ and ‘target’: the
> > image format, conceptually, has nothing to do with whether we’re
> > cross-compiling, compiling for a specific system, etc.  

It depends on what you mean.  How the word "image format" is colloquially used
in the VM world, it very much has to with what guest system (and even which
emulator) this image is for, and that's not at all variable.

But I agree that there are other things that could be variable per image target
system, like the kernel version that actually works, the u-boot that actually
works, the partition layout that actually works, the initrd modules, weird
system packages and/or activation scripts that are required for booting etc.

See buildroot.

Mathieu said:

> On the one hand, I agree that adding "system" and "target" to <image>,
> so that they can override the corresponding arguments doesn't feel
> nice. On the other hand, I think that dealing with system/target is too
> low level for most users.

I agree.  Also, I want to stress that if we do this kind of image generation,
it has to be for Guix images we actually tested.  So the general case with
specifying a random system and target we never saw before cannot be supported
anyway (and will likely not work), especially since bootloaders are anything
but portable in general.

> When using Yocto, Buildroot or even OpenWrt, you say "build me an image
> for that board/machine" and not, "build me an image for that board by
> cross-compiling to this mysterious triplet".

I agree that we should not ask the user to specify a triplet to build a guix
system image.  It's obvious what the triplet is per image, since we tested
it anyway (riiiight?).

> If the user selects the image type "pine64" or "novena", it's obvious
> that the image has to be built for ARM, so I think it makes sense to
> hardcode it somewhere. 

I agree.

[Message part 2 (application/pgp-signature, inline)]

Message #42 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: dannym <at> scratchpost.org, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Tue, 15 Dec 2020 10:58:51 +0100

Hey,

> OTOH, I might want to cross-build a Novena image from x86_64, or I might
> want to build it natively.  Perhaps what could be said is that a Novena
> image can either be built natively on armhf-linux, or cross-built to
> arm-linux-gnueabihf.  Perhaps we should encode this constraint rather
> than a specific ‘system’ or ‘target’?  (I’m thinking out loud…)

Maybe the next step would be to define a <machine> record that encodes
the "system" and "target" constraints for a specific board/machine. The
kernel build options and userland packages options you are mentioning
above could also be part of this record.

As Danny is proposing, we could also have a "--native" argument to "guix
system" that would force native build instead of cross-compiling.

> Regarding ARM boards, do you think some additional abstraction is needed
> to encode cross-cutting concerns that affect not just the partition
> layout and choice of a bootloader, but also kernel build options, and
> maybe options for some userland packages (are there examples of that,
> though?)?
>
> Maybe the best course of action is to add all this info to <image> until
> we have a better idea, after all.

Sure, I agree.

> I guess I’m contributing more questions that answers.  :-)

It helps a lot anyway :)

Thanks,

Mathieu

Message #45 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Tue, 15 Dec 2020 15:11:33 +0100

Hi,

On Sun, 13 Dec 2020 at 15:15, Ludovic Courtès <ludo <at> gnu.org> wrote:

> It was designed at a time where things were quite different (nowadays
> the “vendor” part is almost always useless), and it’s primarily for
> userland software.  It’s well-documented though, no mystery.  ;-)

It is not because it is well-documented that I do not feel confused or
it puzzles me or how difficult it is to understand or know about,
especially when the definition appears now a bit awkward.  Ah it is the
meaning from the dictionary of mystery. ;-)

Kidding aside, thanks for the explanations.  I have bookmarked them.


> --8<---------------cut here---------------start------------->8---
> ;; Description of a platform supported by the GNU system.
> (define-record-type* <platform> platform make-platform
>   platform?
>   (triplet            platform-triplet)            ;"x86_64-linux-gnu"    
>   (system-type        platform-system-type)        ;"x86_64-linux"        
>   (linux-architecture platform-linux-architecture) ;"amd64"               
>   (kernel             platform-kernel)             ;<package>             
>   (ld.so              platform-ld.so)              ;"ld-linux-x86-64.so.2"
>   (gcc                platform-gcc)                ;<package>
>   (binutils           platform-binutils)           ;<package>
>   (libc               platform-transform-libc))    ;<package>   
> --8<---------------cut here---------------end--------------->8---

Naively and what confuse me is the redundancy of the information.  For
example, is it possible to do something else than “gnu”?  Or when one
thing is fixed, other parameters are also fixed, for instance does it
make sense

        "x86_64-linux-gnu"    
        "i686-hurd"        
        "arm"               
        "ld-hurd-arm.so.2"

?

Obviously, the plumbing where it is not a “mystery” for some user is
necessary.  But for user like me, the interface should be simple.


> Currently that info is scattered in various pieces in Guix: in base.scm,
> cross-base.scm, linux.scm, bootstrap.scm, etc.  Having all that in a
> single place would be an improvement.

Yes.

> Of course this is going beyond what was originally discussed in this
> thread and I’m not claiming this is the solution to work on right now.
> It might be a general direction to follow longer-term, though.

I agree.  Aside my usual drift. ;-)


All the best,
simon

Message #48 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: zimoun <zimon.toutoune <at> gmail.com>
Cc: Mathieu Othacehe <othacehe <at> gnu.org>, 45020 <at> debbugs.gnu.org
Subject: Re: [bug#45020] [PATCH 0/2] image: Add system field.
Date: Tue, 15 Dec 2020 22:56:15 +0100

Hi,

zimoun <zimon.toutoune <at> gmail.com> skribis:

>> --8<---------------cut here---------------start------------->8---
>> ;; Description of a platform supported by the GNU system.
>> (define-record-type* <platform> platform make-platform
>>   platform?
>>   (triplet            platform-triplet)            ;"x86_64-linux-gnu"    
>>   (system-type        platform-system-type)        ;"x86_64-linux"        
>>   (linux-architecture platform-linux-architecture) ;"amd64"               
>>   (kernel             platform-kernel)             ;<package>             
>>   (ld.so              platform-ld.so)              ;"ld-linux-x86-64.so.2"
>>   (gcc                platform-gcc)                ;<package>
>>   (binutils           platform-binutils)           ;<package>
>>   (libc               platform-transform-libc))    ;<package>   
>> --8<---------------cut here---------------end--------------->8---
>
> Naively and what confuse me is the redundancy of the information.  For
> example, is it possible to do something else than “gnu”?  Or when one
> thing is fixed, other parameters are also fixed, for instance does it
> make sense
>
>         "x86_64-linux-gnu"    
>         "i686-hurd"        
>         "arm"               
>         "ld-hurd-arm.so.2"
>
> ?

For a given platform, say “GNU/Hurd on i586”, all the parameters are
fixed, with some degrees of liberty on the toolchain, though.

However, currently that information is scattered across different
places, so the goal here would be to gather it all in one place, which
should facilitate porting to new platforms.

Ludo’.

Message #51 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Mathieu Othacehe <othacehe <at> gnu.org>
Cc: dannym <at> scratchpost.org, Ludovic Courtès <ludo <at> gnu.org>,
 45020 <at> debbugs.gnu.org
Subject: Re: bug#45020: [PATCH 0/2] image: Add system field.
Date: Thu, 15 Jul 2021 22:04:39 -0400

Hello,

Mathieu Othacehe <othacehe <at> gnu.org> writes:

> Hey,
>
>> OTOH, I might want to cross-build a Novena image from x86_64, or I might
>> want to build it natively.  Perhaps what could be said is that a Novena
>> image can either be built natively on armhf-linux, or cross-built to
>> arm-linux-gnueabihf.  Perhaps we should encode this constraint rather
>> than a specific ‘system’ or ‘target’?  (I’m thinking out loud…)
>
> Maybe the next step would be to define a <machine> record that encodes
> the "system" and "target" constraints for a specific board/machine. The
> kernel build options and userland packages options you are mentioning
> above could also be part of this record.
>
> As Danny is proposing, we could also have a "--native" argument to "guix
> system" that would force native build instead of cross-compiling.
>
>> Regarding ARM boards, do you think some additional abstraction is needed
>> to encode cross-cutting concerns that affect not just the partition
>> layout and choice of a bootloader, but also kernel build options, and
>> maybe options for some userland packages (are there examples of that,
>> though?)?
>>
>> Maybe the best course of action is to add all this info to <image> until
>> we have a better idea, after all.
>
> Sure, I agree.
>
>> I guess I’m contributing more questions that answers.  :-)
>
> It helps a lot anyway :)

I see this hasn't landed to the repo yet.  Are you still refining it, or
has it fallen into cracks?

Just checking :-).

Thank you,

Maxim

Message #54 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: Vagrant Cascadian <vagrant <at> debian.org>, dannym <at> scratchpost.org,
 Ludovic Courtès <ludo <at> gnu.org>, 45020 <at> debbugs.gnu.org
Subject: Re: bug#45020: [PATCH 0/2] image: Add system field.
Date: Mon, 30 Aug 2021 18:24:27 +0200

[Message part 1 (text/plain, inline)]

Hey,

Here's a patchset based on Ludo suggestion of introduction a platform
record. It is for now limited to system, target, and linux-architecture
fields but we could extend it to add the kernel, gcc, ... fields when
needed.

WDYT?

Thanks,

Mathieu

[0001-gnu-Add-platform-support.patch (text/x-patch, inline)]

From 711861495093a3b52aaf5469faf7f4820dfaa911 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Mon, 30 Aug 2021 17:46:05 +0200
Subject: [PATCH 1/2] gnu: Add platform support.

* gnu/platform.scm: New file.
* gnu/platforms/arm.scm: Ditto.
* gnu/platforms/hurd.scm: Ditto.
* gnu/local.mk (GNU_SYSTEM_MODULES): Add them.
---
 gnu/local.mk           |  4 ++++
 gnu/platform.scm       | 38 ++++++++++++++++++++++++++++++++++++++
 gnu/platforms/arm.scm  | 36 ++++++++++++++++++++++++++++++++++++
 gnu/platforms/hurd.scm | 28 ++++++++++++++++++++++++++++
 4 files changed, 106 insertions(+)
 create mode 100644 gnu/platform.scm
 create mode 100644 gnu/platforms/arm.scm
 create mode 100644 gnu/platforms/hurd.scm

diff --git a/gnu/local.mk b/gnu/local.mk
index 5e4d9518bf..4c2efdf504 100644
--- a/gnu/local.mk
+++ b/gnu/local.mk
@@ -73,6 +73,7 @@ GNU_SYSTEM_MODULES =				\
   %D%/bootloader/depthcharge.scm                \
   %D%/ci.scm					\
   %D%/image.scm					\
+  %D%/platform.scm				\
   %D%/packages.scm				\
   %D%/packages/abduco.scm			\
   %D%/packages/abiword.scm			\
@@ -601,6 +602,9 @@ GNU_SYSTEM_MODULES =				\
   %D%/packages/zile.scm				\
   %D%/packages/zwave.scm			\
 						\
+  %D%/platforms/arm.scm		                \
+  %D%/platforms/hurd.scm	                \
+						\
   %D%/services.scm				\
   %D%/services/admin.scm			\
   %D%/services/audio.scm                        \
diff --git a/gnu/platform.scm b/gnu/platform.scm
new file mode 100644
index 0000000000..bb6519c71a
--- /dev/null
+++ b/gnu/platform.scm
@@ -0,0 +1,38 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Mathieu Othacehe <othacehe <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu platform)
+  #:use-module (guix records)
+  #:export (platform
+            platform?
+            platform-target
+            platform-system
+            platform-linux-architecture))
+
+
+;;;
+;;; Platform record.
+;;;
+
+;; Description of a platform supported by the GNU system.
+(define-record-type* <platform> platform make-platform
+  platform?
+  (target             platform-target)               ;"x86_64-linux-gnu"
+  (system             platform-system)               ;"x86_64-linux"
+  (linux-architecture platform-linux-architecture    ;"amd64"
+                      (default #f)))
diff --git a/gnu/platforms/arm.scm b/gnu/platforms/arm.scm
new file mode 100644
index 0000000000..1e61741a35
--- /dev/null
+++ b/gnu/platforms/arm.scm
@@ -0,0 +1,36 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Mathieu Othacehe <othacehe <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu platforms arm)
+  #:use-module (gnu platform)
+  #:use-module (gnu packages linux)
+  #:use-module (guix records)
+  #:export (armv7-linux
+            aarch64-linux))
+
+(define armv7-linux
+  (platform
+   (target "arm-linux-gnueabihf")
+   (system "armhf-linux")
+   (linux-architecture "arm")))
+
+(define aarch64-linux
+  (platform
+   (target "aarch64-linux-gnu")
+   (system "aarch64-linux")
+   (linux-architecture "arm64")))
diff --git a/gnu/platforms/hurd.scm b/gnu/platforms/hurd.scm
new file mode 100644
index 0000000000..0e5c58fd08
--- /dev/null
+++ b/gnu/platforms/hurd.scm
@@ -0,0 +1,28 @@
+;;; GNU Guix --- Functional package management for GNU
+;;; Copyright © 2021 Mathieu Othacehe <othacehe <at> gnu.org>
+;;;
+;;; This file is part of GNU Guix.
+;;;
+;;; GNU Guix is free software; you can redistribute it and/or modify it
+;;; under the terms of the GNU General Public License as published by
+;;; the Free Software Foundation; either version 3 of the License, or (at
+;;; your option) any later version.
+;;;
+;;; GNU Guix is distributed in the hope that it will be useful, but
+;;; WITHOUT ANY WARRANTY; without even the implied warranty of
+;;; MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+;;; GNU General Public License for more details.
+;;;
+;;; You should have received a copy of the GNU General Public License
+;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
+
+(define-module (gnu platforms hurd)
+  #:use-module (gnu platform)
+  #:use-module (gnu packages linux)
+  #:use-module (guix records)
+  #:export (hurd))
+
+(define hurd
+  (platform
+   (target "i586-pc-gnu")
+   (system "i586-gnu")))
-- 
2.32.0

[0002-image-Add-platform-field.patch (text/x-patch, inline)]

From 8a5c6d75cb2bd9c26ec535229b52a2ab1b86c4b4 Mon Sep 17 00:00:00 2001
From: Mathieu Othacehe <othacehe <at> gnu.org>
Date: Mon, 30 Aug 2021 17:48:10 +0200
Subject: [PATCH 2/2] image: Add platform field.

Fixes: <https://issues.guix.gnu.org/48327>.

* gnu/image.scm (<image>)[target]: Remove this field and replace it with ...
[platform]: ... this new field.
image-target): Remove it.
(image-platform, os+platform->image): New procedures.
* gnu/system/image.scm (arm32-disk-image, arm64-disk-image, arm32-image-type,
arm64-image-type): Remove them.
(raw-with-offset-disk-image, raw-with-offset-image-type): New procedures.
(system-image): Adapt it to use the image platform field.
* gnu/system/images/hurd.scm (hurd-disk-image): Remove the target field.
(hurd-barebones-disk-image, hurd-barebones-qcow2-image): Use
os+platform->image procedure.
* gnu/system/images/novena.scm (novena-image-type,
novena-barebones-raw-image): Use the os+platform->image.
* gnu/system/images/pine64.scm (pine64-image-type,
pine64-barebones-raw-image): Use the os+platform->image.
* gnu/system/images/pinebook-pro.scm (pinebook-pro-image-type,
pinebook-pro-barebones-raw-image): Use the os+platform->image.
* gnu/system/images/rock64.scm (rock64-image-type,
rock64-barebones-raw-image): Use the os+platform->image.
* guix/scripts/system.scm (process-action): Use the image platform field.
---
 gnu/image.scm                      | 13 ++++++--
 gnu/system/image.scm               | 51 ++++++++++++++++++------------
 gnu/system/images/hurd.scm         |  8 +++--
 gnu/system/images/novena.scm       |  6 ++--
 gnu/system/images/pine64.scm       |  6 ++--
 gnu/system/images/pinebook-pro.scm |  6 ++--
 gnu/system/images/rock64.scm       |  8 +++--
 guix/scripts/system.scm            |  5 ++-
 8 files changed, 66 insertions(+), 37 deletions(-)

diff --git a/gnu/image.scm b/gnu/image.scm
index 75d489490d..2381efa208 100644
--- a/gnu/image.scm
+++ b/gnu/image.scm
@@ -17,6 +17,7 @@
 ;;; along with GNU Guix.  If not, see <http://www.gnu.org/licenses/>.
 
 (define-module (gnu image)
+  #:use-module (gnu platform)
   #:use-module (guix records)
   #:export (partition
             partition?
@@ -34,7 +35,7 @@
             image?
             image-name
             image-format
-            image-target
+            image-platform
             image-size
             image-operating-system
             image-partitions
@@ -47,7 +48,8 @@
             image-type-name
             image-type-constructor
 
-            os->image))
+            os->image
+            os+platform->image))
 
 
 ;;;
@@ -78,7 +80,7 @@
   (name               image-name ;symbol
                       (default #f))
   (format             image-format) ;symbol
-  (target             image-target
+  (platform           image-platform ;<platform>
                       (default #f))
   (size               image-size  ;size in bytes as integer
                       (default 'guess))
@@ -112,3 +114,8 @@
 (define* (os->image os #:key type)
   (let ((constructor (image-type-constructor type)))
     (constructor os)))
+
+(define* (os+platform->image os platform #:key type)
+  (image
+   (inherit (os->image os #:type type))
+   (platform platform)))
diff --git a/gnu/system/image.scm b/gnu/system/image.scm
index 1012fa6158..7a807b8226 100644
--- a/gnu/system/image.scm
+++ b/gnu/system/image.scm
@@ -31,6 +31,7 @@
   #:use-module (gnu bootloader)
   #:use-module (gnu bootloader grub)
   #:use-module (gnu image)
+  #:use-module (gnu platform)
   #:use-module (gnu services)
   #:use-module (gnu services base)
   #:use-module (gnu system)
@@ -66,16 +67,14 @@
 
             efi-disk-image
             iso9660-image
-            arm32-disk-image
-            arm64-disk-image
+            raw-with-offset-disk-image
 
             image-with-os
             efi-raw-image-type
             qcow2-image-type
             iso-image-type
             uncompressed-iso-image-type
-            arm32-image-type
-            arm64-image-type
+            raw-with-offset-image-type
 
             image-with-label
             system-image
@@ -128,10 +127,9 @@
            (label "GUIX_IMAGE")
            (flags '(boot)))))))
 
-(define* (arm32-disk-image #:optional (offset root-offset))
+(define* (raw-with-offset-disk-image #:optional (offset root-offset))
   (image
    (format 'disk-image)
-   (target "arm-linux-gnueabihf")
    (partitions
     (list (partition
            (inherit root-partition)
@@ -140,11 +138,6 @@
    ;; fails.
    (volatile-root? #f)))
 
-(define* (arm64-disk-image #:optional (offset root-offset))
-  (image
-   (inherit (arm32-disk-image offset))
-   (target "aarch64-linux-gnu")))
-
 
 ;;;
 ;;; Images types.
@@ -186,15 +179,10 @@ set to the given OS."
                   (compression? #f))
                  <>))))
 
-(define arm32-image-type
-  (image-type
-   (name 'arm32-raw)
-   (constructor (cut image-with-os (arm32-disk-image) <>))))
-
-(define arm64-image-type
+(define raw-with-offset-image-type
   (image-type
-   (name 'arm64-raw)
-   (constructor (cut image-with-os (arm64-disk-image) <>))))
+   (name 'raw-with-offset)
+   (constructor (cut image-with-os (raw-with-offset-disk-image) <>))))
 
 
 ;;
@@ -615,7 +603,30 @@ it can be used for bootloading."
   "Return the derivation of IMAGE.  It can be a raw disk-image or an ISO9660
 image, depending on IMAGE format."
   (define substitutable? (image-substitutable? image))
-  (define target (image-target image))
+  (define platform (image-platform image))
+
+  ;; The image platform definition may provide the appropriate "system"
+  ;; architecture for the image.  If we are already running on this system,
+  ;; the image can be built natively.  If we are running on a different
+  ;; system, then we need to cross-compile, using the "target" provided by the
+  ;; image definition.
+  (define system (and=> platform platform-system))
+  (define target (cond
+                  ;; No defined platform, let's use the user defined
+                  ;; system/target parameters.
+                  ((not platform)
+                   (%current-target-system))
+                  ;; The current system is the same as the platform system, no
+                  ;; need to cross-compile.
+                  ((and system
+                        (string=? system (%current-system)))
+                   #f)
+                  ;; If there is a user defined target let's override the
+                  ;; platform target. Otherwise, we can cross-compile to the
+                  ;; platform target.
+                  (else
+                   (or (%current-target-system)
+                       (and=> platform platform-target)))))
 
   (with-parameters ((%current-target-system target))
     (let* ((os (operating-system-for-image image))
diff --git a/gnu/system/images/hurd.scm b/gnu/system/images/hurd.scm
index fc2dbe3209..77f7ff5e2b 100644
--- a/gnu/system/images/hurd.scm
+++ b/gnu/system/images/hurd.scm
@@ -23,6 +23,7 @@
   #:use-module (gnu bootloader grub)
   #:use-module (gnu image)
   #:use-module (gnu packages ssh)
+  #:use-module (gnu platforms hurd)
   #:use-module (gnu services)
   #:use-module (gnu services ssh)
   #:use-module (gnu system)
@@ -75,7 +76,6 @@
 (define hurd-disk-image
   (image
    (format 'disk-image)
-   (target "i586-pc-gnu")
    (partitions
     (list (partition
            (size 'guess)
@@ -103,13 +103,15 @@
 (define hurd-barebones-disk-image
   (image
    (inherit
-    (os->image hurd-barebones-os #:type hurd-image-type))
+    (os+platform->image hurd-barebones-os hurd
+                        #:type hurd-image-type))
    (name 'hurd-barebones-disk-image)))
 
 (define hurd-barebones-qcow2-image
   (image
    (inherit
-    (os->image hurd-barebones-os #:type hurd-qcow2-image-type))
+    (os+platform->image hurd-barebones-os hurd
+                        #:type hurd-qcow2-image-type))
    (name 'hurd-barebones.qcow2)))
 
 ;; Return the default image.
diff --git a/gnu/system/images/novena.scm b/gnu/system/images/novena.scm
index 63227af509..3ce62fbf3b 100644
--- a/gnu/system/images/novena.scm
+++ b/gnu/system/images/novena.scm
@@ -22,6 +22,7 @@
   #:use-module (gnu bootloader u-boot)
   #:use-module (gnu image)
   #:use-module (gnu packages linux)
+  #:use-module (gnu platforms arm)
   #:use-module (gnu services)
   #:use-module (gnu services base)
   #:use-module (gnu system)
@@ -52,12 +53,13 @@
 (define novena-image-type
   (image-type
    (name 'novena-raw)
-   (constructor (cut image-with-os (arm32-disk-image) <>))))
+   (constructor (cut image-with-os (raw-with-offset-disk-image) <>))))
 
 (define novena-barebones-raw-image
   (image
    (inherit
-    (os->image novena-barebones-os #:type novena-image-type))
+    (os+platform->image novena-barebones-os armv7-linux
+                        #:type novena-image-type))
    (name 'novena-barebones-raw-image)))
 
 ;; Return the default image.
diff --git a/gnu/system/images/pine64.scm b/gnu/system/images/pine64.scm
index 808c71295f..aaec458766 100644
--- a/gnu/system/images/pine64.scm
+++ b/gnu/system/images/pine64.scm
@@ -21,6 +21,7 @@
   #:use-module (gnu bootloader u-boot)
   #:use-module (gnu image)
   #:use-module (gnu packages linux)
+  #:use-module (gnu platforms arm)
   #:use-module (gnu services)
   #:use-module (gnu services base)
   #:use-module (gnu system)
@@ -57,12 +58,13 @@
 (define pine64-image-type
   (image-type
    (name 'pine64-raw)
-   (constructor (cut image-with-os (arm64-disk-image) <>))))
+   (constructor (cut image-with-os (raw-with-offset-disk-image) <>))))
 
 (define pine64-barebones-raw-image
   (image
    (inherit
-    (os->image pine64-barebones-os #:type pine64-image-type))
+    (os+platform->image pine64-barebones-os aarch64-linux
+                        #:type pine64-image-type))
    (name 'pine64-barebones-raw-image)))
 
 ;; Return the default image.
diff --git a/gnu/system/images/pinebook-pro.scm b/gnu/system/images/pinebook-pro.scm
index b6b844cef6..1bfac7a8bb 100644
--- a/gnu/system/images/pinebook-pro.scm
+++ b/gnu/system/images/pinebook-pro.scm
@@ -21,6 +21,7 @@
   #:use-module (gnu bootloader u-boot)
   #:use-module (gnu image)
   #:use-module (gnu packages linux)
+  #:use-module (gnu platforms arm)
   #:use-module (gnu services)
   #:use-module (gnu services base)
   #:use-module (gnu system)
@@ -58,13 +59,14 @@
   (image-type
    (name 'pinebook-pro-raw)
    (constructor (cut image-with-os
-                     (arm64-disk-image (* 9 (expt 2 20))) ;9MiB
+                     (raw-with-offset-disk-image (* 9 (expt 2 20))) ;9MiB
                      <>))))
 
 (define pinebook-pro-barebones-raw-image
   (image
    (inherit
-    (os->image pinebook-pro-barebones-os #:type pinebook-pro-image-type))
+    (os+platform->image pinebook-pro-barebones-os aarch64-linux
+                        #:type pinebook-pro-image-type))
    (name 'pinebook-pro-barebones-raw-image)))
 
 ;; Return the default image.
diff --git a/gnu/system/images/rock64.scm b/gnu/system/images/rock64.scm
index 68d3742adc..d25d55e528 100644
--- a/gnu/system/images/rock64.scm
+++ b/gnu/system/images/rock64.scm
@@ -21,6 +21,7 @@
   #:use-module (gnu bootloader u-boot)
   #:use-module (gnu image)
   #:use-module (gnu packages linux)
+  #:use-module (gnu platforms arm)
   #:use-module (gnu services)
   #:use-module (gnu services base)
   #:use-module (gnu services networking)
@@ -53,12 +54,15 @@
 (define rock64-image-type
   (image-type
    (name 'rock64-raw)
-   (constructor (cut image-with-os (arm64-disk-image (expt 2 24)) <>))))
+   (constructor (cut image-with-os
+                     (raw-with-offset-disk-image (expt 2 24))
+                     <>))))
 
 (define rock64-barebones-raw-image
   (image
    (inherit
-    (os->image rock64-barebones-os #:type rock64-image-type))
+    (os+platform->image rock64-barebones-os aarch64-linux
+                        #:type rock64-image-type))
    (name 'rock64-barebones-raw-image)))
 
 rock64-barebones-raw-image
diff --git a/guix/scripts/system.scm b/guix/scripts/system.scm
index 83bbefd3dc..a98a97e121 100644
--- a/guix/scripts/system.scm
+++ b/guix/scripts/system.scm
@@ -63,6 +63,7 @@
                  (device-module-aliases matching-modules)
   #:use-module (gnu system linux-initrd)
   #:use-module (gnu image)
+  #:use-module (gnu platform)
   #:use-module (gnu system)
   #:use-module (gnu bootloader)
   #:use-module (gnu system file-systems)
@@ -1204,13 +1205,11 @@ resulting from command-line parsing."
                             (base-image (if (operating-system? obj)
                                             (os->image obj
                                                        #:type image-type)
-                                            obj))
-                            (base-target (image-target base-image)))
+                                            obj)))
                         (image
                          (inherit (if label
                                       (image-with-label base-image label)
                                       base-image))
-                         (target (or base-target target))
                          (size image-size)
                          (volatile-root? volatile?))))
          (os          (image-operating-system image))
-- 
2.32.0

Message #57 received at 45020 <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: 45020 <at> debbugs.gnu.org
Cc: Vagrant Cascadian <vagrant <at> debian.org>, dannym <at> scratchpost.org,
 Ludovic Courtès <ludo <at> gnu.org>,
 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: Re: bug#45020: [PATCH 0/2] image: Add system field.
Date: Tue, 05 Oct 2021 08:26:31 +0000

Hey,

> * gnu/platform.scm: New file.
> * gnu/platforms/arm.scm: Ditto.
> * gnu/platforms/hurd.scm: Ditto.
> * gnu/local.mk (GNU_SYSTEM_MODULES): Add them.

I plan to push that one in the next few days.

Thanks,

Mathieu

Message #62 received at 45020-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: 45020-done <at> debbugs.gnu.org
Cc: Vagrant Cascadian <vagrant <at> debian.org>, dannym <at> scratchpost.org,
 Ludovic Courtès <ludo <at> gnu.org>,
 Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Subject: Re: bug#45020: [PATCH 0/2] image: Add system field.
Date: Mon, 11 Oct 2021 12:06:21 +0000

Hey,

> I plan to push that one in the next few days.

Pushed, thanks,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.