GNU bug report logs - #29928
[PATCH 0/5] Optimize profile hooks

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 29928 in the body.
You can then email your comments to 29928 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: 宋文武 <iyzsong <at> member.fsf.org>
To: guix-patches <at> gnu.org
Cc: 宋文武 <iyzsong <at> member.fsf.org>
Subject: [PATCH 0/5] Optimize profile hooks
Date: Mon,  1 Jan 2018 18:33:31 +0800

Hello, these patches make each profile hook run upon its specified interested
inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
install a package without info files won't trigger it.  Thus reduce the chance
and time to rerun them when your profile changed.


One drawback is 'guix package --dry-run' no longer report the derivations of
profile hooks, and the derivation of profile it reports is not the real one.
Addition files will be built when the profiles hooks are run.


Sou Bunnbu (宋文武) (5):
  gexp: Add 'eval-gexp'.
  profiles: info-dir-file: Don't consider unwanted manifest entries.
  guix package: Disable profile hooks on dry runs.
  profiles: Filter out unwanted manifest entries for profile hooks.
  profiles: Sort manifest inputs for profile hooks.

 guix/gexp.scm            |  16 ++++
 guix/profiles.scm        | 188 ++++++++++++++++++++++++++++++++---------------
 guix/scripts/package.scm |   2 +-
 3 files changed, 146 insertions(+), 60 deletions(-)

--
2.13.3

Message #8 received at 29928 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: 宋文武 <iyzsong <at> member.fsf.org>
Cc: 29928 <at> debbugs.gnu.org
Subject: Re: [bug#29928] [PATCH 0/5] Optimize profile hooks
Date: Thu, 11 Jan 2018 23:45:10 +0100

Hello!

宋文武 <iyzsong <at> member.fsf.org> skribis:

> Hello, these patches make each profile hook run upon its specified interested
> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
> install a package without info files won't trigger it.  Thus reduce the chance
> and time to rerun them when your profile changed.

That’s an area in need of improvements, thanks for looking into it!

> One drawback is 'guix package --dry-run' no longer report the derivations of
> profile hooks, and the derivation of profile it reports is not the real one.
> Addition files will be built when the profiles hooks are run.

FWIW I’m not entirely convinced by the approach.  As discussed earlier,
I’d like to experiment with a notion of “build rounds”: the first round
would build a profile without any hooks, the second round would,
depending on what the profile contains, rebuild it with certain hooks.

I don’t have anything to show yet, though…

Ludo’.

Message #11 received at 29928 <at> debbugs.gnu.org (full text, mbox):

From: iyzsong <at> member.fsf.org (宋文武)
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 29928 <at> debbugs.gnu.org
Subject: Re: [bug#29928] [PATCH 0/5] Optimize profile hooks
Date: Fri, 19 Jan 2018 22:42:36 +0800

ludo <at> gnu.org (Ludovic Courtès) writes:

> Hello!
>
> 宋文武 <iyzsong <at> member.fsf.org> skribis:
>
>> Hello, these patches make each profile hook run upon its specified interested
>> inputs, eg: the 'info-dir-file' hook only get inputs with info manuals,
>> install a package without info files won't trigger it.  Thus reduce the chance
>> and time to rerun them when your profile changed.
>
> That’s an area in need of improvements, thanks for looking into it!
>
>> One drawback is 'guix package --dry-run' no longer report the derivations of
>> profile hooks, and the derivation of profile it reports is not the real one.
>> Addition files will be built when the profiles hooks are run.
>
> FWIW I’m not entirely convinced by the approach.

Well.. these patches modify package hooks to:

1. build all manifest inputs first.

2. filter manifest inputs to get interested ones.

3. run hook with its interested inputs.

I think reducing the inputs of hook from the whole manifest to its
interested ones is the only way to avoid unneeded reruns.


> As discussed earlier, I’d like to experiment with a notion of “build
> rounds”: the first round would build a profile without any hooks, the
> second round would, depending on what the profile contains, rebuild it
> with certain hooks.

I think "build rounds" would improve the UI/UX, and by changing inputs
from manifest inputs to a built profile, it would simply the current
implementations of profile hooks, but it won't avoid unneeded reruns
compare to the filtered interested inputs way.

Is my understanding correct?

Message #14 received at 29928 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: iyzsong <at> member.fsf.org (宋文武)
Cc: 29928 <at> debbugs.gnu.org
Subject: Re: [bug#29928] [PATCH 0/5] Optimize profile hooks
Date: Fri, 19 Jan 2018 17:04:38 +0100

Hi,

iyzsong <at> member.fsf.org (宋文武) skribis:

> ludo <at> gnu.org (Ludovic Courtès) writes:

[...]

>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>> profile hooks, and the derivation of profile it reports is not the real one.
>>> Addition files will be built when the profiles hooks are run.
>>
>> FWIW I’m not entirely convinced by the approach.
>
> Well.. these patches modify package hooks to:
>
> 1. build all manifest inputs first.
>
> 2. filter manifest inputs to get interested ones.
>
> 3. run hook with its interested inputs.
>
> I think reducing the inputs of hook from the whole manifest to its
> interested ones is the only way to avoid unneeded reruns.

Agreed.

>> As discussed earlier, I’d like to experiment with a notion of “build
>> rounds”: the first round would build a profile without any hooks, the
>> second round would, depending on what the profile contains, rebuild it
>> with certain hooks.
>
> I think "build rounds" would improve the UI/UX, and by changing inputs
> from manifest inputs to a built profile, it would simply the current
> implementations of profile hooks, but it won't avoid unneeded reruns
> compare to the filtered interested inputs way.
>
> Is my understanding correct?

A “build round” is something that computes derivations based on the
output of previously-built derivations.  So it’s just another way to
structure steps 1/2/3 above; it should not reduce expressivity.

Build rounds would allow us to ensure that ‘build-derivations’ is not
called right in the middle of Guix, and is instead always under the
control of the “top level”.

Ludo’.

Message #17 received at 29928 <at> debbugs.gnu.org (full text, mbox):

From: iyzsong <at> member.fsf.org (宋文武)
To: ludo <at> gnu.org (Ludovic Courtès)
Cc: 29928 <at> debbugs.gnu.org
Subject: Re: [bug#29928] [PATCH 0/5] Optimize profile hooks
Date: Sat, 20 Jan 2018 20:52:26 +0800

ludo <at> gnu.org (Ludovic Courtès) writes:

> Hi,
>
> iyzsong <at> member.fsf.org (宋文武) skribis:
>
>> ludo <at> gnu.org (Ludovic Courtès) writes:
>
> [...]
>
>>>> One drawback is 'guix package --dry-run' no longer report the derivations of
>>>> profile hooks, and the derivation of profile it reports is not the real one.
>>>> Addition files will be built when the profiles hooks are run.
>>>
>>> FWIW I’m not entirely convinced by the approach.
>>
>> Well.. these patches modify package hooks to:
>>
>> 1. build all manifest inputs first.
>>
>> 2. filter manifest inputs to get interested ones.
>>
>> 3. run hook with its interested inputs.
>>
>> I think reducing the inputs of hook from the whole manifest to its
>> interested ones is the only way to avoid unneeded reruns.
>
> Agreed.
>
>>> As discussed earlier, I’d like to experiment with a notion of “build
>>> rounds”: the first round would build a profile without any hooks, the
>>> second round would, depending on what the profile contains, rebuild it
>>> with certain hooks.
>>
>> I think "build rounds" would improve the UI/UX, and by changing inputs
>> from manifest inputs to a built profile, it would simply the current
>> implementations of profile hooks, but it won't avoid unneeded reruns
>> compare to the filtered interested inputs way.
>>
>> Is my understanding correct?
>
> A “build round” is something that computes derivations based on the
> output of previously-built derivations.  So it’s just another way to
> structure steps 1/2/3 above; it should not reduce expressivity.
>
> Build rounds would allow us to ensure that ‘build-derivations’ is not
> called right in the middle of Guix, and is instead always under the
> control of the “top level”.
>

I see, looking forward it, thanks!

Message #22 received at 29928 <at> debbugs.gnu.org (full text, mbox):

From: Leo Prikler <leo.prikler <at> student.tugraz.at>
To: control <at> debbugs.gnu.org, 29928 <at> debbugs.gnu.org
Cc: 宋文武 <iyzsong <at> member.fsf.org>
Subject: Re: [PATCH 0/5] Optimize profile hooks
Date: Tue, 11 May 2021 15:34:55 +0200

merge 29928 29926
merge 29928 29927
thanks

This series (29928 29926 29927 29925 29929 29930) has by now slept on
the mailing list for more than three years.
Should we still try to merge it?  If so, could you send an updated
version, that is not spread across six threads?

Thanks,
Leo

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.