GNU bug report logs - #63459
[PATCH] doc: Rewrite the branching strategy.

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 63459 in the body.
You can then email your comments to 63459 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-patches <at> gnu.org
Subject: [PATCH] doc: Rewrite the branching strategy.
Date: Fri, 12 May 2023 08:55:20 +0100

Move away from using staging and core-updates, and make the strategy
independant of branch names.

Keep the 300 dependent threshold for changes to master, as I don't have any
specific reason to change this.

Most importantly, require using guix-patches issues to coordinate merging of
the branches, as I think that'll address the key issues that have shown up
recently where it's been unclear which branch should be merged next.

* doc/contributing.texi (Submitting Patches): Rewrite branching strategy.
---
 doc/contributing.texi | 58 +++++++++++++++++--------------------------
 1 file changed, 23 insertions(+), 35 deletions(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index 7bf350ee0d..c54910e34d 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -1264,41 +1264,29 @@ Submitting Patches
 @c See <https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00933.html>.
 @cindex branching strategy
 @cindex rebuild scheduling strategy
-Depending on the number of dependent packages and thus the amount of
-rebuilding induced, commits go to different branches, along these lines:
-
-@table @asis
-@item 300 dependent packages or less
-@code{master} branch (non-disruptive changes).
-
-@item between 300 and 1,800 dependent packages
-@code{staging} branch (non-disruptive changes).  This branch is intended
-to be merged in @code{master} every 6 weeks or so.  Topical changes
-(e.g., an update of the GNOME stack) can instead go to a specific branch
-(say, @code{gnome-updates}).  This branch is not expected to be
-buildable or usable until late in its development process.
-
-@item more than 1,800 dependent packages
-@code{core-updates} branch (may include major and potentially disruptive
-changes).  This branch is intended to be merged in @code{master} every
-6 months or so.  This branch is not expected to be buildable or usable
-until late in its development process.
-@end table
-
-All these branches are @uref{https://@value{SUBSTITUTE-SERVER-1},
-tracked by our build farm} and merged into @code{master} once
-everything has been successfully built.  This allows us to fix issues
-before they hit users, and to reduce the window during which pre-built
-binaries are not available.
-
-When we decide to start building the @code{staging} or
-@code{core-updates} branches, they will be forked and renamed with the
-suffix @code{-frozen}, at which time only bug fixes may be pushed to the
-frozen branches.  The @code{core-updates} and @code{staging} branches
-will remain open to accept patches for the next cycle.  Please ask on
-the mailing list or IRC if unsure where to place a patch.
-@c TODO: It would be good with badges on the website that tracks these
-@c branches.  Or maybe even a status page.
+Changes to packages with 300 dependent packages or less can be pushed to
+the @code{master} branch.
+
+Larger changes should be first pushed to a branch other than
+@code{master}. This allows for testing and for the build farms to
+process the changes prior to being pushed to the @code{master} branch.
+
+To help coordinate the merging of branches, you must create a new
+guix-patches issue each time you wish to merge a branch. These issues
+indicate the order in which the branches should be merged, so take a
+look at the open issues for merging branches and mark the issue you
+create as blocked by the issue previously at the back of the queue.
+
+Normally branches will be merged in a ``first come, first merged''
+manor, tracked through the guix-patches issues. If you agree a different
+order with those involved, you can track this by updating which issues
+block which other issues. Therefore, to know which branch is at the
+front of the queue, look for the issue which isn't blocked by any other
+branch merges.
+
+Once a branch is at the front of the queue, wait until sufficient time
+has passed for the build farms to have processed the changes, and for
+the necessary testing to have happened.
 
 @item
 @cindex determinism, of build processes

base-commit: 9d05f9a9f538061e1fdc5aedb0748d260fcf20f7
prerequisite-patch-id: ae24e25c683be86ce0b3fa1fde1bd30e3e08e248
-- 
2.39.1

Message #8 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-devel <at> gnu.org
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: [bug#63459] [PATCH] doc: Rewrite the branching strategy.
Date: Fri, 12 May 2023 09:04:40 +0100

[Message part 1 (text/plain, inline)]

Christopher Baines <mail <at> cbaines.net> writes:

> Move away from using staging and core-updates, and make the strategy
> independant of branch names.
>
> Keep the 300 dependent threshold for changes to master, as I don't have any
> specific reason to change this.
>
> Most importantly, require using guix-patches issues to coordinate merging of
> the branches, as I think that'll address the key issues that have shown up
> recently where it's been unclear which branch should be merged next.
>
> * doc/contributing.texi (Submitting Patches): Rewrite branching strategy.
> ---
>  doc/contributing.texi | 58 +++++++++++++++++--------------------------
>  1 file changed, 23 insertions(+), 35 deletions(-)

Following on from the discussion recently about moving away from staging
and core-updates, I've sent a patch to update the branching strategy.

The key thing is obviously to just remove mentions of staging and
core-updates, making the guidance more generic.

However, I've also added some requirements to use guix-patches issues to
track the intentions to merge branches, as I think that'll help address
some of the issues that came up recently with uncertainty around which
branch will be merged next.

I'm also hoping that these issues then can be used to automate the QA
process, triggering the qa-frontpage to automatically start building the
relevant branches.

Thanks,

Chris

[signature.asc (application/pgp-signature, inline)]

Message #11 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Fri, 19 May 2023 15:22:59 +0200

Hello,

Thanks for this initiative!

Christopher Baines <mail <at> cbaines.net> skribis:

> Move away from using staging and core-updates, and make the strategy
> independant of branch names.
>
> Keep the 300 dependent threshold for changes to master, as I don't have any
> specific reason to change this.
>
> Most importantly, require using guix-patches issues to coordinate merging of
> the branches, as I think that'll address the key issues that have shown up
> recently where it's been unclear which branch should be merged next.
>
> * doc/contributing.texi (Submitting Patches): Rewrite branching strategy.

[...]

> +Changes to packages with 300 dependent packages or less can be pushed to
> +the @code{master} branch.
> +
> +Larger changes should be first pushed to a branch other than
> +@code{master}. This allows for testing and for the build farms to
> +process the changes prior to being pushed to the @code{master} branch.

I’d be more specific:

  Larger changes should first be pushed to a topic branch other than
  @code{master}; the set of changes should be consistent---e.g., ``GNOME
  update'', ``NumPy update'', etc.  This allows for testing: the branch
  will automatically show up at
  @indicateurl{https://qa.guix.gnu.org/branch/@var{branch}}, with an
  indication of its build status on various platforms.

“Automatic” is a bit of an overstatement; that sentence probably needs
to be tweaked.  :-)  But I think it’s good to link to the QA platform to
make things more concrete.

> +To help coordinate the merging of branches, you must create a new
> +guix-patches issue each time you wish to merge a branch. These issues
                                                          ^
+ (@pxref{Tracking Bugs and Patches})

> +indicate the order in which the branches should be merged, so take a
> +look at the open issues for merging branches and mark the issue you
> +create as blocked by the issue previously at the back of the queue.

s/blocked/@dfn{blocked}/

Perhaps add a footnote or paren stating how to “block” an issue in
Debbugs?

> +Normally branches will be merged in a ``first come, first merged''
> +manor, tracked through the guix-patches issues. If you agree a different

s/manor/manner/
s/agree a/agree on a/

> +order with those involved, you can track this by updating which issues
> +block which other issues. Therefore, to know which branch is at the
> +front of the queue, look for the issue which isn't blocked by any other
> +branch merges.
> +
> +Once a branch is at the front of the queue, wait until sufficient time
> +has passed for the build farms to have processed the changes, and for
> +the necessary testing to have happened.

This is a bit technical.  How can I know “which branch is at the front
of the queue”?  Even as a seasoned Debbugs users, I’m not sure what I’m
supposed to do here.  Do you think we could provide ready to use
commands (debbugs.el or ‘mumi’) or at least a sequence of steps to
follow?

Last but not least: two spaces after end-of-sentence period please.  :-)

This is mostly about tweaking words; I think this is a great step
forward, very much in line with what was discussed in February at the
Guix Days.  Thank you!

Ludo’.

Message #14 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Tue, 23 May 2023 18:28:18 +0100

[Message part 1 (text/plain, inline)]

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hello,
>
> Thanks for this initiative!
>
> Christopher Baines <mail <at> cbaines.net> skribis:
>
>> Move away from using staging and core-updates, and make the strategy
>> independant of branch names.
>>
>> Keep the 300 dependent threshold for changes to master, as I don't have any
>> specific reason to change this.
>>
>> Most importantly, require using guix-patches issues to coordinate merging of
>> the branches, as I think that'll address the key issues that have shown up
>> recently where it's been unclear which branch should be merged next.
>>
>> * doc/contributing.texi (Submitting Patches): Rewrite branching strategy.
>
> [...]
>
>> +Changes to packages with 300 dependent packages or less can be pushed to
>> +the @code{master} branch.
>> +
>> +Larger changes should be first pushed to a branch other than
>> +@code{master}. This allows for testing and for the build farms to
>> +process the changes prior to being pushed to the @code{master} branch.
>
> I’d be more specific:
>
>   Larger changes should first be pushed to a topic branch other than
>   @code{master}; the set of changes should be consistent---e.g., ``GNOME
>   update'', ``NumPy update'', etc.  This allows for testing: the branch
>   will automatically show up at
>   @indicateurl{https://qa.guix.gnu.org/branch/@var{branch}}, with an
>   indication of its build status on various platforms.
>
> “Automatic” is a bit of an overstatement; that sentence probably needs
> to be tweaked.  :-)  But I think it’s good to link to the QA platform to
> make things more concrete.

That sounds fine to me. Everything apart from starting the builds is
already automatic, and I want to automate that through the issues
described here.

>> +To help coordinate the merging of branches, you must create a new
>> +guix-patches issue each time you wish to merge a branch. These issues
>                                                           ^
> + (@pxref{Tracking Bugs and Patches})
>
>> +indicate the order in which the branches should be merged, so take a
>> +look at the open issues for merging branches and mark the issue you
>> +create as blocked by the issue previously at the back of the queue.
>
> s/blocked/@dfn{blocked}/
>
> Perhaps add a footnote or paren stating how to “block” an issue in
> Debbugs?

Yeah, I'll try and write something.

>> +Normally branches will be merged in a ``first come, first merged''
>> +manor, tracked through the guix-patches issues. If you agree a different
>
> s/manor/manner/
> s/agree a/agree on a/
>
>> +order with those involved, you can track this by updating which issues
>> +block which other issues. Therefore, to know which branch is at the
>> +front of the queue, look for the issue which isn't blocked by any other
>> +branch merges.
>> +
>> +Once a branch is at the front of the queue, wait until sufficient time
>> +has passed for the build farms to have processed the changes, and for
>> +the necessary testing to have happened.
>
> This is a bit technical.  How can I know “which branch is at the front
> of the queue”?  Even as a seasoned Debbugs users, I’m not sure what I’m
> supposed to do here.  Do you think we could provide ready to use
> commands (debbugs.el or ‘mumi’) or at least a sequence of steps to
> follow?

So, I think there's two technical hurdles to overcome here. The first is
identifying the issues for merging branches, maybe for that we can set
out a format for the title of the bug, but I'm very open to
suggestions. Any way of identifying the open issues should be usable
through debbugs.el and mumi.

The second hurdle is the queuing behaviour, which I think the blocking
behaviour is a natural fit for. Maybe the tooling is lacking but I think
that can be addressed.

I want the qa-frontpage to display the queue of branches (and issues) in
a clear way, as well as providing links to make changes (as it does for
marking issues as moreinfo).

> Last but not least: two spaces after end-of-sentence period please.  :-)
>
> This is mostly about tweaking words; I think this is a great step
> forward, very much in line with what was discussed in February at the
> Guix Days.  Thank you!

Great, thanks for taking a look!

Chris

[signature.asc (application/pgp-signature, inline)]

Message #17 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 63459 <at> debbugs.gnu.org
Cc: Christopher Baines <mail <at> cbaines.net>
Subject: [PATCH v2] doc: Move and rewrite the branching strategy.
Date: Wed, 31 May 2023 10:41:12 +0100

Move away from using staging and core-updates, and make the strategy
independant of branch names.

Keep the 300 dependent threshold for changes to master, as I don't have any
specific reason to change this.

Most importantly, require using guix-patches issues to coordinate merging of
the branches, as I think that'll address the key issues that have shown up
recently where it's been unclear which branch should be merged next.

* doc/contributing.texi (Submitting Patches): Move the branching strategy to a
new Managing Patches and Branches section.
(Managing Patches and Branches): New section.
(Commit Policy): Simplify through referencing the new Managing Patches and
Branches section.

Signed-off-by: Christopher Baines <mail <at> cbaines.net>
---
 doc/contributing.texi | 140 +++++++++++++++++++++---------------------
 doc/guix.texi         |  14 ++---
 2 files changed, 76 insertions(+), 78 deletions(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index f692872c04..96b22b7d9a 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -26,7 +26,7 @@ Contributing
 * Packaging Guidelines::        Growing the distribution.
 * Coding Style::                Hygiene of the contributor.
 * Submitting Patches::          Share your work.
-* Tracking Bugs and Patches::   Keeping it all organized.
+* Tracking Bugs and Changes::   Keeping it all organized.
 * Commit Access::               Pushing to the official repository.
 * Updating the Guix Package::   Updating the Guix package definition.
 * Writing Documentation::       Improving documentation in GNU Guix.
@@ -1161,11 +1161,11 @@ Submitting Patches
 at the section on commit access (@pxref{Commit Access}).
 
 This mailing list is backed by a Debbugs instance, which allows us to
-keep track of submissions (@pxref{Tracking Bugs and Patches}).  Each
-message sent to that mailing list gets a new tracking number assigned;
-people can then follow up on the submission by sending email to
-@code{@var{ISSUE_NUMBER}@@debbugs.gnu.org}, where @var{ISSUE_NUMBER} is
-the tracking number (@pxref{Sending a Patch Series}).
+keep track of submissions (@pxref{Tracking Bugs and Changes}).
+Each message sent to that mailing list gets a new tracking number
+assigned; people can then follow up on the submission by sending email
+to @code{@var{ISSUE_NUMBER}@@debbugs.gnu.org}, where @var{ISSUE_NUMBER}
+is the tracking number (@pxref{Sending a Patch Series}).
 
 Please write commit logs in the ChangeLog format (@pxref{Change Logs,,,
 standards, GNU Coding Standards}); you can check the commit history for
@@ -1257,48 +1257,9 @@ Submitting Patches
 the @code{texlive-tiny} package or @code{texlive-union} procedure instead.
 
 @item
-For important changes, check that dependent packages (if applicable) are
-not affected by the change; @code{guix refresh --list-dependent
-@var{package}} will help you do that (@pxref{Invoking guix refresh}).
-
-@c See <https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00933.html>.
-@cindex branching strategy
-@cindex rebuild scheduling strategy
-Depending on the number of dependent packages and thus the amount of
-rebuilding induced, commits go to different branches, along these lines:
-
-@table @asis
-@item 300 dependent packages or less
-@code{master} branch (non-disruptive changes).
-
-@item between 300 and 1,800 dependent packages
-@code{staging} branch (non-disruptive changes).  This branch is intended
-to be merged in @code{master} every 6 weeks or so.  Topical changes
-(e.g., an update of the GNOME stack) can instead go to a specific branch
-(say, @code{gnome-updates}).  This branch is not expected to be
-buildable or usable until late in its development process.
-
-@item more than 1,800 dependent packages
-@code{core-updates} branch (may include major and potentially disruptive
-changes).  This branch is intended to be merged in @code{master} every
-6 months or so.  This branch is not expected to be buildable or usable
-until late in its development process.
-@end table
-
-All these branches are @uref{https://@value{SUBSTITUTE-SERVER-1},
-tracked by our build farm} and merged into @code{master} once
-everything has been successfully built.  This allows us to fix issues
-before they hit users, and to reduce the window during which pre-built
-binaries are not available.
-
-When we decide to start building the @code{staging} or
-@code{core-updates} branches, they will be forked and renamed with the
-suffix @code{-frozen}, at which time only bug fixes may be pushed to the
-frozen branches.  The @code{core-updates} and @code{staging} branches
-will remain open to accept patches for the next cycle.  Please ask on
-the mailing list or IRC if unsure where to place a patch.
-@c TODO: It would be good with badges on the website that tracks these
-@c branches.  Or maybe even a status page.
+Check that dependent packages (if applicable) are not affected by the
+change; @code{guix refresh --list-dependent @var{package}} will help you
+do that (@pxref{Invoking guix refresh}).
 
 @item
 @cindex determinism, of build processes
@@ -1574,16 +1535,17 @@ Teams
 [env]$ git send-email --to=@var{ISSUE_NUMBER}@@debbugs.gnu.org -2
 @end example
 
-@node Tracking Bugs and Patches
-@section Tracking Bugs and Patches
+@node Tracking Bugs and Changes
+@section Tracking Bugs and Changes
 
-This section describes how the Guix project tracks its bug reports and
-patch submissions.
+This section describes how the Guix project tracks its bug reports,
+patch submissions and topic branches.
 
 @menu
-* The Issue Tracker::           The official bug and patch tracker.
-* Debbugs User Interfaces::     Ways to interact with Debbugs.
-* Debbugs Usertags::            Tag reports with custom labels.
+* The Issue Tracker::             The official bug and patch tracker.
+* Managing Patches and Branches:: How changes to Guix are managed.
+* Debbugs User Interfaces::       Ways to interact with Debbugs.
+* Debbugs Usertags::              Tag reports with custom labels.
 @end menu
 
 @node The Issue Tracker
@@ -1600,6 +1562,51 @@ The Issue Tracker
 against the @code{guix-patches} package by sending email to
 @email{guix-patches@@gnu.org} (@pxref{Submitting Patches}).
 
+@cindex branching strategy
+@cindex rebuild scheduling strategy
+@node Managing Patches and Branches
+@subsection Managing Patches and Branches
+
+Changes should be posted to @email{guix-patches@@gnu.org}.  This mailing
+list fills the patch-tracking database (@pxref{The Issue Tracker}).  It
+also allows patches to be picked up and tested by the quality assurance
+tooling; the result of that testing eventually shows up on the dashboard
+at @indicateurl{https://qa.guix.gnu.org/issue/@var{ISSUE_NUMBER}}, where
+@var{ISSUE_NUMBER} is the number assigned by the issue tracker.  Leave
+time for a review, without committing anything.
+
+As an exception, some changes considered ``trivial'' or ``obvious'' may
+be pushed directly to the @code{master} branch.  This includes changes
+to fix typos and reverting commits that caused immediate problems.  This
+is subject to being adjusted, allowing individuals to commit directly on
+non-controversial changes on parts they’re familiar with.
+
+Changes which affect more than 300 dependent packages (@pxref{Invoking
+guix refresh}) should first be pushed to a topic branch other than
+@code{master}; the set of changes should be consistent---e.g., ``GNOME
+update'', ``NumPy update'', etc.  This allows for testing: the branch
+will automatically show up at
+@indicateurl{https://qa.guix.gnu.org/branch/@var{branch}}, with an
+indication of its build status on various platforms.
+
+To help coordinate the merging of branches, you must create a new
+guix-patches issue each time you wish to merge a branch (@pxref{The
+Issue Tracker}).  These issues indicate the order in which the branches
+should be merged, so take a look at the open issues for merging branches
+and mark the issue you create as @dfn{blocked} by the issue previously
+at the back of the queue.
+
+Normally branches will be merged in a ``first come, first merged''
+manner, tracked through the guix-patches issues.  If you agree on a
+different order with those involved, you can track this by updating
+which issues block which other issues.  Therefore, to know which branch
+is at the front of the queue, look for the issue which isn't
+@dfn{blocked} by any other branch merges.
+
+Once a branch is at the front of the queue, wait until sufficient time
+has passed for the build farms to have processed the changes, and for
+the necessary testing to have happened.
+
 @node Debbugs User Interfaces
 @subsection Debbugs User Interfaces
 
@@ -1816,23 +1823,14 @@ Commit Access
 (discussions of the policy can take place on
 @email{guix-devel@@gnu.org}).
 
-Changes should be posted to @email{guix-patches@@gnu.org}.  This mailing
-list fills the patch-tracking database (@pxref{Tracking Bugs and
-Patches}).  It also allows patches to be picked up and tested by the
-quality assurance tooling; the result of that testing eventually shows
-up on the dashboard at
-@indicateurl{https://qa.guix.gnu.org/issue/@var{ISSUE_NUMBER}}, where
-@var{ISSUE_NUMBER} is the number assigned by the issue tracker.  Leave
-time for a review, without committing anything (@pxref{Submitting
-Patches}).  If you didn’t receive any reply after one week (two weeks
-for more significant changes), and if you're confident, it's OK to
-commit.
+Ensure you're aware of how the changes should be handled
+(@pxref{Managing Patches and Branches}) prior to being pushed to the
+repository, especially for the @code{master} branch.
 
-As an exception, some changes considered ``trivial'' or ``obvious'' may
-be pushed directly.  This includes changes to fix typos and reverting
-commits that caused immediate problems.  This is subject to being
-adjusted, allowing individuals to commit directly on non-controversial
-changes on parts they’re familiar with.
+If you're committing and pushing your own changes, try and wait at least
+one week (two weeks for more significant changes) after you send them
+for review. After this, if no one else is available to review them and
+if you're confident about the changes, it's OK to commit.
 
 When pushing a commit on behalf of somebody else, please add a
 @code{Signed-off-by} line at the end of the commit log message---e.g.,
diff --git a/doc/guix.texi b/doc/guix.texi
index 5fd2449ed5..f93778f358 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -636,18 +636,18 @@ GNU Distribution
 RYF Talos II mainboard}. This platform is available as a "technology
 preview": although it is supported, substitutes are not yet available
 from the build farm (@pxref{Substitutes}), and some packages may fail to
-build (@pxref{Tracking Bugs and Patches}).  That said, the Guix
+build (@pxref{Tracking Bugs and Changes}).  That said, the Guix
 community is actively working on improving this support, and now is a
 great time to try it and get involved!
 
 @item riscv64-linux
 little-endian 64-bit RISC-V processors, specifically RV64GC, and
-Linux-Libre kernel. This platform is available as a "technology preview":
-although it is supported, substitutes are not yet available from the
-build farm (@pxref{Substitutes}), and some packages may fail to build
-(@pxref{Tracking Bugs and Patches}).  That said, the Guix community is
-actively working on improving this support, and now is a great time to
-try it and get involved!
+Linux-Libre kernel. This platform is available as a "technology
+preview": although it is supported, substitutes are not yet available
+from the build farm (@pxref{Substitutes}), and some packages may fail to
+build (@pxref{Tracking Bugs and Changes}).  That said, the Guix
+community is actively working on improving this support, and now is a
+great time to try it and get involved!
 
 @end table
 

base-commit: 63e5975cac15102e35032d15fcd90e43d5610fa4
prerequisite-patch-id: 3576be703371d049aa43170dc47ec8285b8f739e
-- 
2.40.1

Message #20 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 63459 <at> debbugs.gnu.org
Cc: Ludovic Courtès <ludo <at> gnu.org>
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Wed, 31 May 2023 10:46:33 +0100

[Message part 1 (text/plain, inline)]

Christopher Baines <mail <at> cbaines.net> writes:

> Ludovic Courtès <ludo <at> gnu.org> writes:
>
>> Hello,
>>
>> Thanks for this initiative!
>>
>> Christopher Baines <mail <at> cbaines.net> skribis:
>>
>>> +order with those involved, you can track this by updating which issues
>>> +block which other issues. Therefore, to know which branch is at the
>>> +front of the queue, look for the issue which isn't blocked by any other
>>> +branch merges.
>>> +
>>> +Once a branch is at the front of the queue, wait until sufficient time
>>> +has passed for the build farms to have processed the changes, and for
>>> +the necessary testing to have happened.
>>
>> This is a bit technical.  How can I know “which branch is at the front
>> of the queue”?  Even as a seasoned Debbugs users, I’m not sure what I’m
>> supposed to do here.  Do you think we could provide ready to use
>> commands (debbugs.el or ‘mumi’) or at least a sequence of steps to
>> follow?
>
> So, I think there's two technical hurdles to overcome here. The first is
> identifying the issues for merging branches, maybe for that we can set
> out a format for the title of the bug, but I'm very open to
> suggestions. Any way of identifying the open issues should be usable
> through debbugs.el and mumi.
>
> The second hurdle is the queuing behaviour, which I think the blocking
> behaviour is a natural fit for. Maybe the tooling is lacking but I think
> that can be addressed.
>
> I want the qa-frontpage to display the queue of branches (and issues) in
> a clear way, as well as providing links to make changes (as it does for
> marking issues as moreinfo).

I've sent a v2 now which makes more changes, most importantly it pulls
the content out from the "Submitting Patches" section to it's own
section, and also moves content from the Commit Policy in and references
it.

I've also made some progress with the qa-frontpage, it now shows a list
of branches with the corresponding issues on the homepage.

[signature.asc (application/pgp-signature, inline)]

Message #23 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Tue, 06 Jun 2023 17:27:28 +0200

Hi,

Christopher Baines <mail <at> cbaines.net> skribis:

> Move away from using staging and core-updates, and make the strategy
> independant of branch names.
>
> Keep the 300 dependent threshold for changes to master, as I don't have any
> specific reason to change this.
>
> Most importantly, require using guix-patches issues to coordinate merging of
> the branches, as I think that'll address the key issues that have shown up
> recently where it's been unclear which branch should be merged next.
>
> * doc/contributing.texi (Submitting Patches): Move the branching strategy to a
> new Managing Patches and Branches section.
> (Managing Patches and Branches): New section.
> (Commit Policy): Simplify through referencing the new Managing Patches and
> Branches section.
>
> Signed-off-by: Christopher Baines <mail <at> cbaines.net>

Neat!  Only minor comments from me:

> +@cindex branching strategy
> +@cindex rebuild scheduling strategy
> +@node Managing Patches and Branches
> +@subsection Managing Patches and Branches

Index entries should come after the section heading.

> +To help coordinate the merging of branches, you must create a new
> +guix-patches issue each time you wish to merge a branch (@pxref{The
> +Issue Tracker}).  These issues indicate the order in which the branches
> +should be merged, so take a look at the open issues for merging branches
> +and mark the issue you create as @dfn{blocked} by the issue previously
> +at the back of the queue.

IMO we’re still missing a footnote explaining how to block an issue.

> +which issues block which other issues.  Therefore, to know which branch
> +is at the front of the queue, look for the issue which isn't
> +@dfn{blocked} by any other branch merges.

No need for @dfn the second time.  I think this could also benefit from
concrete instructions (like “go to this URL”) or a cross-reference to
the Debbugs User Guide or something.

Anyway, nothing crucial: overall it LGTM!

Thanks,
Ludo’.

Message #26 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 63459 <at> debbugs.gnu.org
Cc: Christopher Baines <mail <at> cbaines.net>
Subject: [PATCH v3] doc: Move and rewrite the branching strategy.
Date: Thu,  8 Jun 2023 15:23:28 +0100

Move away from using staging and core-updates, and make the strategy
independant of branch names.

Keep the 300 dependent threshold for changes to master, as I don't have any
specific reason to change this.

Most importantly, require using guix-patches issues to coordinate merging of
the branches, as I think that'll address the key issues that have shown up
recently where it's been unclear which branch should be merged next.

* doc/contributing.texi (Submitting Patches): Move the branching strategy to a
new Managing Patches and Branches section.
(Managing Patches and Branches): New section.
(Commit Policy): Simplify through referencing the new Managing Patches and
Branches section.

Signed-off-by: Christopher Baines <mail <at> cbaines.net>
---
 doc/contributing.texi | 144 +++++++++++++++++++++---------------------
 doc/guix.texi         |  14 ++--
 2 files changed, 80 insertions(+), 78 deletions(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index f692872c04..1581aa96a1 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -26,7 +26,7 @@ Contributing
 * Packaging Guidelines::        Growing the distribution.
 * Coding Style::                Hygiene of the contributor.
 * Submitting Patches::          Share your work.
-* Tracking Bugs and Patches::   Keeping it all organized.
+* Tracking Bugs and Changes::   Keeping it all organized.
 * Commit Access::               Pushing to the official repository.
 * Updating the Guix Package::   Updating the Guix package definition.
 * Writing Documentation::       Improving documentation in GNU Guix.
@@ -1161,11 +1161,11 @@ Submitting Patches
 at the section on commit access (@pxref{Commit Access}).
 
 This mailing list is backed by a Debbugs instance, which allows us to
-keep track of submissions (@pxref{Tracking Bugs and Patches}).  Each
-message sent to that mailing list gets a new tracking number assigned;
-people can then follow up on the submission by sending email to
-@code{@var{ISSUE_NUMBER}@@debbugs.gnu.org}, where @var{ISSUE_NUMBER} is
-the tracking number (@pxref{Sending a Patch Series}).
+keep track of submissions (@pxref{Tracking Bugs and Changes}).
+Each message sent to that mailing list gets a new tracking number
+assigned; people can then follow up on the submission by sending email
+to @code{@var{ISSUE_NUMBER}@@debbugs.gnu.org}, where @var{ISSUE_NUMBER}
+is the tracking number (@pxref{Sending a Patch Series}).
 
 Please write commit logs in the ChangeLog format (@pxref{Change Logs,,,
 standards, GNU Coding Standards}); you can check the commit history for
@@ -1257,48 +1257,9 @@ Submitting Patches
 the @code{texlive-tiny} package or @code{texlive-union} procedure instead.
 
 @item
-For important changes, check that dependent packages (if applicable) are
-not affected by the change; @code{guix refresh --list-dependent
-@var{package}} will help you do that (@pxref{Invoking guix refresh}).
-
-@c See <https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00933.html>.
-@cindex branching strategy
-@cindex rebuild scheduling strategy
-Depending on the number of dependent packages and thus the amount of
-rebuilding induced, commits go to different branches, along these lines:
-
-@table @asis
-@item 300 dependent packages or less
-@code{master} branch (non-disruptive changes).
-
-@item between 300 and 1,800 dependent packages
-@code{staging} branch (non-disruptive changes).  This branch is intended
-to be merged in @code{master} every 6 weeks or so.  Topical changes
-(e.g., an update of the GNOME stack) can instead go to a specific branch
-(say, @code{gnome-updates}).  This branch is not expected to be
-buildable or usable until late in its development process.
-
-@item more than 1,800 dependent packages
-@code{core-updates} branch (may include major and potentially disruptive
-changes).  This branch is intended to be merged in @code{master} every
-6 months or so.  This branch is not expected to be buildable or usable
-until late in its development process.
-@end table
-
-All these branches are @uref{https://@value{SUBSTITUTE-SERVER-1},
-tracked by our build farm} and merged into @code{master} once
-everything has been successfully built.  This allows us to fix issues
-before they hit users, and to reduce the window during which pre-built
-binaries are not available.
-
-When we decide to start building the @code{staging} or
-@code{core-updates} branches, they will be forked and renamed with the
-suffix @code{-frozen}, at which time only bug fixes may be pushed to the
-frozen branches.  The @code{core-updates} and @code{staging} branches
-will remain open to accept patches for the next cycle.  Please ask on
-the mailing list or IRC if unsure where to place a patch.
-@c TODO: It would be good with badges on the website that tracks these
-@c branches.  Or maybe even a status page.
+Check that dependent packages (if applicable) are not affected by the
+change; @code{guix refresh --list-dependent @var{package}} will help you
+do that (@pxref{Invoking guix refresh}).
 
 @item
 @cindex determinism, of build processes
@@ -1574,16 +1535,17 @@ Teams
 [env]$ git send-email --to=@var{ISSUE_NUMBER}@@debbugs.gnu.org -2
 @end example
 
-@node Tracking Bugs and Patches
-@section Tracking Bugs and Patches
+@node Tracking Bugs and Changes
+@section Tracking Bugs and Changes
 
-This section describes how the Guix project tracks its bug reports and
-patch submissions.
+This section describes how the Guix project tracks its bug reports,
+patch submissions and topic branches.
 
 @menu
-* The Issue Tracker::           The official bug and patch tracker.
-* Debbugs User Interfaces::     Ways to interact with Debbugs.
-* Debbugs Usertags::            Tag reports with custom labels.
+* The Issue Tracker::             The official bug and patch tracker.
+* Managing Patches and Branches:: How changes to Guix are managed.
+* Debbugs User Interfaces::       Ways to interact with Debbugs.
+* Debbugs Usertags::              Tag reports with custom labels.
 @end menu
 
 @node The Issue Tracker
@@ -1600,6 +1562,55 @@ The Issue Tracker
 against the @code{guix-patches} package by sending email to
 @email{guix-patches@@gnu.org} (@pxref{Submitting Patches}).
 
+@node Managing Patches and Branches
+@subsection Managing Patches and Branches
+@cindex branching strategy
+@cindex rebuild scheduling strategy
+
+Changes should be posted to @email{guix-patches@@gnu.org}.  This mailing
+list fills the patch-tracking database (@pxref{The Issue Tracker}).  It
+also allows patches to be picked up and tested by the quality assurance
+tooling; the result of that testing eventually shows up on the dashboard
+at @indicateurl{https://qa.guix.gnu.org/issue/@var{ISSUE_NUMBER}}, where
+@var{ISSUE_NUMBER} is the number assigned by the issue tracker.  Leave
+time for a review, without committing anything.
+
+As an exception, some changes considered ``trivial'' or ``obvious'' may
+be pushed directly to the @code{master} branch.  This includes changes
+to fix typos and reverting commits that caused immediate problems.  This
+is subject to being adjusted, allowing individuals to commit directly on
+non-controversial changes on parts they’re familiar with.
+
+Changes which affect more than 300 dependent packages (@pxref{Invoking
+guix refresh}) should first be pushed to a topic branch other than
+@code{master}; the set of changes should be consistent---e.g., ``GNOME
+update'', ``NumPy update'', etc.  This allows for testing: the branch
+will automatically show up at
+@indicateurl{https://qa.guix.gnu.org/branch/@var{branch}}, with an
+indication of its build status on various platforms.
+
+To help coordinate the merging of branches, you must create a new
+guix-patches issue each time you wish to merge a branch (@pxref{The
+Issue Tracker}).  These issues indicate the order in which the branches
+should be merged, so take a look at the open issues for merging branches
+and mark the issue you create as @dfn{blocked} by the issue previously
+at the back of the queue <at> footnote{You can mark an issue as blocked by
+another by emailing @email{control@@debbugs.gnu.org} with the following
+line in the body of the email: @code{block XXXXX by YYYYY}. Where
+@code{XXXXX} is the number for the blocked issue, and @code{YYYYY} is
+the number for the issue blocking it.}.
+
+Normally branches will be merged in a ``first come, first merged''
+manner, tracked through the guix-patches issues.  If you agree on a
+different order with those involved, you can track this by updating
+which issues block which other issues.  Therefore, to know which branch
+is at the front of the queue, look for the issue which isn't blocked by
+any other branch merges.
+
+Once a branch is at the front of the queue, wait until sufficient time
+has passed for the build farms to have processed the changes, and for
+the necessary testing to have happened.
+
 @node Debbugs User Interfaces
 @subsection Debbugs User Interfaces
 
@@ -1816,23 +1827,14 @@ Commit Access
 (discussions of the policy can take place on
 @email{guix-devel@@gnu.org}).
 
-Changes should be posted to @email{guix-patches@@gnu.org}.  This mailing
-list fills the patch-tracking database (@pxref{Tracking Bugs and
-Patches}).  It also allows patches to be picked up and tested by the
-quality assurance tooling; the result of that testing eventually shows
-up on the dashboard at
-@indicateurl{https://qa.guix.gnu.org/issue/@var{ISSUE_NUMBER}}, where
-@var{ISSUE_NUMBER} is the number assigned by the issue tracker.  Leave
-time for a review, without committing anything (@pxref{Submitting
-Patches}).  If you didn’t receive any reply after one week (two weeks
-for more significant changes), and if you're confident, it's OK to
-commit.
+Ensure you're aware of how the changes should be handled
+(@pxref{Managing Patches and Branches}) prior to being pushed to the
+repository, especially for the @code{master} branch.
 
-As an exception, some changes considered ``trivial'' or ``obvious'' may
-be pushed directly.  This includes changes to fix typos and reverting
-commits that caused immediate problems.  This is subject to being
-adjusted, allowing individuals to commit directly on non-controversial
-changes on parts they’re familiar with.
+If you're committing and pushing your own changes, try and wait at least
+one week (two weeks for more significant changes) after you send them
+for review. After this, if no one else is available to review them and
+if you're confident about the changes, it's OK to commit.
 
 When pushing a commit on behalf of somebody else, please add a
 @code{Signed-off-by} line at the end of the commit log message---e.g.,
diff --git a/doc/guix.texi b/doc/guix.texi
index 01f4e0105f..83cbf004bb 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -637,18 +637,18 @@ GNU Distribution
 RYF Talos II mainboard}. This platform is available as a "technology
 preview": although it is supported, substitutes are not yet available
 from the build farm (@pxref{Substitutes}), and some packages may fail to
-build (@pxref{Tracking Bugs and Patches}).  That said, the Guix
+build (@pxref{Tracking Bugs and Changes}).  That said, the Guix
 community is actively working on improving this support, and now is a
 great time to try it and get involved!
 
 @item riscv64-linux
 little-endian 64-bit RISC-V processors, specifically RV64GC, and
-Linux-Libre kernel. This platform is available as a "technology preview":
-although it is supported, substitutes are not yet available from the
-build farm (@pxref{Substitutes}), and some packages may fail to build
-(@pxref{Tracking Bugs and Patches}).  That said, the Guix community is
-actively working on improving this support, and now is a great time to
-try it and get involved!
+Linux-Libre kernel. This platform is available as a "technology
+preview": although it is supported, substitutes are not yet available
+from the build farm (@pxref{Substitutes}), and some packages may fail to
+build (@pxref{Tracking Bugs and Changes}).  That said, the Guix
+community is actively working on improving this support, and now is a
+great time to try it and get involved!
 
 @end table
 

base-commit: 91454f54a4bf07030b51d718482416518ba5c9a7
prerequisite-patch-id: 0c60b9c48946553181183bd4a16611a27f6e62bf
-- 
2.40.1

Message #29 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: guix-devel <at> gnu.org
Cc: 63459 <at> debbugs.gnu.org
Subject: Changes to the branching/commit policy
Date: Thu, 08 Jun 2023 15:24:37 +0100

[Message part 1 (text/plain, inline)]

Hey!

The changes in #63459 have strayed now in to touching the commit policy
[1]. My intent was to simplify the guidance by grouping it better, but I
think the significant change here is that the commit policy now
references the entire branching strategy, rather than just talking about
sending patches for review.

1: https://guix.gnu.org/manual/devel/en/html_node/Commit-Access.html#Commit-Policy

That new branching strategy makes some "should" requirements on sending
patches for review and pushing to topic branches for larger changes. It
also makes a "must" requirement on opening guix-patches issues to track
and manage merging branches.

I'd like to merge these changes next week since they've been up for a
few weeks, so do comment if you have any thoughts or if you'd like more
time to review them.

Thanks,

Chris

[signature.asc (application/pgp-signature, inline)]

Message #32 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Christopher Baines <mail <at> cbaines.net>
Cc: guix-devel <at> gnu.org, 63459 <at> debbugs.gnu.org
Subject: Re: Changes to the branching/commit policy
Date: Fri, 9 Jun 2023 12:10:01 +0200

Hello Chris,

thanks for taking up this issue! I agreed with Ludovic's comments, so
things look good now for me. A very minor point: In the section on
"trivial" changes, I would drop this sentence (which was already there
before):
"This is subject to being adjusted, allowing individuals to commit directly
on non-controversial changes on parts they’re familiar with."
The sentence is meaningless, as everything is all the time subject to being
adjusted; and we do not have immediate plans to adjust it.

Looking forward to the merge since it clarifies things and removes the
staging and core-updates branches not only from our minds, but also the
texts.

Andreas

Message #35 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Andreas Enge <andreas <at> enge.fr>
Cc: guix-devel <at> gnu.org, 63459 <at> debbugs.gnu.org
Subject: Re: Changes to the branching/commit policy
Date: Sun, 11 Jun 2023 10:37:14 +0100

[Message part 1 (text/plain, inline)]

Andreas Enge <andreas <at> enge.fr> writes:

> thanks for taking up this issue! I agreed with Ludovic's comments, so
> things look good now for me. A very minor point: In the section on
> "trivial" changes, I would drop this sentence (which was already there
> before):
> "This is subject to being adjusted, allowing individuals to commit directly
> on non-controversial changes on parts they’re familiar with."
> The sentence is meaningless, as everything is all the time subject to being
> adjusted; and we do not have immediate plans to adjust it.

My reading of this line is that "adjusted" is probably not the right
word to use, but I think the intent here is to talk about how currently
it's accepted that people can and will push non-controversial changes on
parts they’re familiar with directly to master.

I'm not sure if others read this similarly.

[signature.asc (application/pgp-signature, inline)]

Message #38 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Andreas Enge <andreas <at> enge.fr>
To: Christopher Baines <mail <at> cbaines.net>
Cc: guix-devel <at> gnu.org, 63459 <at> debbugs.gnu.org
Subject: Re: Changes to the branching/commit policy
Date: Sun, 11 Jun 2023 11:53:45 +0200

Am Sun, Jun 11, 2023 at 10:37:14AM +0100 schrieb Christopher Baines:
> My reading of this line is that "adjusted" is probably not the right
> word to use, but I think the intent here is to talk about how currently
> it's accepted that people can and will push non-controversial changes on
> parts they’re familiar with directly to master.

I read it the other way round: Right now it is not accepted, but it might
be adjusted to allow non-controversial changes in the future. Actually
the concept of "non-controversial commits" is probably controversial
in itself...

Andreas

Message #41 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Christopher Baines <mail <at> cbaines.net>
Cc: guix-devel <at> gnu.org, Andreas Enge <andreas <at> enge.fr>, 63459 <at> debbugs.gnu.org
Subject: Re: Changes to the branching/commit policy
Date: Sun, 11 Jun 2023 21:28:38 -0400

Hi Christopher,

Christopher Baines <mail <at> cbaines.net> writes:

> Andreas Enge <andreas <at> enge.fr> writes:
>
>> thanks for taking up this issue! I agreed with Ludovic's comments, so
>> things look good now for me. A very minor point: In the section on
>> "trivial" changes, I would drop this sentence (which was already there
>> before):
>> "This is subject to being adjusted, allowing individuals to commit directly
>> on non-controversial changes on parts they’re familiar with."
>> The sentence is meaningless, as everything is all the time subject to being
>> adjusted; and we do not have immediate plans to adjust it.
>
> My reading of this line is that "adjusted" is probably not the right
> word to use, but I think the intent here is to talk about how currently
> it's accepted that people can and will push non-controversial changes on
> parts they’re familiar with directly to master.
>
> I'm not sure if others read this similarly.

That's how I read it as well.  I like the ability for people to, at
times depending on the situation, choose to push directly to fix or
update something instead of going through the otherwise recommended 1
week QA/review flow.

-- 
Thanks,
Maxim

Message #44 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Sun, 11 Jun 2023 22:37:53 -0400

Hi Christopher,

Christopher Baines <mail <at> cbaines.net> writes:

> Move away from using staging and core-updates, and make the strategy
> independant of branch names.
>
> Keep the 300 dependent threshold for changes to master, as I don't have any
> specific reason to change this.
>
> Most importantly, require using guix-patches issues to coordinate merging of
> the branches, as I think that'll address the key issues that have shown up
> recently where it's been unclear which branch should be merged next.
>
> * doc/contributing.texi (Submitting Patches): Move the branching strategy to a
> new Managing Patches and Branches section.
> (Managing Patches and Branches): New section.
> (Commit Policy): Simplify through referencing the new Managing Patches and
> Branches section.

[...]

> +
> +To help coordinate the merging of branches, you must create a new
> +guix-patches issue each time you wish to merge a branch (@pxref{The
> +Issue Tracker}).  These issues indicate the order in which the branches
> +should be merged, so take a look at the open issues for merging branches
> +and mark the issue you create as @dfn{blocked} by the issue previously
> +at the back of the queue <at> footnote{You can mark an issue as blocked by
> +another by emailing @email{control@@debbugs.gnu.org} with the following
> +line in the body of the email: @code{block XXXXX by YYYYY}. Where
> +@code{XXXXX} is the number for the blocked issue, and @code{YYYYY} is
> +the number for the issue blocking it.}.

Maybe by default, since the strategy would be "first come, first
merged", we can forego with the 'block' tags, as issues will already be
posted in the order (and given an increasing number) they should be
merged?  Then the nitty-gritty details of micro-managing block tags can
be mentioned only when they are useful, e.g. ...

> +Normally branches will be merged in a ``first come, first merged''
> +manner, tracked through the guix-patches issues.  If you agree on a
> +different order with those involved, you can track this by updating
> +which issues block which other issues.  Therefore, to know which branch
> +is at the front of the queue, look for the issue which isn't blocked by
> +any other branch merges.

... here.  Can anyone merge the branches of someone else that posted
them to the tracker but 'hasn't gotten around' to merge them to the repo
(e.g. gone on vacation), although they were fully QA'd, blocking every
other branch merge?

> +Once a branch is at the front of the queue, wait until sufficient time
> +has passed for the build farms to have processed the changes, and for
> +the necessary testing to have happened.

What does that mean concretely?  How can I track the build status of a
change?  Please at least mention the QA badge which is visible from
issues.guix.gnu.org and perhaps other tricks I'm unaware of :-).

Thanks for working on completing the documentation of the new work flow.

-- 
Maxim

Message #47 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: 63459 <at> debbugs.gnu.org
Cc: Christopher Baines <mail <at> cbaines.net>
Subject: [PATCH v4] doc: Move and rewrite the branching strategy.
Date: Mon, 12 Jun 2023 10:01:14 +0100

Move away from using staging and core-updates, and make the strategy
independant of branch names.

Keep the 300 dependent threshold for changes to master, as I don't have any
specific reason to change this.

Most importantly, require using guix-patches issues to coordinate merging of
the branches, as I think that'll address the key issues that have shown up
recently where it's been unclear which branch should be merged next.

* doc/contributing.texi (Submitting Patches): Move the branching strategy to a
new Managing Patches and Branches section.
(Managing Patches and Branches): New section.
(Commit Policy): Simplify through referencing the new Managing Patches and
Branches section.

Signed-off-by: Christopher Baines <mail <at> cbaines.net>
---
 doc/contributing.texi | 144 +++++++++++++++++++++---------------------
 doc/guix.texi         |  14 ++--
 2 files changed, 80 insertions(+), 78 deletions(-)

diff --git a/doc/contributing.texi b/doc/contributing.texi
index 958fc44cbd..3a402c13a9 100644
--- a/doc/contributing.texi
+++ b/doc/contributing.texi
@@ -26,7 +26,7 @@ Contributing
 * Packaging Guidelines::        Growing the distribution.
 * Coding Style::                Hygiene of the contributor.
 * Submitting Patches::          Share your work.
-* Tracking Bugs and Patches::   Keeping it all organized.
+* Tracking Bugs and Changes::   Keeping it all organized.
 * Commit Access::               Pushing to the official repository.
 * Updating the Guix Package::   Updating the Guix package definition.
 * Writing Documentation::       Improving documentation in GNU Guix.
@@ -1161,11 +1161,11 @@ Submitting Patches
 at the section on commit access (@pxref{Commit Access}).
 
 This mailing list is backed by a Debbugs instance, which allows us to
-keep track of submissions (@pxref{Tracking Bugs and Patches}).  Each
-message sent to that mailing list gets a new tracking number assigned;
-people can then follow up on the submission by sending email to
-@code{@var{ISSUE_NUMBER}@@debbugs.gnu.org}, where @var{ISSUE_NUMBER} is
-the tracking number (@pxref{Sending a Patch Series}).
+keep track of submissions (@pxref{Tracking Bugs and Changes}).
+Each message sent to that mailing list gets a new tracking number
+assigned; people can then follow up on the submission by sending email
+to @code{@var{ISSUE_NUMBER}@@debbugs.gnu.org}, where @var{ISSUE_NUMBER}
+is the tracking number (@pxref{Sending a Patch Series}).
 
 Please write commit logs in the ChangeLog format (@pxref{Change Logs,,,
 standards, GNU Coding Standards}); you can check the commit history for
@@ -1257,48 +1257,9 @@ Submitting Patches
 the @code{texlive-tiny} package or @code{texlive-union} procedure instead.
 
 @item
-For important changes, check that dependent packages (if applicable) are
-not affected by the change; @code{guix refresh --list-dependent
-@var{package}} will help you do that (@pxref{Invoking guix refresh}).
-
-@c See <https://lists.gnu.org/archive/html/guix-devel/2016-10/msg00933.html>.
-@cindex branching strategy
-@cindex rebuild scheduling strategy
-Depending on the number of dependent packages and thus the amount of
-rebuilding induced, commits go to different branches, along these lines:
-
-@table @asis
-@item 300 dependent packages or less
-@code{master} branch (non-disruptive changes).
-
-@item between 300 and 1,800 dependent packages
-@code{staging} branch (non-disruptive changes).  This branch is intended
-to be merged in @code{master} every 6 weeks or so.  Topical changes
-(e.g., an update of the GNOME stack) can instead go to a specific branch
-(say, @code{gnome-updates}).  This branch is not expected to be
-buildable or usable until late in its development process.
-
-@item more than 1,800 dependent packages
-@code{core-updates} branch (may include major and potentially disruptive
-changes).  This branch is intended to be merged in @code{master} every
-6 months or so.  This branch is not expected to be buildable or usable
-until late in its development process.
-@end table
-
-All these branches are @uref{https://@value{SUBSTITUTE-SERVER-1},
-tracked by our build farm} and merged into @code{master} once
-everything has been successfully built.  This allows us to fix issues
-before they hit users, and to reduce the window during which pre-built
-binaries are not available.
-
-When we decide to start building the @code{staging} or
-@code{core-updates} branches, they will be forked and renamed with the
-suffix @code{-frozen}, at which time only bug fixes may be pushed to the
-frozen branches.  The @code{core-updates} and @code{staging} branches
-will remain open to accept patches for the next cycle.  Please ask on
-the mailing list or IRC if unsure where to place a patch.
-@c TODO: It would be good with badges on the website that tracks these
-@c branches.  Or maybe even a status page.
+Check that dependent packages (if applicable) are not affected by the
+change; @code{guix refresh --list-dependent @var{package}} will help you
+do that (@pxref{Invoking guix refresh}).
 
 @item
 @cindex determinism, of build processes
@@ -1574,16 +1535,17 @@ Teams
 [env]$ git send-email --to=@var{ISSUE_NUMBER}@@debbugs.gnu.org -2
 @end example
 
-@node Tracking Bugs and Patches
-@section Tracking Bugs and Patches
+@node Tracking Bugs and Changes
+@section Tracking Bugs and Changes
 
-This section describes how the Guix project tracks its bug reports and
-patch submissions.
+This section describes how the Guix project tracks its bug reports,
+patch submissions and topic branches.
 
 @menu
-* The Issue Tracker::           The official bug and patch tracker.
-* Debbugs User Interfaces::     Ways to interact with Debbugs.
-* Debbugs Usertags::            Tag reports with custom labels.
+* The Issue Tracker::             The official bug and patch tracker.
+* Managing Patches and Branches:: How changes to Guix are managed.
+* Debbugs User Interfaces::       Ways to interact with Debbugs.
+* Debbugs Usertags::              Tag reports with custom labels.
 @end menu
 
 @node The Issue Tracker
@@ -1600,6 +1562,55 @@ The Issue Tracker
 against the @code{guix-patches} package by sending email to
 @email{guix-patches@@gnu.org} (@pxref{Submitting Patches}).
 
+@node Managing Patches and Branches
+@subsection Managing Patches and Branches
+@cindex branching strategy
+@cindex rebuild scheduling strategy
+
+Changes should be posted to @email{guix-patches@@gnu.org}.  This mailing
+list fills the patch-tracking database (@pxref{The Issue Tracker}).  It
+also allows patches to be picked up and tested by the quality assurance
+tooling; the result of that testing eventually shows up on the dashboard
+at @indicateurl{https://qa.guix.gnu.org/issue/@var{ISSUE_NUMBER}}, where
+@var{ISSUE_NUMBER} is the number assigned by the issue tracker.  Leave
+time for a review, without committing anything.
+
+As an exception, some changes considered ``trivial'' or ``obvious'' may
+be pushed directly to the @code{master} branch.  This includes changes
+to fix typos and reverting commits that caused immediate problems.  This
+is subject to being adjusted, allowing individuals to commit directly on
+non-controversial changes on parts they’re familiar with.
+
+Changes which affect more than 300 dependent packages (@pxref{Invoking
+guix refresh}) should first be pushed to a topic branch other than
+@code{master}; the set of changes should be consistent---e.g., ``GNOME
+update'', ``NumPy update'', etc.  This allows for testing: the branch
+will automatically show up at
+@indicateurl{https://qa.guix.gnu.org/branch/@var{branch}}, with an
+indication of its build status on various platforms.
+
+To help coordinate the merging of branches, you must create a new
+guix-patches issue each time you wish to merge a branch (@pxref{The
+Issue Tracker}).  Normally branches will be merged in a ``first come,
+first merged'' manner, tracked through the guix-patches issues.
+
+If you agree on a different order with those involved, you can track
+this by updating which issues block <at> footnote{You can mark an issue as
+blocked by another by emailing @email{control@@debbugs.gnu.org} with the
+following line in the body of the email: @code{block XXXXX by YYYYY}.
+Where @code{XXXXX} is the number for the blocked issue, and @code{YYYYY}
+is the number for the issue blocking it.} which other issues.
+Therefore, to know which branch is at the front of the queue, look for
+the oldest issue, or the issue that isn't @dfn{blocked} by any other
+branch merges.  An ordered list of branches with the open issues is
+available at @url{https://qa.guix.gnu.org}.
+
+Once a branch is at the front of the queue, wait until sufficient time
+has passed for the build farms to have processed the changes, and for
+the necessary testing to have happened.  For example, you can check
+@indicateurl{https://qa.guix.gnu.org/branch/@var{branch}} to see
+information on some builds and substitute availability.
+
 @node Debbugs User Interfaces
 @subsection Debbugs User Interfaces
 
@@ -1816,23 +1827,14 @@ Commit Access
 (discussions of the policy can take place on
 @email{guix-devel@@gnu.org}).
 
-Changes should be posted to @email{guix-patches@@gnu.org}.  This mailing
-list fills the patch-tracking database (@pxref{Tracking Bugs and
-Patches}).  It also allows patches to be picked up and tested by the
-quality assurance tooling; the result of that testing eventually shows
-up on the dashboard at
-@indicateurl{https://qa.guix.gnu.org/issue/@var{ISSUE_NUMBER}}, where
-@var{ISSUE_NUMBER} is the number assigned by the issue tracker.  Leave
-time for a review, without committing anything (@pxref{Submitting
-Patches}).  If you didn’t receive any reply after one week (two weeks
-for more significant changes), and if you're confident, it's OK to
-commit.
+Ensure you're aware of how the changes should be handled
+(@pxref{Managing Patches and Branches}) prior to being pushed to the
+repository, especially for the @code{master} branch.
 
-As an exception, some changes considered ``trivial'' or ``obvious'' may
-be pushed directly.  This includes changes to fix typos and reverting
-commits that caused immediate problems.  This is subject to being
-adjusted, allowing individuals to commit directly on non-controversial
-changes on parts they’re familiar with.
+If you're committing and pushing your own changes, try and wait at least
+one week (two weeks for more significant changes) after you send them
+for review. After this, if no one else is available to review them and
+if you're confident about the changes, it's OK to commit.
 
 When pushing a commit on behalf of somebody else, please add a
 @code{Signed-off-by} line at the end of the commit log message---e.g.,
diff --git a/doc/guix.texi b/doc/guix.texi
index 395fc25818..43dffe08c1 100644
--- a/doc/guix.texi
+++ b/doc/guix.texi
@@ -637,18 +637,18 @@ GNU Distribution
 RYF Talos II mainboard}. This platform is available as a "technology
 preview": although it is supported, substitutes are not yet available
 from the build farm (@pxref{Substitutes}), and some packages may fail to
-build (@pxref{Tracking Bugs and Patches}).  That said, the Guix
+build (@pxref{Tracking Bugs and Changes}).  That said, the Guix
 community is actively working on improving this support, and now is a
 great time to try it and get involved!
 
 @item riscv64-linux
 little-endian 64-bit RISC-V processors, specifically RV64GC, and
-Linux-Libre kernel. This platform is available as a "technology preview":
-although it is supported, substitutes are not yet available from the
-build farm (@pxref{Substitutes}), and some packages may fail to build
-(@pxref{Tracking Bugs and Patches}).  That said, the Guix community is
-actively working on improving this support, and now is a great time to
-try it and get involved!
+Linux-Libre kernel. This platform is available as a "technology
+preview": although it is supported, substitutes are not yet available
+from the build farm (@pxref{Substitutes}), and some packages may fail to
+build (@pxref{Tracking Bugs and Changes}).  That said, the Guix
+community is actively working on improving this support, and now is a
+great time to try it and get involved!
 
 @end table
 

base-commit: 259b2e99e7121f05011742955636ff2dd96bf0e8
prerequisite-patch-id: 7a63d38dea972d66bf29acadde23be7e5d8b1b30
-- 
2.40.1

Message #50 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Mon, 12 Jun 2023 10:01:21 +0100

[Message part 1 (text/plain, inline)]

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hi Christopher,
>
> Christopher Baines <mail <at> cbaines.net> writes:
>
>> Move away from using staging and core-updates, and make the strategy
>> independant of branch names.
>>
>> Keep the 300 dependent threshold for changes to master, as I don't have any
>> specific reason to change this.
>>
>> Most importantly, require using guix-patches issues to coordinate merging of
>> the branches, as I think that'll address the key issues that have shown up
>> recently where it's been unclear which branch should be merged next.
>>
>> * doc/contributing.texi (Submitting Patches): Move the branching strategy to a
>> new Managing Patches and Branches section.
>> (Managing Patches and Branches): New section.
>> (Commit Policy): Simplify through referencing the new Managing Patches and
>> Branches section.
>
> [...]
>
>> +
>> +To help coordinate the merging of branches, you must create a new
>> +guix-patches issue each time you wish to merge a branch (@pxref{The
>> +Issue Tracker}).  These issues indicate the order in which the branches
>> +should be merged, so take a look at the open issues for merging branches
>> +and mark the issue you create as @dfn{blocked} by the issue previously
>> +at the back of the queue <at> footnote{You can mark an issue as blocked by
>> +another by emailing @email{control@@debbugs.gnu.org} with the following
>> +line in the body of the email: @code{block XXXXX by YYYYY}. Where
>> +@code{XXXXX} is the number for the blocked issue, and @code{YYYYY} is
>> +the number for the issue blocking it.}.
>
> Maybe by default, since the strategy would be "first come, first
> merged", we can forego with the 'block' tags, as issues will already be
> posted in the order (and given an increasing number) they should be
> merged?  Then the nitty-gritty details of micro-managing block tags can
> be mentioned only when they are useful, e.g. ...

That sounds fine to me.

>> +Normally branches will be merged in a ``first come, first merged''
>> +manner, tracked through the guix-patches issues.  If you agree on a
>> +different order with those involved, you can track this by updating
>> +which issues block which other issues.  Therefore, to know which branch
>> +is at the front of the queue, look for the issue which isn't blocked by
>> +any other branch merges.
>
> ... here.  Can anyone merge the branches of someone else that posted
> them to the tracker but 'hasn't gotten around' to merge them to the repo
> (e.g. gone on vacation), although they were fully QA'd, blocking every
> other branch merge?

I've moved the blocking stuff down.

As for the merging of branches that others have pushed, I'm not sure
there's consensus regarding this. Personally I would like to see this,
being able to merge other committers changes, I raised it on guix-devel
recently [1].

1: https://lists.gnu.org/archive/html/guix-devel/2023-02/msg00263.html

>> +Once a branch is at the front of the queue, wait until sufficient time
>> +has passed for the build farms to have processed the changes, and for
>> +the necessary testing to have happened.
>
> What does that mean concretely?  How can I track the build status of a
> change?  Please at least mention the QA badge which is visible from
> issues.guix.gnu.org and perhaps other tricks I'm unaware of :-).

It's intentionally quite high level and non-concrete. Maybe we'll get to
the point in the future where we have more specific requirements to meet
before merging a branch, but I don't think we have that yet.

qa.guix.gnu.org/branch/NAME is linked to above, and I've added another
link to it here. The QA badge currently doesn't work for branches, but
I'd like to get it working.

I've sent a v4 now, thanks for taking a look!

Chris

[signature.asc (application/pgp-signature, inline)]

Message #53 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Mon, 12 Jun 2023 08:20:27 -0400

Hi Chris,

[...]

>>> +To help coordinate the merging of branches, you must create a new
>>> +guix-patches issue each time you wish to merge a branch (@pxref{The
>>> +Issue Tracker}).  These issues indicate the order in which the branches
>>> +should be merged, so take a look at the open issues for merging branches
>>> +and mark the issue you create as @dfn{blocked} by the issue previously
>>> +at the back of the queue <at> footnote{You can mark an issue as blocked by
>>> +another by emailing @email{control@@debbugs.gnu.org} with the following
>>> +line in the body of the email: @code{block XXXXX by YYYYY}. Where
>>> +@code{XXXXX} is the number for the blocked issue, and @code{YYYYY} is
>>> +the number for the issue blocking it.}.
>>
>> Maybe by default, since the strategy would be "first come, first
>> merged", we can forego with the 'block' tags, as issues will already be
>> posted in the order (and given an increasing number) they should be
>> merged?  Then the nitty-gritty details of micro-managing block tags can
>> be mentioned only when they are useful, e.g. ...
>
> That sounds fine to me.

One disadvantage of this is that people must now manually find the
preceding merge requests on the tracker; but if we have some convention
prefix in the subject, e.g. 'MERGE' or similar (it's always implied we
merge to master branch and nowhere else, correct?), that would still
make it easy.  When the tooling (build coordinator) offers a web view of
the branches to be merged that can be linked as well.

So I think it's a LGTM.

-- 
Thanks,
Maxim

Message #56 received at 63459 <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Maxim Cournoyer <maxim.cournoyer <at> gmail.com>
Cc: 63459 <at> debbugs.gnu.org
Subject: Re: bug#63459: [PATCH] doc: Rewrite the branching strategy.
Date: Mon, 12 Jun 2023 20:53:25 +0100

[Message part 1 (text/plain, inline)]

Maxim Cournoyer <maxim.cournoyer <at> gmail.com> writes:

> Hi Chris,
>
> [...]
>
>>>> +To help coordinate the merging of branches, you must create a new
>>>> +guix-patches issue each time you wish to merge a branch (@pxref{The
>>>> +Issue Tracker}).  These issues indicate the order in which the branches
>>>> +should be merged, so take a look at the open issues for merging branches
>>>> +and mark the issue you create as @dfn{blocked} by the issue previously
>>>> +at the back of the queue <at> footnote{You can mark an issue as blocked by
>>>> +another by emailing @email{control@@debbugs.gnu.org} with the following
>>>> +line in the body of the email: @code{block XXXXX by YYYYY}. Where
>>>> +@code{XXXXX} is the number for the blocked issue, and @code{YYYYY} is
>>>> +the number for the issue blocking it.}.
>>>
>>> Maybe by default, since the strategy would be "first come, first
>>> merged", we can forego with the 'block' tags, as issues will already be
>>> posted in the order (and given an increasing number) they should be
>>> merged?  Then the nitty-gritty details of micro-managing block tags can
>>> be mentioned only when they are useful, e.g. ...
>>
>> That sounds fine to me.
>
> One disadvantage of this is that people must now manually find the
> preceding merge requests on the tracker; but if we have some convention
> prefix in the subject, e.g. 'MERGE' or similar (it's always implied we
> merge to master branch and nowhere else, correct?), that would still
> make it easy.  When the tooling (build coordinator) offers a web view of
> the branches to be merged that can be linked as well.

There's already a webpage featuring the branches and corresponding
issues, they feature in a table on [1]. The qa-frontpage makes the
assumption that the issue titles include the string "Request for
merging" and have the branch name in quotes, but that's just because
that was used as the title for [2].

1: https://qa.guix.gnu.org/
2: https://issues.guix.gnu.org/63521

As you say, it would be good to settle on a convention and mandate this
in contributing.texi.

As for where you're merging, yes, I'm assuming you're merging to master
here.

> So I think it's a LGTM.

Great, thanks for taking a look.

[signature.asc (application/pgp-signature, inline)]

Message #61 received at 63459-done <at> debbugs.gnu.org (full text, mbox):

From: Christopher Baines <mail <at> cbaines.net>
To: Christopher Baines <mail <at> cbaines.net>
Cc: 63459-done <at> debbugs.gnu.org
Subject: Re: [bug#63459] [PATCH] doc: Rewrite the branching strategy.
Date: Mon, 12 Jun 2023 21:19:42 +0100

[Message part 1 (text/plain, inline)]

I've now pushed this to master as
0ea096ae23fa81f05ce97e5e61c15647c0a475ec.

There's still lots to improve, both within the guidance and in addition
to it.

Top on my list is making some requirements about the issues to open when
you want to merge a branch (e.g. specifying the title format so that
qa.guix.gnu.org can detect them).

Thanks,

Chris

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.