GNU bug report logs - #32575
[Cuirass] Filter results by architecture

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 32575 in the body.
You can then email your comments to 32575 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: bug-guix <at> gnu.org
Subject: [Cuirass] Filter results by architecture
Date: Wed, 29 Aug 2018 15:54:47 +0200

The Cuirass web interface shows the number of successful, failed, and
pending builds for each evaluation.  Looking at just these numbers it is
impossible to tell, how each of the supported architectures is affected.

It would be good if we could separate the view by architecture.  Then we
could more easily determine that a change broke many builds for one
architecture while fixing builds on another.

One way to do this would be to accept an optional query variable, e.g.

    http://ci.guix.info/jobset/guix-master?system=x86_64-linux

This could be selected from a drop-down on the page or exposed through a
number of links.

--
Ricardo

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Joshua Branson <jbranso <at> fastmail.com>
To: bug-guix <at> gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Wed, 29 Aug 2018 16:49:13 -0400

Ricardo Wurmus <rekado <at> elephly.net> writes:

> The Cuirass web interface shows the number of successful, failed, and
> pending builds for each evaluation.  Looking at just these numbers it is
> impossible to tell, how each of the supported architectures is affected.
>
> It would be good if we could separate the view by architecture.  Then we
> could more easily determine that a change broke many builds for one
> architecture while fixing builds on another.
>
> One way to do this would be to accept an optional query variable, e.g.
>
>     http://ci.guix.info/jobset/guix-master?system=x86_64-linux

That is an option.  Another one is using a REST API.  It seems to have
all the hype these days.  So the URL would turn into:

     http://ci.guix.info/jobset/guix-master/system/x86_64-linux

Though I freely admit, I don't completely understand the benefits of REST.

>
> This could be selected from a drop-down on the page or exposed through a
> number of links.
>
> --
> Ricardo

Message #11 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: Gábor Boskovits <boskovits <at> gmail.com>
To: Joshua Branson <jbranso <at> fastmail.com>
Cc: 32575 <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 30 Aug 2018 07:56:26 +0200

[Message part 1 (text/plain, inline)]

Joshua Branson <jbranso <at> fastmail.com> ezt írta (időpont: 2018. aug. 29.,
Sze, 22:41):

> Ricardo Wurmus <rekado <at> elephly.net> writes:
>
> > The Cuirass web interface shows the number of successful, failed, and
> > pending builds for each evaluation.  Looking at just these numbers it is
> > impossible to tell, how each of the supported architectures is affected.
> >
> > It would be good if we could separate the view by architecture.  Then we
> > could more easily determine that a change broke many builds for one
> > architecture while fixing builds on another.
> >
> > One way to do this would be to accept an optional query variable, e.g.
> >
> >     http://ci.guix.info/jobset/guix-master?system=x86_64-linux
>
> That is an option.  Another one is using a REST API.  It seems to have
> all the hype these days.  So the URL would turn into:
>
>      http://ci.guix.info/jobset/guix-master/system/x86_64-linux
>
> Though I freely admit, I don't completely understand the benefits of REST.
>
> Actually there are some more options to do this, but I think this should
go with
a more generic filtering/sorting capability, using a uniform
implementation. I
noticed this in a writeup before, Ludo asked me to turn that to a TODO on
the Cuirass repository, and I will do that once back from vacation.

> >
> > This could be selected from a drop-down on the page or exposed through a
> > number of links.
> >
> > --
> > Ricardo
>
>
>
>

[Message part 2 (text/html, inline)]

Message #14 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: Danny Milosavljevic <dannym <at> scratchpost.org>
To: Ricardo Wurmus <rekado <at> elephly.net>
Cc: 32575 <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 30 Aug 2018 11:41:35 +0200

[Message part 1 (text/plain, inline)]

Hi Ricardo,

On Wed, 29 Aug 2018 15:54:47 +0200
Ricardo Wurmus <rekado <at> elephly.net> wrote:

> The Cuirass web interface shows the number of successful, failed, and
> pending builds for each evaluation.  Looking at just these numbers it is
> impossible to tell, how each of the supported architectures is affected.
> 
> It would be good if we could separate the view by architecture.  Then we
> could more easily determine that a change broke many builds for one
> architecture while fixing builds on another.
> 
> One way to do this would be to accept an optional query variable, e.g.
> 
>     http://ci.guix.info/jobset/guix-master?system=x86_64-linux
> 
> This could be selected from a drop-down on the page or exposed through a
> number of links.

I agree.

Also, in the Javascript frontend I had a list of architecture links for each package.

The filter could be applied to show only a given set of architectures.

I think that for a portable package, the architecture it runs on is
an implementation detail - it should build on all of them.  If it doesn't,
that should show up as an error.

So I had

  hello      [x86_64-checkbox-log] [armhf-checkbox-log] [aarch64-checkbox-log]

and not

  hello.x86_64 [checkbox-log]
  hello.armhf [checkbox-log]
  hello.aarch64 [checkbox-log]

The latter looks more like these are different packages with different purposes -
which they really aren't from a user standpoint.

[Message part 2 (application/pgp-signature, inline)]

Message #17 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Danny Milosavljevic <dannym <at> scratchpost.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>, 32575 <at> debbugs.gnu.org,
 clement <at> lassieur.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 30 Aug 2018 14:14:07 +0200

Hi Danny & all,

Danny Milosavljevic <dannym <at> scratchpost.org> skribis:

> I think that for a portable package, the architecture it runs on is
> an implementation detail - it should build on all of them.  If it doesn't,
> that should show up as an error.
>
> So I had
>
>   hello      [x86_64-checkbox-log] [armhf-checkbox-log] [aarch64-checkbox-log]
>
> and not
>
>   hello.x86_64 [checkbox-log]
>   hello.armhf [checkbox-log]
>   hello.aarch64 [checkbox-log]
>
> The latter looks more like these are different packages with different purposes -
> which they really aren't from a user standpoint.

The difficulty is that, from Cuirass’ viewpoint, “hello.x86_64-linux”
and “hello.armhf-linux” are just two different unrelated jobs.

Perhaps what we would need is to internally change how jobs are
represented in the database: we could have one job, “hello”, connected
to one or more “builds”, each with its own system.

I think it would amount to splitting the “Builds” table into two tables:
“Builds” and “Jobs”.  Clément, does that make sense?

Ludo’.

Message #20 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 Danny Milosavljevic <dannym <at> scratchpost.org>, 32575 <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 30 Aug 2018 14:50:45 +0200

Ludovic Courtès <ludo <at> gnu.org> writes:

> Hi Danny & all,
>
> Danny Milosavljevic <dannym <at> scratchpost.org> skribis:
>
>> I think that for a portable package, the architecture it runs on is
>> an implementation detail - it should build on all of them.  If it doesn't,
>> that should show up as an error.
>>
>> So I had
>>
>>   hello      [x86_64-checkbox-log] [armhf-checkbox-log] [aarch64-checkbox-log]
>>
>> and not
>>
>>   hello.x86_64 [checkbox-log]
>>   hello.armhf [checkbox-log]
>>   hello.aarch64 [checkbox-log]
>>
>> The latter looks more like these are different packages with different purposes -
>> which they really aren't from a user standpoint.
>
> The difficulty is that, from Cuirass’ viewpoint, “hello.x86_64-linux”
> and “hello.armhf-linux” are just two different unrelated jobs.
>
> Perhaps what we would need is to internally change how jobs are
> represented in the database: we could have one job, “hello”, connected
> to one or more “builds”, each with its own system.
>
> I think it would amount to splitting the “Builds” table into two tables:
> “Builds” and “Jobs”.  Clément, does that make sense?

The 'job' word already has a meaning in Cuirass: it is the thing that is
returned from the evaluation.  For example, if Cuirass builds foo and
bar for x86_64 and i686, there will be exactly 4 jobs produced at each
evaluation :

 - foo.x86_64-linux     
 - foo.i686-linux       
 - bar.x86_64-linux     
 - bar.i686-linux       

(10 evaluations means 40 jobs produced, etc.)

(Most of them have a derivation file associated that already exists, so
they won't be added in the Build table.)

I don't think we should change that meaning because it will make
everything more difficult to understand.

But the Builds table looks like this:

CREATE TABLE Builds (
  derivation    TEXT NOT NULL PRIMARY KEY,
  evaluation    INTEGER NOT NULL,
  job_name      TEXT NOT NULL,
  system        TEXT NOT NULL,
  nix_name      TEXT NOT NULL,
  log           TEXT NOT NULL,
  status        INTEGER NOT NULL,
  timestamp     INTEGER NOT NULL,
  starttime     INTEGER NOT NULL,
  stoptime      INTEGER NOT NULL,
  FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
);

We even have the 'system' column, so to me we have everything we need,
and we could display on one line all the builds that have the same
'nix_name' for a given evaluation.

Does it make sense?
Clément

Message #23 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: ludo <at> gnu.org (Ludovic Courtès)
To: Clément Lassieur <clement <at> lassieur.org>
Cc: Ricardo Wurmus <rekado <at> elephly.net>,
 Danny Milosavljevic <dannym <at> scratchpost.org>, 32575 <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 30 Aug 2018 22:33:49 +0200

Clément Lassieur <clement <at> lassieur.org> skribis:

> Ludovic Courtès <ludo <at> gnu.org> writes:

[...]

>> Perhaps what we would need is to internally change how jobs are
>> represented in the database: we could have one job, “hello”, connected
>> to one or more “builds”, each with its own system.
>>
>> I think it would amount to splitting the “Builds” table into two tables:
>> “Builds” and “Jobs”.  Clément, does that make sense?
>
> The 'job' word already has a meaning in Cuirass: it is the thing that is
> returned from the evaluation.  For example, if Cuirass builds foo and
> bar for x86_64 and i686, there will be exactly 4 jobs produced at each
> evaluation :
>
>  - foo.x86_64-linux     
>  - foo.i686-linux       
>  - bar.x86_64-linux     
>  - bar.i686-linux       

Yes.

> CREATE TABLE Builds (
>   derivation    TEXT NOT NULL PRIMARY KEY,
>   evaluation    INTEGER NOT NULL,
>   job_name      TEXT NOT NULL,
>   system        TEXT NOT NULL,
>   nix_name      TEXT NOT NULL,
>   log           TEXT NOT NULL,
>   status        INTEGER NOT NULL,
>   timestamp     INTEGER NOT NULL,
>   starttime     INTEGER NOT NULL,
>   stoptime      INTEGER NOT NULL,
>   FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
> );
>
> We even have the 'system' column, so to me we have everything we need,
> and we could display on one line all the builds that have the same
> 'nix_name' for a given evaluation.

Hmm, probably, indeed (though ‘nix_name’ is meant as hint, not as a
key.)

Right now, build-aux/hydra/*.scm returns a list of jobs like this:

  (hello-2.10.x86_64-linux
    (derivation
      .
      "/gnu/store/2dl7n4l0l0vjzpjnv67fbb7vf24kw0ap-hello-2.10.drv")
    (description …)
    (long-description …)
    (license …)
    (home-page …)
    (maintainers "bug-guix <at> gnu.org")
    (max-silent-time . 3600)
    (timeout . 72000))
  ;; … likewise for ‘hello.i686-linux’, etc.

My proposal would be for build-aux/hydra/*.scm to return jobs that look
like this:

  (hello-2.10     ; <- no special naming convention
    (derivations
      .
      (("x86_64-linux" . /gnu/store/…-hello-2.10.drv")
       ("i686-linux" . /gnu/store/…-hello-2.10.drv")))
    (description …)
    (long-description …)
    (license …)
    (home-page …)
    (maintainers "bug-guix <at> gnu.org")
    (max-silent-time . 3600)
    (timeout . 72000))

Conceptually, that models the situation better, IMO.

But like you write, we probably already have everything to do something
along the lines of what Danny proposed.  The change above can come later
(it would be incompatible with Hydra, too.)

Thoughts?

Ludo’.

Message #26 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: Clément Lassieur <clement <at> lassieur.org>
To: Ludovic Courtès <ludo <at> gnu.org>
Cc: 32575 <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Fri, 31 Aug 2018 20:35:32 +0200

Ludovic Courtès <ludo <at> gnu.org> writes:

>> CREATE TABLE Builds (
>>   derivation    TEXT NOT NULL PRIMARY KEY,
>>   evaluation    INTEGER NOT NULL,
>>   job_name      TEXT NOT NULL,
>>   system        TEXT NOT NULL,
>>   nix_name      TEXT NOT NULL,
>>   log           TEXT NOT NULL,
>>   status        INTEGER NOT NULL,
>>   timestamp     INTEGER NOT NULL,
>>   starttime     INTEGER NOT NULL,
>>   stoptime      INTEGER NOT NULL,
>>   FOREIGN KEY (evaluation) REFERENCES Evaluations (id)
>> );
>>
>> We even have the 'system' column, so to me we have everything we need,
>> and we could display on one line all the builds that have the same
>> 'nix_name' for a given evaluation.
>
> Hmm, probably, indeed (though ‘nix_name’ is meant as hint, not as a
> key.)
>
> Right now, build-aux/hydra/*.scm returns a list of jobs like this:
>
>   (hello-2.10.x86_64-linux
>     (derivation
>       .
>       "/gnu/store/2dl7n4l0l0vjzpjnv67fbb7vf24kw0ap-hello-2.10.drv")
>     (description …)
>     (long-description …)
>     (license …)
>     (home-page …)
>     (maintainers "bug-guix <at> gnu.org")
>     (max-silent-time . 3600)
>     (timeout . 72000))
>   ;; … likewise for ‘hello.i686-linux’, etc.
>
> My proposal would be for build-aux/hydra/*.scm to return jobs that look
> like this:
>
>   (hello-2.10     ; <- no special naming convention
      ^
This is 'nix-name' (which is 'derivation-name')

>     (derivations
>       .
>       (("x86_64-linux" . /gnu/store/…-hello-2.10.drv")
>        ("i686-linux" . /gnu/store/…-hello-2.10.drv")))
              ^
This is 'system' (which is 'derivation-system')

>     (description …)
>     (long-description …)
>     (license …)
>     (home-page …)
>     (maintainers "bug-guix <at> gnu.org")
>     (max-silent-time . 3600)
>     (timeout . 72000))

So everything is already in the derivations that are in Cuirass.  Why
would we need to change the interface with the evaluator
(build-aux/hydra/*.scm)?

Clément

> Conceptually, that models the situation better, IMO.
>
> But like you write, we probably already have everything to do something
> along the lines of what Danny proposed.  The change above can come later
> (it would be incompatible with Hydra, too.)
>
> Thoughts?
>
> Ludo’.

Message #29 received at 32575 <at> debbugs.gnu.org (full text, mbox):

From: Ricardo Wurmus <rekado <at> elephly.net>
To: Joshua Branson <jbranso <at> fastmail.com>
Cc: 32575 <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 30 Aug 2018 10:09:17 +0200

Hi Joshua,

> Ricardo Wurmus <rekado <at> elephly.net> writes:
>
>> The Cuirass web interface shows the number of successful, failed, and
>> pending builds for each evaluation.  Looking at just these numbers it is
>> impossible to tell, how each of the supported architectures is affected.
>>
>> It would be good if we could separate the view by architecture.  Then we
>> could more easily determine that a change broke many builds for one
>> architecture while fixing builds on another.
>>
>> One way to do this would be to accept an optional query variable, e.g.
>>
>>     http://ci.guix.info/jobset/guix-master?system=x86_64-linux
>
> That is an option.  Another one is using a REST API.  It seems to have
> all the hype these days.  So the URL would turn into:
>
>      http://ci.guix.info/jobset/guix-master/system/x86_64-linux
>
> Though I freely admit, I don't completely understand the benefits of REST.

REST doesn’t quite apply here, because we only use GET — the Cuirass web
interface is read-only.  A big part of REST is to use HTTP verbs in an
appropriate manner and keep the URLs as resource identifiers the same
for all verbs.

What you refer to is the related trend to using Clean URLs:

    https://en.wikipedia.org/wiki/Clean_URL

These are often used with a RESTful API.

I think that filtering of a dynamic resource could very well be done
with a GET query string.  A Clean URL would make more sense for
something that doesn’t change as quickly (e.g. a particular product in a
catalogue).

--
Ricardo

Message #34 received at 32575-done <at> debbugs.gnu.org (full text, mbox):

From: Mathieu Othacehe <othacehe <at> gnu.org>
To: Clément Lassieur <clement <at> lassieur.org>
Cc: Ludovic Courtès <ludo <at> gnu.org>, 32575-done <at> debbugs.gnu.org
Subject: Re: bug#32575: [Cuirass] Filter results by architecture
Date: Thu, 25 Mar 2021 14:32:11 +0100

Hello,

The Builds table now has as "system" field on Cuirass master. The job
system is displayed on http://ci.guix.gnu.org/eval/xxx page. With
1ed93601089e774df849bc4ffab718bb1f142d34 it is also possible to sort by
table columns, including the "System" column.

Closing this one,

Thanks,

Mathieu

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.