GNU bug report logs -
#41708
"guix weather" : 504 error
Previous Next
Reported by: zimoun <zimon.toutoune <at> gmail.com>
Date: Thu, 4 Jun 2020 14:36:02 UTC
Severity: normal
Done: Christopher Baines <mail <at> cbaines.net>
Bug is archived. No further changes may be made.
To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 41708 in the body.
You can then email your comments to 41708 AT debbugs.gnu.org in the normal way.
Toggle the display of automated, internal messages from the tracker.
Report forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Thu, 04 Jun 2020 14:36:02 GMT)
Full text and
rfc822 format available.
Acknowledgement sent
to
zimoun <zimon.toutoune <at> gmail.com>
:
New bug report received and forwarded. Copy sent to
bug-guix <at> gnu.org
.
(Thu, 04 Jun 2020 14:36:02 GMT)
Full text and
rfc822 format available.
Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):
Dear,
By default, "guix weather" returns:
--8<---------------cut here---------------start------------->8---
'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
--8<---------------cut here---------------end--------------->8---
which is not fatal but annoying. Especially, it takes time.
As discussed on IRC [1], it seems that it is a bug server side.
In addition, something appears similar with Bayfront, well the 4
substitutes servers that I know returns:
--8<---------------cut here---------------start------------->8---
'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
'https://bayfront.guix.gnu.org/api/queue?nr=1000' returned 504
("Gateway Time-out")
(continuous integration information unavailable)
'https://guix.tobias.gr/api/queue?nr=1000' returned 410 ("Gone")
--8<---------------cut here---------------end--------------->8---
All the best,
simon
[1] http://logs.guix.gnu.org/guix/2020-06-03.log#201511
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Tue, 09 Jun 2020 21:13:01 GMT)
Full text and
rfc822 format available.
Message #8 received at 41708 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
zimoun <zimon.toutoune <at> gmail.com> writes:
> By default, "guix weather" returns:
>
> --8<---------------cut here---------------start------------->8---
> 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
> --8<---------------cut here---------------end--------------->8---
>
> which is not fatal but annoying. Especially, it takes time.
>
> As discussed on IRC [1], it seems that it is a bug server side.
>
> In addition, something appears similar with Bayfront, well the 4
> substitutes servers that I know returns:
>
> --8<---------------cut here---------------start------------->8---
> 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
> 'https://bayfront.guix.gnu.org/api/queue?nr=1000' returned 504
> ("Gateway Time-out")
> (continuous integration information unavailable)
> 'https://guix.tobias.gr/api/queue?nr=1000' returned 410 ("Gone")
> --8<---------------cut here---------------end--------------->8---
Hey,
So I think I've got a patch [1] to Cuirass to "fix" this.
1: https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00117.html
I expected this was due to the complicated part of the db-get-builds
query, but I think it's actually down to the simple part that fetches
all the outputs for all the builds. Fetching the outputs for a build is
slow, because at the moment, there's no index on the Outputs field, so
looking up the outputs requires reading through the whole table, and the
Outputs table can be quite large.
The performance should improve with the additional query. To see why,
you can use EXPLAIN QUERY PLAN to see what SQLite plans to do when
processing the query:
sqlite> EXPLAIN QUERY PLAN SELECT name, path FROM Outputs WHERE derivation = 'foo';
QUERY PLAN
`--SCAN TABLE Outputs
sqlite> CREATE INDEX Outputs_derivation_index ON Outputs (derivation);
sqlite> EXPLAIN QUERY PLAN SELECT name, path FROM Outputs WHERE derivation = 'foo';
QUERY PLAN
`--SEARCH TABLE Outputs USING INDEX Outputs_derivation_index (derivation=?)
I believe that searching the table using an index is going to be faster
than scanning the table, and testing locally and on bayfront suggests
this will resolve the performance issue.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Sat, 13 Jun 2020 13:20:01 GMT)
Full text and
rfc822 format available.
Message #11 received at 41708 <at> debbugs.gnu.org (full text, mbox):
Hi Chris,
On Tue, 09 Jun 2020 at 22:12, Christopher Baines <mail <at> cbaines.net> wrote:
> So I think I've got a patch [1] to Cuirass to "fix" this.
>
> 1: https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00117.html
[...]
> I believe that searching the table using an index is going to be faster
> than scanning the table, and testing locally and on bayfront suggests
> this will resolve the performance issue.
Is the patch installed on Bayfront?
If yes, I still see 'returned 504 ("Gateway Time-out")' when quering
Bayfront as '--substitute-urls'.
If no, is it possible to install it? for testing it in "real life".
Cheers,
simon
ps:
It is an old request because I complained [1] about it one year ago.
Maybe something related to the summer coming. ;-)
1: https://lists.gnu.org/archive/html/help-guix/2019-05/msg00355.html
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Sat, 13 Jun 2020 16:24:02 GMT)
Full text and
rfc822 format available.
Message #14 received at 41708 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
zimoun <zimon.toutoune <at> gmail.com> writes:
> On Tue, 09 Jun 2020 at 22:12, Christopher Baines <mail <at> cbaines.net> wrote:
>
>> So I think I've got a patch [1] to Cuirass to "fix" this.
>>
>> 1: https://lists.gnu.org/archive/html/guix-devel/2020-06/msg00117.html
>
> [...]
>
>> I believe that searching the table using an index is going to be faster
>> than scanning the table, and testing locally and on bayfront suggests
>> this will resolve the performance issue.
>
> Is the patch installed on Bayfront?
> If yes, I still see 'returned 504 ("Gateway Time-out")' when quering
> Bayfront as '--substitute-urls'.
> If no, is it possible to install it? for testing it in "real life".
>
>
> Cheers,
> simon
>
> ps:
> It is an old request because I complained [1] about it one year ago.
> Maybe something related to the summer coming. ;-)
>
> 1: https://lists.gnu.org/archive/html/help-guix/2019-05/msg00355.html
I've now deployed the change to Bayfront, let me know if you see a
difference? :)
Thanks,
Chris
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Sat, 13 Jun 2020 17:57:01 GMT)
Full text and
rfc822 format available.
Message #17 received at 41708 <at> debbugs.gnu.org (full text, mbox):
Hi Chris,
On Sat, 13 Jun 2020 at 17:23, Christopher Baines <mail <at> cbaines.net> wrote:
> I've now deployed the change to Bayfront, let me know if you see a
> difference? :)
Yes! :-)
--8<---------------cut here---------------start------------->8---
$ guix weather guix --substitute-urls='https://ci.guix.gnu.org https://bayfront.guix.gnu.org'
computing 1 package derivations for x86_64-linux...
looking for 1 store items on https://ci.guix.gnu.org...
https://ci.guix.gnu.org
100.0% substitutes available (1 out of 1)
at least 106.4 MiB of nars (compressed)
278.7 MiB on disk (uncompressed)
0.002 seconds per request (0.0 seconds in total)
543.8 requests per second
'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
looking for 1 store items on https://bayfront.guix.gnu.org...
https://bayfront.guix.gnu.org
100.0% substitutes available (1 out of 1)
at least 106.3 MiB of nars (compressed)
278.6 MiB on disk (uncompressed)
0.001 seconds per request (0.0 seconds in total)
791.8 requests per second
at least 1,000 queued builds
x86_64-linux: 1000 (100.0%)
build rate: 9.22 builds per hour
x86_64-linux: 9.22 builds per hour
--8<---------------cut here---------------end--------------->8---
I will try a couple more of queries. But I propose to push the patch on
Berlin too. WDYT?
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Mon, 15 Jun 2020 18:44:02 GMT)
Full text and
rfc822 format available.
Message #20 received at 41708 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
zimoun <zimon.toutoune <at> gmail.com> writes:
> Hi Chris,
>
> On Sat, 13 Jun 2020 at 17:23, Christopher Baines <mail <at> cbaines.net> wrote:
>
>> I've now deployed the change to Bayfront, let me know if you see a
>> difference? :)
>
> Yes! :-)
>
> --8<---------------cut here---------------start------------->8---
> $ guix weather guix --substitute-urls='https://ci.guix.gnu.org https://bayfront.guix.gnu.org'
> computing 1 package derivations for x86_64-linux...
>
> looking for 1 store items on https://ci.guix.gnu.org...
> https://ci.guix.gnu.org
> 100.0% substitutes available (1 out of 1)
> at least 106.4 MiB of nars (compressed)
> 278.7 MiB on disk (uncompressed)
> 0.002 seconds per request (0.0 seconds in total)
> 543.8 requests per second
> 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
>
> looking for 1 store items on https://bayfront.guix.gnu.org...
> https://bayfront.guix.gnu.org
> 100.0% substitutes available (1 out of 1)
> at least 106.3 MiB of nars (compressed)
> 278.6 MiB on disk (uncompressed)
> 0.001 seconds per request (0.0 seconds in total)
> 791.8 requests per second
>
> at least 1,000 queued builds
> x86_64-linux: 1000 (100.0%)
> build rate: 9.22 builds per hour
> x86_64-linux: 9.22 builds per hour
> --8<---------------cut here---------------end--------------->8---
>
> I will try a couple more of queries. But I propose to push the patch on
> Berlin too. WDYT?
I believe it's been deployed now. It was working earlier today at least,
however all of Cuirass seems to be stuck now.
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Mon, 15 Jun 2020 20:04:02 GMT)
Full text and
rfc822 format available.
Message #23 received at 41708 <at> debbugs.gnu.org (full text, mbox):
On Mon, 15 Jun 2020 at 19:43, Christopher Baines <mail <at> cbaines.net> wrote:
> I believe it's been deployed now. It was working earlier today at least,
> however all of Cuirass seems to be stuck now.
Well, it seems working. Feel free to close the bug.
--8<---------------cut here---------------start------------->8---
$ guix weather
computing 13,814 package derivations for x86_64-linux...
looking for 14,354 store items on https://ci.guix.gnu.org...
updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
https://ci.guix.gnu.org
88.2% substitutes available (12,661 out of 14,354)
at least 72,495.6 MiB of nars (compressed)
118,865.0 MiB on disk (uncompressed)
0.021 seconds per request (303.4 seconds in total)
47.3 requests per second
0.0% (0 out of 1,693) of the missing items are queued
at least 1,000 queued builds
i686-linux: 310 (31.0%)
aarch64-linux: 230 (23.0%)
armhf-linux: 313 (31.3%)
x86_64-linux: 147 (14.7%)
build rate: 39.53 builds per hour
x86_64-linux: 9.53 builds per hour
aarch64-linux: 9.57 builds per hour
armhf-linux: 8.54 builds per hour
i686-linux: 11.90 builds per hour
--8<---------------cut here---------------end--------------->8---
Cheers,
simon
Reply sent
to
Christopher Baines <mail <at> cbaines.net>
:
You have taken responsibility.
(Mon, 15 Jun 2020 20:24:02 GMT)
Full text and
rfc822 format available.
Notification sent
to
zimoun <zimon.toutoune <at> gmail.com>
:
bug acknowledged by developer.
(Mon, 15 Jun 2020 20:24:02 GMT)
Full text and
rfc822 format available.
Message #28 received at 41708-done <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
zimoun <zimon.toutoune <at> gmail.com> writes:
> On Mon, 15 Jun 2020 at 19:43, Christopher Baines <mail <at> cbaines.net> wrote:
>
>> I believe it's been deployed now. It was working earlier today at least,
>> however all of Cuirass seems to be stuck now.
>
> Well, it seems working. Feel free to close the bug.
Great, closing :)
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Tue, 16 Jun 2020 10:16:02 GMT)
Full text and
rfc822 format available.
Message #31 received at 41708 <at> debbugs.gnu.org (full text, mbox):
zimoun <zimon.toutoune <at> gmail.com> skribis:
> On Mon, 15 Jun 2020 at 19:43, Christopher Baines <mail <at> cbaines.net> wrote:
>
>> I believe it's been deployed now. It was working earlier today at least,
>> however all of Cuirass seems to be stuck now.
>
> Well, it seems working. Feel free to close the bug.
>
> $ guix weather
> computing 13,814 package derivations for x86_64-linux...
> looking for 14,354 store items on https://ci.guix.gnu.org...
> updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> https://ci.guix.gnu.org
> 88.2% substitutes available (12,661 out of 14,354)
> at least 72,495.6 MiB of nars (compressed)
> 118,865.0 MiB on disk (uncompressed)
> 0.021 seconds per request (303.4 seconds in total)
> 47.3 requests per second
>
> 0.0% (0 out of 1,693) of the missing items are queued
> at least 1,000 queued builds
> i686-linux: 310 (31.0%)
> aarch64-linux: 230 (23.0%)
> armhf-linux: 313 (31.3%)
> x86_64-linux: 147 (14.7%)
> build rate: 39.53 builds per hour
> x86_64-linux: 9.53 builds per hour
> aarch64-linux: 9.57 builds per hour
> armhf-linux: 8.54 builds per hour
> i686-linux: 11.90 builds per hour
Awesome, thank you Chris!
Ludo’.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Sat, 20 Jun 2020 00:05:02 GMT)
Full text and
rfc822 format available.
Message #34 received at 41708 <at> debbugs.gnu.org (full text, mbox):
Dear Chris,
On Mon, 15 Jun 2020 at 21:23, Christopher Baines <mail <at> cbaines.net> wrote:
> zimoun <zimon.toutoune <at> gmail.com> writes:
>> On Mon, 15 Jun 2020 at 19:43, Christopher Baines <mail <at> cbaines.net> wrote:
>>> I believe it's been deployed now. It was working earlier today at least,
>>> however all of Cuirass seems to be stuck now.
>>
>> Well, it seems working. Feel free to close the bug.
>
> Great, closing :)
I do not know if it is only this evening, but something appears wrong:
--8<---------------cut here---------------start------------->8---
$ guix weather
computing 13,814 package derivations for x86_64-linux...
looking for 14,354 store items on https://ci.guix.gnu.org...
updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
https://ci.guix.gnu.org
88.3% substitutes available (12,672 out of 14,354)
at least 72,508.1 MiB of nars (compressed)
118,880.3 MiB on disk (uncompressed)
0.001 seconds per request (12.8 seconds in total)
1,123.2 requests per second
'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
--8<---------------cut here---------------end--------------->8---
For the record:
--8<---------------cut here---------------start------------->8---
$ guix describe
Generation 36 Jun 08 2020 00:18:47 (current)
guix e782756
repository URL: https://git.savannah.gnu.org/git/guix.git
branch: master
commit: e78275608065ef073775fabb9f1a757da65851f2
--8<---------------cut here---------------end--------------->8---
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Sat, 20 Jun 2020 08:23:01 GMT)
Full text and
rfc822 format available.
Message #37 received at 41708 <at> debbugs.gnu.org (full text, mbox):
[Message part 1 (text/plain, inline)]
zimoun <zimon.toutoune <at> gmail.com> writes:
> Dear Chris,
>
> On Mon, 15 Jun 2020 at 21:23, Christopher Baines <mail <at> cbaines.net> wrote:
>> zimoun <zimon.toutoune <at> gmail.com> writes:
>>> On Mon, 15 Jun 2020 at 19:43, Christopher Baines <mail <at> cbaines.net> wrote:
>
>>>> I believe it's been deployed now. It was working earlier today at least,
>>>> however all of Cuirass seems to be stuck now.
>>>
>>> Well, it seems working. Feel free to close the bug.
>>
>> Great, closing :)
>
> I do not know if it is only this evening, but something appears wrong:
>
> --8<---------------cut here---------------start------------->8---
> $ guix weather
> computing 13,814 package derivations for x86_64-linux...
> looking for 14,354 store items on https://ci.guix.gnu.org...
> updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> https://ci.guix.gnu.org
> 88.3% substitutes available (12,672 out of 14,354)
> at least 72,508.1 MiB of nars (compressed)
> 118,880.3 MiB on disk (uncompressed)
> 0.001 seconds per request (12.8 seconds in total)
> 1,123.2 requests per second
> 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
> --8<---------------cut here---------------end--------------->8---
Did you check if any pages loaded for Cuirass, or was it just this API
endpoint that was timing out?
Thanks,
Chris
[signature.asc (application/pgp-signature, inline)]
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Wed, 24 Jun 2020 10:11:01 GMT)
Full text and
rfc822 format available.
Message #40 received at 41708 <at> debbugs.gnu.org (full text, mbox):
Hi Chris,
On Sat, 20 Jun 2020 at 10:22, Christopher Baines <mail <at> cbaines.net> wrote:
> > --8<---------------cut here---------------start------------->8---
> > $ guix weather
> > computing 13,814 package derivations for x86_64-linux...
> > looking for 14,354 store items on https://ci.guix.gnu.org...
> > updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
> > https://ci.guix.gnu.org
> > 88.3% substitutes available (12,672 out of 14,354)
> > at least 72,508.1 MiB of nars (compressed)
> > 118,880.3 MiB on disk (uncompressed)
> > 0.001 seconds per request (12.8 seconds in total)
> > 1,123.2 requests per second
> > 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
> > --8<---------------cut here---------------end--------------->8---
>
> Did you check if any pages loaded for Cuirass, or was it just this API
> endpoint that was timing out?
I am not sure to understand your both questions.
What I see as an end-user is again "returned 504 ("Gateway
Time-out")". And I do not know if something has changed on
ci.guix.gnu.org or if there is another problem.
Tell me if I can do something for helping.
Cheers,
simon
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Wed, 24 Jun 2020 12:59:01 GMT)
Full text and
rfc822 format available.
Message #43 received at 41708 <at> debbugs.gnu.org (full text, mbox):
zimoun <zimon.toutoune <at> gmail.com> writes:
> Hi Chris,
>
> On Sat, 20 Jun 2020 at 10:22, Christopher Baines <mail <at> cbaines.net> wrote:
>
>> > --8<---------------cut here---------------start------------->8---
>> > $ guix weather
>> > computing 13,814 package derivations for x86_64-linux...
>> > looking for 14,354 store items on https://ci.guix.gnu.org...
>> > updating substitutes from 'https://ci.guix.gnu.org'... 100.0%
>> > https://ci.guix.gnu.org
>> > 88.3% substitutes available (12,672 out of 14,354)
>> > at least 72,508.1 MiB of nars (compressed)
>> > 118,880.3 MiB on disk (uncompressed)
>> > 0.001 seconds per request (12.8 seconds in total)
>> > 1,123.2 requests per second
>> > 'https://ci.guix.gnu.org/api/queue?nr=1000' returned 504 ("Gateway Time-out")
>> > --8<---------------cut here---------------end--------------->8---
>>
>> Did you check if any pages loaded for Cuirass, or was it just this API
>> endpoint that was timing out?
>
> I am not sure to understand your both questions.
> What I see as an end-user is again "returned 504 ("Gateway
> Time-out")". And I do not know if something has changed on
> ci.guix.gnu.org or if there is another problem.
> Tell me if I can do something for helping.
Right now I see a response when I visit that URL.
But recently the web interface has been a bit flakey and I had to
restart it not long ago.
Information forwarded
to
bug-guix <at> gnu.org
:
bug#41708
; Package
guix
.
(Wed, 24 Jun 2020 13:07:02 GMT)
Full text and
rfc822 format available.
Message #46 received at 41708 <at> debbugs.gnu.org (full text, mbox):
Hi Ricardo,
On Wed, 24 Jun 2020 at 14:58, Ricardo Wurmus <rekado <at> elephly.net> wrote:
> Right now I see a response when I visit that URL.
> But recently the web interface has been a bit flakey and I had to
> restart it not long ago.
Thank you for the notification.
Now it works again as expected for me. I have read on #guix that Ludo
restarted the Web Interface at 14:32 and just after the 504
disappeared.
Let say that now this bug is close. :-)
Cheers,
simon
bug archived.
Request was from
Debbugs Internal Request <help-debbugs <at> gnu.org>
to
internal_control <at> debbugs.gnu.org
.
(Thu, 23 Jul 2020 11:24:04 GMT)
Full text and
rfc822 format available.
This bug report was last modified 3 years and 249 days ago.
Previous Next
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997,2003 nCipher Corporation Ltd,
1994-97 Ian Jackson.