GNU bug report logs - #46877
Guix assumes ideal network exists, does not

Previous Next

To reply to this bug, email your comments to 46877 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Tobias Geerinckx-Rice <me <at> tobias.gr>
To: bug-guix <at> gnu.org
Subject: Guix assumes ideal network exists, does not
Date: Tue, 02 Mar 2021 17:32:50 +0100

[Message part 1 (text/plain, inline)]

Guixfriends,

Errors like the one below are omnipresent.  It's a problem.

Expecting an ever-increasing (16k+?) number of requests to finish 
flawlessly or aborting a very long (20+ minutes here) process is 
doomed.

We don't seem to have a plan forward (I include myself!).  Looking 
at other package managers, a blunt ‘retry a finite number of 
times’ seems common.  Guix's ‘pretend that "transient" is a magic 
powerword’ approach less so.

What do people think?  Does the heavy pipelining complicate 
matters?

--8<---------------cut here---------------start------------->8---
~ $ guix weather --substitute-urls=https://guix.tobias.gr
computing 16,745 package derivations for x86_64-linux...
looking for 18,127 store items on https://guix.tobias.gr...
updating substitutes from 'https://guix.tobias.gr'... 
41.6%Backtrace:
         11 (primitive-load 
         "/home/nckx/.config/guix/current/bin/guix")
In guix/ui.scm:
 2164:12 10 (run-guix-command _ . _)
In ice-9/boot-9.scm:
 1736:10  9 (with-exception-handler _ _ #:unwind? _ 
 #:unwind-for-type _)
 1731:15  8 (with-exception-handler #<procedure 7fe8a173d7e0 at 
 ice-9/boot-9.scm:1815:7 (exn)> _ #:unwind? _ …)
In guix/scripts/weather.scm:
   546:9  7 (_)
In guix/build/utils.scm:
  569:23  6 (every* #<procedure 7fe875fdd4e0 at 
  guix/scripts/weather.scm:546:17 (server)> _)
In guix/scripts/weather.scm:
  547:19  5 (_ "https://guix.tobias.gr")
  116:17  4 (report-server-coverage _ _ #:display-missing? _)
In unknown file:
          3 (_ #<procedure 7fe86e3cbbe0 at 
          guix/scripts/weather.scm:184:2 ()> #<procedure list _> 
          . #w())
In guix/scripts/substitute.scm:
  411:23  2 (lookup-narinfos _ _ #:open-connection _)
  371:26  1 (fetch-narinfos _ _ #:open-connection _)
In ice-9/boot-9.scm:
 1669:16  0 (raise-exception _ #:continuable? _)

ice-9/boot-9.scm:1669:16: In procedure raise-exception:
Throw to key `gnutls-error' with args `(#<gnutls-error-enum Error 
in the pull function.> read_from_session_record_port)'.
--8<---------------cut here---------------end--------------->8---

Kind regards,

T G-R

[signature.asc (application/pgp-signature, inline)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Leo Famulari <leo <at> famulari.name>
To: Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: 46877 <at> debbugs.gnu.org
Subject: Re: bug#46877: Guix assumes ideal network exists, does not
Date: Tue, 2 Mar 2021 14:55:02 -0500

On Tue, Mar 02, 2021 at 05:32:50PM +0100, Tobias Geerinckx-Rice via Bug reports for GNU Guix wrote:
> Guixfriends,
> 
> Errors like the one below are omnipresent.  It's a problem.
> 
> Expecting an ever-increasing (16k+?) number of requests to finish flawlessly
> or aborting a very long (20+ minutes here) process is doomed.
> 
> We don't seem to have a plan forward (I include myself!).  Looking at other
> package managers, a blunt ‘retry a finite number of times’ seems common.
> Guix's ‘pretend that "transient" is a magic powerword’ approach less so.
> 
> What do people think?  Does the heavy pipelining complicate matters?
> 
> --8<---------------cut here---------------start------------->8---
> ~ $ guix weather --substitute-urls=https://guix.tobias.gr

I sometimes see similar failures with `guix weather`.

To clarify, do you also see them when substituting? Or just when
checking for substitutes?

Message #14 received at 46877 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 46877 <at> debbugs.gnu.org
Subject: Re: bug#46877: Guix assumes ideal network exists, does not
Date: Tue, 2 Mar 2021 21:00:57 +0100

Hi Tobias,

On Tue, 2 Mar 2021 at 17:33, Tobias Geerinckx-Rice via Bug reports for
GNU Guix <bug-guix <at> gnu.org> wrote:

> We don't seem to have a plan forward (I include myself!).  Looking
> at other package managers, a blunt ‘retry a finite number of
> times’ seems common.  Guix's ‘pretend that "transient" is a magic
> powerword’ approach less so.

If I read correctly, the issue comes from (guix scripts substitutes)
in 'fetch-narinfos':

--8<---------------cut here---------------start------------->8---
                          (call-with-connection-error-handling
                           uri
                           (lambda ()
                             (http-multiple-get uri
                                                handle-narinfo-response '()
                                                requests
                                                #:open-connection
open-connection
                                                #:verify-certificate? #f))))))
--8<---------------cut here---------------end--------------->8---

where 'requests' can be really large (all the missing information from
local cache).

Instead of processing the list in one go, we could imagine split it
into several chunks, say X, and so process only X requests by X
requests.

WDYT?


Cheers,
simon

Message #17 received at 46877 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Tobias Geerinckx-Rice <me <at> tobias.gr>
Cc: 46877 <at> debbugs.gnu.org
Subject: Re: bug#46877: Guix assumes ideal network exists, does not
Date: Tue, 2 Mar 2021 21:18:17 +0100

On Tue, 2 Mar 2021 at 21:00, zimoun <zimon.toutoune <at> gmail.com> wrote:

> Instead of processing the list in one go, we could imagine split it
> into several chunks, say X, and so process only X requests by X
> requests.

My bad, it is already the case where X=1000 in (guix http-client).
Well, the error / retry seems poorly handled in 'http-multiple-get'.

Cheers,
simon

Message #20 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
Cc: Tobias Geerinckx-Rice <me <at> tobias.gr>, 46877 <at> debbugs.gnu.org
Subject: Re: bug#46877: Guix assumes ideal network exists, does not
Date: Wed, 10 Mar 2021 12:17:41 +0100

Hi,

Tobias Geerinckx-Rice via Bug reports for GNU Guix <bug-guix <at> gnu.org>
skribis:

> ~ $ guix weather --substitute-urls=https://guix.tobias.gr
> computing 16,745 package derivations for x86_64-linux...
> looking for 18,127 store items on https://guix.tobias.gr...
> updating substitutes from 'https://guix.tobias.gr'... 
> 41.6%Backtrace:
>           11 (primitive-load 
>           "/home/nckx/.config/guix/current/bin/guix")
> In guix/ui.scm:
>   2164:12 10 (run-guix-command _ . _)
> In ice-9/boot-9.scm:
>   1736:10  9 (with-exception-handler _ _ #:unwind? _ 
>   #:unwind-for-type _)
>   1731:15  8 (with-exception-handler #<procedure 7fe8a173d7e0 at 
>   ice-9/boot-9.scm:1815:7 (exn)> _ #:unwind? _ …)
> In guix/scripts/weather.scm:
>     546:9  7 (_)
> In guix/build/utils.scm:
>    569:23  6 (every* #<procedure 7fe875fdd4e0 at 
>    guix/scripts/weather.scm:546:17 (server)> _)
> In guix/scripts/weather.scm:
>    547:19  5 (_ "https://guix.tobias.gr")
>    116:17  4 (report-server-coverage _ _ #:display-missing? _)
> In unknown file:
>            3 (_ #<procedure 7fe86e3cbbe0 at 
>            guix/scripts/weather.scm:184:2 ()> #<procedure list _> 
>            . #w())
> In guix/scripts/substitute.scm:
>    411:23  2 (lookup-narinfos _ _ #:open-connection _)
>    371:26  1 (fetch-narinfos _ _ #:open-connection _)
> In ice-9/boot-9.scm:
>   1669:16  0 (raise-exception _ #:continuable? _)
>
> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
> Throw to key `gnutls-error' with args `(#<gnutls-error-enum Error 
> in the pull function.> read_from_session_record_port)'.

Hmm isn’t it a regression?

First, in (guix scripts substitute) there’s a ‘with-networking’ macro to
nicely present networking errors; perhaps we’re missing it here?

Second, is it a genuine bug?  Do you see the same thing from (guix
scripts substitute)?

Thanks,
Ludo’.

Message #26 received at 46877 <at> debbugs.gnu.org (full text, mbox):

From: zimoun <zimon.toutoune <at> gmail.com>
To: Ludovic Courtès <ludo <at> gnu.org>, 46877 <at> debbugs.gnu.org
Subject: Re: bug#46877: Guix assumes ideal network exists, does not
Date: Wed, 10 Mar 2021 23:04:59 +0100

Hi,

On Wed, 10 Mar 2021 at 12:17, Ludovic Courtès <ludo <at> gnu.org> wrote:


>> In guix/scripts/substitute.scm:
>>    411:23  2 (lookup-narinfos _ _ #:open-connection _)
>>    371:26  1 (fetch-narinfos _ _ #:open-connection _)
>> In ice-9/boot-9.scm:
>>   1669:16  0 (raise-exception _ #:continuable? _)
>>
>> ice-9/boot-9.scm:1669:16: In procedure raise-exception:
>> Throw to key `gnutls-error' with args `(#<gnutls-error-enum Error 
>> in the pull function.> read_from_session_record_port)'.
>
> Hmm isn’t it a regression?
>
> First, in (guix scripts substitute) there’s a ‘with-networking’ macro to
> nicely present networking errors; perhaps we’re missing it here?
>
> Second, is it a genuine bug?  Do you see the same thing from (guix
> scripts substitute)?

I do not know if it is related, the bug#47055 seems “similar”:

<http://issues.guix.gnu.org/issue/47055>


Cheers,
simon

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.