X-Loop: help-debbugs@HIDDEN Subject: bug#34033: Offloading sometimes hangs Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-guix@HIDDEN Resent-Date: Thu, 10 Jan 2019 16:10:02 +0000 Resent-Message-ID: <handler.34033.B.154713658627317 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: report 34033 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 34033 <at> debbugs.gnu.org X-Debbugs-Original-To: bug-guix@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.154713658627317 (code B ref -1); Thu, 10 Jan 2019 16:10:02 +0000 Received: (at submit) by debbugs.gnu.org; 10 Jan 2019 16:09:46 +0000 Received: from localhost ([127.0.0.1]:54111 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1ghctt-00076W-UC for submit <at> debbugs.gnu.org; Thu, 10 Jan 2019 11:09:46 -0500 Received: from eggs.gnu.org ([209.51.188.92]:55842) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1ghcts-00076J-Gx for submit <at> debbugs.gnu.org; Thu, 10 Jan 2019 11:09:44 -0500 Received: from lists.gnu.org ([209.51.188.17]:40213) by eggs.gnu.org with esmtps (TLS1.0:RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1ghctn-0005bJ-9n for submit <at> debbugs.gnu.org; Thu, 10 Jan 2019 11:09:39 -0500 Received: from eggs.gnu.org ([209.51.188.92]:37807) by lists.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1ghctm-0006Fe-6V for bug-guix@HIDDEN; Thu, 10 Jan 2019 11:09:39 -0500 X-Spam-Checker-Version: SpamAssassin 3.3.2 (2011-06-06) on eggs.gnu.org X-Spam-Level: X-Spam-Status: No, score=-0.0 required=5.0 tests=BAYES_20 autolearn=disabled version=3.3.2 Received: from Debian-exim by eggs.gnu.org with spam-scanned (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1ghctl-0005Zd-7u for bug-guix@HIDDEN; Thu, 10 Jan 2019 11:09:38 -0500 Received: from hera.aquilenet.fr ([2a0c:e300::1]:60310) by eggs.gnu.org with esmtps (TLS1.0:DHE_RSA_AES_256_CBC_SHA1:32) (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1ghctk-0005XJ-TI for bug-guix@HIDDEN; Thu, 10 Jan 2019 11:09:37 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 75E0E195E for <bug-guix@HIDDEN>; Thu, 10 Jan 2019 17:09:33 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id pvs2nnhO917r for <bug-guix@HIDDEN>; Thu, 10 Jan 2019 17:09:32 +0100 (CET) Received: from ribbon (unknown [IPv6:2001:660:6102:320:e120:2c8f:8909:cdfe]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 46F56193C for <bug-guix@HIDDEN>; Thu, 10 Jan 2019 17:09:32 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 21 =?UTF-8?Q?Niv=C3=B4se?= an 227 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Thu, 10 Jan 2019 17:09:31 +0100 Message-ID: <87o98obikk.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: Genre and OS details not recognized. X-Received-From: 2a0c:e300::1 X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.6.x X-Spam-Score: -0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello, So there=E2=80=99s another situation where offloading regularly hangs on berlin. The =E2=80=98guix offload=E2=80=99 process looks like this: --8<---------------cut here---------------start------------->8--- (gdb) bt #0 0x00007f1f715686a1 in __GI___poll (fds=3D0x14e9b30, nfds=3D1, timeout= =3D-1) at ../sysdeps/unix/sysv/linux/poll.c:29 #1 0x00007f1f673b94e7 in ssh_poll (timeout=3D<optimized out>, nfds=3D<opti= mized out>, fds=3D<optimized out>) at /tmp/guix-build-libssh-0.7.7.drv-0/libssh-0.7.7-checkout/src/poll.c:= 98 #2 ssh_poll_ctx_dopoll (ctx=3Dctx@entry=3D0x14ee2e0, timeout=3Dtimeout@ent= ry=3D-1) at /tmp/guix-build-libssh-0.7.7.drv-0/libssh-0.7.7-checkout/src/poll.c:= 612 #3 0x00007f1f673ba449 in ssh_handle_packets (session=3Dsession@entry=3D0x2= 249360, timeout=3Dtimeout@entry=3D-1) at /tmp/guix-build-libssh-0.7.7.drv-0/libssh-0.7.7-checkout/src/session= .c:634 #4 0x00007f1f673ba51d in ssh_handle_packets_termination (session=3Dsession= @entry=3D0x2249360, timeout=3D<optimized out>, timeout@entry=3D-3, fct=3Dfct@entry=3D0x7f1f673a4430 <ssh_channel_read_= termination>, user=3Duser@entry=3D0x7ffce23953f0) at /tmp/guix-build-libssh-0.7.7.drv-0/libssh-0.7.7-checkout/src/session= .c:696 #5 0x00007f1f673a6aaf in ssh_channel_read_timeout (channel=3D0x224e360, de= st=3Ddest@entry=3D0x18ef020, count=3Dcount@entry=3D8, is_stderr=3D<optimized out>, timeout=3D-3, tim= eout@entry=3D-1) at /tmp/guix-build-libssh-0.7.7.drv-0/libssh-0.7.7-checkout/src/channel= s.c:2705 #6 0x00007f1f673a6bbb in ssh_channel_read (channel=3D<optimized out>, dest= =3Ddest@entry=3D0x18ef020, count=3Dcount@entry=3D8, is_stderr=3D<optimized out>) at /tmp/guix-build-libssh-0.7.7.drv-0/libs= sh-0.7.7-checkout/src/channels.c:2621 #7 0x00007f1f67413a23 in read_from_channel_port ( channel=3D<error reading variable: ERROR: In procedure gdbscm_memory_po= rt_fill_input: error reading memory>0x22f01a0, dst=3D<optimized out>, start= =3D0, count=3D8) at channel-type.c:161 #8 0x00007f1f71b65287 in scm_i_read_bytes ( port=3Dport@entry=3D<error reading variable: ERROR: In procedure gdbscm= _memory_port_fill_input: error reading memory>0x22f01a0, dst=3Ddst@entry=3D= "#<vu8vector>" =3D {...}, start=3Dstart@entry=3D0, count=3Dcount@entry=3D8)= at ports.c:1559 #9 0x00007f1f71b6996c in scm_c_read_bytes ( port=3Dport@entry=3D<error reading variable: ERROR: In procedure gdbscm= _memory_port_fill_input: error reading memory>0x22f01a0, dst=3Ddst@entry=3D= "#<vu8vector>" =3D {...}, start=3Dstart@entry=3D0, count=3Dcount@entry=3D8)= at ports.c:1639 #10 0x00007f1f71b6fd80 in scm_get_bytevector_n ( port=3D<error reading variable: ERROR: In procedure gdbscm_memory_port_= fill_input: error reading memory>0x22f01a0, count=3D<optimized out>) at r6rs-ports.c:421 #11 0x00007f1f71ba4715 in vm_regular_engine (thread=3D0x14e9b30, vp=3D0xc31= f30, registers=3D0xffffffff, resume=3D1901495969) at vm-engine.c:786 [...] (gdb) p *fds $1 =3D {fd =3D 15, events =3D 1, revents =3D 0} (gdb) shell ls -l /proc/12185/fd total 0 lr-x------ 1 root root 64 Jan 10 16:56 0 -> 'pipe:[76778016]' l-wx------ 1 root root 64 Jan 10 16:56 1 -> 'pipe:[76778015]' lr-x------ 1 root root 64 Jan 10 16:56 10 -> 'pipe:[76838317]' l-wx------ 1 root root 64 Jan 10 16:56 11 -> 'pipe:[76838317]' lr-x------ 1 root root 64 Jan 10 16:56 12 -> 'pipe:[76851360]' l-wx------ 1 root root 64 Jan 10 16:56 13 -> 'pipe:[76851360]' l-wx------ 1 root root 64 Jan 10 16:56 14 -> /var/guix/offload/overdrive1.g= uixsd.org/1 lrwx------ 1 root root 64 Jan 10 16:56 15 -> 'socket:[76860702]' lr-x------ 1 root root 64 Jan 10 16:56 16 -> /dev/urandom l-wx------ 1 root root 64 Jan 10 16:56 2 -> 'pipe:[76778015]' lr-x------ 1 root root 64 Jan 10 16:56 3 -> 'pipe:[76838313]' l-wx------ 1 root root 64 Jan 10 16:56 4 -> 'pipe:[76778017]' l-wx------ 1 root root 64 Jan 10 16:56 5 -> 'pipe:[76838313]' lr-x------ 1 root root 64 Jan 10 16:56 6 -> 'pipe:[76838316]' l-wx------ 1 root root 64 Jan 10 16:56 7 -> 'pipe:[76838316]' lr-x------ 1 root root 64 Jan 10 16:56 8 -> 'pipe:[76841414]' l-wx------ 1 root root 64 Jan 10 16:56 9 -> 'pipe:[76841414]' --8<---------------cut here---------------end--------------->8--- It=E2=80=99s a =E2=80=98get-bytevector-n=E2=80=99 for 8 bytes, so it looks = like the daemon protocol. At that point the socket is actually dead: if I connect on the remote machine (overdrive1.guixsd.org) I can see that there are no other open SSH sessions. A simple thing would be to somehow get libssh to pass POLLIN | POLLRDHUP instead of just POLLIN. Additionally, we could change Guile-SSH so that we can specify a timeout when reading from a channel. Ludo=E2=80=99.
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Subject: bug#34033: Acknowledgement (Offloading sometimes hangs) Message-ID: <handler.34033.B.154713658627317.ack <at> debbugs.gnu.org> References: <87o98obikk.fsf@HIDDEN> X-Gnu-PR-Message: ack 34033 X-Gnu-PR-Package: guix Reply-To: 34033 <at> debbugs.gnu.org Date: Thu, 10 Jan 2019 16:10:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): bug-guix@HIDDEN If you wish to submit further information on this problem, please send it to 34033 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 34033: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D34033 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN Subject: bug#34033: Offloading sometimes hangs Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-guix@HIDDEN Resent-Date: Mon, 14 Jan 2019 22:47:01 +0000 Resent-Message-ID: <handler.34033.B34033.154750596110221 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 34033 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: 34033 <at> debbugs.gnu.org Received: via spool by 34033-submit <at> debbugs.gnu.org id=B34033.154750596110221 (code B ref 34033); Mon, 14 Jan 2019 22:47:01 +0000 Received: (at 34033) by debbugs.gnu.org; 14 Jan 2019 22:46:01 +0000 Received: from localhost ([127.0.0.1]:59426 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1gjAzZ-0002ej-5Z for submit <at> debbugs.gnu.org; Mon, 14 Jan 2019 17:46:01 -0500 Received: from hera.aquilenet.fr ([185.233.100.1]:33768) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1gjAzX-0002eW-E6 for 34033 <at> debbugs.gnu.org; Mon, 14 Jan 2019 17:46:00 -0500 Received: from localhost (localhost [127.0.0.1]) by hera.aquilenet.fr (Postfix) with ESMTP id 0623C1BB2 for <34033 <at> debbugs.gnu.org>; Mon, 14 Jan 2019 23:45:58 +0100 (CET) X-Virus-Scanned: Debian amavisd-new at aquilenet.fr Received: from hera.aquilenet.fr ([127.0.0.1]) by localhost (hera.aquilenet.fr [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id 0Gm0SI52e8ux for <34033 <at> debbugs.gnu.org>; Mon, 14 Jan 2019 23:45:57 +0100 (CET) Received: from ribbon (unknown [IPv6:2a01:e0a:1d:7270:af76:b9b:ca24:c465]) by hera.aquilenet.fr (Postfix) with ESMTPSA id 15B0E130C for <34033 <at> debbugs.gnu.org>; Mon, 14 Jan 2019 23:45:56 +0100 (CET) From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> References: <87o98obikk.fsf@HIDDEN> Date: Mon, 14 Jan 2019 23:45:56 +0100 In-Reply-To: <87o98obikk.fsf@HIDDEN> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Thu, 10 Jan 2019 17:09:31 +0100") Message-ID: <87fttuq2mz.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.1 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 1.0 (+) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -0.0 (/) Hello, Ludovic Court=C3=A8s <ludo@HIDDEN> skribis: > A simple thing would be to somehow get libssh to pass POLLIN | POLLRDHUP > instead of just POLLIN. Reported here: https://www.libssh.org/archive/libssh/2019-01/0000000.html A fix has been proposed by upstream and should be committed shortly. > Additionally, we could change Guile-SSH so that we can specify a timeout > when reading from a channel. Turns out we can set a per-session timeout, which we already do (see #:timeout in =E2=80=98open-ssh-session=E2=80=99 in (guix scripts offload)) = but =E2=80=98ssh_channel_read=E2=80=99 would ignore it and instead pass an infi= nite timeout to poll(2): https://www.libssh.org/archive/libssh/2019-01/0000001.html This issue happens to be fixed in libssh 0.8.x, so I upgraded our libssh package in commit a8b0556ea1e439c89dc1ba33c8864e8b9b811f08. (That still doesn=E2=80=99t tell us why our =E2=80=98guix offload=E2=80=99 = processes would occasionally be stuck but at least it ensures the build farm keeps making progress even when that happens.) Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN Subject: bug#34033: Offloading sometimes hangs Resent-From: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-guix@HIDDEN Resent-Date: Sat, 22 Feb 2020 04:38:01 +0000 Resent-Message-ID: <handler.34033.B34033.15823462378066 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 34033 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Cc: 34033 <at> debbugs.gnu.org Received: via spool by 34033-submit <at> debbugs.gnu.org id=B34033.15823462378066 (code B ref 34033); Sat, 22 Feb 2020 04:38:01 +0000 Received: (at 34033) by debbugs.gnu.org; 22 Feb 2020 04:37:17 +0000 Received: from localhost ([127.0.0.1]:48514 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1j5MXU-000261-8M for submit <at> debbugs.gnu.org; Fri, 21 Feb 2020 23:37:17 -0500 Received: from mail-qt1-f195.google.com ([209.85.160.195]:46260) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maxim.cournoyer@HIDDEN>) id 1j5MXS-00025p-6w for 34033 <at> debbugs.gnu.org; Fri, 21 Feb 2020 23:37:14 -0500 Received: by mail-qt1-f195.google.com with SMTP id i14so2857912qtv.13 for <34033 <at> debbugs.gnu.org>; Fri, 21 Feb 2020 20:37:14 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=+M54D5f6sHD468ltGLg9X1XriouDmI73jTK3ZOVSA5Q=; b=KwubyUdFe2GL8wKFtKX3cKDL+MbAXIxeXSbWw2LgHwPgwOq/wVzLBKaTUhFj0F9CSy byK/xbpKMHS9i7Cdy/9/i59oir60jIh10wLE7inKV6HuhoflaoLYp/jC5h1rZICm8H6f UrH8ElMqNlKumytex8e28MQ7R7JiKJyAn8WQ+9J01F3xQj4NyZkeyYjkMp5hcNEjLNXo 06UBmERXG9wMTmNdImiVQUvn255W2ruO5A2iFqXKGbSijpXelA4UvnTQ9Uz+NRHPSkKK kdOic+pFbD1fARLaAHs6WUDToc+fLSpQ5Ew97P7EZqvkiPLFhcUBhD0dv6xT5QJD6BW+ vdtA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=+M54D5f6sHD468ltGLg9X1XriouDmI73jTK3ZOVSA5Q=; b=r0zX5ZmEX2fut28XDkXGrV7ndZodxBlUYorUkFWMtfENb8XuiQCgmv03UMrz5VoiH9 Jij0o/gOqQDUu3HvX8O/asRir1dgUX0vbJ8z/7xC2ite1bKwZ0gZhzXR3tfMunQ8BHqU VSX9AyGJWPtYmnX/MVwV5MjAQoBXQveE7Xs45FcwkfEwWyXFdjsqeKfnyE2zo7yTCO+8 dlAbrl8LlATflPbrDVMtc3X1IKmlzR8F8wcEm2Ymr0lc9HI0Ufv3KBT7DQvVwxi8JwED RF1XCL1Tv0ncGnCADANwoRfleOkK1HPCSLr9IuQgb+nubitR4fvxpLrWe2xV8eFsWLVg h4bg== X-Gm-Message-State: APjAAAVC/jFJGRl2tY/tvqbWIrFpWllDjyxc/aPpotVej36x01OQmgJ/ ZXdZFr96cP4ZLtj+e4q5a2WnHftb X-Google-Smtp-Source: APXvYqz3lmUktckC4FBk10dOmHCfi4i3a33Hl80aUWvzHSVnRAv6F+wa6aYidFog9g625ly6PRW9Dw== X-Received: by 2002:ac8:4505:: with SMTP id q5mr34667985qtn.84.1582346228300; Fri, 21 Feb 2020 20:37:08 -0800 (PST) Received: from apteryx (dsl-236-125-236.b2b2c.ca. [207.236.125.236]) by smtp.gmail.com with ESMTPSA id d20sm2546082qkg.8.2020.02.21.20.37.06 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Fri, 21 Feb 2020 20:37:07 -0800 (PST) From: Maxim Cournoyer <maxim.cournoyer@HIDDEN> X-Google-Original-From: Maxim Cournoyer <maxim@HIDDEN> References: <87o98obikk.fsf@HIDDEN> <87fttuq2mz.fsf@HIDDEN> Date: Fri, 21 Feb 2020 23:37:06 -0500 In-Reply-To: <87fttuq2mz.fsf@HIDDEN> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Mon, 14 Jan 2019 23:45:56 +0100") Message-ID: <87wo8fqlu5.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello Ludovic, Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > Hello, > > Ludovic Court=C3=A8s <ludo@HIDDEN> skribis: > >> A simple thing would be to somehow get libssh to pass POLLIN | POLLRDHUP >> instead of just POLLIN. > > Reported here: > > https://www.libssh.org/archive/libssh/2019-01/0000000.html > > A fix has been proposed by upstream and should be committed shortly. > >> Additionally, we could change Guile-SSH so that we can specify a timeout >> when reading from a channel. > > Turns out we can set a per-session timeout, which we already do (see > #:timeout in =E2=80=98open-ssh-session=E2=80=99 in (guix scripts offload)= ) but > =E2=80=98ssh_channel_read=E2=80=99 would ignore it and instead pass an in= finite timeout > to poll(2): > > https://www.libssh.org/archive/libssh/2019-01/0000001.html > > This issue happens to be fixed in libssh 0.8.x, so I upgraded our libssh > package in commit a8b0556ea1e439c89dc1ba33c8864e8b9b811f08. > > (That still doesn=E2=80=99t tell us why our =E2=80=98guix offload=E2=80= =99 processes would > occasionally be stuck but at least it ensures the build farm keeps > making progress even when that happens.) > > Ludo=E2=80=99. Seems the patch in the response at the URL you linked is awaiting some feedback/review. Is this the reason 'guix substitute' hangs for so long when the substitute server is down? (like 1 minute or so). Maxim
X-Loop: help-debbugs@HIDDEN Subject: bug#34033: Offloading sometimes hangs Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-guix@HIDDEN Resent-Date: Sat, 22 Feb 2020 20:36:01 +0000 Resent-Message-ID: <handler.34033.B34033.15824037598669 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 34033 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Cc: 34033 <at> debbugs.gnu.org Received: via spool by 34033-submit <at> debbugs.gnu.org id=B34033.15824037598669 (code B ref 34033); Sat, 22 Feb 2020 20:36:01 +0000 Received: (at 34033) by debbugs.gnu.org; 22 Feb 2020 20:35:59 +0000 Received: from localhost ([127.0.0.1]:50081 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1j5bVG-0002Fk-TY for submit <at> debbugs.gnu.org; Sat, 22 Feb 2020 15:35:59 -0500 Received: from eggs.gnu.org ([209.51.188.92]:48872) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <ludo@HIDDEN>) id 1j5bVF-0002FX-2n for 34033 <at> debbugs.gnu.org; Sat, 22 Feb 2020 15:35:57 -0500 Received: from fencepost.gnu.org ([2001:470:142:3::e]:56284) by eggs.gnu.org with esmtp (Exim 4.71) (envelope-from <ludo@HIDDEN>) id 1j5bV9-0007Y4-UN; Sat, 22 Feb 2020 15:35:51 -0500 Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=46544 helo=ribbon) by fencepost.gnu.org with esmtpsa (TLS1.2:RSA_AES_256_CBC_SHA1:256) (Exim 4.82) (envelope-from <ludo@HIDDEN>) id 1j5bV9-0006jo-HX; Sat, 22 Feb 2020 15:35:51 -0500 From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> References: <87o98obikk.fsf@HIDDEN> <87fttuq2mz.fsf@HIDDEN> <87wo8fqlu5.fsf@HIDDEN> X-URL: http://www.fdn.fr/~lcourtes/ X-Revolutionary-Date: 4 =?UTF-8?Q?Vent=C3=B4se?= an 228 de la =?UTF-8?Q?R=C3=A9volution?= X-PGP-Key-ID: 0x090B11993D9AEBB5 X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4 0CFB 090B 1199 3D9A EBB5 X-OS: x86_64-pc-linux-gnu Date: Sat, 22 Feb 2020 21:35:50 +0100 In-Reply-To: <87wo8fqlu5.fsf@HIDDEN> (Maxim Cournoyer's message of "Fri, 21 Feb 2020 23:37:06 -0500") Message-ID: <87v9nyuzq1.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-detected-operating-system: by eggs.gnu.org: GNU/Linux 2.2.x-3.x [generic] X-Spam-Score: -0.7 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.7 (-) Hi Maxim, Maxim Cournoyer <maxim.cournoyer@HIDDEN> skribis: > Ludovic Court=C3=A8s <ludo@HIDDEN> writes: > >> Hello, >> >> Ludovic Court=C3=A8s <ludo@HIDDEN> skribis: >> >>> A simple thing would be to somehow get libssh to pass POLLIN | POLLRDHUP >>> instead of just POLLIN. >> >> Reported here: >> >> https://www.libssh.org/archive/libssh/2019-01/0000000.html >> >> A fix has been proposed by upstream and should be committed shortly. >> >>> Additionally, we could change Guile-SSH so that we can specify a timeout >>> when reading from a channel. >> >> Turns out we can set a per-session timeout, which we already do (see >> #:timeout in =E2=80=98open-ssh-session=E2=80=99 in (guix scripts offload= )) but >> =E2=80=98ssh_channel_read=E2=80=99 would ignore it and instead pass an i= nfinite timeout >> to poll(2): >> >> https://www.libssh.org/archive/libssh/2019-01/0000001.html >> >> This issue happens to be fixed in libssh 0.8.x, so I upgraded our libssh >> package in commit a8b0556ea1e439c89dc1ba33c8864e8b9b811f08. >> >> (That still doesn=E2=80=99t tell us why our =E2=80=98guix offload=E2=80= =99 processes would >> occasionally be stuck but at least it ensures the build farm keeps >> making progress even when that happens.) >> >> Ludo=E2=80=99. > > Seems the patch in the response at the URL you linked is awaiting some > feedback/review. Is this the reason 'guix substitute' hangs for so long > when the substitute server is down? (like 1 minute or so). The issues above are in libssh and were fixed a while ago. =E2=80=98guix substitute=E2=80=99 doesn=E2=80=99t use Guile-SSH/libssh, so the problem yo= u=E2=80=99re seeing must be something different. What do you mean by =E2=80=9Cthe substitute server is down=E2=80=9D? You m= ean =E2=80=98guix publish=E2=80=99 is not running, or the machine is unavailable altogether? Thanks, Ludo=E2=80=99.
X-Loop: help-debbugs@HIDDEN Subject: bug#34033: Offloading sometimes hangs Resent-From: Maxim Cournoyer <maxim.cournoyer@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-guix@HIDDEN Resent-Date: Mon, 24 Feb 2020 14:00:02 +0000 Resent-Message-ID: <handler.34033.B34033.15825527752756 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 34033 X-GNU-PR-Package: guix X-GNU-PR-Keywords: To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN> Cc: 34033 <at> debbugs.gnu.org Received: via spool by 34033-submit <at> debbugs.gnu.org id=B34033.15825527752756 (code B ref 34033); Mon, 24 Feb 2020 14:00:02 +0000 Received: (at 34033) by debbugs.gnu.org; 24 Feb 2020 13:59:35 +0000 Received: from localhost ([127.0.0.1]:53016 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1j6EGk-0000iO-V2 for submit <at> debbugs.gnu.org; Mon, 24 Feb 2020 08:59:35 -0500 Received: from mail-qk1-f175.google.com ([209.85.222.175]:35383) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <maxim.cournoyer@HIDDEN>) id 1j6EGj-0000iC-LB for 34033 <at> debbugs.gnu.org; Mon, 24 Feb 2020 08:59:33 -0500 Received: by mail-qk1-f175.google.com with SMTP id 145so2556830qkl.2 for <34033 <at> debbugs.gnu.org>; Mon, 24 Feb 2020 05:59:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:references:date:in-reply-to:message-id :user-agent:mime-version:content-transfer-encoding; bh=Q2i62VaZr0MAmsMom2oBc4xoXmFqCMuoNU9ugu5pWV0=; b=T5NWECQdbzAk5M7RF2uTLAOH3zCPET/tY+ODrCDJyslIN1yDTlC1Ck9lm9nuh0IA3U teTeuI5aKY5GhejxUKZ8K3txM7pfzcFiFZlWFAtalhBb/kRQ66biPX1n37Wppx/Wl/EI ShFCQrtbdHJQRiL2ECl90FCBS3MTyYiANBOXBPFbPwzySRgQwFDlG0yJtu/bcKyAsBvm E1+nHiCsKXLCezjOS4JyjN806Sux363Bgdu3pYwHrAcPX6Ku7uEVT39OnL7kPRylF5QV rJqo+ZLoVv515BNcO3aNku8o8wsxBL1ztXjTi77vTkV+jpbK01wNWdQ8swQXLOmG9BO5 GLUQ== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20161025; h=x-gm-message-state:from:to:cc:subject:references:date:in-reply-to :message-id:user-agent:mime-version:content-transfer-encoding; bh=Q2i62VaZr0MAmsMom2oBc4xoXmFqCMuoNU9ugu5pWV0=; b=gcwot4zTMBKsejOdH4GWwrjmJk9GEqmeew429sWYCZRaIlD5MOOSxiYwSB8Fq2ajgO uA4VTsdMIQ/6Ah5um/uR0qx3flDfZnpE/EZ+YaV/JU2q91EiCKGvlxeMt3jxkN79rpXK SOYqaaoiHObwJscrFysF25CY+9uEGRe//K4vwbLu+olmHrP6eXCrcUzZwQ2j0RX/Jn9U e8TjrbtGMusBrRBNjkaRiqj95ptIMdV4OVGwF+e/jfugd3IF9xfeHHEfqhy61B3R1zmb 1uEwA6Szvnk23nqSpnQOFmtjZDW10FTrRj87+oHrMX8yJuuYs5iA7V8ASdZW75elJa+O 7d6g== X-Gm-Message-State: APjAAAUdeO/D3evJFv3GCjF0UCgS+RI1HKkxDqD6wXxARCt3u/CX2pwv eR7EWnAIGZ+p8yp/lWoNMiLQn1KR X-Google-Smtp-Source: APXvYqys9lbGuiyk3mG3xbPs1G0Gf8+xXiucMV88pQprUY8oHvc8Wm/6N3Rq1afAXk2LQKe37OrwKg== X-Received: by 2002:a05:620a:545:: with SMTP id o5mr48640851qko.27.1582552767628; Mon, 24 Feb 2020 05:59:27 -0800 (PST) Received: from raisin ([2607:fad8:4:6:235e:8579:8464:aacc]) by smtp.gmail.com with ESMTPSA id f59sm5961646qtb.75.2020.02.24.05.59.26 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 24 Feb 2020 05:59:26 -0800 (PST) From: Maxim Cournoyer <maxim.cournoyer@HIDDEN> References: <87o98obikk.fsf@HIDDEN> <87fttuq2mz.fsf@HIDDEN> <87wo8fqlu5.fsf@HIDDEN> <87v9nyuzq1.fsf@HIDDEN> Date: Mon, 24 Feb 2020 08:59:25 -0500 In-Reply-To: <87v9nyuzq1.fsf@HIDDEN> ("Ludovic \=\?utf-8\?Q\?Court\=C3\=A8s\=22'\?\= \=\?utf-8\?Q\?s\?\= message of "Sat, 22 Feb 2020 21:35:50 +0100") Message-ID: <87blpof5mq.fsf@HIDDEN> User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/26.3 (gnu/linux) MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: quoted-printable X-Spam-Score: 0.0 (/) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -1.0 (-) Hello Ludovic, Ludovic Court=C3=A8s <ludo@HIDDEN> writes: [...] > The issues above are in libssh and were fixed a while ago. =E2=80=98guix > substitute=E2=80=99 doesn=E2=80=99t use Guile-SSH/libssh, so the problem = you=E2=80=99re seeing > must be something different. OK, good to know! > What do you mean by =E2=80=9Cthe substitute server is down=E2=80=9D? You= mean =E2=80=98guix > publish=E2=80=99 is not running, or the machine is unavailable altogether? The machine is turned off (i.e., the machine is unavailable altogether :-). It doesn't hang forever, but the timeout is a rather long. I'm using that machine as both a substitute and an offload server. Maxim
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.