Received: (at 74381) by debbugs.gnu.org; 19 Dec 2024 16:54:23 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Thu Dec 19 11:54:23 2024 Received: from localhost ([127.0.0.1]:39864 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1tOJn4-0001NH-RX for submit <at> debbugs.gnu.org; Thu, 19 Dec 2024 11:54:23 -0500 Received: from mout01.posteo.de ([185.67.36.65]:51087) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dariqq@HIDDEN>) id 1tOJmz-0001Mw-Qk for 74381 <at> debbugs.gnu.org; Thu, 19 Dec 2024 11:54:21 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout01.posteo.de (Postfix) with ESMTPS id A5C63240027 for <74381 <at> debbugs.gnu.org>; Thu, 19 Dec 2024 17:54:10 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1734627251; bh=dYoQ9O5VT60PtWyZm1bW9q6rE83vqFxWAXTCjUnmkGU=; h=Message-ID:Date:MIME-Version:To:Subject:From:Content-Type: Content-Transfer-Encoding:From; b=C9v9OpKmARpR6PaoIKKMyCakRtYO0kKJ85HEoWOlgpL0gVe0sa+N7hH/itpVeL+Bm d5m9CCfQPO57K97PMRvxBZ4kkI2U+XECbmcMRricJ2Z1oOurckMX5mpc4YwrS9lMu9 HcTzq+398OX2lf6l9QH0z2Yp+oyZpJaCMVAgJwRhUAp7wrAmopXK5C5c0an+w/IKYG xF1IN+pxGxo2hikNvVFTTw1xQygBy8uOXVUbxOnj4ZjNN+lphEVdp39weizeOrat89 CD87Tg4U4WW3PNFQa4wlgwaZe2n676/SmQAeouFdLMpsopYo0fUTqaRA//mxrsGuYL EFcP4glt3S/ww== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4YDc8L4PvPz9rxV for <74381 <at> debbugs.gnu.org>; Thu, 19 Dec 2024 17:54:10 +0100 (CET) Message-ID: <a9f06663-01e4-4b47-8a1e-7330135d5c4c@HIDDEN> Date: Thu, 19 Dec 2024 16:54:04 +0000 MIME-Version: 1.0 To: 74381 <at> debbugs.gnu.org Subject: High memory usage during guix pull (i686-linux, guile jit) Content-Language: en-US From: Dariqq <dariqq@HIDDEN> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: 74381 X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) It turns out disabling guile jit does not work reliably and even without it guix pull may oom building guix-packages-base using 3GB+ of RAM. My workaround for now is to run 'guix pull -s i686-linux -p /tmp/whatever' on an x86_64 machine (to have more RAM available) with guile jit disabled multiple times until it succeeds (not exactly sure why that works, is something getting cached somewhere?) and use that machine to serve substitutes. After mentioning this on irc today janneke mentioned this commit https://git.savannah.gnu.org/cgit/guix.git/commit/?id=15c5f1a2c20b21de0f19f42db1ccab4c42117ebb which reduces the number of files per chunk from 25 to 10 to get guix pull working on the 32 bit hurd. I tried reducing that even further to 5 in my local checkout and the max RAM usage went to (a little bit more reasonable) 1.5-2 GB (both on i686 natively and when emulating with -s i686 with and without guile jit). This would be still not enough for my little machine but might be enough for the substitute servers to build it reliably. I also wanted to try this in a i586-gnu childhurd but guix pull immediately segfaulted.
bug-guix@HIDDEN:bug#74381; Package guix.
Full text available.Received: (at submit) by debbugs.gnu.org; 16 Nov 2024 12:58:34 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Nov 16 07:58:34 2024 Received: from localhost ([127.0.0.1]:52439 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1tCINl-00019k-OU for submit <at> debbugs.gnu.org; Sat, 16 Nov 2024 07:58:34 -0500 Received: from lists.gnu.org ([209.51.188.17]:52516) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <dariqq@HIDDEN>) id 1tCINj-00019c-Gi for submit <at> debbugs.gnu.org; Sat, 16 Nov 2024 07:58:32 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <dariqq@HIDDEN>) id 1tCINj-00079x-3a for bug-guix@HIDDEN; Sat, 16 Nov 2024 07:58:31 -0500 Received: from mout02.posteo.de ([185.67.36.66]) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <dariqq@HIDDEN>) id 1tCINg-0005UZ-Fj for bug-guix@HIDDEN; Sat, 16 Nov 2024 07:58:30 -0500 Received: from submission (posteo.de [185.67.36.169]) by mout02.posteo.de (Postfix) with ESMTPS id 8E08B240101 for <bug-guix@HIDDEN>; Sat, 16 Nov 2024 13:58:24 +0100 (CET) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=posteo.net; s=2017; t=1731761904; bh=VpYofyjWuA2w7HCm/zgJ1TWnUfAeiQ1C/6pQy8H+3cY=; h=Message-ID:Date:MIME-Version:To:From:Subject:Content-Type: Content-Transfer-Encoding:From; b=LT5oWMKWBiKZHzUurqkBHirRSA8orT+i/YdmSSTXeprwWHCfj3EpOqqiJrJ564bUH SvhaY8l76FCDg5WoT2LwM8M1MgX/l2KFcJi3Bh9iwhYTdtX2U69Whc2ND74xRlAxt+ zLpliByZRSBZrkVljkgunG3rZMduml1uINv+qSU+J5j9lxbMhYnFzyimvVlxu5htJV BREYAS4rGtI4zqm7ZKXSqj+mImZIYy9HCvn3mw9DeClvEdPbR2o9qeUCARZE18ouN5 gspFMpy0+vMcIjey8lWQRYPR3f4QCoDSvvAH6UnbnKViRlPEXaVI4t9ZwTNBU4OURX I1dOX0D+IaXHg== Received: from customer (localhost [127.0.0.1]) by submission (posteo.de) with ESMTPSA id 4XrDTX0dWbz6twx for <bug-guix@HIDDEN>; Sat, 16 Nov 2024 13:58:23 +0100 (CET) Message-ID: <3faad437-c87c-44d0-ad77-9e42418bf2dd@HIDDEN> Date: Sat, 16 Nov 2024 12:58:14 +0000 MIME-Version: 1.0 Content-Language: en-US To: bug-guix@HIDDEN From: Dariqq <dariqq@HIDDEN> Subject: High memory usage during guix pull (i686-linux, guile jit) Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Received-SPF: pass client-ip=185.67.36.66; envelope-from=dariqq@HIDDEN; helo=mout02.posteo.de X-Spam_score_int: -43 X-Spam_score: -4.4 X-Spam_bar: ---- X-Spam_report: (-4.4 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1, DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=0.001, RCVD_IN_MSPIKE_WL=0.001, RCVD_IN_VALIDITY_CERTIFIED_BLOCKED=0.001, RCVD_IN_VALIDITY_RPBL_BLOCKED=0.001, SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -1.3 (-) X-Debbugs-Envelope-To: submit X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -2.3 (--) Hi, Using 'guix pull' on i686-linux regularly fails when building 'guix-packages-base': E.g this recent run on ci.guix.gnu.org: https://ci.guix.gnu.org/build/6472396/details Log from me trying to pull 3e8d3d80f41e016cdfe80e488a78c2351c94fef8: Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 8192 KiB GC Warning: Failed to expand heap by 64 KiB GC Warning: Out of Memory! Heap size: 3436 MiB. Returning NULL! Warning: Unwind-only out of memory exception; skipping pre-unwind handler. Warning: Unwind-only out of memory exception; skipping pre-unwind handler. [ 19/ 20] compiling... 90.0% of 10 filesallocating JIT code buffer failed: Cannot allocate memory JIT failed due to resource exhaustion disabling automatic JIT compilation allocate_stack failed: Cannot allocate memory Warning: Unwind-only stack overflow exception; skipping pre-unwind handler. [...] The error suggests this might be related to guile jit compiling things Disabling it by adding GUILE_JIT_THRESHOLD=-1 to the guix-daemon environment got things working for me again.
Dariqq <dariqq@HIDDEN>:bug-guix@HIDDEN.
Full text available.bug-guix@HIDDEN:bug#74381; Package guix.
Full text available.
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.