GNU bug report logs - #48005
low performance when building profile of already built packages

Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.

Package: guix; Severity: important; Reported by: Ricardo Wurmus <rekado@HIDDEN>; merged with #41702; dated Sat, 24 Apr 2021 20:40:02 UTC; Maintainer for guix is bug-guix@HIDDEN.
Merged 41702 48005. Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> Full text available.
Severity set to 'important' from 'normal' Request was from Ludovic Courtès <ludo@HIDDEN> to control <at> Full text available.

Message received at submit <at>

Received: (at submit) by; 24 Apr 2021 20:39:16 +0000
From debbugs-submit-bounces <at> Sat Apr 24 16:39:16 2021
Received: from localhost ([]:41580
	by with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at>>)
	id 1laP3b-0007JZ-Qb
	for submit <at>; Sat, 24 Apr 2021 16:39:16 -0400
Received: from ([]:41314)
 by with esmtp (Exim 4.84_2)
 (envelope-from <rekado@HIDDEN>) id 1laP3Y-0007JQ-8W
 for submit <at>; Sat, 24 Apr 2021 16:39:14 -0400
Received: from ([2001:470:142:3::10]:39280)
 by with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <rekado@HIDDEN>)
 id 1laP3X-0008DT-SG
 for bug-guix@HIDDEN; Sat, 24 Apr 2021 16:39:12 -0400
Received: from ([]:21154)
 by with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <rekado@HIDDEN>)
 id 1laP3V-0004Q0-Pq
 for bug-guix@HIDDEN; Sat, 24 Apr 2021 16:39:11 -0400
ARC-Seal: i=1; a=rsa-sha256; t=1619296743; cv=none;; s=zohoarc; 
ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed;;
 s=zohoarc; t=1619296743;
ARC-Authentication-Results: i=1;;
 spf=pass  smtp.mailfrom=rekado@HIDDEN;
 dmarc=pass header.from=<rekado@HIDDEN> header.from=<rekado@HIDDEN>
DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; t=1619296743; 
 s=zoho;; i=rekado@HIDDEN;
Received: from localhost ( []) by with SMTPS id 1619296741170139.57686629621355;
 Sat, 24 Apr 2021 13:39:01 -0700 (PDT)
User-agent: mu4e 1.4.15; emacs 27.2
From: Ricardo Wurmus <rekado@HIDDEN>
To: bug-guix@HIDDEN
Subject: low performance when building profile of already built packages
X-PGP-Fingerprint: BCA6 89B6 3655 3801 C3C6  2150 197A 5888 235F ACAC
Date: Sat, 24 Apr 2021 22:38:58 +0200
Message-ID: <87zgxnpgd9.fsf@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: quoted-printable
X-ZohoMailClient: External
Received-SPF: pass client-ip=; envelope-from=rekado@HIDDEN;
X-Spam_score_int: -20
X-Spam_score: -2.1
X-Spam_bar: --
X-Spam_report: (-2.1 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 SPF_HELO_NONE=0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-Debbugs-Envelope-To: submit
X-BeenThere: debbugs-submit <at>
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <>
List-Unsubscribe: <>, 
 <mailto:debbugs-submit-request <at>>
List-Archive: <>
List-Post: <mailto:debbugs-submit <at>>
List-Help: <mailto:debbugs-submit-request <at>>
List-Subscribe: <>, 
 <mailto:debbugs-submit-request <at>>
Errors-To: debbugs-submit-bounces <at>
Sender: "Debbugs-submit" <debbugs-submit-bounces <at>>
X-Spam-Score: -2.3 (--)

Here is a manifest that isn=E2=80=99t unusual for a bioinformatics=20

--8<---------------cut here---------------start------------->8---
(define packages
  (list "bash-minimal"

(specifications->manifest packages)
--8<---------------cut here---------------end--------------->8---

When all these packages already exist and nothing is to be done=20
when Guix is asked to instantiate this manifest, more than 13=20
seconds pass.

--8<---------------cut here---------------start------------->8---
$ [env] export GUIX_PROFILING=3D"object-cache=20
add-data-to-store-cache rpc"
$ [env] time guix package -m manifest-test.scm -p /tmp/foo
The following packages will be installed:
   bash-minimal           5.0.16
   fastqc                 0.11.5
   gzip                   1.10
   htseq                  0.9.1
   multiqc                1.5
   r-corrplot             0.84
   r-crosstalk            1.1.1
   r-data-table           1.14.0
   r-deseq2               1.30.1
   r-dt                   0.17
   r-genomicalignments    1.26.0
   r-genomicranges        1.42.0
   r-ggplot2              3.3.3
   r-ggrepel              0.9.1
   r-gprofiler            0.7.0
   r-knitr                1.31
   r-pheatmap             1.0.12
   r-plotly               4.9.3
   r-reshape2             1.4.4
   r-rmarkdown            2.7
   r-rsamtools            2.6.0
   r-rtracklayer          1.50.0
   r-s4vectors            0.28.1
   r-scales               1.1.1
   r-summarizedexperiment 1.20.0
   r-tximport             1.18.0
   samtools               1.11
   sed                    4.8
   trim-galore            0.6.6
[[most of the time passes here]]

nothing to be done
Store object cache:
  fresh caches:     2
  lookups:      45493
  hits:         43943 (96.6%)
  cache size:    1549 entries

'add-data-to-store' cache:
  lookups:       7755
  hits:          3411 (44.0%)
  .drv files:    4357 (56.2%)
  Scheme files:  3343 (43.1%)
Remote procedure call summary: 5283 RPCs
  built-in-builders              ...     1
  add-to-store/tree              ...    27
  add-to-store                   ...   213
  query-references               ...   698
  add-text-to-store              ...  4344

real	0m13.379s
user	0m16.414s
sys	0m0.253s
--8<---------------cut here---------------end--------------->8---

Some of these R packages have a lot of transitive inputs, such as=20
r-deseq2 with 81 transitive propagated inputs.  So there=E2=80=99s a lot=20
of propagation here, which I suspect contributes to things being=20
so unexpectedly slow.

This report may be related to performance issues reported for=20
=E2=80=9Cguix environment=E2=80=9D in, wh=
also involves R packages (such as r-learnr with 46 transitive=20
propagated inputs).


Acknowledgement sent to Ricardo Wurmus <rekado@HIDDEN>:
New bug report received and forwarded. Copy sent to bug-guix@HIDDEN. Full text available.
Report forwarded to bug-guix@HIDDEN:
bug#48005; Package guix. Full text available.
Please note: This is a static page, with minimal formatting, updated once a day.
Click here to see this page with the latest information and nicer formatting.
Last modified: Thu, 27 May 2021 16:45:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.