GNU logs - #49421, boring messages

Message sent to guix-patches@HIDDEN:

Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Maxime Devos <maximedevos@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Mon, 05 Jul 2021 19:07:01 +0000
Resent-Message-ID: <handler.49421.B.162551200813844 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 49421 <at> debbugs.gnu.org
Message-ID: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
From: Maxime Devos <maximedevos@HIDDEN>
Date: Mon, 05 Jul 2021 21:06:32 +0200
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-eL9RW6M64D0urJpMFyVi"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
Received-SPF: pass client-ip=2a02:1800:110:4::f00:19;
 envelope-from=maximedevos@HIDDEN; helo=laurent.telenet-ops.be
X-Spam_score_int: -27
X-Spam_score: -2.8
X-Spam_bar: --
X-Spam_report: (-2.8 / 5.0 requ) BAYES_00=-1.9, DKIM_SIGNED=0.1,
 DKIM_VALID=-0.1, DKIM_VALID_AU=-0.1, DKIM_VALID_EF=-0.1, FREEMAIL_FROM=0.001,
 RCVD_IN_DNSWL_LOW=-0.7, SPF_HELO_NONE=0.001,
 SPF_PASS=-0.001 autolearn=ham autolearn_force=no
X-Spam_action: no action
X-Spam-Score: 0.6 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)


--=-eL9RW6M64D0urJpMFyVi
Content-Type: multipart/mixed; boundary="=-hq4KR5p1OcuVD7uhGQNP"


--=-hq4KR5p1OcuVD7uhGQNP
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Hi guix,

These two patches should speed up profile generation
by optimising the 'fonts-dir-file' hook.  The first patch
is the most important; from the commit message:

=E2=80=98Only let the build G-exp refer to inputs that might actually
have fonts.  That way, if the list of fonts in the manifest
didn't change and the new manifest is built, the font files
will not be rebuilt.=E2=80=99

The second patch doesn't bring much.

To test, you can test whether things like ...

$ ./pre-inst-env guix environment --ad-hoc --pure lagrange -- lagrange

still work (lagrange is a graphical application using fonts).
Not sure what a good =E2=80=98benchmark=E2=80=99 would be.

Unfortunately, this does not help with the "guix package -i" case,
as in that case, the code doesn't have access to all the package objects,
and will have to satisfy itself with the store paths, in which case the
profile code pessimistically assumes the store item has fonts ...

I suppose this restriction could be lifted if/when the gs-fonts package
is renamed to font-ghostscript or something like that ...

However, the optimisation should work in the "guix environment ... --ad-hoc=
 ...",
"guix system reconfigure ..." and (I'd presume) the guix home-manager cases=
,
though I only tested the first.

Greetings,
Maxime.

--=-hq4KR5p1OcuVD7uhGQNP
Content-Disposition: inline; filename="0001-profiles-Optimise-fonts-dir-file.patch"
Content-Type: text/x-patch; name="0001-profiles-Optimise-fonts-dir-file.patch";
	charset="UTF-8"
Content-Transfer-Encoding: base64

RnJvbSA0ZmUxZTMwZTMzYzAxYmU5ZmQxN2NmMjQwNzMyYjMzNTFjN2IwZmE0IE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBNYXhpbWUgRGV2b3MgPG1heGltZWRldm9zQHRlbGVuZXQuYmU+
CkRhdGU6IE1vbiwgNSBKdWwgMjAyMSAxODo1NTozMSArMDIwMApTdWJqZWN0OiBbUEFUQ0ggMS8y
XSBwcm9maWxlczogT3B0aW1pc2UgJ2ZvbnRzLWRpci1maWxlJy4KCk9ubHkgbGV0IHRoZSBidWls
ZCBHLWV4cCByZWZlciB0byBpbnB1dHMgdGhhdCBtaWdodCBhY3R1YWxseQpoYXZlIGZvbnRzLiAg
VGhhdCB3YXksIGlmIHRoZSBsaXN0IG9mIGZvbnRzIGluIHRoZSBtYW5pZmVzdApkaWRuJ3QgY2hh
bmdlIGFuZCB0aGUgbmV3IG1hbmlmZXN0IGlzIGJ1aWx0LCB0aGUgZm9udCBmaWxlcwp3aWxsIG5v
dCBiZSByZWJ1aWx0LgoKKiBndWl4L3Byb2ZpbGVzLnNjbQogIChmb250cy1kaXItZmlsZSlbaGFz
LWZvbnRzP106IE5ldyBwcmVkaWNhdGUuCiAgKGZvbnRzLWRpci1maWxlKVtyZWxldmFudC1pbnB1
dHNdOiBOZXcgdmFyaWFibGUuCiAgKGZvbnRzLWRpci1maWxlKVtidWlsZF06IFVzZSAncmVsZXZh
bnQtaW5wdXRzJyBpbnN0ZWFkIG9mCiAgJ21hbmlmZXN0LWlucHV0cycuCiogZG9jL2NvbnRyaWJ1
dGluZy50ZXhpIChGb250cyk6IE5vdGUgdGhlICdmb250cy0nIG5hbWluZwogIGNvbnZlbnRpb24g
aXMgdGVjaG5pY2FsbHkgaW1wb3J0YW50IG5vdy4KKiBnbnUvcGFja2FnZXMvZ2hvc3RzY3JpcHQu
c2NtIChncy1mb250cyk6IFdvcmstYXJvdW5kIHRoZQogIHBhY2thZ2UgbmFtZSBjb250cmF2ZW5p
bmcgdGhlIGNvbnZlbnRpb24uCi0tLQogZG9jL2NvbnRyaWJ1dGluZy50ZXhpICAgICAgICB8ICA4
ICsrKysrKysrCiBnbnUvcGFja2FnZXMvZ2hvc3RzY3JpcHQuc2NtIHwgMTAgKysrKysrKysrLQog
Z3VpeC9wcm9maWxlcy5zY20gICAgICAgICAgICB8IDIyICsrKysrKysrKysrKysrKysrKysrKy0K
IDMgZmlsZXMgY2hhbmdlZCwgMzggaW5zZXJ0aW9ucygrKSwgMiBkZWxldGlvbnMoLSkKCmRpZmYg
LS1naXQgYS9kb2MvY29udHJpYnV0aW5nLnRleGkgYi9kb2MvY29udHJpYnV0aW5nLnRleGkKaW5k
ZXggMWYwMTYwYTcwNy4uOTAzY2ZkMzkwYiAxMDA2NDQKLS0tIGEvZG9jL2NvbnRyaWJ1dGluZy50
ZXhpCisrKyBiL2RvYy9jb250cmlidXRpbmcudGV4aQpAQCAtODg2LDYgKzg4NiwxNCBAQCBpcyBh
ZGRlZCB0byB0aGUgcGFja2FnZSBuYW1lLiAgV2UgdXNlIEBjb2Rley10dGZ9IGZvciBUcnVlVHlw
ZSBmb250cywKIEBjb2Rley1vdGZ9IGZvciBPcGVuVHlwZSBmb250cyBhbmQgQGNvZGV7LXR5cGUx
fSBmb3IgUG9zdFNjcmlwdCBUeXBlIDEKIGZvbnRzLgogCitUaGVyZSBhcmUgaW1wb3J0YW50IHRl
Y2huaWNhbCByZWFzb25zIGZvciB0aGlzIG5hbWluZyBjb252ZW50aW9uIGFzIHdlbGw6Citmb3Ig
ZWZmaWNpZW5jeSByZWFzb25zLCB0aGUgcHJvZmlsZSBnZW5lcmF0aW9uIGNvZGUgbmVlZHMgdG8g
a25vdyBpZiBhCitwYWNrYWdlIGhhcyBmb250cyBhbmQgbG9va3MgYXQgdGhlIHBhY2thZ2UgbmFt
ZSB0byBkZXRlcm1pbmUgdGhpcy4KKworQGMgXiBUaGVyZSBpcyBhbiBlc2NhcGUgaGF0Y2ggKHRo
ZSAnaGFzLWZvbnRzPycgcGFja2FnZSBwcm9wZXJ0eSksCitAYyBidXQgbGV0J3Mga2VlcCBpdCB1
bmRvY3VtZW50ZWQgdW50aWwgaXQgdHVybnMgb3V0IHRvIGJlIGFjdHVhbGx5CitAYyBuZWVkZWQg
c29tZXdoZXJlIGVsc2UgdGhhbiBpbiB0aGUgaW5jb3JyZWN0bHktbmFtZWQgJ2dzLWZvbnRzJwor
QGMgcGFja2FnZSAuLi4KIAogQG5vZGUgQ29kaW5nIFN0eWxlCiBAc2VjdGlvbiBDb2RpbmcgU3R5
bGUKZGlmZiAtLWdpdCBhL2dudS9wYWNrYWdlcy9naG9zdHNjcmlwdC5zY20gYi9nbnUvcGFja2Fn
ZXMvZ2hvc3RzY3JpcHQuc2NtCmluZGV4IDAzYTUxNmRjNTIuLmE5ZjFjNTJjNjYgMTAwNjQ0Ci0t
LSBhL2dudS9wYWNrYWdlcy9naG9zdHNjcmlwdC5zY20KKysrIGIvZ251L3BhY2thZ2VzL2dob3N0
c2NyaXB0LnNjbQpAQCAtMzg0LDcgKzM4NCwxNSBAQCBhcmNoaXRlY3R1cmUuIikKICAgICAiR2hv
c3RzY3JpcHQgZm9udHMgcHJvdmlkZXMgZm9udHMgYW5kIGZvbnQgbWV0cmljcyBjdXN0b21hcmls
eSBkaXN0cmlidXRlZCB3aXRoCiBHaG9zdHNjcmlwdC4gIEl0IGN1cnJlbnRseSBpbmNsdWRlcyB0
aGUgMzUgc3RhbmRhcmQgUG9zdFNjcmlwdCBmb250cy4iKQogICAgKGxpY2Vuc2UgbGljZW5zZTpn
cGwyKQotICAgKGhvbWUtcGFnZSAiaHR0cHM6Ly9zb3VyY2Vmb3JnZS5uZXQvcHJvamVjdHMvZ3Mt
Zm9udHMvIikpKQorICAgKGhvbWUtcGFnZSAiaHR0cHM6Ly9zb3VyY2Vmb3JnZS5uZXQvcHJvamVj
dHMvZ3MtZm9udHMvIikKKyAgIChwcm9wZXJ0aWVzCisgICAgOzsgVE9ETzogZXhwbGljaXRlbHkg
dGVsbCAnZm9udHMtZGlyLWZpbGUnIHRoYXQgdGhpcyBpcyBhIGZvbnQgcGFja2FnZSwKKyAgICA7
OyBhcyB0aGlzIHBhY2thZ2UgdmlvbGF0ZXMgdGhlIGNvbnZlbnRpb24gdGhhdCBmb250IHBhY2th
Z2UgbmFtZXMgYXJlCisgICAgOzsgcHJlZml4ZWQgd2l0aCAnZm9udC0nLgorICAgIDs7CisgICAg
OzsgVE9ETyhjb3JlLXVwZGF0ZXMpOiBSZW5hbWUgcGFja2FnZSB0byAnZm9udHMtZ2hvc3RzY3Jp
cHQnIG9yIHNvbWV0aGluZworICAgIDs7IGxpa2UgdGhhdC4KKyAgICBgKChoYXMtZm9udHM/IC4g
I3QpKSkpKQogCiAoZGVmaW5lLXB1YmxpYyBsaWJzcGVjdHJlCiAgIChwYWNrYWdlCmRpZmYgLS1n
aXQgYS9ndWl4L3Byb2ZpbGVzLnNjbSBiL2d1aXgvcHJvZmlsZXMuc2NtCmluZGV4IDhjMDIxNDlj
NmYuLjBmOWRmNjhmNDIgMTAwNjQ0Ci0tLSBhL2d1aXgvcHJvZmlsZXMuc2NtCisrKyBiL2d1aXgv
cHJvZmlsZXMuc2NtCkBAIC0xMiw2ICsxMiw3IEBACiA7OzsgQ29weXJpZ2h0IMKpIDIwMTkgTWF0
aGlldSBPdGhhY2VoZSA8bS5vdGhhY2VoZUBnbWFpbC5jb20+CiA7OzsgQ29weXJpZ2h0IMKpIDIw
MjAgRGFubnkgTWlsb3NhdmxqZXZpYyA8ZGFubnltQHNjcmF0Y2hwb3N0Lm9yZz4KIDs7OyBDb3B5
cmlnaHQgwqkgMjAxNCBEYXZpZCBUaG9tcHNvbiA8ZGF2ZXRAZ251Lm9yZz4KKzs7OyBDb3B5cmln
aHQgwqkgMjAyMSBNYXhpbWUgRGV2b3MgPG1heGltZWRldm9zQHRlbGVuZXQuYmU+CiA7OzsKIDs7
OyBUaGlzIGZpbGUgaXMgcGFydCBvZiBHTlUgR3VpeC4KIDs7OwpAQCAtMzMsNiArMzQsOCBAQAog
ICAjOnVzZS1tb2R1bGUgKChndWl4IHV0aWxzKSAjOmhpZGUgKHBhY2thZ2UtbmFtZS0+bmFtZSt2
ZXJzaW9uKSkKICAgIzp1c2UtbW9kdWxlICgoZ3VpeCBidWlsZCB1dGlscykKICAgICAgICAgICAg
ICAgICAjOnNlbGVjdCAocGFja2FnZS1uYW1lLT5uYW1lK3ZlcnNpb24gbWtkaXItcCkpCisgICM6
dXNlLW1vZHVsZSAoKGd1aXggYnVpbGQtc3lzdGVtKQorICAgICAgICAgICAgICAgICM6c2VsZWN0
IChidWlsZC1zeXN0ZW0tbmFtZSkpCiAgICM6dXNlLW1vZHVsZSAoKGd1aXggZGlhZ25vc3RpY3Mp
ICM6c2VsZWN0ICgmZml4LWhpbnQpKQogICAjOnVzZS1tb2R1bGUgKGd1aXggaTE4bikKICAgIzp1
c2UtbW9kdWxlIChndWl4IHJlY29yZHMpCkBAIC0xNTE0LDYgKzE1MTcsMjMgQEAgZmlsZXMgZm9y
IHRoZSBmb250cyBvZiB0aGUgQHZhcnttYW5pZmVzdH0gZW50cmllcy4iCiAgIChkZWZpbmUgbWtm
b250ZGlyCiAgICAgKG1vZHVsZS1yZWYgKHJlc29sdmUtaW50ZXJmYWNlICcoZ251IHBhY2thZ2Vz
IHhvcmcpKSAnbWtmb250ZGlyKSkKIAorICAoZGVmaW5lIChoYXMtZm9udHM/IGlucHV0KQorICAg
IChkZWZpbmUgdGhpbmcgKGdleHAtaW5wdXQtdGhpbmcgaW5wdXQpKQorICAgIChpZiAocGFja2Fn
ZT8gdGhpbmcpCisgICAgICAgIChvciAoc3RyaW5nLXByZWZpeD8gImZvbnQtIiAocGFja2FnZS1u
YW1lIHRoaW5nKSkKKyAgICAgICAgICAgIDs7IEluIHRoZSB1cHN0cmVhbSAnZ3VpeCcgY2hhbm5l
bCwgZm9udCBwYWNrYWdlcyBzaG91bGQKKyAgICAgICAgICAgIDs7IGJlIG5hbWVkIGZvbnQtU09N
RVRISU5HLiAgQnV0IGlmIGFub3RoZXIgY2hhbm5lbAorICAgICAgICAgICAgOzsgbmFtZXMgaXRz
IGZvbnRzIGRpZmZlcmVudGx5IGJ1dCB1c2VzIGZvbnQtYnVpbGQtc3lzdGVtLAorICAgICAgICAg
ICAgOzsgYWNjZXB0aW5nIHRoYXQgc2VlbXMgZnJpZW5kbHkuCisgICAgICAgICAgICAoZXE/ICdm
b250IChidWlsZC1zeXN0ZW0tbmFtZSAocGFja2FnZS1idWlsZC1zeXN0ZW0gdGhpbmcpKSkKKyAg
ICAgICAgICAgIDs7IEZJWE1FKGNvcmUtdXBkYXRlcykgZXNjYXBlIGhhdGNoIGZvciB0aGUgaW5j
b3JyZWN0bHktbmFtZWQKKyAgICAgICAgICAgIDs7ICdncy1mb250cycgcGFja2FnZQorICAgICAg
ICAgICAgKGFzc3EtcmVmIChwYWNrYWdlLXByb3BlcnRpZXMgdGhpbmcpICdoYXMtZm9udHM/KSkK
KyAgICAgICAgOzsgUGVzc2ltaXN0aWNhbGx5IGFzc3VtZSB0aGUgaW5wdXQgbWlnaHQgaGF2ZSBm
b250cy4KKyAgICAgICAgI3QpKQorCisgIChkZWZpbmUgcmVsZXZhbnQtaW5wdXRzIChmaWx0ZXIg
aGFzLWZvbnRzPyAobWFuaWZlc3QtaW5wdXRzIG1hbmlmZXN0KSkpCisKICAgKGRlZmluZSBidWls
ZAogICAgICN+KGJlZ2luCiAgICAgICAgICh1c2UtbW9kdWxlcyAoc3JmaSBzcmZpLTI2KQpAQCAt
MTUyMiw3ICsxNTQyLDcgQEAgZmlsZXMgZm9yIHRoZSBmb250cyBvZiB0aGUgQHZhcnttYW5pZmVz
dH0gZW50cmllcy4iCiAgICAgICAgIChsZXQgKChmb250cy1kaXJzIChmaWx0ZXIgZmlsZS1leGlz
dHM/CiAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKG1hcCAoY3V0IHN0cmluZy1h
cHBlbmQgPD4KICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIi9z
aGFyZS9mb250cyIpCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAnIyQo
bWFuaWZlc3QtaW5wdXRzIG1hbmlmZXN0KSkpKSkKKyAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgICAgICAgICcjJHJlbGV2YW50LWlucHV0cykpKSkKICAgICAgICAgICAobWtkaXIgIyRv
dXRwdXQpCiAgICAgICAgICAgKGlmIChudWxsPyBmb250cy1kaXJzKQogICAgICAgICAgICAgICAo
ZXhpdCAjdCkKLS0gCjIuMzIuMAoK


--=-hq4KR5p1OcuVD7uhGQNP
Content-Disposition: inline;
	filename*0=0002-profiles-Avoid-dependency-on-mkfontdir-and-friend-wh.pat;
	filename*1=ch
Content-Type: text/x-patch;
	name="0002-profiles-Avoid-dependency-on-mkfontdir-and-friend-wh.patch";
	charset="UTF-8"
Content-Transfer-Encoding: base64

RnJvbSBhNzA2YjhiZTRmNTQ1MzBiMWNkMTJjMDNhMWJmMzk0MWJlNDNiZTNjIE1vbiBTZXAgMTcg
MDA6MDA6MDAgMjAwMQpGcm9tOiBNYXhpbWUgRGV2b3MgPG1heGltZWRldm9zQHRlbGVuZXQuYmU+
CkRhdGU6IE1vbiwgNSBKdWwgMjAyMSAyMDoxNjoyMiArMDIwMApTdWJqZWN0OiBbUEFUQ0ggMi8y
XSBwcm9maWxlczogQXZvaWQgZGVwZW5kZW5jeSBvbiAnbWtmb250ZGlyJyBhbmQgZnJpZW5kIHdo
ZW4KIHVudXNlZC4KCkF2b2lkIGRlcGVuZGluZyBvbiBvbiAibWtmb250ZGlyIiBhbmQgIm1rZm9u
dHNjYWxlIiBpZgp0aGV5IHdvbid0IGFjdHVhbGx5IGJlIHVzZWQsIHRvIGF2b2lkIGJ1aWxkaW5n
IHRoZQphZm9yZW1lbnRpb25lZCBwYWNrYWdlcyB3aGVuIHRoZXkgYXJlIHVwZGF0ZWQuCgoqIGd1
aXgvcHJvZmlsZXMuc2NtCiAgKGZvbnRzLWRpci1maWxlKVtidWlsZF06IE5vdGUgd2h5IHRoZSAn
KG51bGw/IGZvbnRzLWRpciknIGNoZWNrCiAgY2Fubm90IGJlIHJlbW92ZWQuCiAgKGZvbnRzLWRp
ci1maWxlKTogV2hlbiB0aGUgJ3JlbGV2YW50LWlucHV0cycgbGlzdCBpcyBlbXB0eSwKICBqdXN0
IGJ1aWxkIGFuIGVtcHR5IGRpcmVjdG9yeS4KLS0tCiBndWl4L3Byb2ZpbGVzLnNjbSB8IDI1ICsr
KysrKysrKysrKysrKystLS0tLS0tLS0KIDEgZmlsZSBjaGFuZ2VkLCAxNiBpbnNlcnRpb25zKCsp
LCA5IGRlbGV0aW9ucygtKQoKZGlmZiAtLWdpdCBhL2d1aXgvcHJvZmlsZXMuc2NtIGIvZ3VpeC9w
cm9maWxlcy5zY20KaW5kZXggMGY5ZGY2OGY0Mi4uNmNmNDgwZGRmMiAxMDA2NDQKLS0tIGEvZ3Vp
eC9wcm9maWxlcy5zY20KKysrIGIvZ3VpeC9wcm9maWxlcy5zY20KQEAgLTE1NDQsNiArMTU0NCw4
IEBAIGZpbGVzIGZvciB0aGUgZm9udHMgb2YgdGhlIEB2YXJ7bWFuaWZlc3R9IGVudHJpZXMuIgog
ICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAiL3NoYXJlL2ZvbnRz
IikKICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICcjJHJlbGV2YW50LWlu
cHV0cykpKSkKICAgICAgICAgICAobWtkaXIgIyRvdXRwdXQpCisgICAgICAgICAgOzsgaGFzLWZv
bnRzPyBjYW4gaGF2ZSBmYWxzZSBwb3NpdGl2ZXMsCisgICAgICAgICAgOzsgc28gdGhpcyBjaGVj
ayBpcyBuZWNlc3NhcnkuCiAgICAgICAgICAgKGlmIChudWxsPyBmb250cy1kaXJzKQogICAgICAg
ICAgICAgICAoZXhpdCAjdCkKICAgICAgICAgICAgICAgKGxldCogKChzaGFyZS1kaXIgICAoc3Ry
aW5nLWFwcGVuZCAjJG91dHB1dCAiL3NoYXJlIikpCkBAIC0xNTg1LDE1ICsxNTg3LDIwIEBAIGZp
bGVzIGZvciB0aGUgZm9udHMgb2YgdGhlIEB2YXJ7bWFuaWZlc3R9IGVudHJpZXMuIgogICAgICAg
ICAgICAgICAgICAgICAgICAgICAgICAgICAgIChkZWxldGUtZmlsZSBmb250cy1kaXItZmlsZSkp
KSkKICAgICAgICAgICAgICAgICAgICAgICAgICAgICBkaXJlY3RvcmllcykpKSkpKSkKIAotICAo
Z2V4cC0+ZGVyaXZhdGlvbiAiZm9udHMtZGlyIiBidWlsZAotICAgICAgICAgICAgICAgICAgICAj
Om1vZHVsZXMgJygoZ3VpeCBidWlsZCB1dGlscykKLSAgICAgICAgICAgICAgICAgICAgICAgICAg
ICAgICAgKGd1aXggYnVpbGQgdW5pb24pCi0gICAgICAgICAgICAgICAgICAgICAgICAgICAgICAg
IChzcmZpIHNyZmktMjYpKQotICAgICAgICAgICAgICAgICAgICAjOmxvY2FsLWJ1aWxkPyAjdAot
ICAgICAgICAgICAgICAgICAgICAjOnN1YnN0aXR1dGFibGU/ICNmCi0gICAgICAgICAgICAgICAg
ICAgICM6cHJvcGVydGllcwotICAgICAgICAgICAgICAgICAgICBgKCh0eXBlIC4gcHJvZmlsZS1o
b29rKQotICAgICAgICAgICAgICAgICAgICAgIChob29rIC4gZm9udHMtZGlyKSkpKQorICAoaWYg
KG51bGw/IHJlbGV2YW50LWlucHV0cykKKyAgICAgIDs7IEF2b2lkIGRlcGVuZGluZyBvbiBvbiAi
bWtmb250ZGlyIiBhbmQgIm1rZm9udHNjYWxlIiBpZgorICAgICAgOzsgdGhleSB3b24ndCBhY3R1
YWxseSBiZSB1c2VkLCB0byBhdm9pZCBidWlsZGluZyB0aGUgYWZvcmVtZW50aW9uZWQKKyAgICAg
IDs7IHBhY2thZ2VzIHdoZW4gdGhleSBhcmUgdXBkYXRlZC4KKyAgICAgIChsb3dlci1vYmplY3Qg
KGZpbGUtdW5pb24gImZvbnRzLWRpciIgJygpKSkKKyAgICAgIChnZXhwLT5kZXJpdmF0aW9uICJm
b250cy1kaXIiIGJ1aWxkCisgICAgICAgICAgICAgICAgICAgICAgICAjOm1vZHVsZXMgJygoZ3Vp
eCBidWlsZCB1dGlscykKKyAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgIChndWl4
IGJ1aWxkIHVuaW9uKQorICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgICAgKHNyZmkg
c3JmaS0yNikpCisgICAgICAgICAgICAgICAgICAgICAgICAjOmxvY2FsLWJ1aWxkPyAjdAorICAg
ICAgICAgICAgICAgICAgICAgICAgIzpzdWJzdGl0dXRhYmxlPyAjZgorICAgICAgICAgICAgICAg
ICAgICAgICAgIzpwcm9wZXJ0aWVzCisgICAgICAgICAgICAgICAgICAgICAgICBgKCh0eXBlIC4g
cHJvZmlsZS1ob29rKQorICAgICAgICAgICAgICAgICAgICAgICAgICAoaG9vayAuIGZvbnRzLWRp
cikpKSkpCiAKIChkZWZpbmUgKG1hbnVhbC1kYXRhYmFzZSBtYW5pZmVzdCkKICAgIlJldHVybiBh
IGRlcml2YXRpb24gdGhhdCBidWlsZHMgdGhlIG1hbnVhbCBwYWdlIGRhdGFiYXNlIChcIm1hbmRi
XCIpIGZvcgotLSAKMi4zMi4wCgo=


--=-hq4KR5p1OcuVD7uhGQNP--

--=-eL9RW6M64D0urJpMFyVi
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYONYOBccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7lJtAP9nvaqaE+zY3PdMOWeNm3KRB0gn
Jg7olk/UOUwGsYucgAEA2flnKzsocky7i3yjttW1kFKdfasoNfjiUnMOziukKw0=
=6xxl
-----END PGP SIGNATURE-----

--=-eL9RW6M64D0urJpMFyVi--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: Maxime Devos <maximedevos@HIDDEN>
Subject: bug#49421: Acknowledgement ([PATCH] profiles: Optimise
 'fonts-dir-file'.)
Message-ID: <handler.49421.B.162551200813844.ack <at> debbugs.gnu.org>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
X-Gnu-PR-Message: ack 49421
X-Gnu-PR-Package: guix-patches
X-Gnu-PR-Keywords: patch
Reply-To: 49421 <at> debbugs.gnu.org
Date: Mon, 05 Jul 2021 19:07:02 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 guix-patches@HIDDEN

If you wish to submit further information on this problem, please
send it to 49421 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
49421: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D49421
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to guix-patches@HIDDEN:

Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 07 Jul 2021 16:54:02 +0000
Resent-Message-ID: <handler.49421.B49421.162567678716539 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Maxime Devos <maximedevos@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
Date: Wed, 07 Jul 2021 18:52:54 +0200
In-Reply-To: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 (Maxime Devos's message of "Mon, 05 Jul 2021 21:06:32 +0200")
Message-ID: <87a6myoyl5.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

Hi,

Maxime Devos <maximedevos@HIDDEN> skribis:

> These two patches should speed up profile generation
> by optimising the 'fonts-dir-file' hook.  The first patch
> is the most important; from the commit message:
>
> =E2=80=98Only let the build G-exp refer to inputs that might actually
> have fonts.  That way, if the list of fonts in the manifest
> didn't change and the new manifest is built, the font files
> will not be rebuilt.=E2=80=99
>
> The second patch doesn't bring much.
>
> To test, you can test whether things like ...
>
> $ ./pre-inst-env guix environment --ad-hoc --pure lagrange -- lagrange
>
> still work (lagrange is a graphical application using fonts).
> Not sure what a good =E2=80=98benchmark=E2=80=99 would be.
>
> Unfortunately, this does not help with the "guix package -i" case,
> as in that case, the code doesn't have access to all the package objects,
> and will have to satisfy itself with the store paths, in which case the
> profile code pessimistically assumes the store item has fonts ...
>
> I suppose this restriction could be lifted if/when the gs-fonts package
> is renamed to font-ghostscript or something like that ...

Do you know what=E2=80=99s taking time in the =E2=80=98fonts-dir-file=E2=80=
=99 hook?
(On my x86_64 laptop with SSD, it runs in 4.4s on a cold cache for a
profile containing 14 font packages, and in 1.2s on a warm cache.)

IIUC, the hook completes almost instantaneously when creating a profile
without fonts because (null? fonts-dirs) is true.

In other cases, one option to speed things up (and possibly remove the
hook=E2=80=99s dependency on mkfontdir) would be to create =E2=80=98fonts.d=
ir=E2=80=99 and
=E2=80=98fonts.scale=E2=80=99 in a build phase of each font package, *provi=
ded* there=E2=80=99s
a cheap way to assemble several such files, such as mere concatenation.
That way, the profile hook would only need to assemble these files as
opposed to traversing all the font files.

Given that =E2=80=98fonts.dir=E2=80=99 and =E2=80=98fonts.scale=E2=80=99 ar=
e mere text files with one
line per font, this approach looks doable.

WDYT?

> From 4fe1e30e33c01be9fd17cf240732b3351c7b0fa4 Mon Sep 17 00:00:00 2001
> From: Maxime Devos <maximedevos@HIDDEN>
> Date: Mon, 5 Jul 2021 18:55:31 +0200
> Subject: [PATCH 1/2] profiles: Optimise 'fonts-dir-file'.
>
> Only let the build G-exp refer to inputs that might actually
> have fonts.  That way, if the list of fonts in the manifest
> didn't change and the new manifest is built, the font files
> will not be rebuilt.
>
> * guix/profiles.scm
>   (fonts-dir-file)[has-fonts?]: New predicate.
>   (fonts-dir-file)[relevant-inputs]: New variable.
>   (fonts-dir-file)[build]: Use 'relevant-inputs' instead of
>   'manifest-inputs'.
> * doc/contributing.texi (Fonts): Note the 'fonts-' naming
>   convention is technically important now.
> * gnu/packages/ghostscript.scm (gs-fonts): Work-around the
>   package name contravening the convention.

To me this approach should be a last resort because it relies on
conventions and manual annotations, and it=E2=80=99s easy to forget these a=
nd
get incorrect results (we could end up having to deal with reports like
=E2=80=9CI installed font X, why doesn=E2=80=99t in show up in the font lis=
t?=E2=80=9D).

> +            ;; In the upstream 'guix' channel, font packages should
> +            ;; be named font-SOMETHING.  But if another channel
> +            ;; names its fonts differently but uses font-build-system,
> +            ;; accepting that seems friendly.
> +            (eq? 'font (build-system-name (package-build-system thing)))

In general =E2=80=98build-system-name=E2=80=99 shouldn=E2=80=99t be used; t=
he name is meant to
be a debugging aid.

Thanks,
Ludo=E2=80=99.

Message sent to guix-patches@HIDDEN:

Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Maxime Devos <maximedevos@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Thu, 15 Jul 2021 16:29:01 +0000
Resent-Message-ID: <handler.49421.B49421.162636650917646 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
Message-ID: <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
From: Maxime Devos <maximedevos@HIDDEN>
Date: Thu, 15 Jul 2021 18:28:16 +0200
In-Reply-To: <87a6myoyl5.fsf@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 <87a6myoyl5.fsf@HIDDEN>
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-f8AOwVr+vtZtUE5SEUdt"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>


--=-f8AOwVr+vtZtUE5SEUdt
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

Ludovic Court=C3=A8s schreef op wo 07-07-2021 om 18:52 [+0200]:
> Hi,
>=20
> Maxime Devos <maximedevos@HIDDEN> skribis:
>=20
> > These two patches should speed up profile generation
> > by optimising the 'fonts-dir-file' hook.  The first patch
> > is the most important; from the commit message:
> >=20
> > =E2=80=98Only let the build G-exp refer to inputs that might actually
> > have fonts.  That way, if the list of fonts in the manifest
> > didn't change and the new manifest is built, the font files
> > will not be rebuilt.=E2=80=99
> >=20
> > The second patch doesn't bring much.
> >=20
> > To test, you can test whether things like ...
> >=20
> > $ ./pre-inst-env guix environment --ad-hoc --pure lagrange -- lagrange
> >=20
> > still work (lagrange is a graphical application using fonts).
> > Not sure what a good =E2=80=98benchmark=E2=80=99 would be.
> >=20
> > Unfortunately, this does not help with the "guix package -i" case,
> > as in that case, the code doesn't have access to all the package object=
s,
> > and will have to satisfy itself with the store paths, in which case the
> > profile code pessimistically assumes the store item has fonts ...
> >=20
> > I suppose this restriction could be lifted if/when the gs-fonts package
> > is renamed to font-ghostscript or something like that ...
>=20
> Do you know what=E2=80=99s taking time in the =E2=80=98fonts-dir-file=E2=
=80=99 hook?
> (On my x86_64 laptop with SSD, it runs in 4.4s on a cold cache for a
> profile containing 14 font packages, and in 1.2s on a warm cache.)

I noticed =E2=80=98building fonts directories=E2=80=99 was taking long (9 m=
inutes or so,
when using "guix package -r two packages that don't have fonts"),
so I took the opportunity to "strace" the process, though I was a little la=
te.
Some statistics:

% time     seconds  usecs/call     calls    errors syscall
------ ----------- ----------- --------- --------- ----------------
 27,14    1,676551         117     14297           rename
 19,60    1,211012          13     90829         8 lstat
 18,34    1,133182          63     17787        15 statx
 11,80    0,729185          51     14297           link
  6,65    0,411065       68510         6           wait4
  5,08    0,313632          10     28651           chmod
  3,66    0,226258          14     15114           readlink
  2,18    0,134913           8     15193           utimensat
  1,78    0,110257          32      3431           rmdir
  0,91    0,056504           7      7370           getdents64
  0,82    0,050817           3     14295           getpid
  0,47    0,029074           7      3736           openat
  0,46    0,028547           8      3479      3428 unlink
  0,36    0,021940           5      3747           close
  0,26    0,016355           4      3704           fstat
  0,16    0,009752          10       898           lchown
  0,12    0,007242           4      1800           geteuid
  0,07    0,004203          62        67           read
  0,06    0,003512           3       900           getegid
  0,03    0,001959         326         6           clone
  0,02    0,001133          18        62           write
  0,01    0,000488           3       127         1 fcntl
  0,01    0,000451           3       136           lseek
  0,00    0,000197          14        14           mkdir
  0,00    0,000065          13         5           select
  0,00    0,000045           5         8           chown
  0,00    0,000044           7         6           brk
  0,00    0,000027          13         2           munmap
  0,00    0,000023          11         2           pipe
  0,00    0,000013           6         2           stat
  0,00    0,000007           7         1           futex
  0,00    0,000004           2         2           getuid
  0,00    0,000000           0         1           rt_sigreturn
------ ----------- ----------- --------- --------- ----------------
100,00    6,178457          25    239975      3452 total

And a part of the log:

<start of log>
lstat("/gnu/store/.links/[...]2i", {st_mode=3DS_IFLNK|0777, st_size=3D124, =
...}) =3D 0
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", {st_mode=3DS_IFDIR|0555, st_size=3D4112, ...>
chmod("/gnu/store/[..]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AMS=
/Regular/283", 040755) =3D 0
getpid()                                =3D 26371
link("/gnu/store/.links/[..]2i", "/gnu/store/.tmp-link-26371-632297102") =
=3D 0
rename("/gnu/store/.tmp-link-26371-632297102", "/gnu/store/[...]nm-fonts-di=
r/share/fonts/mathjax/HTML-CSS/TeX/png/AMS/Regular/283/2>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", {st_mode=3DS_IFDIR|0755, st_size=3D4112, ...>
chmod("/gnu/store/[...]-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AMS/=
Regular/283", 040555) =3D 0
utimensat(AT_FDCWD, "/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-=
CSS/TeX/png/AMS/Regular/283", [{tv_sec=3D1626363034, tv_ns>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283/2223.png", {st_mode=3DS_IFLNK|0777, st_size=3D>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283/2223.png", {st_mode=3DS_IFLNK|0777, st_size=3D>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283/2223.png", {st_mode=3DS_IFLNK|0777, st_size=3D>
readlink("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png=
/AMS/Regular/283/2223.png", "/gnu/store/bf582s56ldb3y34p>
statx(AT_FDCWD, "/gnu/store/.links/[...]v3", AT_STATX_SYNC_AS_STAT|AT_SYMLI=
NK_NOFOLLOW, 0, {stx_mask=3DSTATX_ALL|>
lstat("/gnu/store/.links/[...]v3", {st_mode=3DS_IFLNK|0777, st_size=3D124, =
...}) =3D 0
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", {st_mode=3DS_IFDIR|0555, st_size=3D4112, ...>
chmod("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", 040755) =3D 0
getpid()                                =3D 26371
link("/gnu/store/.links/[...]v3", "/gnu/store/.tmp-link-26371-114005693") =
=3D 0
rename("/gnu/store/.tmp-link-26371-114005693", "/gnu/store/[...]nm-fonts-di=
r/share/fonts/mathjax/HTML-CSS/TeX/png/AMS/Regular/283/2>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", {st_mode=3DS_IFDIR|0755, st_size=3D4112, ...>
chmod("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", 040555) =3D 0
utimensat(AT_FDCWD, "/gnu/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/Te=
X/png/AMS/Regular/283", [{tv_sec=3D1626363034, tv_ns>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283/22CC.png", {st_mode=3DS_IFLNK|0777, st_size=3D>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283/22CC.png", {st_mode=3DS_IFLNK|0777, st_size=3D>
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283/22CC.png", {st_mode=3DS_IFLNK|0777, st_size=3D>
readlink("/gnu/store/h6681qq9s5kqjvdzkhqhddqbjj3q48nm-fonts-dir/share/fonts=
/mathjax/HTML-CSS/TeX/png/AMS/Regular/283/22CC.png", "/gnu/store/bf582s56ld=
b3y34p>
statx(AT_FDCWD, "/gnu/store/.links/[...]my", AT_STATX_SYNC_AS_STAT|AT_SYMLI=
NK_NOFOLLOW, 0, {stx_mask=3DSTATX_ALL|>
lstat("/gnu/store/.links/[...]my", {st_mode=3DS_IFLNK|0777, st_size=3D124, =
...}) =3D 0
lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", {st_mode=3DS_IFDIR|0555, st_size=3D4112, ...>
chmod("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/AM=
S/Regular/283", 040755) =3D 0
getpid()                                =3D 26371
<end of log>

It seems like =E2=80=98duplication=E2=80=99 is taking a lot of time!
(See nix/libstore/optimise-store.cc and (guix store deduplication).)

> IIUC, the hook completes almost instantaneously when creating a profile
> without fonts because (null? fonts-dirs) is true.
>=20
> In other cases, one option to speed things up (and possibly remove the
> hook=E2=80=99s dependency on mkfontdir) would be to create =E2=80=98fonts=
.dir=E2=80=99 and
> =E2=80=98fonts.scale=E2=80=99 in a build phase of each font package, *pro=
vided* there=E2=80=99s
> a cheap way to assemble several such files, such as mere concatenation.
> That way, the profile hook would only need to assemble these files as
> opposed to traversing all the font files.

Looking at ~/.guix-profile/share/fonts/truetype/fonts.dir,
it seems to be simply a list of entries lke

   DejaVuSans-Bold.ttf -misc-dejavu sans-bold-r-normal--0-0-0-0-p-0-koi8-ru

with additionally as first line the number of fonts.  It appears to be
sorted by font file name (DejaVuSans-Bold.ttf) though, though that only
seems to be to not depend on file system ordering.  Not quite _mere_
concatenation, but should be easy enough, I think.

(Likewise for fonts.scaale)

I would need to look at the code of "mkfontscale" and "mkfontdir" to be sur=
e
though.

> Given that =E2=80=98fonts.dir=E2=80=99 and =E2=80=98fonts.scale=E2=80=99 =
are mere text files with one
> line per font, this approach looks doable.
>=20
> WDYT?

That seems a good idea, reducing the amount of 'stat'ing, 'readdir'ing,
and 'read'ing (mkfontscale needs to actually read the font files)
which should help on spinning disks (and other disks, but on SSDs it's prob=
ably
less noticable).

> > From 4fe1e30e33c01be9fd17cf240732b3351c7b0fa4 Mon Sep 17 00:00:00 2001
> > From: Maxime Devos <maximedevos@HIDDEN>
> > Date: Mon, 5 Jul 2021 18:55:31 +0200
> > Subject: [PATCH 1/2] profiles: Optimise 'fonts-dir-file'.
> >=20
> > Only let the build G-exp refer to inputs that might actually
> > have fonts.  That way, if the list of fonts in the manifest
> > didn't change and the new manifest is built, the font files
> > will not be rebuilt.
> >=20
> > * guix/profiles.scm
> >   (fonts-dir-file)[has-fonts?]: New predicate.
> >   (fonts-dir-file)[relevant-inputs]: New variable.
> >   (fonts-dir-file)[build]: Use 'relevant-inputs' instead of
> >   'manifest-inputs'.
> > * doc/contributing.texi (Fonts): Note the 'fonts-' naming
> >   convention is technically important now.
> > * gnu/packages/ghostscript.scm (gs-fonts): Work-around the
> >   package name contravening the convention.
>=20
> To me this approach should be a last resort because it relies on
> conventions and manual annotations, and it=E2=80=99s easy to forget these=
 and
> get incorrect results (we could end up having to deal with reports like
> =E2=80=9CI installed font X, why doesn=E2=80=99t in show up in the font l=
ist?=E2=80=9D).

AFAIK, all font packages in Guix (except gs-fonts) are currently
correctly-named.  They aren't going to spontanuously rename theirselves.
So to me, the only potential problem seems to be _new_ font packages
ignoring the conventions.  But new packages should always be reviewed and
tested, so such reports don't seem plausible to me.

(Also, the manual annotation would go away once 'gs-fonts' is renamed.
I'll run "echo /gnu/store/*/share/fonts > atchie-fonts" though to see
if there are other packages not following the convention.)

> > +            ;; In the upstream 'guix' channel, font packages should
> > +            ;; be named font-SOMETHING.  But if another channel
> > +            ;; names its fonts differently but uses font-build-system,
> > +            ;; accepting that seems friendly.
> > +            (eq? 'font (build-system-name (package-build-system thing)=
))
>=20
> In general =E2=80=98build-system-name=E2=80=99 shouldn=E2=80=99t be used;=
 the name is meant to
> be a debugging aid.

I could do (eq? font-build-system (packag-build-system thing)) instead
or drop this clause.

> Thanks,
> Ludo=E2=80=99.

--=-f8AOwVr+vtZtUE5SEUdt
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYPBiIBccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7vYQAP4p5PTgHgfrNPzE6lB/cIAg8WQP
2UTmyDLOQNOQ8qnVLwEAvkqoSME3w0PP5KnZloAY5BsT0wd2rzaWpBkRzWE4NA4=
=1Vm6
-----END PGP SIGNATURE-----

--=-f8AOwVr+vtZtUE5SEUdt--

Message sent to guix-patches@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Maxime Devos <maximedevos@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Thu, 15 Jul 2021 18:49:02 +0000
Resent-Message-ID: <handler.49421.B49421.162637490524113 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 49421
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
Received: via spool by 49421-submit <at> debbugs.gnu.org id=B49421.162637490524113
          (code B ref 49421); Thu, 15 Jul 2021 18:49:02 +0000
Received: (at 49421) by debbugs.gnu.org; 15 Jul 2021 18:48:25 +0000
Received: from localhost ([127.0.0.1]:49725 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1m46PJ-0006Gr-4p
	for submit <at> debbugs.gnu.org; Thu, 15 Jul 2021 14:48:25 -0400
Received: from michel.telenet-ops.be ([195.130.137.88]:57740)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maximedevos@HIDDEN>) id 1m46PF-0006Gf-Ft
 for 49421 <at> debbugs.gnu.org; Thu, 15 Jul 2021 14:48:24 -0400
Received: from butterfly.local ([213.119.236.63])
 by michel.telenet-ops.be with bizsmtp
 id VWoJ2500L1Nl63106WoKyU; Thu, 15 Jul 2021 20:48:20 +0200
Message-ID: <369206a276e225478ed1bbce0b883255dccef369.camel@HIDDEN>
From: Maxime Devos <maximedevos@HIDDEN>
Date: Thu, 15 Jul 2021 20:48:09 +0200
In-Reply-To: <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 <87a6myoyl5.fsf@HIDDEN>
 <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
Content-Type: multipart/signed; micalg="pgp-sha512";
 protocol="application/pgp-signature"; boundary="=-CbfvmF6qgDJPbAHSoYqp"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r21;
 t=1626374900; bh=cLsUGn6jBBuJ8JOaVGkzFELFr1c87eH+cEfbFVAY2ZU=;
 h=Subject:From:To:Cc:Date:In-Reply-To:References;
 b=bOJS2+qoZ0FyP80wxOVeS3W6rzhikMdcvU/hk5tcsjVcZ+puHIQXA2jz+JXaede1l
 BiCnzCRfl3CJn3gYpZJXrVkRgIzhHCxEBfHrJsV+CWZ5LxhxgJ++sPotuve8vCFErY
 8j97b3LcsmZVPzRjUblSZRoykkPyat2KMXHNvDg9LV5HbR9z9m841z9uM+AcSFyv71
 8aAXGVSBzfKpkySe7IVF4nbURem4DTH6vteiPs+FzHUF5vRoSJ4gDD8w6uKSDsxG5J
 gI9kGIPrgd2OKWzkNYbVkjKgO+++lW5hUUbUWkS7QUQuU9WXsO5b7ZP0zgMgSrmaNE
 mEvNPC0dy9s1Q==
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)


--=-CbfvmF6qgDJPbAHSoYqp
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

> (Also, the manual annotation would go away once 'gs-fonts' is renamed.
> I'll run "echo /gnu/store/*/share/fonts > atchie-fonts" though to see
> if there are other packages not following the convention.)

Scheme code:

(use-modules (ice-9 ftw) (srfi srfi-1))
(define (has-share/fonts? o)
  (file-exists? (string-append "/gnu/store/" o "/share/fonts")))
(define t (scandir "/gnu/store" has-share/fonts?))
(define t2 (map (lambda (x) (substring x 33)) t))
(define t3 (delete-duplicates t2))
(define (fonty? x) (string-prefix? "font-" x))
(partition (lambda (x) (string-prefix? "font-" x)) t3)

Output on my system:

$1 =3D ("font-alias-1.0.3" [...])
$2 =3D ("fonts-dir" "profile" "mate-1.24.1" "denemo-2.5.0" "mate-1.24.0" "g=
s-fonts-8.11")

"fonts-dir", "profile" and "gs-fonts" are expected.

"mate" uses "union-build" and has "font-cantarell" in 'inputs'.
Apparently "denemo" has some font files as well.
Something will need to be figured out for these cases,
if the approach of my patch is followed.

I think it's better to optimise 'union-build' and the store deduplication
code though (maybe subdirectories can be process concurrently, to maximise =
I/O
utilisation?). That should improve performance outside the 'fonts-dir-file'=
 hook
as well.

Greetings,
Maxime.

--=-CbfvmF6qgDJPbAHSoYqp
Content-Type: application/pgp-signature; name="signature.asc"
Content-Description: This is a digitally signed message part
Content-Transfer-Encoding: 7bit

-----BEGIN PGP SIGNATURE-----

iI0EABYKADUWIQTB8z7iDFKP233XAR9J4+4iGRcl7gUCYPCC6RccbWF4aW1lZGV2
b3NAdGVsZW5ldC5iZQAKCRBJ4+4iGRcl7sFIAQCihDN5aMkr4uoKyseSCramBWrf
1uRwf2AHfKk6o2Qx5AD/TJygfcxSi/UZQimJKTtBuduiomtY1ouxpVaqkYDrkwY=
=iA5C
-----END PGP SIGNATURE-----

--=-CbfvmF6qgDJPbAHSoYqp--

Message sent to guix-patches@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Fri, 16 Jul 2021 16:06:02 +0000
Resent-Message-ID: <handler.49421.B49421.162645156112981 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 49421
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Maxime Devos <maximedevos@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
Received: via spool by 49421-submit <at> debbugs.gnu.org id=B49421.162645156112981
          (code B ref 49421); Fri, 16 Jul 2021 16:06:02 +0000
Received: (at 49421) by debbugs.gnu.org; 16 Jul 2021 16:06:01 +0000
Received: from localhost ([127.0.0.1]:52216 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1m4QLg-0003NJ-MF
	for submit <at> debbugs.gnu.org; Fri, 16 Jul 2021 12:06:00 -0400
Received: from eggs.gnu.org ([209.51.188.92]:46006)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1m4QLf-0003N6-Fj
 for 49421 <at> debbugs.gnu.org; Fri, 16 Jul 2021 12:06:00 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:34184)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1m4QLZ-00056F-QE; Fri, 16 Jul 2021 12:05:53 -0400
Received: from vpn-0-27.aquilenet.fr ([2a0c:e300:4:27::]:38218 helo=ribbon)
 by fencepost.gnu.org with esmtpsa (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256)
 (Exim 4.90_1) (envelope-from <ludo@HIDDEN>)
 id 1m4QLZ-0004vh-AM; Fri, 16 Jul 2021 12:05:53 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 <87a6myoyl5.fsf@HIDDEN>
 <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
X-URL: http://www.fdn.fr/~lcourtes/
X-Revolutionary-Date: 28 Messidor an 229 de la =?UTF-8?Q?R=C3=A9volution?=
X-PGP-Key-ID: 0x090B11993D9AEBB5
X-PGP-Key: http://www.fdn.fr/~lcourtes/ludovic.asc
X-PGP-Fingerprint: 3CE4 6455 8A84 FDC6 9DB4  0CFB 090B 1199 3D9A EBB5
X-OS: x86_64-pc-linux-gnu
Date: Fri, 16 Jul 2021 18:05:49 +0200
In-Reply-To: <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
 (Maxime Devos's message of "Thu, 15 Jul 2021 18:28:16 +0200")
Message-ID: <87pmviw8f6.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi!

Maxime Devos <maximedevos@HIDDEN> skribis:

> Ludovic Court=C3=A8s schreef op wo 07-07-2021 om 18:52 [+0200]:

[...]

>> Do you know what=E2=80=99s taking time in the =E2=80=98fonts-dir-file=E2=
=80=99 hook?
>> (On my x86_64 laptop with SSD, it runs in 4.4s on a cold cache for a
>> profile containing 14 font packages, and in 1.2s on a warm cache.)
>
> I noticed =E2=80=98building fonts directories=E2=80=99 was taking long (9=
 minutes or so,
> when using "guix package -r two packages that don't have fonts"),

Ouch, that=E2=80=99s way more than what I=E2=80=99m observing.  :-)  Do you=
 have a
manifest or package list to reproduce that?  Were those timings on a
spinning disk?

> so I took the opportunity to "strace" the process, though I was a little =
late.
> Some statistics:
>
> % time     seconds  usecs/call     calls    errors syscall
> ------ ----------- ----------- --------- --------- ----------------
>  27,14    1,676551         117     14297           rename
>  19,60    1,211012          13     90829         8 lstat
>  18,34    1,133182          63     17787        15 statx
>  11,80    0,729185          51     14297           link

[...]

> statx(AT_FDCWD, "/gnu/store/.links/[...]my", AT_STATX_SYNC_AS_STAT|AT_SYM=
LINK_NOFOLLOW, 0, {stx_mask=3DSTATX_ALL|>
> lstat("/gnu/store/.links/[...]my", {st_mode=3DS_IFLNK|0777, st_size=3D124=
, ...}) =3D 0
> lstat("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/=
AMS/Regular/283", {st_mode=3DS_IFDIR|0555, st_size=3D4112, ...>
> chmod("/gnu/store/[...]nm-fonts-dir/share/fonts/mathjax/HTML-CSS/TeX/png/=
AMS/Regular/283", 040755) =3D 0
> getpid()                                =3D 26371
> <end of log>
>
> It seems like =E2=80=98duplication=E2=80=99 is taking a lot of time!

It surely does but we need more data before we can draw a conclusion
(the system time above accounts for ~6 seconds, not 9 minutes).

Anyway, it looks like we should be able to push most of the I/O work to
packages.  Let=E2=80=99s see!

Thanks,
Ludo=E2=80=99.

Message sent to guix-patches@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Maxime Devos <maximedevos@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Sat, 17 Jul 2021 14:43:02 +0000
Resent-Message-ID: <handler.49421.B49421.162653297827267 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 49421
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
Received: via spool by 49421-submit <at> debbugs.gnu.org id=B49421.162653297827267
          (code B ref 49421); Sat, 17 Jul 2021 14:43:02 +0000
Received: (at 49421) by debbugs.gnu.org; 17 Jul 2021 14:42:58 +0000
Received: from localhost ([127.0.0.1]:54522 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1m4lWr-00075j-Oj
	for submit <at> debbugs.gnu.org; Sat, 17 Jul 2021 10:42:58 -0400
Received: from michel.telenet-ops.be ([195.130.137.88]:37820)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <maximedevos@HIDDEN>) id 1m4lWp-00075a-7R
 for 49421 <at> debbugs.gnu.org; Sat, 17 Jul 2021 10:42:56 -0400
Received: from ptr-bvsjgyjmffd7q9timvx.18120a2.ip6.access.telenet.be
 ([IPv6:2a02:1811:8c09:9d00:aaf1:9810:a0b8:a55d])
 by michel.telenet-ops.be with bizsmtp
 id WEir2500f0mfAB406EisBZ; Sat, 17 Jul 2021 16:42:53 +0200
Message-ID: <5af3b4c9975873800e371da77c6ba929f7bd90d7.camel@HIDDEN>
From: Maxime Devos <maximedevos@HIDDEN>
Date: Sat, 17 Jul 2021 16:42:51 +0200
In-Reply-To: <87pmviw8f6.fsf@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 <87a6myoyl5.fsf@HIDDEN>
 <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
 <87pmviw8f6.fsf@HIDDEN>
Content-Type: multipart/mixed; boundary="=-dDORbdbkc6k0U8yUg3Gy"
User-Agent: Evolution 3.34.2 
MIME-Version: 1.0
DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=telenet.be; s=r21;
 t=1626532973; bh=rQC0tulfIpTZxhAOrPIJg9LtWeLWtmAqo50yYr3A0oE=;
 h=Subject:From:To:Cc:Date:In-Reply-To:References;
 b=IE29gGNCpo0rwjMDAeA7fFpnmPzqak1YAt7bK9vuUBfDjQeC7Z+KjlXkO6Wxlg9vR
 RNcrQDCa89yLHAxh+rhebzd+LPKhB4hD+F/afEhbRTGKBNojQ34pMSIPgypB7hxqRn
 FrMk8na/a79WQXHo4qtCckL+252e7j/Rzwm45MypL3B6/2Ll9ktvTmrBWuDMPbp4yh
 9y8QK36rYRv1g6Lti5/jLgk9vNmojVcECqNJEEKVAkfu46hoo+VFx6qYyGgW9ENac7
 dd+w40AOEp2SqZjT9mc92FF9V2qr5cJ1ez9V5HvVk150oqPeWBfONX5dR0PCymszuN
 bExHwCLqKL+ig==
X-Spam-Score: -0.7 (/)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -1.7 (-)


--=-dDORbdbkc6k0U8yUg3Gy
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: 8bit

Ludovic Courtès schreef op vr 16-07-2021 om 18:05 [+0200]:
> Ouch, that’s way more than what I’m observing.  :-)  Do you have a
> manifest or package list to reproduce that?  Were those timings on a
> spinning disk?

It is on a spinning disk.  I attached the output of "guix package --export-manifest".

Greetings,
Maxime.

--=-dDORbdbkc6k0U8yUg3Gy
Content-Disposition: attachment; filename="mani"
Content-Transfer-Encoding: base64
Content-Type: text/plain; name="mani"; charset="UTF-8"

OzsgRGl0IOKAmG1hbmlmZXN04oCZIGthbiB3b3JkZW4gZG9vcmdlZ2V2ZW4gYWFuIOKAmGd1aXgg
cGFja2FnZSAtbeKAmSBvbSBkZSBpbmhvdWQKOzsgdmFuIGplIHByb2ZpZWwgdGUgcmVwcm9kdWNl
cmVuLiAgRGl0IGlzIOKAmHN5bWJvbGlzY2jigJk6IGhldCB2ZXJ3aWpzdCBlbmtlbAo7OyBuYWFy
IGRlIHBha2tldG5hbWVuLiAgT20gcHJlY2llcyBoZXR6ZWxmZGUgcHJvZmllbCB0ZSByZXByb2R1
Y2VyZW4sIG1vZXQKOzsgamUgb29rIGRlIGxpanN0IHZhbiBnZWJydWlrdGUga2FuYWxlbiBiZXdh
cmVuLCB6b2FscyB0ZXJ1Z2dlZ2V2ZW4gZG9vciDigJhndWl4IGRlc2NyaWJl4oCZLgo7OyBaaWUg
ZGUgc2VjdGllIOKAmEd1aXggcmVwcm9kdWNlcmVu4oCZIGluIGRlIGhhbmRsZWlkaW5nLgoKKHNw
ZWNpZmljYXRpb25zLT5tYW5pZmVzdAogIChsaXN0ICJqYW1pLWdub21lIgogICAgICAgICJ0ZXht
YWNzIgogICAgICAgICJnaXQ6c2VuZC1lbWFpbCIKICAgICAgICAibGFncmFuZ2UiCiAgICAgICAg
InZsYyIKICAgICAgICAiYmx1ZW1hbiIKICAgICAgICAiYml0bGJlZS1wdXJwbGUiCiAgICAgICAg
Im5ldHdvcmstbWFuYWdlci1vcGVuY29ubmVjdCIKICAgICAgICAiYXVkYWNpdHkiCiAgICAgICAg
ImV2b2x1dGlvbiIKICAgICAgICAia2l3aXgtZGVza3RvcCIKICAgICAgICAiaWNlY2F0IgogICAg
ICAgICJzaW1wbGUtc2NhbiIKICAgICAgICAiZ3VpbGUtc3R1ZGlvIgogICAgICAgICJ1bmdvb2ds
ZWQtY2hyb21pdW0iCiAgICAgICAgInBpZGdpbi1vdHIiCiAgICAgICAgImVtYWNzLWNhbGlicmVk
YiIKICAgICAgICAiY2FsaWJyZSIKICAgICAgICAiYnRyZnMtcHJvZ3MiCiAgICAgICAgInRpbGVk
IgogICAgICAgICJjaGVlc2UiCiAgICAgICAgInNlYWhvcnNlIgogICAgICAgICJ3aXJlc2hhcmsi
CiAgICAgICAgImdpdCIKICAgICAgICAicmV1c2UiCiAgICAgICAgImd6b2NoaSIKICAgICAgICAi
emJhciIKICAgICAgICAidGV4bWFrZXIiCiAgICAgICAgImdudW1lcmljIgogICAgICAgICJncGEi
CiAgICAgICAgInRleGxpdmUtYmFzZSIKICAgICAgICAiZ2ltcCIKICAgICAgICAicGluZW50cnki
CiAgICAgICAgImVtYWNzLWV4d20iCiAgICAgICAgImdlYW55IgogICAgICAgICJza3kiCiAgICAg
ICAgInRleGxpdmUiCiAgICAgICAgImdudXJvYm90cyIKICAgICAgICAiZ3JhcGh2aXoiCiAgICAg
ICAgImd1aWxlLXNkbCIKICAgICAgICAiZ3VpbGUtY2hpY2thZGVlIgogICAgICAgICJlbWFjcyIK
ICAgICAgICAiZW1hY3MtbWFnaXQtb3JnLXRvZG9zLWVsIgogICAgICAgICJlbWFjcy1tYWdpdC1h
bm5leCIKICAgICAgICAiZW1hY3MtbWFnaXQtZ2Vycml0IgogICAgICAgICJlbWFjcy1tYWdpdCIK
ICAgICAgICAidG9tZTQiCiAgICAgICAgImVtYWNzLWd1aXgiCiAgICAgICAgInppbGUtb24tZ3Vp
bGUiCiAgICAgICAgImVtYWNzLWRlYmJ1Z3MiCiAgICAgICAgImVtYWNzLWhlbG0tYmlidGV4Igog
ICAgICAgICJlbWFjcy1lYmRiIgogICAgICAgICJlbWFjcy1oZWxtLWZpcmVmb3giCiAgICAgICAg
ImVtYWNzLWhlbG0tZXd3IgogICAgICAgICJlbWFjcy1oZWxtIgogICAgICAgICJlbWFjcy1oZWxt
LXdpa2lwZWRpYSIKICAgICAgICAibWFuLXBhZ2VzIgogICAgICAgICJlbWFjcy1ub3YtZWwiCiAg
ICAgICAgImdudXBnIgogICAgICAgICJyciIKICAgICAgICAib3BlbnNzaCIKICAgICAgICAiaXJz
c2kiCiAgICAgICAgImRlZSIKICAgICAgICAiZW1hY3MtYmx1ZXRvb3RoIgogICAgICAgICJndWls
ZS1jYWlybyIKICAgICAgICAieGRnLWRidXMtcHJveHkiCiAgICAgICAgImZldGNobWFpbCIKICAg
ICAgICAibGliaW1vYmlsZWRldmljZSIKICAgICAgICAiZW1hY3MtbGVkZ2VyLW1vZGUiCiAgICAg
ICAgImdkYiIKICAgICAgICAidmFsZ3JpbmQiCiAgICAgICAgInBsb3R1dGlscyIKICAgICAgICAi
eGRnLXV0aWxzIgogICAgICAgICJhcnRhbmlzIgogICAgICAgICJnaG9zdHNjcmlwdCIKICAgICAg
ICAiZGJ1czpkb2MiCiAgICAgICAgImRidXMiCiAgICAgICAgImhhdW50IgogICAgICAgICJuZXRo
YWNrIgogICAgICAgICJnY2MtdG9vbGNoYWluIgogICAgICAgICJlbWFjcy1jb21wYW55LWNvcSIK
ICAgICAgICAiZW1hY3MtZmx5Y2hlY2siCiAgICAgICAgImVtYWNzLWZseWNoZWNrLWd1aWxlIgog
ICAgICAgICJlbWFjcy1naHViIgogICAgICAgICJlbWFjcy1idWlsZC1mYXJtIgogICAgICAgICJl
bWFjcy1lYmliIgogICAgICAgICJlbWFjcy1jb21wYW55LW1hdGgiCiAgICAgICAgImNyeXB0c2V0
dXAiCiAgICAgICAgImVtYWNzLXBhcmVkaXQiCiAgICAgICAgImVtYWNzLWFjLWdlaXNlciIKICAg
ICAgICAiZW1hY3MtZ2Vpc2VyIgogICAgICAgICJlbWFjcy1nY21oIgogICAgICAgICJlbWFjcy1q
dWxpYS1tb2RlIgogICAgICAgICJlbWFjcy15YXNuaXBwZXQiCiAgICAgICAgImVtYWNzLW94LWVw
dWIiCiAgICAgICAgImVtYWNzLWJvbmdvIgogICAgICAgICJlbWFjcy1lbGYtbW9kZSIKICAgICAg
ICAiZW1hY3MtanMyLW1vZGUiCiAgICAgICAgImVtYWNzLWNvbXBhbnktZWJkYiIKICAgICAgICAi
ZW1hY3MtYmJkYiIKICAgICAgICAiZW1hY3MtZmZhcC1yZmMtc3BhY2UiCiAgICAgICAgInozIgog
ICAgICAgICJ0ZXhsaXZlLWxhdGV4LWJhYmVsIgogICAgICAgICJ0ZXhsaXZlLWxhdGV4LWdlb21l
dHJ5IgogICAgICAgICJ0ZXhsaXZlLWxhdGV4LWh5cGVycmVmIgogICAgICAgICJ0ZXhsaXZlLWxh
dGV4LXBnZiIKICAgICAgICAidGV4bGl2ZS1sYXRleC1ncmFwaGljcyIKICAgICAgICAiZ2xpYmM6
ZGVidWciCiAgICAgICAgImdsaWJjIgogICAgICAgICJndWlsZTpkZWJ1ZyIKICAgICAgICAiZ3Vp
bGUiCiAgICAgICAgImFzcGVsbCIKICAgICAgICAiYXNwZWxsLWRpY3QtZGUiCiAgICAgICAgImFz
cGVsbC1kaWN0LW5sIgogICAgICAgICJhc3BlbGwtZGljdC1mciIKICAgICAgICAiYXNwZWxsLWRp
Y3QtZW4iCiAgICAgICAgInh4ZCIKICAgICAgICAiZ28taXBmcyIKICAgICAgICAidW56aXAiCiAg
ICAgICAgInNjcmVlbiIKICAgICAgICAibGlicmVvZmZpY2UiCiAgICAgICAgImh0b3AiCiAgICAg
ICAgInNjc2giCiAgICAgICAgImd1aWxlLW1pbmlhZGFwdG9uIgogICAgICAgICJzaWNwIgogICAg
ICAgICJtaWciCiAgICAgICAgInRleGxpdmUtZXBzZiIKICAgICAgICAiZm9udC1tYXRoamF4Igog
ICAgICAgICJndWlsZS1hYy1kLWJ1cyIKICAgICAgICAiZ2FzaCIKICAgICAgICAiemlwIgogICAg
ICAgICJqYWNhbCIKICAgICAgICAiZ2FtYml0LWMiCiAgICAgICAgInN0cmFjZSIKICAgICAgICAi
Zm9udC1kZWphdnUiCiAgICAgICAgImd1aWxlLWZpYmVycyIKICAgICAgICAicGtnLWNvbmZpZyIK
ICAgICAgICAibmN1cnNlcyIpKQo=


--=-dDORbdbkc6k0U8yUg3Gy--

Message sent to guix-patches@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 21 Jul 2021 14:49:01 +0000
Resent-Message-ID: <handler.49421.B49421.162687893320316 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 49421
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Maxime Devos <maximedevos@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
Received: via spool by 49421-submit <at> debbugs.gnu.org id=B49421.162687893320316
          (code B ref 49421); Wed, 21 Jul 2021 14:49:01 +0000
Received: (at 49421) by debbugs.gnu.org; 21 Jul 2021 14:48:53 +0000
Received: from localhost ([127.0.0.1]:38230 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1m6DWn-0005Hb-Ey
	for submit <at> debbugs.gnu.org; Wed, 21 Jul 2021 10:48:53 -0400
Received: from eggs.gnu.org ([209.51.188.92]:53256)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1m6DWl-0005HK-Rs
 for 49421 <at> debbugs.gnu.org; Wed, 21 Jul 2021 10:48:52 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:50362)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1m6DWf-000397-Nh; Wed, 21 Jul 2021 10:48:45 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45270 helo=ribbon)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1m6DWf-0002a1-GE; Wed, 21 Jul 2021 10:48:45 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 <87a6myoyl5.fsf@HIDDEN>
 <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
 <87pmviw8f6.fsf@HIDDEN>
 <5af3b4c9975873800e371da77c6ba929f7bd90d7.camel@HIDDEN>
Date: Wed, 21 Jul 2021 16:48:43 +0200
In-Reply-To: <5af3b4c9975873800e371da77c6ba929f7bd90d7.camel@HIDDEN>
 (Maxime Devos's message of "Sat, 17 Jul 2021 16:42:51 +0200")
Message-ID: <875yx3hgdw.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hello,

Maxime Devos <maximedevos@HIDDEN> skribis:

> Ludovic Court=C3=A8s schreef op vr 16-07-2021 om 18:05 [+0200]:
>> Ouch, that=E2=80=99s way more than what I=E2=80=99m observing.  :-)  Do =
you have a
>> manifest or package list to reproduce that?  Were those timings on a
>> spinning disk?
>
> It is on a spinning disk.  I attached the output of "guix package --expor=
t-manifest".

I isolated =E2=80=98fonts-dir.drv=E2=80=99 for that profile and here=E2=80=
=99s what I get with a
~5yo SSD:

--8<---------------cut here---------------start------------->8---
$ time guix build  /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.dr=
v --check
The following profile hook will be built:
   /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.drv
building fonts directory...
successfully built /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.drv
successfully built /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.drv
/gnu/store/fgjmwpgrmp9m83qxvqrs1ghwasjrjxpc-fonts-dir

real    0m24.651s
user    0m2.120s
sys     0m0.164s
$ time guix build  /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.dr=
v --check
The following profile hook will be built:
   /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.drv
building fonts directory...
successfully built /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.drv
successfully built /gnu/store/nmr6clkx4n554a0gynr6sb4a43dj6mfa-fonts-dir.drv
/gnu/store/fgjmwpgrmp9m83qxvqrs1ghwasjrjxpc-fonts-dir

real    0m12.025s
user    0m1.924s
sys     0m0.036s
--8<---------------cut here---------------end--------------->8---

That=E2=80=99s fairly significant (it baffles me that it=E2=80=99s still 20=
x less than
what you see with a spinning disk!).

There are only 327 fonts in there:

--8<---------------cut here---------------start------------->8---
$ wc -l /gnu/store/fgjmwpgrmp9m83qxvqrs1ghwasjrjxpc-fonts-dir/share/fonts/t=
ruetype/fonts.*
  328 /gnu/store/fgjmwpgrmp9m83qxvqrs1ghwasjrjxpc-fonts-dir/share/fonts/tru=
etype/fonts.dir
  328 /gnu/store/fgjmwpgrmp9m83qxvqrs1ghwasjrjxpc-fonts-dir/share/fonts/tru=
etype/fonts.scale
  656 totalo
--8<---------------cut here---------------end--------------->8---

Thanks,
Ludo=E2=80=99.

Message sent to guix-patches@HIDDEN:

X-Loop: help-debbugs@HIDDEN
Subject: [bug#49421] [PATCH] profiles: Optimise 'fonts-dir-file'.
Resent-From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: guix-patches@HIDDEN
Resent-Date: Wed, 21 Jul 2021 14:59:02 +0000
Resent-Message-ID: <handler.49421.B49421.162687953121526 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
X-GNU-PR-Message: followup 49421
X-GNU-PR-Package: guix-patches
X-GNU-PR-Keywords: patch
To: Maxime Devos <maximedevos@HIDDEN>
Cc: 49421 <at> debbugs.gnu.org
Received: via spool by 49421-submit <at> debbugs.gnu.org id=B49421.162687953121526
          (code B ref 49421); Wed, 21 Jul 2021 14:59:02 +0000
Received: (at 49421) by debbugs.gnu.org; 21 Jul 2021 14:58:51 +0000
Received: from localhost ([127.0.0.1]:38270 helo=debbugs.gnu.org)
	by debbugs.gnu.org with esmtp (Exim 4.84_2)
	(envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>)
	id 1m6DgR-0005b7-AY
	for submit <at> debbugs.gnu.org; Wed, 21 Jul 2021 10:58:51 -0400
Received: from eggs.gnu.org ([209.51.188.92]:56096)
 by debbugs.gnu.org with esmtp (Exim 4.84_2)
 (envelope-from <ludo@HIDDEN>) id 1m6DgP-0005av-LU
 for 49421 <at> debbugs.gnu.org; Wed, 21 Jul 2021 10:58:50 -0400
Received: from fencepost.gnu.org ([2001:470:142:3::e]:50760)
 by eggs.gnu.org with esmtp (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1m6DgK-0001yE-Cr; Wed, 21 Jul 2021 10:58:44 -0400
Received: from [2a01:e0a:1d:7270:af76:b9b:ca24:c465] (port=45272 helo=ribbon)
 by fencepost.gnu.org with esmtpsa
 (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1)
 (envelope-from <ludo@HIDDEN>)
 id 1m6DgJ-0006Hk-Uf; Wed, 21 Jul 2021 10:58:44 -0400
From: Ludovic =?UTF-8?Q?Court=C3=A8s?= <ludo@HIDDEN>
References: <e1af6b02effdeaf4a05f1ddb602003d57961ec62.camel@HIDDEN>
 <87a6myoyl5.fsf@HIDDEN>
 <5ec75ed429133cbc73dc7916959302033fe70a13.camel@HIDDEN>
 <369206a276e225478ed1bbce0b883255dccef369.camel@HIDDEN>
Date: Wed, 21 Jul 2021 16:58:42 +0200
In-Reply-To: <369206a276e225478ed1bbce0b883255dccef369.camel@HIDDEN>
 (Maxime Devos's message of "Thu, 15 Jul 2021 20:48:09 +0200")
Message-ID: <87y29zg1ct.fsf_-_@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.2 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable
X-Spam-Score: -2.3 (--)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -3.3 (---)

Hi,

Maxime Devos <maximedevos@HIDDEN> skribis:

> I think it's better to optimise 'union-build' and the store deduplication
> code though (maybe subdirectories can be process concurrently, to maximis=
e I/O
> utilisation?). That should improve performance outside the 'fonts-dir-fil=
e' hook
> as well.

I=E2=80=99m afraid there=E2=80=99s little room for improvement on =E2=80=98=
union-build=E2=80=99 (Mark
put quite some effort on it back in commit
12129998689648923b58c426362a1bc875da75f9), though with a fresh eye
perhaps you=E2=80=99ll find optimization opportunities.

As for deduplication, there are two cases: deduplication for
substitutes, and deduplication upon build completion (what we see
here).

In the former case, deduplication is part of the substitute unpacking
pipeline (see
<https://guix.gnu.org/en/blog/2021/getting-bytes-to-disk-more-quickly/>)
and I think syscalls are reduced to the minimum necessary (that=E2=80=99s (=
guix
store deduplication)).

The strace snippet you sent suggests that, in the latter case, the
daemon=E2=80=99s code stats the same file several times, so maybe there=E2=
=80=99s room
for improvement here.

Thanks,
Ludo=E2=80=99.

Last modified: Wed, 21 Jul 2021 15:00:02 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.