GNU bug report logs - #34016
Git checkouts managed by (guix git) grow indefinitely?

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 34016 in the body.
You can then email your comments to 34016 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludovic.courtes <at> inria.fr>
To: Bug Guix <bug-guix <at> gnu.org>
Cc: Erik Edrosa <erik.edrosa <at> gmail.com>
Subject: Git checkouts managed by (guix git) grow indefinitely?
Date: Tue, 08 Jan 2019 17:56:55 +0100

Hello,

On berlin I noticed that Cuirass would run out of file descriptors
almost immediately once started.  I managed to get the list of open
files:

--8<---------------cut here---------------start------------->8---
ludo <at> berlin ~$ sudo ls -l /proc/$(sudo herd status cuirass|grep value|sed -es'/^.* \([0-9]\+\)\.$/\1/g')/fd
[…]
alwq/.git/objects/pack/pack-dde01a576697a783fd12eb73228921153ba18b4b.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 978 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-d65f5c22414e4864a00ec7bee4cff5ff0b4a2c31.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 979 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-d78f9f47a68d4254a81d57d6183fe9c9be41752d.pack
l-wx------ 1 cuirass cuirass 64 Jan  8 17:46 98 -> 'pipe:[70917334]'
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 980 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-58f83b77dbe392bba277f10a1bdd7ed469946feb.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 981 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-d28d075e1dd259683747b5723a232fa4d60cd1a9.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 982 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-cffbdad6def559b60bd928173916f3edbb1a4a8f.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 983 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-ac42d4dd95b02bb54d03687c984c67bacf0e1feb.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 984 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-e5e26a7aeb5a33889f6ac8cff8bef182b5b8a0b2.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 985 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-ac66439d6b07a0e4cf2e4ccef3be21e8a1c96d55.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 986 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-f73b60b056634b58f44fb65f20416e22a6c936e8.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 987 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-f8f3313f5ec872e43cb1e12a8b6509e7f0a13512.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 988 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-55e720e2a4ec09cc996599c5089be871eba8f829.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 989 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-d2441e647df889422906a30d06d6d195e522cbd1.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 99 -> 'pipe:[70908233]'
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 990 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-61fed4082491ff66716c92fe49fe499cd6a5fa78.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 991 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-0bcefb4f7953471a0bc944f0f0c3bbe9143df22b.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 992 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-6a5fbea2046bdc8c94926e9d3d516e1b5351b645.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 993 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-add66d8bdd96e4cc67e78a23c428f7b4e46974df.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 994 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-740728c75faa0e9179ad977d4e4b2eb6bfa51345.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 995 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-c78417a98abdc6a7396fb3f6e4d5633de20c819c.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 996 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-df4cae77b58d600538f2dc51b7b31b61a92cab23.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 997 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-83559c192d41400596d42587bb50d409c121969d.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 998 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-1f46268a2ba6bd3b97d9d7cea4e18233212cc75e.pack
lr-x------ 1 cuirass cuirass 64 Jan  8 17:46 999 -> /var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq/.git/objects/pack/pack-55a94a468fa892b0128656fc5aa953c667161fcd.pack
--8<---------------cut here---------------end--------------->8---

So it’s Guile-Git, called from ‘update-cached-checkout’¹, that would open
all these pack files, quickly running out of file descriptors.  I wonder
if we’re doing something wrong here or if we hit a limitation of
libgit2.  Ideas?

In the meantime, the obvious workaround was to “rm -rf
/var/cache/cuirass/pjmkglp4t7znuugeurpurzikxq3tnlaywmisyr27shj7apsnalwq”.

Thanks,
Ludo’.

¹ https://git.savannah.gnu.org/cgit/guix.git/tree/guix/git.scm#n122

Message #10 received at 34016 <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: 34016 <at> debbugs.gnu.org
Cc: Erik Edrosa <erik.edrosa <at> gmail.com>
Subject: Re: bug#34016: Git checkouts managed by (guix git) grow indefinitely?
Date: Fri, 09 Aug 2019 11:36:08 +0200

Hello,

Ludovic Courtès <ludovic.courtes <at> inria.fr> skribis:

> So it’s Guile-Git, called from ‘update-cached-checkout’¹, that would open
> all these pack files, quickly running out of file descriptors.  I wonder
> if we’re doing something wrong here or if we hit a limitation of
> libgit2.  Ideas?

Forwarded here: <https://gitlab.com/guile-git/guile-git/issues/16>.

Ludo’.

Message #13 received at 34016 <at> debbugs.gnu.org (full text, mbox):

From: Brendan Tildesley <btild <at> mailbox.org>
To: "34016 <at> debbugs.gnu.org" <34016 <at> debbugs.gnu.org>
Cc: "ludo <at> gnu.org" <ludo <at> gnu.org>
Subject: Git checkouts managed by (guix git) grow indefinitely
Date: Fri, 23 Apr 2021 13:11:31 +0200 (CEST)

[Message part 1 (text/plain, inline)]

Since this bug report 1.1.0 was released. The Changelog includes this note:
"Repositories with a large number of packfiles no longer exhaust the
number of file descriptors."

https://github.com/libgit2/libgit2/pull/5396

May or may not be related to this.

[Message part 2 (text/html, inline)]

Message #18 received at 34016-done <at> debbugs.gnu.org (full text, mbox):

From: Ludovic Courtès <ludo <at> gnu.org>
To: Brendan Tildesley <btild <at> mailbox.org>
Cc: "34016 <at> debbugs.gnu.org" <34016-done <at> debbugs.gnu.org>
Subject: Re: Git checkouts managed by (guix git) grow indefinitely
Date: Fri, 23 Apr 2021 16:59:59 +0200

Hi Brendan,

Brendan Tildesley <btild <at> mailbox.org> skribis:

> Since this bug report 1.1.0 was released. The Changelog includes this note:
> "Repositories with a large number of packfiles no longer exhaust the
> number of file descriptors."
>
> https://github.com/libgit2/libgit2/pull/5396
>
> May or may not be related to this.

That may be the case because I’ve not experienced it in more than a year.
Closing!

Thanks for the heads-up,
Ludo’.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.