GNU bug report logs - #40628
Optimize admin/nt dependency computation

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 40628 in the body.
You can then email your comments to 40628 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Noam Postavsky <npostavs <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Optimize admin/nt dependency computation
Date: Tue, 14 Apr 2020 12:41:43 -0400

[Message part 1 (text/plain, inline)]

Severity: wishlist
Tags: patch
X-Debbugs-CC: Phillip Lord <phillip.lord <at> russet.org.uk>

While checking the patch I suggested in
https://debbugs.gnu.org/40003#11, I noticed the dependency extraction
seemed pretty slow.  With the patch attached below it's much faster
(1m20.437s to 0m3.695s according to 'time ./build-dep-zips.py -l').

[0001-Optimize-admin-nt-dependency-computation.patch (text/plain, attachment)]

Message #8 received at 40628 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 40628 <at> debbugs.gnu.org, phillip lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#40628: Optimize admin/nt dependency computation
Date: Sat, 08 Aug 2020 14:57:00 +0200

Noam Postavsky <npostavs <at> gmail.com> writes:

> While checking the patch I suggested in
> https://debbugs.gnu.org/40003#11, I noticed the dependency extraction
> seemed pretty slow.  With the patch attached below it's much faster
> (1m20.437s to 0m3.695s according to 'time ./build-dep-zips.py -l').
>
>>From 5312f51e315cc147ca901d499aa3c3cc64aa3340 Mon Sep 17 00:00:00 2001
> From: Noam Postavsky <npostavs <at> gmail.com>
> Date: Tue, 14 Apr 2020 12:27:34 -0400
> Subject: [PATCH] Optimize admin/nt dependency computation
>
> admin/nt/dist-build/build-dep-zips.py (immediate_deps)
> (extract_deps): Gather package dependency info in batches, rather than
> one at a time.  This reduces the number of invocations of 'pacman -Si
> ...' to the depth of the dependency tree, rather than the number of
> dependent packages.
> (top-level): Don't call 'extract_deps' when given the '-l' option.

This was four months ago, but the patch was apparently not applied?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #11 received at 40628 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 40628 <at> debbugs.gnu.org, phillip lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#40628: Optimize admin/nt dependency computation
Date: Tue, 18 Aug 2020 15:58:32 +0200

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Noam Postavsky <npostavs <at> gmail.com> writes:
>
>> While checking the patch I suggested in
>> https://debbugs.gnu.org/40003#11, I noticed the dependency extraction
>> seemed pretty slow.  With the patch attached below it's much faster
>> (1m20.437s to 0m3.695s according to 'time ./build-dep-zips.py -l').

[...]

> This was four months ago, but the patch was apparently not applied?

I don't have any Windows machines -- has anybody given this patch a test
to see whether it introduces any problems?

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #14 received at 40628 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Noam Postavsky <npostavs <at> gmail.com>
Cc: 40628 <at> debbugs.gnu.org, phillip lord <phillip.lord <at> russet.org.uk>
Subject: Re: bug#40628: Optimize admin/nt dependency computation
Date: Fri, 12 Nov 2021 10:23:17 +0100

Lars Ingebrigtsen <larsi <at> gnus.org> writes:

> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
>
>> Noam Postavsky <npostavs <at> gmail.com> writes:
>>
>>> While checking the patch I suggested in
>>> https://debbugs.gnu.org/40003#11, I noticed the dependency extraction
>>> seemed pretty slow.  With the patch attached below it's much faster
>>> (1m20.437s to 0m3.695s according to 'time ./build-dep-zips.py -l').
>
> [...]
>
>> This was four months ago, but the patch was apparently not applied?
>
> I don't have any Windows machines -- has anybody given this patch a test
> to see whether it introduces any problems?

I tested it myself, and it brought the generation time down from ~3
minutes to about 20 seconds, so I've applied the patch.  I don't see any
difference in the results, but then again, if failed with this both
before and after the change:

subprocess.CalledProcessError: Command 'wget -a ../download.log -O ../emacs-src-cache/mingw-w64-libtiff-4.3.0-6.src.tar.gz https://sourceforge.net/projects/msys2/files/REPOS/MINGW/Sources/mingw-w64-libtiff-4.3.0-6.src.tar.gz/download' returned non-zero exit status 8.


-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Message #19 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Andy Moreton <andrewjmoreton <at> gmail.com>
To: bug-gnu-emacs <at> gnu.org
Subject: Re: bug#40628: Optimize admin/nt dependency computation
Date: Fri, 12 Nov 2021 09:55:02 +0000

On Fri 12 Nov 2021, Lars Ingebrigtsen wrote:

> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
>
>> Lars Ingebrigtsen <larsi <at> gnus.org> writes:
>>
>>> Noam Postavsky <npostavs <at> gmail.com> writes:
>>>
>>>> While checking the patch I suggested in
>>>> https://debbugs.gnu.org/40003#11, I noticed the dependency extraction
>>>> seemed pretty slow.  With the patch attached below it's much faster
>>>> (1m20.437s to 0m3.695s according to 'time ./build-dep-zips.py -l').
>>
>> [...]
>>
>>> This was four months ago, but the patch was apparently not applied?
>>
>> I don't have any Windows machines -- has anybody given this patch a test
>> to see whether it introduces any problems?
>
> I tested it myself, and it brought the generation time down from ~3
> minutes to about 20 seconds, so I've applied the patch.  I don't see any
> difference in the results, but then again, if failed with this both
> before and after the change:
>
> subprocess.CalledProcessError: Command 'wget -a ../download.log -O
> ../emacs-src-cache/mingw-w64-libtiff-4.3.0-6.src.tar.gz
> https://sourceforge.net/projects/msys2/files/REPOS/MINGW/Sources/mingw-w64-libtiff-4.3.0-6.src.tar.gz/download'
> returned non-zero exit status 8.

The MSYS2 project has moved some stuff away from sourceforge to its own
site. The sources are now at:

  https://repo.msys2.org/mingw/sources/

For the libtiff download above, that results in:

  https://repo.msys2.org/mingw/sources/mingw-w64-libtiff-4.3.0-6.src.tar.gz

The script will need adjusting to change the URL, and drop the
"/download" suffix.

    AndyM 

Message #22 received at 40628 <at> debbugs.gnu.org (full text, mbox):

From: Eli Zaretskii <eliz <at> gnu.org>
To: Lars Ingebrigtsen <larsi <at> gnus.org>
Cc: 40628 <at> debbugs.gnu.org, npostavs <at> gmail.com, phillip.lord <at> russet.org.uk
Subject: Re: bug#40628: Optimize admin/nt dependency computation
Date: Fri, 12 Nov 2021 14:03:16 +0200

> From: Lars Ingebrigtsen <larsi <at> gnus.org>
> Date: Fri, 12 Nov 2021 10:23:17 +0100
> Cc: 40628 <at> debbugs.gnu.org, phillip lord <phillip.lord <at> russet.org.uk>
> 
> I tested it myself, and it brought the generation time down from ~3
> minutes to about 20 seconds, so I've applied the patch.  I don't see any
> difference in the results, but then again, if failed with this both
> before and after the change:
> 
> subprocess.CalledProcessError: Command 'wget -a ../download.log -O ../emacs-src-cache/mingw-w64-libtiff-4.3.0-6.src.tar.gz https://sourceforge.net/projects/msys2/files/REPOS/MINGW/Sources/mingw-w64-libtiff-4.3.0-6.src.tar.gz/download' returned non-zero exit status 8.

According to my records, 8 means "Server issued an error response".
Whatever that means.

Message #25 received at 40628 <at> debbugs.gnu.org (full text, mbox):

From: Lars Ingebrigtsen <larsi <at> gnus.org>
To: Andy Moreton <andrewjmoreton <at> gmail.com>
Cc: 40628 <at> debbugs.gnu.org
Subject: Re: bug#40628: Optimize admin/nt dependency computation
Date: Sun, 14 Nov 2021 02:06:56 +0100

Andy Moreton <andrewjmoreton <at> gmail.com> writes:

> The MSYS2 project has moved some stuff away from sourceforge to its own
> site. The sources are now at:
>
>   https://repo.msys2.org/mingw/sources/
>
> For the libtiff download above, that results in:
>
>   https://repo.msys2.org/mingw/sources/mingw-w64-libtiff-4.3.0-6.src.tar.gz
>
> The script will need adjusting to change the URL, and drop the
> "/download" suffix.

Thanks; I've now updated the script with the new URL (and dropped the
/download).

-- 
(domestic pets only, the antidote for overdose, milk.)
   bloggy blog: http://lars.ingebrigtsen.no

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.