X-Loop: help-debbugs@HIDDEN Subject: bug#62069: Support xz parallel compression in dist-xz Resent-From: Sam James <sam@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-automake@HIDDEN Resent-Date: Thu, 09 Mar 2023 08:50:02 +0000 Resent-Message-ID: <handler.62069.B.167835179221949 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: report 62069 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: 62069 <at> debbugs.gnu.org X-Debbugs-Original-To: bug-automake@HIDDEN Received: via spool by submit <at> debbugs.gnu.org id=B.167835179221949 (code B ref -1); Thu, 09 Mar 2023 08:50:02 +0000 Received: (at submit) by debbugs.gnu.org; 9 Mar 2023 08:49:52 +0000 Received: from localhost ([127.0.0.1]:50862 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1paByC-0005hx-4x for submit <at> debbugs.gnu.org; Thu, 09 Mar 2023 03:49:52 -0500 Received: from lists.gnu.org ([209.51.188.17]:48732) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <sam@HIDDEN>) id 1paBy9-0005ho-WD for submit <at> debbugs.gnu.org; Thu, 09 Mar 2023 03:49:50 -0500 Received: from eggs.gnu.org ([2001:470:142:3::10]) by lists.gnu.org with esmtps (TLS1.2:ECDHE_RSA_AES_256_GCM_SHA384:256) (Exim 4.90_1) (envelope-from <sam@HIDDEN>) id 1paBy8-0007cw-1A for bug-automake@HIDDEN; Thu, 09 Mar 2023 03:49:49 -0500 Received: from woodpecker.gentoo.org ([140.211.166.183] helo=smtp.gentoo.org) by eggs.gnu.org with esmtps (TLS1.2:ECDHE_RSA_CHACHA20_POLY1305:256) (Exim 4.90_1) (envelope-from <sam@HIDDEN>) id 1paBy5-0001Og-Er for bug-automake@HIDDEN; Thu, 09 Mar 2023 03:49:47 -0500 User-agent: mu4e 1.8.14; emacs 29.0.60 From: Sam James <sam@HIDDEN> Date: Thu, 09 Mar 2023 08:47:23 +0000 Message-ID: <87ttyumhr1.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; boundary="=-=-="; micalg=pgp-sha512; protocol="application/pgp-signature" Received-SPF: pass client-ip=140.211.166.183; envelope-from=sam@HIDDEN; helo=smtp.gentoo.org X-Spam_score_int: -41 X-Spam_score: -4.2 X-Spam_bar: ---- X-Spam_report: (-4.2 / 5.0 requ) BAYES_00=-1.9, RCVD_IN_DNSWL_MED=-2.3, RCVD_IN_MSPIKE_H3=-0.01, RCVD_IN_MSPIKE_WL=-0.01, SPF_HELO_PASS=-0.001, SPF_PASS=-0.001 autolearn=ham autolearn_force=no X-Spam_action: no action X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) --=-=-= Content-Type: text/plain xz supports parallel compression which obviously speeds up the time taken to run the 'dist-xz' rule, but it also speeds up *decompression* too, as the parallel compressor creates different output which can be decompressed in parallel. There's two options, I think, for automake: 1. Pass -T+1 to xz if xz is >= 5.4.0 (a runtime check would be needed, but it's cheap to do) as this uses 1 thread, but uses the parallel compressor to do it, so allows decompression in parallel by users; 2. Pass -T0 (or -Tn, n >= 1) to allow autodetection of the number of available CPU cores (or n if specified) which also then naturally uses the threaded compressor. --=-=-= Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iOUEARYKAI0WIQQlpruI3Zt2TGtVQcJzhAn1IN+RkAUCZAmdol8UgAAAAAAuAChp c3N1ZXItZnByQG5vdGF0aW9ucy5vcGVucGdwLmZpZnRoaG9yc2VtYW4ubmV0MjVB NkJCODhERDlCNzY0QzZCNTU0MUMyNzM4NDA5RjUyMERGOTE5MA8cc2FtQGdlbnRv by5vcmcACgkQc4QJ9SDfkZCbQgEAqAkclb+KblJO4HGJKJLn+LY9CjhDmVETzbvY PhdjK4gA/3gNoFZ+KfPZrP1tXIjHM0RHOKi9ctCWhG6fj7tiVlwA =oQGl -----END PGP SIGNATURE----- --=-=-=--
Content-Disposition: inline Content-Transfer-Encoding: quoted-printable MIME-Version: 1.0 X-Mailer: MIME-tools 5.505 (Entity 5.505) Content-Type: text/plain; charset=utf-8 X-Loop: help-debbugs@HIDDEN From: help-debbugs@HIDDEN (GNU bug Tracking System) To: Sam James <sam@HIDDEN> Subject: bug#62069: Acknowledgement (Support xz parallel compression in dist-xz) Message-ID: <handler.62069.B.167835179221949.ack <at> debbugs.gnu.org> References: <87ttyumhr1.fsf@HIDDEN> X-Gnu-PR-Message: ack 62069 X-Gnu-PR-Package: automake Reply-To: 62069 <at> debbugs.gnu.org Date: Thu, 09 Mar 2023 08:50:02 +0000 Thank you for filing a new bug report with debbugs.gnu.org. This is an automatically generated reply to let you know your message has been received. Your message is being forwarded to the package maintainers and other interested parties for their attention; they will reply in due course. Your message has been sent to the package maintainer(s): bug-automake@HIDDEN If you wish to submit further information on this problem, please send it to 62069 <at> debbugs.gnu.org. Please do not send mail to help-debbugs@HIDDEN unless you wish to report a problem with the Bug-tracking system. --=20 62069: https://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D62069 GNU Bug Tracking System Contact help-debbugs@HIDDEN with problems
X-Loop: help-debbugs@HIDDEN Subject: bug#62069: Support xz parallel compression in dist-xz Resent-From: Mike Frysinger <vapier@HIDDEN> Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> Resent-CC: bug-automake@HIDDEN Resent-Date: Sat, 13 Jan 2024 06:06:01 +0000 Resent-Message-ID: <handler.62069.B62069.170512595927101 <at> debbugs.gnu.org> Resent-Sender: help-debbugs@HIDDEN X-GNU-PR-Message: followup 62069 X-GNU-PR-Package: automake X-GNU-PR-Keywords: To: Sam James <sam@HIDDEN> Cc: 62069 <at> debbugs.gnu.org Received: via spool by 62069-submit <at> debbugs.gnu.org id=B62069.170512595927101 (code B ref 62069); Sat, 13 Jan 2024 06:06:01 +0000 Received: (at 62069) by debbugs.gnu.org; 13 Jan 2024 06:05:59 +0000 Received: from localhost ([127.0.0.1]:38163 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOX9a-000733-LV for submit <at> debbugs.gnu.org; Sat, 13 Jan 2024 01:05:58 -0500 Received: from woodpecker.gentoo.org ([140.211.166.183]:55364 helo=smtp.gentoo.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <vapier@HIDDEN>) id 1rOX9X-00072Z-AV for 62069 <at> debbugs.gnu.org; Sat, 13 Jan 2024 01:05:56 -0500 Received: by smtp.gentoo.org (Postfix, from userid 559) id 69561343184; Sat, 13 Jan 2024 06:05:50 +0000 (UTC) Date: Sat, 13 Jan 2024 01:05:49 -0500 From: Mike Frysinger <vapier@HIDDEN> Message-ID: <ZaIoPUs0VD6g8DB3@vapier> References: <87ttyumhr1.fsf@HIDDEN> MIME-Version: 1.0 Content-Type: multipart/signed; micalg=pgp-sha256; protocol="application/pgp-signature"; boundary="4S1hJ4y1GbueWUoA" Content-Disposition: inline In-Reply-To: <87ttyumhr1.fsf@HIDDEN> X-Spam-Score: -2.3 (--) X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) --4S1hJ4y1GbueWUoA Content-Type: text/plain; charset=utf-8 Content-Disposition: inline Content-Transfer-Encoding: quoted-printable On 09 Mar 2023 08:47, Sam James wrote: > xz supports parallel compression which obviously speeds up > the time taken to run the 'dist-xz' rule, but it also > speeds up *decompression* too, as the parallel compressor > creates different output which can be decompressed in parallel. >=20 > There's two options, I think, for automake: > 1. Pass -T+1 to xz if xz is >=3D 5.4.0 (a runtime check > would be needed, but it's cheap to do) as this uses > 1 thread, but uses the parallel compressor to do it, > so allows decompression in parallel by users; >=20 > 2. Pass -T0 (or -Tn, n >=3D 1) to allow autodetection > of the number of available CPU cores (or n if specified) > which also then naturally uses the threaded compressor. it should be a configure check when dist-xz is enabled in the call to AM_INIT_AUTOMAKE, and we should default to `-T0`. this would be the default XZ_OPT value if it hasn't been set/overridden (where we use -e now). -mike --4S1hJ4y1GbueWUoA Content-Type: application/pgp-signature; name="signature.asc" -----BEGIN PGP SIGNATURE----- iQIzBAABCAAdFiEEuQK1JxMl+JKsJRrUQWM7n+g39YEFAmWiKD0ACgkQQWM7n+g3 9YFsyg/8C3rJsG3N5m/lTkTPXKA20xAaTkBCFKAFpOAZXzz92jZfjxJHsHrC5UCn 5TtOgQuv7iEUIo1k6HfirlWUicbt9AkpJnQsqdmu0X3qzgXapM/ns7rbObkohNin hAtDotj8prEshOW86y95B9mPhbTI/61wJtgJ2cVDouR0mTAako0WqibAYnLZj6rV 2mmu4gkckmYB4fKl3ZjI58pCOQ2q6MulQ76zBfdjgNzt+EeVlnV0bH5fsJUEQA0c 38P0yLVOcf35G/4KeMhONj4rqu7K+n7ehbnwszfVxCQeNeel12fqIag3+BK7D3DL C735b+A21zo4rYp4BNIm1G8mz8np1MxnK3MZ2QZhepMlt7EIDH92NmU/9006Wl2U 6/zxfPs26jDH4dtAWDKS73pT4N6/VlIvwz48nEpElJ6n8R0bkX3CkwnEo0mmWDrA JRmSg/HSxb9z2Mzs5vTJifofEkGJH2YP29g8Gze4tGcne1tbNGMl310AADrsSaQz FkNDAa1xz07ua+my+4gEJeGXIiRPgWd+ooJIWqM6kelYHq3nzFf7qLItqztpxI8g cmgZ0kllJQPaU2UQ77+K5YNRXR8HVbmEkNC23ET/OlI5GBwVhJ6SG1Kxl5+6pWQN 0+nTyonXMAvgs8NOtkKSIFLUM25BQlj0E/oAKY/6sKF9uPzKq0o= =trVG -----END PGP SIGNATURE----- --4S1hJ4y1GbueWUoA--
Received: (at control) by debbugs.gnu.org; 13 Jan 2024 06:05:55 +0000 From debbugs-submit-bounces <at> debbugs.gnu.org Sat Jan 13 01:05:55 2024 Received: from localhost ([127.0.0.1]:38160 helo=debbugs.gnu.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <debbugs-submit-bounces <at> debbugs.gnu.org>) id 1rOX9W-00072l-IM for submit <at> debbugs.gnu.org; Sat, 13 Jan 2024 01:05:55 -0500 Received: from woodpecker.gentoo.org ([2001:470:ea4a:1:5054:ff:fec7:86e4]:47259 helo=smtp.gentoo.org) by debbugs.gnu.org with esmtp (Exim 4.84_2) (envelope-from <vapier@HIDDEN>) id 1rOX9Q-00072T-ME for control <at> debbugs.gnu.org; Sat, 13 Jan 2024 01:05:53 -0500 Received: by smtp.gentoo.org (Postfix, from userid 559) id 2B0F8343183; Sat, 13 Jan 2024 06:05:44 +0000 (UTC) From: Mike Frysinger <vapier@HIDDEN> To: control <at> debbugs.gnu.org Subject: Control message User-Agent: GNU debbugs/0 Message-Id: <20240113060544.2B0F8343183@HIDDEN> Date: Sat, 13 Jan 2024 06:05:44 +0000 (UTC) X-Spam-Score: -2.3 (--) X-Debbugs-Envelope-To: control X-BeenThere: debbugs-submit <at> debbugs.gnu.org X-Mailman-Version: 2.1.18 Precedence: list List-Id: <debbugs-submit.debbugs.gnu.org> List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe> List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/> List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org> List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help> List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe> Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org> X-Spam-Score: -3.3 (---) severity 62069 wishlist thankyou
GNU bug tracking system
Copyright (C) 1999 Darren O. Benham,
1997 nCipher Corporation Ltd,
1994-97 Ian Jackson.