GNU bug report logs - #74847
change default tar format from v7 to ustar (was: Re: reproducible tar archives)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 74847 in the body.
You can then email your comments to 74847 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Simon Josefsson <simon <at> josefsson.org>
To: bug-automake <at> gnu.org
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, Bruno Haible <bruno <at> clisp.org>
Subject: change default tar format from v7 to ustar (was: Re: reproducible
 tar archives)
Date: Fri, 13 Dec 2024 12:29:27 +0100

[Message part 1 (text/plain, inline)]

Hi automake folks,

What do you think about changing Automake's default tar format from v7
to ustar?

Are you aware of any system 'tar' that does doesn't cope with ustar?

What do you think about defaulting to the --format=posix PAX format?  It
may be just as safe as ustar, and only if a package uses some feature
(like >255 long filenames) that require PAX then it would be enabled,
otherwise it will remain ustar-compatible.

Discussion on the gnulib list suggests that at least ustar ought to be
safe to use these days.

/Simon

Bruno Haible via Gnulib discussion list <bug-gnulib <at> gnu.org> writes:

> Simon Josefsson wrote:
>> Possibly automake can move
>> from v7 to ustar.  Adding this to gnulib now allows us to gain
>> experience with it.
>
> I have already experience with it: Since 2018, all tarballs of GNU gettext
> were made with --format=ustar. I have tested them on all possible platforms,
> from HP-UX 11 to IRIX 6.5, from AIX 7 to NetBSD, and not encountered a
> problem — nor received a complaint or bug report because of it.
>
> So, from my point of view, we can ask the Automake team to adopt
> --format=ustar now.

[signature.asc (application/pgp-signature, inline)]

Message #8 received at 74847 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: simon <at> josefsson.org
Cc: eggert <at> cs.ucla.edu, bruno <at> clisp.org, 74847 <at> debbugs.gnu.org
Subject: Re: bug#74847: change default tar format from v7 to ustar (was: Re:
 reproducible tar archives)
Date: Fri, 13 Dec 2024 16:45:41 -0700

    >> Possibly automake can move from v7 to ustar.

Possibly. What's the advantage?  --thanks, karl.

Message #11 received at 74847 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: simon <at> josefsson.org
Cc: eggert <at> cs.ucla.edu, bruno <at> clisp.org, 74847 <at> debbugs.gnu.org
Subject: Re: bug#74847: change default tar format from v7 to ustar (was: Re:
 reproducible tar archives)
Date: Mon, 24 Feb 2025 15:41:54 -0700

    Back on this suggestion:
    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=74847

    What do you think about changing Automake's default tar format from v7
    to ustar?

Given Bruno's experience, ok, let's try. I won't be surprised if
problems come out of the woodwork, but maybe it will be fine.

I think the change below is all that's needed to implement this?
I pushed it. Let me know if something more comes to mind.

    What do you think about defaulting to the --format=posix PAX format?

Given the draconian warnings in the automake manual (not written by me),
I suspect it is still premature. In particular, I doubt the situation
has changed on Solaris:

    > @option{tar-pax} selects the new pax interchange format defined by POSIX
    > 1003.1-2001.  It does not limit the length of file names.
    > ...
    > As of 2018, this format [tar-pax] is supported by the native @code{tar}
    > command only on GNU, FreeBSD, and OpenBSD systems; it is not
    > supported by the native @code{tar} command on NetBSD, AIX, HP-UX, or
    > Solaris. 

which is followed by:

    > There are moves to change the pax format in an upward-compatible
    > way, so this option may refer to a more recent version in the future.

Did any change to pax formats ever come to pass? --thanks, karl.

-----------------------------------------------------------------------------
dist: change tar format default to ustar (!).

Suggested in https://bugs.gnu.org74847.

* m4/init.m4 (tar-ustar): make ustar the default instead of tar-v7.
* doc/automake.texi (tar-formats): document this.
* NEWS: mention this.
* t/tar-override.sh: force tar-v7 for purposes of this test.
diff --git a/NEWS b/NEWS
index da6b5e493..796a9030f 100644
--- a/NEWS
+++ b/NEWS
@@ -6,13 +6,17 @@ New in 1.x:

 * New features added

+  - Default tar format is now ustar, to support longer filenames;
+    the tar-v7 and other explicit options to force a particular tar
+    format are unchanged. (bug#74847)
+
+  - New option dist-bzip3 for bzip3 compression of distributions. (bug#73795)
+
   - New option --stderr-prefix for tap-driver.sh, to prefix each line of
     stderr from a test script with a given string. (bug#72536)

   - Support for Algol 68 added, based on the GNU Algol 68 compiler. (bug#75807)

-  - New option dist-bzip3 for bzip3 compression of distributions. (bug#73795)
-
 * Bugs fixed

   - Do not make Perl warnings fatal, per Perl's recommendation.
diff --git a/doc/automake.texi b/doc/automake.texi
index 6dcd94826..51190ea50 100644
--- a/doc/automake.texi
+++ b/doc/automake.texi
@@ -11373,7 +11373,7 @@ These options must be passed as arguments to @code{AM_INIT_AUTOMAKE}
 Automake will complain if it sees such options in an
 @code{AUTOMAKE_OPTIONS} variable.

-@option{tar-v7} selects the old V7 tar format.  This is the historical
+@option{tar-v7} selects the old V7 tar format.  This was the historical
 default.  This antiquated format is understood by all tar
 implementations and supports file names with up to 99 characters.  When
 given longer file names some tar implementations will diagnose the
@@ -11383,7 +11383,8 @@ directories.  When using this format, consider using the
 @option{filename-length-max=99} option to catch file names too long.

 @option{tar-ustar} selects the ustar format defined by POSIX
-1003.1-1988.  This format is old enough to be portable:
+1003.1-1988.  This format became the default in Automake version 1.18
+(released in 2025).
 As of 2018, it is supported by the native @code{tar} command on
 GNU, FreeBSD, NetBSD, OpenBSD, AIX, HP-UX, and Solaris, at least.
 It fully supports empty directories.  It can store file names with up
diff --git a/m4/init.m4 b/m4/init.m4
index 5da8f9e13..ed10dabcf 100644
--- a/m4/init.m4
+++ b/m4/init.m4
@@ -100,8 +100,9 @@ AC_REQUIRE([AC_PROG_AWK])dnl
 AC_REQUIRE([AC_PROG_MAKE_SET])dnl
 AC_REQUIRE([AM_SET_LEADING_DOT])dnl
 _AM_IF_OPTION([tar-ustar], [_AM_PROG_TAR([ustar])],
-	      [_AM_IF_OPTION([tar-pax], [_AM_PROG_TAR([pax])],
-			     [_AM_PROG_TAR([v7])])])
+  [_AM_IF_OPTION([tar-pax], [_AM_PROG_TAR([pax])],
+    [_AM_IF_OPTION([tar-v7], [_AM_PROG_TAR([v7])],
+      [_AM_PROG_TAR([ustar])])])])
 _AM_IF_OPTION([no-dependencies],,
 [AC_PROVIDE_IFELSE([AC_PROG_CC],
 		  [_AM_DEPENDENCIES([CC])],
diff --git a/t/tar-override.sh b/t/tar-override.sh
index 941a599da..e02c255ca 100644
--- a/t/tar-override.sh
+++ b/t/tar-override.sh
@@ -16,14 +16,21 @@

 # Check that the user can override the tar program used by "make dist"
 # at runtime, by redefining the 'TAR' environment variable.
-# NOTE: currently this works only when the tar format used is 'v7'
-#       (which is the default one).
+# 
+# Currently this works only when the tar format used is 'v7';
+# as of 2025 (automake-1.18), this is no longer the default,
+# so force that format in our test setup.

 . test-init.sh

 cwd=$(pwd) || fatal_ "getting current working directory"

-echo AC_OUTPUT >> configure.ac
+cat > configure.ac << 'END'
+AC_INIT([tar-override], [1.0])
+AM_INIT_AUTOMAKE([tar-v7])
+AC_CONFIG_FILES([Makefile])
+AC_OUTPUT
+END

 cat > am--tar <<'END'
 #!/bin/sh

compile finished at Mon Feb 24 14:39:19 2025

Message #14 received at 74847 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Karl Berry <karl <at> freefriends.org>, simon <at> josefsson.org
Cc: bruno <at> clisp.org, 74847 <at> debbugs.gnu.org
Subject: Re: bug#74847: change default tar format from v7 to ustar
Date: Mon, 24 Feb 2025 16:13:35 -0800

On 2/24/25 14:41, Karl Berry wrote:
> Did any change to pax formats ever come to pass?

None that would affect this issue with Solaris etc.

Message #19 received at 74847-done <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: 74847-done <at> debbugs.gnu.org
Subject: Re: bug#74847: change default tar format from v7 to ustar
Date: Tue, 25 Feb 2025 15:16:07 -0700

Message #22 received at 74847 <at> debbugs.gnu.org (full text, mbox):

From: Simon Josefsson <simon <at> josefsson.org>
To: Karl Berry <karl <at> freefriends.org>
Cc: eggert <at> cs.ucla.edu, bruno <at> clisp.org, 74847 <at> debbugs.gnu.org
Subject: Re: bug#74847: change default tar format from v7 to ustar
Date: Tue, 25 Feb 2025 23:32:28 +0100

[Message part 1 (text/plain, inline)]

Karl Berry <karl <at> freefriends.org> writes:

> Given Bruno's experience, ok, let's try. I won't be surprised if
> problems come out of the woodwork, but maybe it will be fine.
>
> I think the change below is all that's needed to implement this?
> I pushed it. Let me know if something more comes to mind.

Thank you!

>     What do you think about defaulting to the --format=posix PAX format?

I think defaulting to 'ustar' is the right choice today.  I'm not aware
of any practical advantage that is relevant to software release archives
with PAX over ustar today.

/Simon

[signature.asc (application/pgp-signature, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.