GNU bug report logs - #73316
Numeric user ID too large when generating the tarball

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 73316 in the body.
You can then email your comments to 73316 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: glorenzetti <at> cefca.es
To: bug-automake <at> gnu.org
Subject: Numeric user ID too large when generating the tarball
Date: Tue, 17 Sep 2024 10:34:01 +0200

Good morning,

GNU Astronomy Utilities (Gnuastro) uses GNU Autotools. A Gnuastro 
developer encountered the "Numeric user ID too large" error when 
generating a tarball with the 'make dist' command on his macOS (Monterey 
12.3) laptop. His user id is an integer with 9 digits, which is too big 
for the old v7 and ustar formats of tar.

The issue is discussed in Gnuastro's bug tracker 
(https://savannah.gnu.org/bugs/?65578), where the work-around conclusion 
was to use a conditional that adds 'tar-pax' to 'AM_INIT_AUTOMAKE' when 
the user ID is so large. For reference, here is Gnuastro's configure.ac: 
https://git.savannah.gnu.org/cgit/gnuastro.git/tree/configure.ac#n44
The issue was also erroneusly reported in autoconf's bug tracker 
(https://savannah.gnu.org/support/?111123).

Even though we encountered this problem in macOS, this problem may not 
be limited to it. Other user management systems who use such large IDs 
may also cause this issue.

While it is possible to write such workaround downstream in Gnuastro, it 
would be nice to have the issue solved upstream in the GNU Autotools 
since it can happen for any other developer of other software that uses 
Autotools.

Best Regards,
Giacomo Lorenzetti

Message #8 received at 73316 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: glorenzetti <at> cefca.es
Cc: 73316 <at> debbugs.gnu.org
Subject: Re: bug#73316: Numeric user ID too large when generating the tarball
Date: Tue, 17 Sep 2024 15:40:53 -0600

Hi Giacomo - thanks for the report. Unfortunately, I (still) don't see a
good way for Automake can automatically deal with this problem. After
reading the various links in your message, it seems even less feasible.

Automatically switching to tar-pax if the current id is too large feels
like too big of a change to me. The user might well want to fix it in
another way. I know I would.

Does tar report an error for you with a too-large-for-v7 uid value?
I get, e.g.,
$ tar -cf x.tar --owner=31863269 --format=v7 x.html
tar: value 31863269 out of uid_t range 0..2097151
tar: Exiting with failure status due to previous errors

So I don't see anything to be gained by automake trying to detect the
situation just to issue a different error.

All I can see to do is to add tar-pax as another workaround for the
problem in the manual. (Glad you had that idea.)

*

Regarding "privileges" needed for --owner=0 --group=0, as you reported
in https://savannah.gnu.org/bugs/?65578#comment5 ... that seems
strange. It's just writing a file with certain bits. It does not happen
for me with GNU tar 1.34 (compiled from the original source).

But given that that was the case for you, that's another argument for
automake not to try to do anything automatically. For instance, you
could (hopefully) use --owner=1 --group=1 instead, if your tar somehow
does something special about [ug]id 0.

*

Regarding the TAR_OPTIONS envvar, it works for me with the
same tar 1.34:

$ TAR_OPTIONS=-v tar cf test.tar test.txt
test.txt

So I don't know why it didn't work for you. Puzzling.

Happy hacking,
Karl

Message #11 received at 73316 <at> debbugs.gnu.org (full text, mbox):

From: glorenzetti <at> cefca.es
To: Karl Berry <karl <at> freefriends.org>
Cc: 73316 <at> debbugs.gnu.org
Subject: Re: bug#73316: Numeric user ID too large when generating the tarball
Date: Thu, 19 Sep 2024 18:00:23 +0200

Hi Karl,

thanks for the detailed answers.
We've run few more tests, and we've found a couple more issues.

1. As you pointed out, TAR_OPTIONS is working fine from the command 
line, but it doesn't work when written inside Makefile.am, neither on 
Mac, nor on Linux.
$ TAR_OPTIONS="--version"
$ export TAR_OPTIONS

2. The workaround we were attempting uses an AS_IF to initialize a 
variable to be used as argument in AM_INIT_AUTOMAKE, but 
AM_INIT_AUTOMAKE is reading that variable as an option. Are we missing 
something regarding the syntax of autotools?

3. Feature request: it would be nice to have a "tar-no-owner" option for 
AM_INIT_AUTOMAKE that sets user and group ids to 0. (The "privileges" 
issue apparently is only when extracting with --same-owner, so intended)

Best,
Giacomo

Message #14 received at 73316 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: glorenzetti <at> cefca.es
Cc: 73316 <at> debbugs.gnu.org
Subject: Re: bug#73316: Numeric user ID too large when generating the tarball
Date: Sun, 23 Feb 2025 14:34:43 -0700

Hi Giacomo - back on your report from last year (sorry):

    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73316
    
    1. As you pointed out, TAR_OPTIONS is working fine from the command 
    line, but it doesn't work when written inside Makefile.am, 

You need to add an "export" directive to the Makefile, which requires
GNU make.  As in:

TAR_OPTIONS = --owner=0 --group=0
export TAR_OPTIONS

FWIW, I added this to the manual for the automake-1.17 release
(https://gnu.org//automake/manual/automake.html#Basics-of-Distribution)
following another user's report, https://bugs.gnu.org/19615.

I've added your finding of tax-pax to the manual. I don't know if
tar-pax is portable enough now for general use. It wasn't when it was
invented.

    2. The workaround we were attempting uses an AS_IF to initialize a 
    variable to be used as argument in AM_INIT_AUTOMAKE, but 
    AM_INIT_AUTOMAKE is reading that variable as an option. Are we missing 
    something regarding the syntax of autotools?

As far as I can see, I don't think AM_INIT_AUTOMAKE can handle
"computed" options. Looking at Automake/Options.pm, it is just
considering the options as string constants and checking for validity
using regexps. I don't see a feasible way to do otherwise.

    3. Feature request: it would be nice to have a "tar-no-owner" option
    for AM_INIT_AUTOMAKE that sets user and group ids to 0. (The
    "privileges" issue apparently is only when extracting with
    --same-owner, so intended)

It would be nice. The problem is that there's no portable way to do it,
and it's not simple to detect and change the tar invocation.

If you're willing to use GNU tar, the above should suffice.
If you need to be portable to all tars, then, well, I don't know how/if
it's possible with any other tar.

I'm going to close this for lack of anything more to do, but please
reply/reopen issue as desired. --thanks, karl.

Message #17 received at 73316 <at> debbugs.gnu.org (full text, mbox):

From: Karl Berry <karl <at> freefriends.org>
To: glorenzetti <at> cefca.es
Cc: 73316 <at> debbugs.gnu.org
Subject: Re: bug#73316: Numeric user ID too large when generating the tarball
Date: Mon, 24 Feb 2025 10:04:54 -0700

[I mailed this yesterday, but the bug has not been updated. Disturbing.
originally mailed on Date: Sun, 23 Feb 2025 13:35:17 -0800.
Trying again.]

Hi Giacomo - back on your report from last year (sorry):

    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=73316
    
    1. As you pointed out, TAR_OPTIONS is working fine from the command 
    line, but it doesn't work when written inside Makefile.am, 

You need to add an "export" directive to the Makefile, which requires
GNU make.  As in:

TAR_OPTIONS = --owner=0 --group=0
export TAR_OPTIONS

FWIW, I added this to the manual for the automake-1.17 release
(https://gnu.org//automake/manual/automake.html#Basics-of-Distribution)
following another user's report, https://bugs.gnu.org/19615.

I've added your finding of tax-pax to the manual. I don't know if
tar-pax is portable enough now for general use. It wasn't when it was
invented.

    2. The workaround we were attempting uses an AS_IF to initialize a 
    variable to be used as argument in AM_INIT_AUTOMAKE, but 
    AM_INIT_AUTOMAKE is reading that variable as an option. Are we missing 
    something regarding the syntax of autotools?

As far as I can see, I don't think AM_INIT_AUTOMAKE can handle
"computed" options. Looking at Automake/Options.pm, it is just
considering the options as string constants and checking for validity
using regexps. I don't see a feasible way to do otherwise.

    3. Feature request: it would be nice to have a "tar-no-owner" option
    for AM_INIT_AUTOMAKE that sets user and group ids to 0. (The
    "privileges" issue apparently is only when extracting with
    --same-owner, so intended)

It would be nice. The problem is that there's no portable way to do it,
and it's not simple to detect and change the tar invocation.

If you're willing to use GNU tar, the above should suffice.
If you need to be portable to all tars, then, well, I don't know how/if
it's possible with any other tar.

I'm going to close this for lack of anything more to do, but please
reply/reopen issue as desired. --thanks, karl.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.