GNU bug report logs - #55023
Issue with CP empty folder after y2038 on 32-bits Kernel

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 55023 in the body.
You can then email your comments to 55023 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>
To: bug-coreutils <at> gnu.org
Subject: Re: Issue with CP empty folder after y2038 on 32-bits Kernel
Date: Tue, 19 Apr 2022 09:47:25 +0200

[Message part 1 (text/plain, inline)]

Hello,

I did not received any feedback from this request right now. Have you
made any progress on this subject ?

Please let me know the progress for this, or contact me for additional
information if needed.

I'd like to have a ticket link to follow the advancement of this issue
(if possible). I'm available to test a patch if you are able to provide
me one.

Best regards,

On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>
> Hello,
>
> I'm working for a client to generate embedded 32-bits Linux Kernel
> working after y2038 issue.
>
> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>
> The Kernel pass y2038 but I found an issue with cp:
> After analysis, the error occurs when trying to move an empty folder
> without all user mode rights.
>
> Here how to reproduce:
>
> # mkdir -p test/test1 folder
> # chmod u-w test/test1
> # date -s "2040-04-02"
> # cp -a test/* folder/
> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>
> Note: The folder is copied before the error occurs. The copy works
> fine before y2038.
>
>
> The issue comes from coreutils-9.0/src/cp.c
>
> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>
> FYI I had a previous issue while calling lstat function from
> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
> as a workaround.
>
>
> Keep me in touch if you need more information.
>
> --
>
> *Arnaud PANAÏOTIS* | Lead Developer Freelance
> +33 6 34 82 12 62 | arnaud.panaiotis <at> gmx.fr <mailto:Arnaud Panaïotis
> <arnaud.panaiotis <at> gmx.fr>>
>
> 18 place Jean Moulin - 38000 Grenoble - France
> APsudo - www.panaiotis.fr <https://www.panaiotis.fr>
--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaiotis <at> gmx.fr <mailto:Arnaud Panaïotis
<arnaud.panaiotis <at> gmx.fr>>

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr <https://www.panaiotis.fr>

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus

[Message part 2 (text/html, inline)]

Message #8 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Tue, 19 Apr 2022 15:00:25 +0100

On 19/04/2022 08:47, Arnaud Panaïotis wrote:
> Hello,
> 
> I did not received any feedback from this request right now. Have you
> made any progress on this subject ?
> 
> Please let me know the progress for this, or contact me for additional
> information if needed.
> 
> I'd like to have a ticket link to follow the advancement of this issue
> (if possible). I'm available to test a patch if you are able to provide
> me one.
> 
> Best regards,
> 
> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>
>> Hello,
>>
>> I'm working for a client to generate embedded 32-bits Linux Kernel
>> working after y2038 issue.
>>
>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>
>> The Kernel pass y2038 but I found an issue with cp:
>> After analysis, the error occurs when trying to move an empty folder
>> without all user mode rights.
>>
>> Here how to reproduce:
>>
>> # mkdir -p test/test1 folder
>> # chmod u-w test/test1
>> # date -s "2040-04-02"
>> # cp -a test/* folder/
>> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>>
>> Note: The folder is copied before the error occurs. The copy works
>> fine before y2038.
>>
>>
>> The issue comes from coreutils-9.0/src/cp.c
>>
>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>
>> FYI I had a previous issue while calling lstat function from
>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>> as a workaround.
>>
>>
>> Keep me in touch if you need more information.

The original mail seems to not have hit the lists, sorry.

The error suggests that the fstatat() done within lchmod()
is using a 32 bit time_t component of the stat structure.
Your kernel is new enough to support the 64 bit equivalent,
but you don't mention glibc. Can you ensure you're using
at least glibc 2.34, which added support for the 64 bit variants.

coreutils is configured by default to enable use
of the 64 bit variants where available, and you've confirmed
this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.

An strace of the cp command would be useful to confirm the problematic syscall.

thanks,
Pádraig

Message #11 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>
To: Pádraig Brady <P <at> draigBrady.com>, 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Tue, 19 Apr 2022 17:01:35 +0200

[Message part 1 (text/plain, inline)]

On 19/04/2022 16:00, Pádraig Brady wrote:
> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>> Hello,
>>
>> I did not received any feedback from this request right now. Have you
>> made any progress on this subject ?
>>
>> Please let me know the progress for this, or contact me for additional
>> information if needed.
>>
>> I'd like to have a ticket link to follow the advancement of this issue
>> (if possible). I'm available to test a patch if you are able to provide
>> me one.
>>
>> Best regards,
>>
>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>
>>> Hello,
>>>
>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>> working after y2038 issue.
>>>
>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>
>>> The Kernel pass y2038 but I found an issue with cp:
>>> After analysis, the error occurs when trying to move an empty folder
>>> without all user mode rights.
>>>
>>> Here how to reproduce:
>>>
>>> # mkdir -p test/test1 folder
>>> # chmod u-w test/test1
>>> # date -s "2040-04-02"
>>> # cp -a test/* folder/
>>> cp: setting permissions for 'folder/test1' : Value too large for
>>> defined data type
>>>
>>> Note: The folder is copied before the error occurs. The copy works
>>> fine before y2038.
>>>
>>>
>>> The issue comes from coreutils-9.0/src/cp.c
>>>
>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>
>>> FYI I had a previous issue while calling lstat function from
>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>> as a workaround.
>>>
>>>
>>> Keep me in touch if you need more information.
>
> The original mail seems to not have hit the lists, sorry.
>
> The error suggests that the fstatat() done within lchmod()
> is using a 32 bit time_t component of the stat structure.
> Your kernel is new enough to support the 64 bit equivalent,
> but you don't mention glibc. Can you ensure you're using
> at least glibc 2.34, which added support for the 64 bit variants.
>
> coreutils is configured by default to enable use
> of the 64 bit variants where available, and you've confirmed
> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are
> defined.
>
> An strace of the cp command would be useful to confirm the problematic
> syscall.
>
> thanks,
> Pádraig

Hello,

Thanks for the quick answer.

My glibc is 2.34-9 (2.34-9-g9acab0bba6a5a57323b1f94bf95b21618a9e5aa4).

Strace attached.

Except for the stat.h, I encountered no issue with Kernel, mysql, my
application...

I indeed suspect stat.h to use a 32 bits time_t somewhere.

Regards,

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaiotis <at> gmx.fr <mailto:Arnaud Panaïotis
<arnaud.panaiotis <at> gmx.fr>>

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr <https://www.panaiotis.fr>

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus

[Message part 2 (text/html, inline)]

[strace-cp2038.log (text/plain, attachment)]

Message #14 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 21 Apr 2022 13:36:12 +0100

On 19/04/2022 16:01, Arnaud Panaïotis wrote:
> On 19/04/2022 16:00, Pádraig Brady wrote:
>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>> Hello,
>>>
>>> I did not received any feedback from this request right now. Have you
>>> made any progress on this subject ?
>>>
>>> Please let me know the progress for this, or contact me for additional
>>> information if needed.
>>>
>>> I'd like to have a ticket link to follow the advancement of this issue
>>> (if possible). I'm available to test a patch if you are able to provide
>>> me one.
>>>
>>> Best regards,
>>>
>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>
>>>> Hello,
>>>>
>>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>>> working after y2038 issue.
>>>>
>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>
>>>> The Kernel pass y2038 but I found an issue with cp:
>>>> After analysis, the error occurs when trying to move an empty folder
>>>> without all user mode rights.
>>>>
>>>> Here how to reproduce:
>>>>
>>>> # mkdir -p test/test1 folder
>>>> # chmod u-w test/test1
>>>> # date -s "2040-04-02"
>>>> # cp -a test/* folder/
>>>> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>>>>
>>>> Note: The folder is copied before the error occurs. The copy works
>>>> fine before y2038.
>>>>
>>>>
>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>
>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>
>>>> FYI I had a previous issue while calling lstat function from
>>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>>> as a workaround.
>>>>
>>>>
>>>> Keep me in touch if you need more information.
>>
>> The original mail seems to not have hit the lists, sorry.
>>
>> The error suggests that the fstatat() done within lchmod()
>> is using a 32 bit time_t component of the stat structure.
>> Your kernel is new enough to support the 64 bit equivalent,
>> but you don't mention glibc. Can you ensure you're using
>> at least glibc 2.34, which added support for the 64 bit variants.
>>
>> coreutils is configured by default to enable use
>> of the 64 bit variants where available, and you've confirmed
>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.
>>
>> An strace of the cp command would be useful to confirm the problematic syscall.

That suggests the kernel (statx) returns fine,
but glibc is returning the EOVERFLOW.
That suggests fstatat() rather than fstatat64() is being used
(inferring that by comparing
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)

So why is fstatat() being used if compiling with
-D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64

You might confirm what statat is being called with:
nm cp | grep U.*statat

The redirect from fstatat() in code to the appropriate 64 bit interface
should be done with asm redirects and so immune to any undef etc.
that gnulib may be doing.

So in summary please look at how fstatat() is being referenced
on your system (by gnulib).

thanks,
Pádraig

Message #17 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>
To: Pádraig Brady <P <at> draigBrady.com>, 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 21 Apr 2022 16:53:47 +0200

[Message part 1 (text/plain, inline)]

On 21/04/2022 14:36, Pádraig Brady wrote:
> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>> On 19/04/2022 16:00, Pádraig Brady wrote:
>>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>>> Hello,
>>>>
>>>> I did not received any feedback from this request right now. Have you
>>>> made any progress on this subject ?
>>>>
>>>> Please let me know the progress for this, or contact me for additional
>>>> information if needed.
>>>>
>>>> I'd like to have a ticket link to follow the advancement of this issue
>>>> (if possible). I'm available to test a patch if you are able to
>>>> provide
>>>> me one.
>>>>
>>>> Best regards,
>>>>
>>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>>
>>>>> Hello,
>>>>>
>>>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>>>> working after y2038 issue.
>>>>>
>>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>>
>>>>> The Kernel pass y2038 but I found an issue with cp:
>>>>> After analysis, the error occurs when trying to move an empty folder
>>>>> without all user mode rights.
>>>>>
>>>>> Here how to reproduce:
>>>>>
>>>>> # mkdir -p test/test1 folder
>>>>> # chmod u-w test/test1
>>>>> # date -s "2040-04-02"
>>>>> # cp -a test/* folder/
>>>>> cp: setting permissions for 'folder/test1' : Value too large for
>>>>> defined data type
>>>>>
>>>>> Note: The folder is copied before the error occurs. The copy works
>>>>> fine before y2038.
>>>>>
>>>>>
>>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>>
>>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>>
>>>>> FYI I had a previous issue while calling lstat function from
>>>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>>>> as a workaround.
>>>>>
>>>>>
>>>>> Keep me in touch if you need more information.
>>>
>>> The original mail seems to not have hit the lists, sorry.
>>>
>>> The error suggests that the fstatat() done within lchmod()
>>> is using a 32 bit time_t component of the stat structure.
>>> Your kernel is new enough to support the 64 bit equivalent,
>>> but you don't mention glibc. Can you ensure you're using
>>> at least glibc 2.34, which added support for the 64 bit variants.
>>>
>>> coreutils is configured by default to enable use
>>> of the 64 bit variants where available, and you've confirmed
>>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are
>>> defined.
>>>
>>> An strace of the cp command would be useful to confirm the
>>> problematic syscall.
>
> That suggests the kernel (statx) returns fine,
> but glibc is returning the EOVERFLOW.
> That suggests fstatat() rather than fstatat64() is being used
> (inferring that by comparing
> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>
> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>
>
> So why is fstatat() being used if compiling with
> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>
> You might confirm what statat is being called with:
> nm cp | grep U.*statat
>
> The redirect from fstatat() in code to the appropriate 64 bit interface
> should be done with asm redirects and so immune to any undef etc.
> that gnulib may be doing.
>
> So in summary please look at how fstatat() is being referenced
> on your system (by gnulib).
>
> thanks,
> Pádraig

Hi,

I had to add -D option to nm to avoid "no symbols" error.

        U __fstatat64_time64 <at> GLIBC_2.34

All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.

Thanks,

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaiotis <at> gmx.fr <mailto:Arnaud Panaïotis
<arnaud.panaiotis <at> gmx.fr>>

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr <https://www.panaiotis.fr>

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus

[Message part 2 (text/html, inline)]

Message #20 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023 <at> debbugs.gnu.org
Cc: Adhemerval Zanella <adhemerval.zanella <at> linaro.org>
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 21 Apr 2022 16:25:49 +0100

On 21/04/2022 15:53, Arnaud Panaïotis wrote:
> 
> On 21/04/2022 14:36, Pádraig Brady wrote:
>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>>> On 19/04/2022 16:00, Pádraig Brady wrote:
>>>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>>>> Hello,
>>>>>
>>>>> I did not received any feedback from this request right now. Have you
>>>>> made any progress on this subject ?
>>>>>
>>>>> Please let me know the progress for this, or contact me for additional
>>>>> information if needed.
>>>>>
>>>>> I'd like to have a ticket link to follow the advancement of this issue
>>>>> (if possible). I'm available to test a patch if you are able to provide
>>>>> me one.
>>>>>
>>>>> Best regards,
>>>>>
>>>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>>>>> working after y2038 issue.
>>>>>>
>>>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>>>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>>>
>>>>>> The Kernel pass y2038 but I found an issue with cp:
>>>>>> After analysis, the error occurs when trying to move an empty folder
>>>>>> without all user mode rights.
>>>>>>
>>>>>> Here how to reproduce:
>>>>>>
>>>>>> # mkdir -p test/test1 folder
>>>>>> # chmod u-w test/test1
>>>>>> # date -s "2040-04-02"
>>>>>> # cp -a test/* folder/
>>>>>> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>>>>>>
>>>>>> Note: The folder is copied before the error occurs. The copy works
>>>>>> fine before y2038.
>>>>>>
>>>>>>
>>>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>>>
>>>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>>>
>>>>>> FYI I had a previous issue while calling lstat function from
>>>>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>>>>> as a workaround.
>>>>>>
>>>>>>
>>>>>> Keep me in touch if you need more information.
>>>>
>>>> The original mail seems to not have hit the lists, sorry.
>>>>
>>>> The error suggests that the fstatat() done within lchmod()
>>>> is using a 32 bit time_t component of the stat structure.
>>>> Your kernel is new enough to support the 64 bit equivalent,
>>>> but you don't mention glibc. Can you ensure you're using
>>>> at least glibc 2.34, which added support for the 64 bit variants.
>>>>
>>>> coreutils is configured by default to enable use
>>>> of the 64 bit variants where available, and you've confirmed
>>>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.
>>>>
>>>> An strace of the cp command would be useful to confirm the problematic syscall.
>>
>> That suggests the kernel (statx) returns fine,
>> but glibc is returning the EOVERFLOW.
>> That suggests fstatat() rather than fstatat64() is being used
>> (inferring that by comparing
>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>
>> So why is fstatat() being used if compiling with
>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>
>> You might confirm what statat is being called with:
>> nm cp | grep U.*statat
>>
>> The redirect from fstatat() in code to the appropriate 64 bit interface
>> should be done with asm redirects and so immune to any undef etc.
>> that gnulib may be doing.
>>
>> So in summary please look at how fstatat() is being referenced
>> on your system (by gnulib).
>>
>> thanks,
>> Pádraig
> 
> Hi,
> 
> I had to add -D option to nm to avoid "no symbols" error.
> 
>          U __fstatat64_time64 <at> GLIBC_2.34
> 
> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.

That looks correct.

So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
You might get to the issue more quickly by installing debug symbols for
your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args

thanks,
Pádraig.

Message #23 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Adhemerval Zanella <adhemerval.zanella <at> linaro.org>,
 Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 21 Apr 2022 21:59:04 +0100

On 21/04/2022 18:41, Adhemerval Zanella wrote:
> 
> 
> On 21/04/2022 12:25, Pádraig Brady wrote:
>> On 21/04/2022 15:53, Arnaud Panaïotis wrote:
>>>
>>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>>>>> On 19/04/2022 16:00, Pádraig Brady wrote:
>>>>>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>>>>>> Hello,
>>>>>>>
>>>>>>> I did not received any feedback from this request right now. Have you
>>>>>>> made any progress on this subject ?
>>>>>>>
>>>>>>> Please let me know the progress for this, or contact me for additional
>>>>>>> information if needed.
>>>>>>>
>>>>>>> I'd like to have a ticket link to follow the advancement of this issue
>>>>>>> (if possible). I'm available to test a patch if you are able to provide
>>>>>>> me one.
>>>>>>>
>>>>>>> Best regards,
>>>>>>>
>>>>>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>>>>>
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>>>>>>> working after y2038 issue.
>>>>>>>>
>>>>>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>>>>>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>>>>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>>>>>
>>>>>>>> The Kernel pass y2038 but I found an issue with cp:
>>>>>>>> After analysis, the error occurs when trying to move an empty folder
>>>>>>>> without all user mode rights.
>>>>>>>>
>>>>>>>> Here how to reproduce:
>>>>>>>>
>>>>>>>> # mkdir -p test/test1 folder
>>>>>>>> # chmod u-w test/test1
>>>>>>>> # date -s "2040-04-02"
>>>>>>>> # cp -a test/* folder/
>>>>>>>> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>>>>>>>>
>>>>>>>> Note: The folder is copied before the error occurs. The copy works
>>>>>>>> fine before y2038.
>>>>>>>>
>>>>>>>>
>>>>>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>>>>>
>>>>>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>>>>>
>>>>>>>> FYI I had a previous issue while calling lstat function from
>>>>>>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>>>>>>> as a workaround.
>>>>>>>>
>>>>>>>>
>>>>>>>> Keep me in touch if you need more information.
>>>>>>
>>>>>> The original mail seems to not have hit the lists, sorry.
>>>>>>
>>>>>> The error suggests that the fstatat() done within lchmod()
>>>>>> is using a 32 bit time_t component of the stat structure.
>>>>>> Your kernel is new enough to support the 64 bit equivalent,
>>>>>> but you don't mention glibc. Can you ensure you're using
>>>>>> at least glibc 2.34, which added support for the 64 bit variants.
>>>>>>
>>>>>> coreutils is configured by default to enable use
>>>>>> of the 64 bit variants where available, and you've confirmed
>>>>>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.
>>>>>>
>>>>>> An strace of the cp command would be useful to confirm the problematic syscall.
>>>>
>>>> That suggests the kernel (statx) returns fine,
>>>> but glibc is returning the EOVERFLOW.
>>>> That suggests fstatat() rather than fstatat64() is being used
>>>> (inferring that by comparing
>>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>>>
>>>> So why is fstatat() being used if compiling with
>>>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>>>
>>>> You might confirm what statat is being called with:
>>>> nm cp | grep U.*statat
>>>>
>>>> The redirect from fstatat() in code to the appropriate 64 bit interface
>>>> should be done with asm redirects and so immune to any undef etc.
>>>> that gnulib may be doing.
>>>>
>>>> So in summary please look at how fstatat() is being referenced
>>>> on your system (by gnulib).
>>>>
>>>> thanks,
>>>> Pádraig
>>>
>>> Hi,
>>>
>>> I had to add -D option to nm to avoid "no symbols" error.
>>>
>>>           U __fstatat64_time64 <at> GLIBC_2.34
>>>
>>> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.
>>
>> That looks correct.
>>
>> So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
>> You might get to the issue more quickly by installing debug symbols for
>> your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args
>>
> 
> Assuming a default config option (without --enable-kernel) the
> _fstatat64_time64 should first try statx and then the old fstatat64
> if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
> only happens for later, assuming kernel does not returns anything
> bogus.
> 
> What strace shows in this scenario?

strace shows statx() returning non error on this 5.15 kernel

Message #26 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Adhemerval Zanella <adhemerval.zanella <at> linaro.org>
To: Pádraig Brady <P <at> draigBrady.com>,
 Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 21 Apr 2022 14:41:29 -0300

On 21/04/2022 12:25, Pádraig Brady wrote:
> On 21/04/2022 15:53, Arnaud Panaïotis wrote:
>>
>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>>>> On 19/04/2022 16:00, Pádraig Brady wrote:
>>>>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>>>>> Hello,
>>>>>>
>>>>>> I did not received any feedback from this request right now. Have you
>>>>>> made any progress on this subject ?
>>>>>>
>>>>>> Please let me know the progress for this, or contact me for additional
>>>>>> information if needed.
>>>>>>
>>>>>> I'd like to have a ticket link to follow the advancement of this issue
>>>>>> (if possible). I'm available to test a patch if you are able to provide
>>>>>> me one.
>>>>>>
>>>>>> Best regards,
>>>>>>
>>>>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>>>>
>>>>>>> Hello,
>>>>>>>
>>>>>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>>>>>> working after y2038 issue.
>>>>>>>
>>>>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>>>>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>>>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>>>>
>>>>>>> The Kernel pass y2038 but I found an issue with cp:
>>>>>>> After analysis, the error occurs when trying to move an empty folder
>>>>>>> without all user mode rights.
>>>>>>>
>>>>>>> Here how to reproduce:
>>>>>>>
>>>>>>> # mkdir -p test/test1 folder
>>>>>>> # chmod u-w test/test1
>>>>>>> # date -s "2040-04-02"
>>>>>>> # cp -a test/* folder/
>>>>>>> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>>>>>>>
>>>>>>> Note: The folder is copied before the error occurs. The copy works
>>>>>>> fine before y2038.
>>>>>>>
>>>>>>>
>>>>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>>>>
>>>>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>>>>
>>>>>>> FYI I had a previous issue while calling lstat function from
>>>>>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>>>>>> as a workaround.
>>>>>>>
>>>>>>>
>>>>>>> Keep me in touch if you need more information.
>>>>>
>>>>> The original mail seems to not have hit the lists, sorry.
>>>>>
>>>>> The error suggests that the fstatat() done within lchmod()
>>>>> is using a 32 bit time_t component of the stat structure.
>>>>> Your kernel is new enough to support the 64 bit equivalent,
>>>>> but you don't mention glibc. Can you ensure you're using
>>>>> at least glibc 2.34, which added support for the 64 bit variants.
>>>>>
>>>>> coreutils is configured by default to enable use
>>>>> of the 64 bit variants where available, and you've confirmed
>>>>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.
>>>>>
>>>>> An strace of the cp command would be useful to confirm the problematic syscall.
>>>
>>> That suggests the kernel (statx) returns fine,
>>> but glibc is returning the EOVERFLOW.
>>> That suggests fstatat() rather than fstatat64() is being used
>>> (inferring that by comparing
>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>>
>>> So why is fstatat() being used if compiling with
>>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>>
>>> You might confirm what statat is being called with:
>>> nm cp | grep U.*statat
>>>
>>> The redirect from fstatat() in code to the appropriate 64 bit interface
>>> should be done with asm redirects and so immune to any undef etc.
>>> that gnulib may be doing.
>>>
>>> So in summary please look at how fstatat() is being referenced
>>> on your system (by gnulib).
>>>
>>> thanks,
>>> Pádraig
>>
>> Hi,
>>
>> I had to add -D option to nm to avoid "no symbols" error.
>>
>>          U __fstatat64_time64 <at> GLIBC_2.34
>>
>> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.
> 
> That looks correct.
> 
> So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
> You might get to the issue more quickly by installing debug symbols for
> your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args
>

Assuming a default config option (without --enable-kernel) the 
_fstatat64_time64 should first try statx and then the old fstatat64
if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
only happens for later, assuming kernel does not returns anything
bogus.

What strace shows in this scenario?

Message #29 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>
To: Pádraig Brady <P <at> draigBrady.com>,
 Adhemerval Zanella <adhemerval.zanella <at> linaro.org>, 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Tue, 26 Apr 2022 14:44:37 +0200

[Message part 1 (text/plain, inline)]

On 21/04/2022 22:59, Pádraig Brady wrote:
> On 21/04/2022 18:41, Adhemerval Zanella wrote:
>>
>>
>> On 21/04/2022 12:25, Pádraig Brady wrote:
>>> On 21/04/2022 15:53, Arnaud Panaïotis wrote:
>>>>
>>>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>>>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>>>>>> On 19/04/2022 16:00, Pádraig Brady wrote:
>>>>>>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>>>>>>> Hello,
>>>>>>>>
>>>>>>>> I did not received any feedback from this request right now.
>>>>>>>> Have you
>>>>>>>> made any progress on this subject ?
>>>>>>>>
>>>>>>>> Please let me know the progress for this, or contact me for
>>>>>>>> additional
>>>>>>>> information if needed.
>>>>>>>>
>>>>>>>> I'd like to have a ticket link to follow the advancement of
>>>>>>>> this issue
>>>>>>>> (if possible). I'm available to test a patch if you are able to
>>>>>>>> provide
>>>>>>>> me one.
>>>>>>>>
>>>>>>>> Best regards,
>>>>>>>>
>>>>>>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>>>>>>
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I'm working for a client to generate embedded 32-bits Linux
>>>>>>>>> Kernel
>>>>>>>>> working after y2038 issue.
>>>>>>>>>
>>>>>>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils
>>>>>>>>> 9.0, GCC
>>>>>>>>> 11.2.0, Binutils 2.37 and CFLAGS -D_LARGEFILE_SOURCE
>>>>>>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>>>>>>
>>>>>>>>> The Kernel pass y2038 but I found an issue with cp:
>>>>>>>>> After analysis, the error occurs when trying to move an empty
>>>>>>>>> folder
>>>>>>>>> without all user mode rights.
>>>>>>>>>
>>>>>>>>> Here how to reproduce:
>>>>>>>>>
>>>>>>>>> # mkdir -p test/test1 folder
>>>>>>>>> # chmod u-w test/test1
>>>>>>>>> # date -s "2040-04-02"
>>>>>>>>> # cp -a test/* folder/
>>>>>>>>> cp: setting permissions for 'folder/test1' : Value too large
>>>>>>>>> for defined data type
>>>>>>>>>
>>>>>>>>> Note: The folder is copied before the error occurs. The copy
>>>>>>>>> works
>>>>>>>>> fine before y2038.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>>>>>>
>>>>>>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>>>>>>
>>>>>>>>> FYI I had a previous issue while calling lstat function from
>>>>>>>>> <sys/stat.h> which is included in lib/lchmod.c. I used
>>>>>>>>> /usr/bin/stat
>>>>>>>>> as a workaround.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Keep me in touch if you need more information.
>>>>>>>
>>>>>>> The original mail seems to not have hit the lists, sorry.
>>>>>>>
>>>>>>> The error suggests that the fstatat() done within lchmod()
>>>>>>> is using a 32 bit time_t component of the stat structure.
>>>>>>> Your kernel is new enough to support the 64 bit equivalent,
>>>>>>> but you don't mention glibc. Can you ensure you're using
>>>>>>> at least glibc 2.34, which added support for the 64 bit variants.
>>>>>>>
>>>>>>> coreutils is configured by default to enable use
>>>>>>> of the 64 bit variants where available, and you've confirmed
>>>>>>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>>>>>> are defined.
>>>>>>>
>>>>>>> An strace of the cp command would be useful to confirm the
>>>>>>> problematic syscall.
>>>>>
>>>>> That suggests the kernel (statx) returns fine,
>>>>> but glibc is returning the EOVERFLOW.
>>>>> That suggests fstatat() rather than fstatat64() is being used
>>>>> (inferring that by comparing
>>>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>>>>>
>>>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>>>>
>>>>>
>>>>> So why is fstatat() being used if compiling with
>>>>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>>>>
>>>>> You might confirm what statat is being called with:
>>>>> nm cp | grep U.*statat
>>>>>
>>>>> The redirect from fstatat() in code to the appropriate 64 bit
>>>>> interface
>>>>> should be done with asm redirects and so immune to any undef etc.
>>>>> that gnulib may be doing.
>>>>>
>>>>> So in summary please look at how fstatat() is being referenced
>>>>> on your system (by gnulib).
>>>>>
>>>>> thanks,
>>>>> Pádraig
>>>>
>>>> Hi,
>>>>
>>>> I had to add -D option to nm to avoid "no symbols" error.
>>>>
>>>>           U __fstatat64_time64 <at> GLIBC_2.34
>>>>
>>>> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.
>>>
>>> That looks correct.
>>>
>>> So perhaps the EOVERFLOW is within your glibc's fstatat64
>>> implementation?
>>> You might get to the issue more quickly by installing debug symbols for
>>> your coreutils and/or glibc and using: gdb -tui -args cp rest of cp
>>> args
>>>
>>
>> Assuming a default config option (without --enable-kernel) the
>> _fstatat64_time64 should first try statx and then the old fstatat64
>> if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
>> only happens for later, assuming kernel does not returns anything
>> bogus.
>>
>> What strace shows in this scenario?
>
> strace shows statx() returning non error on this 5.15 kernel
Hi,

Sorry for the delay, had trouble to generate all requirements within
Buildroot (now with gdb+coreutils debugs). I can add glibc debug if needed.

quotearg_buffer_restyled (buffer=buffer <at> entry=0x41b5a0 <slot0> "",
 buffersize=<optimized out>, buffersize <at> entry=256,
arg=arg <at> entry=0x41c5f0 "folder/test1", argsize=4294967295,

    quoting_style=<optimized out>, flags=<optimized out>,
quote_these_too=<optimized out>, left_quote=0x0, right_quote=0x0)
at lib/quotearg.c:262

(gdb)
quotearg_n_options (n=n <at> entry=0, arg=arg <at> entry=0x41c5f0 "folder/test1",
argsize=argsize <at> entry=4294967295, options=0xbffff630) at
lib/quotearg.c:905

0x00406b46 in copy_reg (src_sb=<optimized out>,
new_dst=<optimized out>, omitted_permissions=<optimized
out>, dst_mode=<optimized out>, x=<optimized out>,
dst_name=<optimized out>,

    src_name=<optimized out>) at src/copy.c:1146

/bin/cp: setting permissions for 'folder/test1': Value too large for defined data type

I don't have layout src working but layout asm works.

(breakpoint at fstatat, then step by step).

Let me know if you need more details, I'm available for a shared screen
visio if needed (I'm in UTC+2).

Best regards,

--

*Arnaud PANAÏOTIS* | Lead Developer Freelance
+33 6 34 82 12 62 | arnaud.panaiotis <at> gmx.fr <mailto:Arnaud Panaïotis
<arnaud.panaiotis <at> gmx.fr>>

18 place Jean Moulin - 38000 Grenoble
APsudo - www.panaiotis.fr <https://www.panaiotis.fr>

--
L'absence de virus dans ce courrier électronique a été vérifiée par le logiciel antivirus Avast.
https://www.avast.com/antivirus

[Message part 2 (text/html, inline)]

Message #34 received at 55023-done <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Adhemerval Zanella <adhemerval.zanella <at> linaro.org>,
 Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023-done <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Wed, 27 Apr 2022 17:42:47 +0100

Marking this as done in the coreutils bug tracker,
now that this is being tracked in glibc.

thank you!

On 27/04/2022 13:46, Adhemerval Zanella wrote:

>>>>>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>>>>>> That suggests the kernel (statx) returns fine,
>>>>>>> but glibc's fstatat64 is returning the EOVERFLOW.

> It seems to be a glibc missing support indeed.  The coreutils issues indicates
> that lchmodat failed somehow:
> 
>                if (lchmodat (dst_dirfd, dst_relname, dst_mode | S_IRWXU) != 0)
>                  {
>                    error (0, errno, _("setting permissions for %s"),
>                           quoteaf (dst_name));
>                    goto un_backup;
>                  }
> 
> And lchmodat is a gnulib wrapper for fchmodat:
> 
> CHMODAT_INLINE int
> lchmodat (int fd, char const *file, mode_t mode)
> {
>    return fchmodat (fd, file, mode, AT_SYMLINK_NOFOLLOW);
> }
> 
> And since Linux fchmodat syscall does not provide a 'flag' argument (to
> handle AT_SYMLINK_NOFOLLOW), glibc emulates it through opening a procfs file
> descriptor, issuing fstatat to check if it is link (since some kernels and
> filesystem it returns in inconsistent results), and then issue chmod.
> 
> However, the glibc internal fstat does not use the 64-bit version, which
> then results in EOVERFLOW.
> 
> I have opened https://sourceware.org/bugzilla/show_bug.cgi?id=29097 and I will
> fix it upstream and backport to 2.34 and 2.35.

Message #37 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Adhemerval Zanella <adhemerval.zanella <at> linaro.org>
To: Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 Pádraig Brady <P <at> draigBrady.com>, 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Wed, 27 Apr 2022 09:46:48 -0300

On 26/04/2022 09:44, Arnaud Panaïotis wrote:
> 
> On 21/04/2022 22:59, Pádraig Brady wrote:
>> On 21/04/2022 18:41, Adhemerval Zanella wrote:
>>>
>>>
>>> On 21/04/2022 12:25, Pádraig Brady wrote:
>>>> On 21/04/2022 15:53, Arnaud Panaïotis wrote:
>>>>>
>>>>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>>>>> On 19/04/2022 16:01, Arnaud Panaïotis wrote:
>>>>>>> On 19/04/2022 16:00, Pádraig Brady wrote:
>>>>>>>> On 19/04/2022 08:47, Arnaud Panaïotis wrote:
>>>>>>>>> Hello,
>>>>>>>>>
>>>>>>>>> I did not received any feedback from this request right now. Have you
>>>>>>>>> made any progress on this subject ?
>>>>>>>>>
>>>>>>>>> Please let me know the progress for this, or contact me for additional
>>>>>>>>> information if needed.
>>>>>>>>>
>>>>>>>>> I'd like to have a ticket link to follow the advancement of this issue
>>>>>>>>> (if possible). I'm available to test a patch if you are able to provide
>>>>>>>>> me one.
>>>>>>>>>
>>>>>>>>> Best regards,
>>>>>>>>>
>>>>>>>>> On 01/04/2022 15:55, Arnaud Panaïotis wrote:
>>>>>>>>>>
>>>>>>>>>> Hello,
>>>>>>>>>>
>>>>>>>>>> I'm working for a client to generate embedded 32-bits Linux Kernel
>>>>>>>>>> working after y2038 issue.
>>>>>>>>>>
>>>>>>>>>> I generated a 5.15 Kernel thought Buildroot with Coreutils 9.0, GCC
>>>>>>>>>> 11.2.0, Binutils 2.37 and CFLAGS  -D_LARGEFILE_SOURCE
>>>>>>>>>> -D_FILE_OFFSET_BITS=64  -D_TIME_BITS=64
>>>>>>>>>>
>>>>>>>>>> The Kernel pass y2038 but I found an issue with cp:
>>>>>>>>>> After analysis, the error occurs when trying to move an empty folder
>>>>>>>>>> without all user mode rights.
>>>>>>>>>>
>>>>>>>>>> Here how to reproduce:
>>>>>>>>>>
>>>>>>>>>> # mkdir -p test/test1 folder
>>>>>>>>>> # chmod u-w test/test1
>>>>>>>>>> # date -s "2040-04-02"
>>>>>>>>>> # cp -a test/* folder/
>>>>>>>>>> cp: setting permissions for 'folder/test1' : Value too large for defined data type
>>>>>>>>>>
>>>>>>>>>> Note: The folder is copied before the error occurs. The copy works
>>>>>>>>>> fine before y2038.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> The issue comes from coreutils-9.0/src/cp.c
>>>>>>>>>>
>>>>>>>>>> Line 512 : if (lchmod (dir, stats.st_mode | S_IRWXU) != 0)
>>>>>>>>>>
>>>>>>>>>> FYI I had a previous issue while calling lstat function from
>>>>>>>>>> <sys/stat.h> which is included in lib/lchmod.c. I used /usr/bin/stat
>>>>>>>>>> as a workaround.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Keep me in touch if you need more information.
>>>>>>>>
>>>>>>>> The original mail seems to not have hit the lists, sorry.
>>>>>>>>
>>>>>>>> The error suggests that the fstatat() done within lchmod()
>>>>>>>> is using a 32 bit time_t component of the stat structure.
>>>>>>>> Your kernel is new enough to support the 64 bit equivalent,
>>>>>>>> but you don't mention glibc. Can you ensure you're using
>>>>>>>> at least glibc 2.34, which added support for the 64 bit variants.
>>>>>>>>
>>>>>>>> coreutils is configured by default to enable use
>>>>>>>> of the 64 bit variants where available, and you've confirmed
>>>>>>>> this as you say both -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64 are defined.
>>>>>>>>
>>>>>>>> An strace of the cp command would be useful to confirm the problematic syscall.
>>>>>>
>>>>>> That suggests the kernel (statx) returns fine,
>>>>>> but glibc is returning the EOVERFLOW.
>>>>>> That suggests fstatat() rather than fstatat64() is being used
>>>>>> (inferring that by comparing
>>>>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat.c
>>>>>> https://sourceware.org/git/?p=glibc.git;a=history;f=sysdeps/unix/sysv/linux/fstatat64.c)
>>>>>>
>>>>>> So why is fstatat() being used if compiling with
>>>>>> -D_FILE_OFFSET_BITS=64 and -D_TIME_BITS=64
>>>>>>
>>>>>> You might confirm what statat is being called with:
>>>>>> nm cp | grep U.*statat
>>>>>>
>>>>>> The redirect from fstatat() in code to the appropriate 64 bit interface
>>>>>> should be done with asm redirects and so immune to any undef etc.
>>>>>> that gnulib may be doing.
>>>>>>
>>>>>> So in summary please look at how fstatat() is being referenced
>>>>>> on your system (by gnulib).
>>>>>>
>>>>>> thanks,
>>>>>> Pádraig
>>>>>
>>>>> Hi,
>>>>>
>>>>> I had to add -D option to nm to avoid "no symbols" error.
>>>>>
>>>>>           U __fstatat64_time64 <at> GLIBC_2.34
>>>>>
>>>>> All fstat, fstatat, lstat and stat are 64_time64 with the same GLIBC.
>>>>
>>>> That looks correct.
>>>>
>>>> So perhaps the EOVERFLOW is within your glibc's fstatat64 implementation?
>>>> You might get to the issue more quickly by installing debug symbols for
>>>> your coreutils and/or glibc and using: gdb -tui -args cp rest of cp args
>>>>
>>>
>>> Assuming a default config option (without --enable-kernel) the
>>> _fstatat64_time64 should first try statx and then the old fstatat64
>>> if statx fails with ENOSYS (on kernel older than 4.11).  The EOVERFLOW
>>> only happens for later, assuming kernel does not returns anything
>>> bogus.
>>>
>>> What strace shows in this scenario?
>>
>> strace shows statx() returning non error on this 5.15 kernel
> Hi,
> 
> Sorry for the delay, had trouble to generate all requirements within Buildroot (now with gdb+coreutils debugs). I can add glibc debug if needed.
> 
> quotearg_buffer_restyled (buffer=buffer <at> entry=0x41b5a0 <slot0> "",
>  buffersize=<optimized out>, buffersize <at> entry=256,
> arg=arg <at> entry=0x41c5f0 "folder/test1", argsize=4294967295,
> 
>     quoting_style=<optimized out>, flags=<optimized out>,
> quote_these_too=<optimized out>, left_quote=0x0, right_quote=0x0)
> at lib/quotearg.c:262
> 
> (gdb)
> quotearg_n_options (n=n <at> entry=0, arg=arg <at> entry=0x41c5f0 "folder/test1",
> argsize=argsize <at> entry=4294967295, options=0xbffff630) at
> lib/quotearg.c:905
> 
> 0x00406b46 in copy_reg (src_sb=<optimized out>,
> new_dst=<optimized out>, omitted_permissions=<optimized
> out>, dst_mode=<optimized out>, x=<optimized out>,
> dst_name=<optimized out>,
> 
>     src_name=<optimized out>) at src/copy.c:1146
> 
> /bin/cp: setting permissions for 'folder/test1': Value too large for defined data type
> 
> I don't have layout src working but layout asm works.
> 
> (breakpoint at fstatat, then step by step).
> 
> Let me know if you need more details, I'm available for a shared screen visio if needed (I'm in UTC+2).

It seems to be a glibc missing support indeed.  The coreutils issues indicates
that lchmodat failed somehow:

              if (lchmodat (dst_dirfd, dst_relname, dst_mode | S_IRWXU) != 0)
                {
                  error (0, errno, _("setting permissions for %s"),
                         quoteaf (dst_name));
                  goto un_backup;
                }

And lchmodat is a gnulib wrapper for fchmodat:

CHMODAT_INLINE int
lchmodat (int fd, char const *file, mode_t mode)
{
  return fchmodat (fd, file, mode, AT_SYMLINK_NOFOLLOW);
}

And since Linux fchmodat syscall does not provide a 'flag' argument (to
handle AT_SYMLINK_NOFOLLOW), glibc emulates it through opening a procfs file
descriptor, issuing fstatat to check if it is link (since some kernels and
filesystem it returns in inconsistent results), and then issue chmod.

However, the glibc internal fstat does not use the 64-bit version, which
then results in EOVERFLOW. 

I have opened https://sourceware.org/bugzilla/show_bug.cgi?id=29097 and I will
fix it upstream and backport to 2.34 and 2.35.

Message #40 received at 55023-done <at> debbugs.gnu.org (full text, mbox):

From: Adhemerval Zanella <adhemerval.zanella <at> linaro.org>
To: Pádraig Brady <P <at> draigBrady.com>,
 Arnaud Panaïotis <arnaud.panaiotis <at> gmx.fr>,
 55023-done <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 28 Apr 2022 10:22:11 -0300

On 27/04/2022 13:42, Pádraig Brady wrote:
> Marking this as done in the coreutils bug tracker,
> now that this is being tracked in glibc.
> 
> thank you!
> 
> On 27/04/2022 13:46, Adhemerval Zanella wrote:
> 
>>>>>>> On 21/04/2022 14:36, Pádraig Brady wrote:
>>>>>>>> That suggests the kernel (statx) returns fine,
>>>>>>>> but glibc's fstatat64 is returning the EOVERFLOW.
> 
>> It seems to be a glibc missing support indeed.  The coreutils issues indicates
>> that lchmodat failed somehow:
>>
>>                if (lchmodat (dst_dirfd, dst_relname, dst_mode | S_IRWXU) != 0)
>>                  {
>>                    error (0, errno, _("setting permissions for %s"),
>>                           quoteaf (dst_name));
>>                    goto un_backup;
>>                  }
>>
>> And lchmodat is a gnulib wrapper for fchmodat:
>>
>> CHMODAT_INLINE int
>> lchmodat (int fd, char const *file, mode_t mode)
>> {
>>    return fchmodat (fd, file, mode, AT_SYMLINK_NOFOLLOW);
>> }
>>
>> And since Linux fchmodat syscall does not provide a 'flag' argument (to
>> handle AT_SYMLINK_NOFOLLOW), glibc emulates it through opening a procfs file
>> descriptor, issuing fstatat to check if it is link (since some kernels and
>> filesystem it returns in inconsistent results), and then issue chmod.
>>
>> However, the glibc internal fstat does not use the 64-bit version, which
>> then results in EOVERFLOW.
>>
>> I have opened https://sourceware.org/bugzilla/show_bug.cgi?id=29097 and I will
>> fix it upstream and backport to 2.34 and 2.35.
> 

It is now fixed on master, 2.35, and 2.34 branches.

Message #43 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: 55023 <at> debbugs.gnu.org, P <at> draigBrady.com, arnaud.panaiotis <at> gmx.fr
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Thu, 28 Apr 2022 16:56:56 -0700

On 4/27/22 09:42, Pádraig Brady wrote:
> Marking this as done in the coreutils bug tracker,
> now that this is being tracked in glibc.

This could also be worked around Gnulib for the benefit of 32-bit apps 
running with unpatched glibc 2.34 and 2.35, or glibc older than 2.34. 
Not sure it's worth the trouble, though, as the fixed glibc should be 
universal pretty much everywhere before the year 2038 rolls around.

Message #46 received at 55023 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, 55023 <at> debbugs.gnu.org
Subject: Re: bug#55023: Issue with CP empty folder after y2038 on 32-bits
 Kernel
Date: Fri, 29 Apr 2022 13:56:24 +0100

On 29/04/2022 00:56, Paul Eggert wrote:
> On 4/27/22 09:42, Pádraig Brady wrote:
>> Marking this as done in the coreutils bug tracker,
>> now that this is being tracked in glibc.
> 
> This could also be worked around Gnulib for the benefit of 32-bit apps
> running with unpatched glibc 2.34 and 2.35, or glibc older than 2.34.
> Not sure it's worth the trouble, though, as the fixed glibc should be
> universal pretty much everywhere before the year 2038 rolls around.

Yes I was thinking about that,
and also thought it not necessary to deal with in gnulib.

thanks,
Pádraig.

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.