GNU bug report logs - #34199
Small bug in cp (for win64)

Previous Next

To add a comment to this bug, you must first unarchive it, by sending
a message to control AT debbugs.gnu.org, with unarchive 34199 in the body.
You can then email your comments to 34199 AT debbugs.gnu.org in the normal way.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Chris Kalish <chris <at> thekalishes.com>
To: CP Bugs <bug-coreutils <at> gnu.org>
Subject: Small bug in cp (for win64)
Date: Fri, 25 Jan 2019 12:14:42 -0500

[Message part 1 (text/plain, inline)]

Hi, guys ... I use cp to backup source systems to an external drive.  It
works great (and the --update=number function is a key differentiator).
However, I noticed that (for NTFS file  systems) long directory names
(\abc\def\ghi\jkl\mno\pqrstuvwxyz\blahblah\longlonglongdirectorystring) are
not supported (they throw "no such file or directory errors").  I assume
you're making an assumption on a max static var size (i.e.,
szDirectory[100]) ... can you either up that allocation or malloc() the
memory to the input string?  I believe the NTFS fully-cascading filename
limit is 32,000 characters.

(actual example):

*cp: cannot create regular file
`C:\\XDrive\\temp\\delete\\KalishCMirror\\XDrive\\Temp\\delete\\GDGBackups\\Presariokids\\LastBackup\\EBackups\\Devin\\DAS\\Documents\\eclipseWS\\2cov2cor\\.settings\\org.eclipse.cdt.managedbuilder.core.prefs':
No such file or directory*


It will copy if I subst the directory name into a virtual drive letter, but
that is not a reasonable solution to recusing my entire drive.

Thanks!

-chris

[Message part 2 (text/html, inline)]

Message #12 received at 34199-done <at> debbugs.gnu.org (full text, mbox):

From: Eric Blake <eblake <at> redhat.com>
To: Chris Kalish <chris <at> thekalishes.com>, 34199-done <at> debbugs.gnu.org
Subject: Re: bug#34199: Small bug in cp (for win64)
Date: Fri, 25 Jan 2019 14:33:57 -0600

[Message part 1 (text/plain, inline)]

tag 34199 notabug
thanks

On 1/25/19 11:14 AM, Chris Kalish wrote:
> Hi, guys ... I use cp to backup source systems to an external drive.  It
> works great (and the --update=number function is a key differentiator).
> However, I noticed that (for NTFS file  systems) long directory names
> (\abc\def\ghi\jkl\mno\pqrstuvwxyz\blahblah\longlonglongdirectorystring) are
> not supported (they throw "no such file or directory errors").  I assume
> you're making an assumption on a max static var size (i.e.,
> szDirectory[100]) ... can you either up that allocation or malloc() the
> memory to the input string?  I believe the NTFS fully-cascading filename
> limit is 32,000 characters.

You are incorrect about upstream cp itself having a fixed-size buffer;
however, the underlying operating system and/or file system may have
limits that you are tripping over.  I know that on Windows, whether an
application was compiled against ASCII vs. Unicode functions from libc
can make a difference on the maximum file name that program can use.

However, I can also state that at least the cygwin build of cp (from
cygwin.com) does not suffer from the limitation on Windows, and it uses
the same upstream cp sources.  So I assume that you are using a
pre-built cp from some other site than cygwin, and that the limitation
is inherent to the porting job of whatever you are using.  Therefore,
you are better off reporting this to the downstream distributor of the
pre-built binary you are using, as upstream is not the problem.  I'm
marking this as not a bug to avoid it staying open forever in our bug
database, but feel free to reply to this with further details, and we
can reopen it if you provide more details that points back at the
upstream code doing something that interferes with proper long file name
usage.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[signature.asc (application/pgp-signature, attachment)]

Message #15 received at 34199 <at> debbugs.gnu.org (full text, mbox):

From: Chris Kalish <chris <at> thekalishes.com>
To: 34199 <at> debbugs.gnu.org
Subject: Re: bug#34199: closed (Re: bug#34199: Small bug in cp (for win64))
Date: Sun, 27 Jan 2019 21:38:53 -0500

[Message part 1 (text/plain, inline)]

Hmmm ... not sure of the distribution, but the help file pointed me at this
address:

C:\> cp --version

cp (GNU coreutils) 5.3.0

Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering.


Copyright (C) 2005 Free Software Foundation, Inc.

This is free software; see the source for copying conditions.  There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.


C:\> cp --help
.... ..... .... ....
As a special case, cp makes a backup of SOURCE when the force and backup
options are given and SOURCE and DEST are the same name for an existing,
regular file.

Report bugs to <bug-coreutils <at> gnu.org>.


-chris

On Fri, Jan 25, 2019 at 3:35 PM GNU bug Tracking System <
help-debbugs <at> gnu.org> wrote:

> Your bug report
>
> #34199: Small bug in cp (for win64)
>
> which was filed against the coreutils package, has been closed.
>
> The explanation is attached below, along with your original report.
> If you require more details, please reply to 34199 <at> debbugs.gnu.org.
>
> --
> 34199: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=34199
> GNU Bug Tracking System
> Contact help-debbugs <at> gnu.org with problems
>
>
>
> ---------- Forwarded message ----------
> From: Eric Blake <eblake <at> redhat.com>
> To: Chris Kalish <chris <at> thekalishes.com>, 34199-done <at> debbugs.gnu.org
> Cc:
> Bcc:
> Date: Fri, 25 Jan 2019 14:33:57 -0600
> Subject: Re: bug#34199: Small bug in cp (for win64)
> tag 34199 notabug
> thanks
>
> On 1/25/19 11:14 AM, Chris Kalish wrote:
> > Hi, guys ... I use cp to backup source systems to an external drive.  It
> > works great (and the --update=number function is a key differentiator).
> > However, I noticed that (for NTFS file  systems) long directory names
> > (\abc\def\ghi\jkl\mno\pqrstuvwxyz\blahblah\longlonglongdirectorystring)
> are
> > not supported (they throw "no such file or directory errors").  I assume
> > you're making an assumption on a max static var size (i.e.,
> > szDirectory[100]) ... can you either up that allocation or malloc() the
> > memory to the input string?  I believe the NTFS fully-cascading filename
> > limit is 32,000 characters.
>
> You are incorrect about upstream cp itself having a fixed-size buffer;
> however, the underlying operating system and/or file system may have
> limits that you are tripping over.  I know that on Windows, whether an
> application was compiled against ASCII vs. Unicode functions from libc
> can make a difference on the maximum file name that program can use.
>
> However, I can also state that at least the cygwin build of cp (from
> cygwin.com) does not suffer from the limitation on Windows, and it uses
> the same upstream cp sources.  So I assume that you are using a
> pre-built cp from some other site than cygwin, and that the limitation
> is inherent to the porting job of whatever you are using.  Therefore,
> you are better off reporting this to the downstream distributor of the
> pre-built binary you are using, as upstream is not the problem.  I'm
> marking this as not a bug to avoid it staying open forever in our bug
> database, but feel free to reply to this with further details, and we
> can reopen it if you provide more details that points back at the
> upstream code doing something that interferes with proper long file name
> usage.
>
> --
> Eric Blake, Principal Software Engineer
> Red Hat, Inc.           +1-919-301-3226
> Virtualization:  qemu.org | libvirt.org
>
>
>
>
> ---------- Forwarded message ----------
> From: Chris Kalish <chris <at> thekalishes.com>
> To: CP Bugs <bug-coreutils <at> gnu.org>
> Cc:
> Bcc:
> Date: Fri, 25 Jan 2019 12:14:42 -0500
> Subject: Small bug in cp (for win64)
> Hi, guys ... I use cp to backup source systems to an external drive.  It
> works great (and the --update=number function is a key differentiator).
> However, I noticed that (for NTFS file  systems) long directory names
> (\abc\def\ghi\jkl\mno\pqrstuvwxyz\blahblah\longlonglongdirectorystring) are
> not supported (they throw "no such file or directory errors").  I assume
> you're making an assumption on a max static var size (i.e.,
> szDirectory[100]) ... can you either up that allocation or malloc() the
> memory to the input string?  I believe the NTFS fully-cascading filename
> limit is 32,000 characters.
>
> (actual example):
>
> *cp: cannot create regular file
> `C:\\XDrive\\temp\\delete\\KalishCMirror\\XDrive\\Temp\\delete\\GDGBackups\\Presariokids\\LastBackup\\EBackups\\Devin\\DAS\\Documents\\eclipseWS\\2cov2cor\\.settings\\org.eclipse.cdt.managedbuilder.core.prefs':
> No such file or directory*
>
>
> It will copy if I subst the directory name into a virtual drive letter,
> but that is not a reasonable solution to recusing my entire drive.
>
> Thanks!
>
> -chris
>
>

[Message part 2 (text/html, inline)]

Message #18 received at 34199 <at> debbugs.gnu.org (full text, mbox):

From: Chris Kalish <chris <at> thekalishes.com>
To: 34199 <at> debbugs.gnu.org
Subject: Re: bug#34199: closed (Re: bug#34199: Small bug in cp (for win64))
Date: Sat, 9 Feb 2019 14:10:59 -0500

[Message part 1 (text/plain, inline)]

Hi, guys ... any word on this? (see below)

-chris


On Sun, Jan 27, 2019 at 9:38 PM Chris Kalish <chris <at> thekalishes.com> wrote:

> Hmmm ... not sure of the distribution, but the help file pointed me at
> this address:
>
> C:\> cp --version
>
> cp (GNU coreutils) 5.3.0
>
> Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering.
>
>
> Copyright (C) 2005 Free Software Foundation, Inc.
>
> This is free software; see the source for copying conditions.  There is NO
>
> warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
>
>
> C:\> cp --help
> .... ..... .... ....
> As a special case, cp makes a backup of SOURCE when the force and backup
> options are given and SOURCE and DEST are the same name for an existing,
> regular file.
>
> Report bugs to <bug-coreutils <at> gnu.org>.
>
>
> -chris
>
> On Fri, Jan 25, 2019 at 3:35 PM GNU bug Tracking System <
> help-debbugs <at> gnu.org> wrote:
>
>> Your bug report
>>
>> #34199: Small bug in cp (for win64)
>>
>> which was filed against the coreutils package, has been closed.
>>
>> The explanation is attached below, along with your original report.
>> If you require more details, please reply to 34199 <at> debbugs.gnu.org.
>>
>> --
>> 34199: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=34199
>> GNU Bug Tracking System
>> Contact help-debbugs <at> gnu.org with problems
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Eric Blake <eblake <at> redhat.com>
>> To: Chris Kalish <chris <at> thekalishes.com>, 34199-done <at> debbugs.gnu.org
>> Cc:
>> Bcc:
>> Date: Fri, 25 Jan 2019 14:33:57 -0600
>> Subject: Re: bug#34199: Small bug in cp (for win64)
>> tag 34199 notabug
>> thanks
>>
>> On 1/25/19 11:14 AM, Chris Kalish wrote:
>> > Hi, guys ... I use cp to backup source systems to an external drive.  It
>> > works great (and the --update=number function is a key differentiator).
>> > However, I noticed that (for NTFS file  systems) long directory names
>> > (\abc\def\ghi\jkl\mno\pqrstuvwxyz\blahblah\longlonglongdirectorystring)
>> are
>> > not supported (they throw "no such file or directory errors").  I assume
>> > you're making an assumption on a max static var size (i.e.,
>> > szDirectory[100]) ... can you either up that allocation or malloc() the
>> > memory to the input string?  I believe the NTFS fully-cascading filename
>> > limit is 32,000 characters.
>>
>> You are incorrect about upstream cp itself having a fixed-size buffer;
>> however, the underlying operating system and/or file system may have
>> limits that you are tripping over.  I know that on Windows, whether an
>> application was compiled against ASCII vs. Unicode functions from libc
>> can make a difference on the maximum file name that program can use.
>>
>> However, I can also state that at least the cygwin build of cp (from
>> cygwin.com) does not suffer from the limitation on Windows, and it uses
>> the same upstream cp sources.  So I assume that you are using a
>> pre-built cp from some other site than cygwin, and that the limitation
>> is inherent to the porting job of whatever you are using.  Therefore,
>> you are better off reporting this to the downstream distributor of the
>> pre-built binary you are using, as upstream is not the problem.  I'm
>> marking this as not a bug to avoid it staying open forever in our bug
>> database, but feel free to reply to this with further details, and we
>> can reopen it if you provide more details that points back at the
>> upstream code doing something that interferes with proper long file name
>> usage.
>>
>> --
>> Eric Blake, Principal Software Engineer
>> Red Hat, Inc.           +1-919-301-3226
>> Virtualization:  qemu.org | libvirt.org
>>
>>
>>
>>
>> ---------- Forwarded message ----------
>> From: Chris Kalish <chris <at> thekalishes.com>
>> To: CP Bugs <bug-coreutils <at> gnu.org>
>> Cc:
>> Bcc:
>> Date: Fri, 25 Jan 2019 12:14:42 -0500
>> Subject: Small bug in cp (for win64)
>> Hi, guys ... I use cp to backup source systems to an external drive.  It
>> works great (and the --update=number function is a key differentiator).
>> However, I noticed that (for NTFS file  systems) long directory names
>> (\abc\def\ghi\jkl\mno\pqrstuvwxyz\blahblah\longlonglongdirectorystring) are
>> not supported (they throw "no such file or directory errors").  I assume
>> you're making an assumption on a max static var size (i.e.,
>> szDirectory[100]) ... can you either up that allocation or malloc() the
>> memory to the input string?  I believe the NTFS fully-cascading filename
>> limit is 32,000 characters.
>>
>> (actual example):
>>
>> *cp: cannot create regular file
>> `C:\\XDrive\\temp\\delete\\KalishCMirror\\XDrive\\Temp\\delete\\GDGBackups\\Presariokids\\LastBackup\\EBackups\\Devin\\DAS\\Documents\\eclipseWS\\2cov2cor\\.settings\\org.eclipse.cdt.managedbuilder.core.prefs':
>> No such file or directory*
>>
>>
>> It will copy if I subst the directory name into a virtual drive letter,
>> but that is not a reasonable solution to recusing my entire drive.
>>
>> Thanks!
>>
>> -chris
>>
>>

[Message part 2 (text/html, inline)]

Message #21 received at 34199 <at> debbugs.gnu.org (full text, mbox):

From: Bob Proulx <bob <at> proulx.com>
To: Chris Kalish <chris <at> thekalishes.com>
Cc: 34199 <at> debbugs.gnu.org
Subject: Re: bug#34199: closed (Re: bug#34199: Small bug in cp (for win64))
Date: Sun, 10 Feb 2019 15:17:39 -0700

Chris Kalish wrote:
> Hmmm ... not sure of the distribution, but the help file pointed me at this
> address:

> C:\> cp --version
> cp (GNU coreutils) 5.3.0

I always hate it when I am on your side of things and upstream says
this to me.  But here I am on the other side and going to say almost
exactly the thing I hate to hear.

Coreutils 5.3.0 was released on 2005-01-08 and today is 2019-02-10
making that version of the program you are running 14 years old!  That
is a very long time ago.  Since you are running on MS-Windows I will
say that was probably five whole versions of Microsoft ago!  It would
not be practically possible for most of us to recreate that version on
MS-Windows-XP of that era.  This makes it difficult to impossible to
do anything about even if we had an alive build system from 2005 still
running.  Plus here we are concerned about software on free(dom)
licensed platforms and Microsoft is a closed source proprietary
platform.  That was always supported by other teams doing ports to
non-free operating systems.  What's a developer to do? :-(

Perhaps I should ignore all of the years and simply say, yes, that is
a bug.  (I don't really know.  But I will say it.  Searching the
changelogs will show that 5.3.0 did introduce a number of bugs.)  And
we have fixed it!  The new version is v8.30 and that bug is fixed.
Eric reported that it was not a problem for Cygwin on MS-Windows.
Please upgrade to it and confirm with us that it is working for you
there.  Maybe that would be a less unpleasant to hear? :-)

> C:\> cp --help
> Report bugs to <bug-coreutils <at> gnu.org>.

We are happy to have bugs reported here.  But often in ports the
behavior is dependent upon the port environment.  That is outside of
our control.

Please do upgrade to a newer version.  Cygwin tends to be the most
capable version.  Although there are other ports too.  We would
appreciate hearing about how this worked out for you regardless.

And maybe along the way you might consider upgrading to a free(dom)
software licensed operating system?  Then you would have upgrades
available by default. :-)

Bob

Message #24 received at 34199 <at> debbugs.gnu.org (full text, mbox):

From: Chris Kalish <chris <at> thekalishes.com>
To: Bob Proulx <bob <at> proulx.com>
Cc: 34199 <at> debbugs.gnu.org
Subject: Re: bug#34199: closed (Re: bug#34199: Small bug in cp (for win64))
Date: Tue, 12 Feb 2019 13:14:21 -0500

[Message part 1 (text/plain, inline)]

Hi, Bob ... thanks so much for the info! (very entertaining!) 🤓

You were right ... I had installed the newer version of Coreutils from
cygwin and that works better (but still has some NTFS-related bugs, I
think).

New version:

C:\temp\x\x>cp --version
cp (GNU coreutils) 8.26
Packaged by Cygwin (8.26-2)
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html
>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

Written by Torbjorn Granlund, David MacKenzie, and Jim Meyering.



1. Does not seem to like directories or filenames with European characters:


*Windows Copy (works):*

C:\temp\x\x>copy /b "t:\xdrive\SameTimeChatTranscripts\CN=Linda
Bañuelos,OU=Corporate,O=GeneralElectric" .

t:\xdrive\SameTimeChatTranscripts\CN=Linda
Bañuelos,OU=Corporate,O=GeneralElectric\ChatHistory.properties

        1 file(s) copied.


*CP (does not work):*

C:\temp\x\x>cp "t:\xdrive\SameTimeChatTranscripts\CN=Linda
Bañuelos,OU=Corporate,O=GeneralElectric" .

cp: cannot stat '"t:\xdrive\SameTimeChatTranscripts\CN=Linda
Ba'$'\303\261''uelos,OU=Corporate,O=GeneralElectric"': No such file or
directory


There seems to be a problem with long directory names, still, but I'm still
working on isolating that.  Can you give any insight into the above?

Thanks!!!!

-chris

On Sun, Feb 10, 2019 at 5:17 PM Bob Proulx <bob <at> proulx.com> wrote:

> Chris Kalish wrote:
> > Hmmm ... not sure of the distribution, but the help file pointed me at
> this
> > address:
>
> > C:\> cp --version
> > cp (GNU coreutils) 5.3.0
>
> I always hate it when I am on your side of things and upstream says
> this to me.  But here I am on the other side and going to say almost
> exactly the thing I hate to hear.
>
> Coreutils 5.3.0 was released on 2005-01-08 and today is 2019-02-10
> making that version of the program you are running 14 years old!  That
> is a very long time ago.  Since you are running on MS-Windows I will
> say that was probably five whole versions of Microsoft ago!  It would
> not be practically possible for most of us to recreate that version on
> MS-Windows-XP of that era.  This makes it difficult to impossible to
> do anything about even if we had an alive build system from 2005 still
> running.  Plus here we are concerned about software on free(dom)
> licensed platforms and Microsoft is a closed source proprietary
> platform.  That was always supported by other teams doing ports to
> non-free operating systems.  What's a developer to do? :-(
>
> Perhaps I should ignore all of the years and simply say, yes, that is
> a bug.  (I don't really know.  But I will say it.  Searching the
> changelogs will show that 5.3.0 did introduce a number of bugs.)  And
> we have fixed it!  The new version is v8.30 and that bug is fixed.
> Eric reported that it was not a problem for Cygwin on MS-Windows.
> Please upgrade to it and confirm with us that it is working for you
> there.  Maybe that would be a less unpleasant to hear? :-)
>
> > C:\> cp --help
> > Report bugs to <bug-coreutils <at> gnu.org>.
>
> We are happy to have bugs reported here.  But often in ports the
> behavior is dependent upon the port environment.  That is outside of
> our control.
>
> Please do upgrade to a newer version.  Cygwin tends to be the most
> capable version.  Although there are other ports too.  We would
> appreciate hearing about how this worked out for you regardless.
>
> And maybe along the way you might consider upgrading to a free(dom)
> software licensed operating system?  Then you would have upgrades
> available by default. :-)
>
> Bob
>

[Message part 2 (text/html, inline)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.