GNU bug report logs - #62572
cp --no-clobber behavior has changed

Previous Next

To reply to this bug, email your comments to 62572 AT debbugs.gnu.org.

Toggle the display of automated, internal messages from the tracker.

Message #5 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Alberto Salvia Novella <es20490446e <at> gmail.com>
To: bug-coreutils <at> gnu.org
Subject: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 19:01:09 +0200

[Message part 1 (text/plain, inline)]

In the past if you did:
cp --no-clobber $in $out

And "out" existed, "cp" exited with 0. But now, with coreutils 9.2, it
exists with 1.

Is this on purpose?

(When replying include my email in the field "to", as I'm not subscribed to
this list)

[Message part 2 (text/html, inline)]

Message #8 received at submit <at> debbugs.gnu.org (full text, mbox):

From: Sam James <sam <at> gentoo.org>
To: Alberto Salvia Novella <es20490446e <at> gmail.com>
Cc: bug-coreutils <at> gnu.org, 62572 <at> debbugs.gnu.org,
 Eli Schwartz <eschwartz93 <at> gmail.com>
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 19:30:47 +0100

[Message part 1 (text/plain, inline)]

Alberto Salvia Novella <es20490446e <at> gmail.com> writes:

> In the past if you did:
> cp --no-clobber $in $out
>
> And "out" existed, "cp" exited with 0. But now, with coreutils 9.2, it
> exists with 1.
>
> Is this on purpose?
>
> (When replying include my email in the field "to", as I'm not subscribed to
> this list)

We hit this in Gentoo at https://bugs.gentoo.org/902751. We concluded
the usage in the ebuild ("build script") was wrong, given that the
coreutils-9.2 NEWS says:
> 'cp -n' and 'mv -n' now exit with nonzero status if they skip their
> action because the destination exists, and likewise for 'cp -i',
> 'ln -i', and 'mv -i' when the user declines.  (POSIX specifies
> this for 'cp -i' and 'mv -i'.)

It still was a bit unexpected though given POSIX doesn't specify -n.

[signature.asc (application/pgp-signature, inline)]

Message #14 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Alberto Salvia Novella <es20490446e <at> gmail.com>
Cc: 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 13:01:07 -0700

On 2023-03-31 10:01, Alberto Salvia Novella wrote:
> Is this on purpose?

Yes, part of the idea was to let shell programmers easily test whether 
cp successfully copied the data. Having cp -i conform to POSIX was a 
lesser consideration, though it's a bit nicer if -n and -i are somewhat 
consistent.

For what it's worth, the old behavior wasn't documented and the new 
behavior is.

Message #17 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Sven Joachim <svenjoac <at> gmx.de>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 22:37:58 +0200

On 2023-03-31 13:01 -0700, Paul Eggert wrote:

> On 2023-03-31 10:01, Alberto Salvia Novella wrote:
>> Is this on purpose?
>
> Yes, part of the idea was to let shell programmers easily test whether
> cp successfully copied the data.

By making them stop using the '-n' option, since they cannot rely on the
exit code anyway?

> Having cp -i conform to POSIX was a
> lesser consideration, though it's a bit nicer if -n and -i are
> somewhat consistent.

It is not so nice that this is probably going to break several dozen
packages in Debian alone. :-(

Cheers,
       Sven

Message #20 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Alberto Salvia Novella <es20490446e <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 22:53:31 +0200

[Message part 1 (text/plain, inline)]

https://youtu.be/o_kh1_gOkwk

On Fri, 31 Mar 2023 at 22:01, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> On 2023-03-31 10:01, Alberto Salvia Novella wrote:
> > Is this on purpose?
>
> Yes, part of the idea was to let shell programmers easily test whether
> cp successfully copied the data. Having cp -i conform to POSIX was a
> lesser consideration, though it's a bit nicer if -n and -i are somewhat
> consistent.
>
> For what it's worth, the old behavior wasn't documented and the new
> behavior is.
>

[Message part 2 (text/html, inline)]

Message #23 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 14:15:47 -0700

On 2023-03-31 13:37, Sven Joachim wrote:
> On 2023-03-31 13:01 -0700, Paul Eggert wrote:
> 
>> part of the idea was to let shell programmers easily test whether
>> cp successfully copied the data.
> 
> By making them stop using the '-n' option, since they cannot rely on the
> exit code anyway?

Portable code could not rely on the exit status anyway, as FreeBSD cp 
agrees with the new GNU behavior, not the old. See:

https://bugs.gnu.org/61105

It was a messy situation where there was no perfect solution. That being 
said, there is an advantage of consistency with FreeBSD, and the new GNU 
behavior does avoid the race mentioned in Bug#61105.

Message #26 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 22:32:12 +0100

On 31/03/2023 22:15, Paul Eggert wrote:
> On 2023-03-31 13:37, Sven Joachim wrote:
>> On 2023-03-31 13:01 -0700, Paul Eggert wrote:
>>
>>> part of the idea was to let shell programmers easily test whether
>>> cp successfully copied the data.
>>
>> By making them stop using the '-n' option, since they cannot rely on the
>> exit code anyway?
> 
> Portable code could not rely on the exit status anyway, as FreeBSD cp
> agrees with the new GNU behavior, not the old. See:
> 
> https://bugs.gnu.org/61105
> 
> It was a messy situation where there was no perfect solution. That being
> said, there is an advantage of consistency with FreeBSD, and the new GNU
> behavior does avoid the race mentioned in Bug#61105.

Perhaps we should support:
  --no-clobber[={skip, fail (default)}]

so then users can at least easily change -n to --no-clobber=skip
to get the old behavior?

An alternative would be to augment the --update option to support:
  --update[={none, older (default)}]
where --update=none would be the equivalent of the old -n behavior.

Perhaps we should also diagnose files skipped in the -n fail case,
to make it easier for users to see what the issue is.

cheers,
Pádraig

Message #29 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 31 Mar 2023 16:29:03 -0700

On 2023-03-31 14:32, Pádraig Brady wrote:

> Perhaps we should support:
>    --no-clobber[={skip, fail (default)}]
> 
> so then users can at least easily change -n to --no-clobber=skip
> to get the old behavior?
> 
> An alternative would be to augment the --update option to support:
>    --update[={none, older (default)}]
> where --update=none would be the equivalent of the old -n behavior.

The latter sounds a bit better but I suppose either would work. We could 
generalize it a bit further, e.g.:

  --skip-diagnose[={yes,no}]
     Whether to diagnose a copying action being skipped.
  --skip-fail[={yes,no}]
     Whether exit status should be 1 when skipping a copying action.

Presumably similar options would apply to ln and mv.

All these extra options might be overkill, though.


> Perhaps we should also diagnose files skipped in the -n fail case,
> to make it easier for users to see what the issue is.

FreeBSD cp -n doesn't diagnose, and GNU cp -n has never diagnosed, so 
it's probably better to leave sleeping dogs lie.

Message #32 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Christoph Anton Mitterer <calestyo <at> scientia.org>
To: 62572 <at> debbugs.gnu.org
Subject: Re: cp --no-clobber behavior has changed
Date: Sat, 01 Apr 2023 02:05:15 +0200

In principle I have no strong opinion on what the behaviour should be.

But if one strictly follows the POSIX wording:

>    The following exit values shall be returned:
>     0
>        All input files were [copied/moved] successfully.
>    >0
>        An error occurred.

The change seems to make sense, as one could argue that the above ain't
the case with -n if the destination already exists.

It's however open to debate whether --no-clobber - were it in POSIX -
would have fallen into that spirit.


But at least as of corutils 9.1 's cp info page it was documented as:
>An exit status of zero indicates success, and a nonzero value
>indicates failure.

together with:

> ‘-n’
> ‘--no-clobber’
>     Do not overwrite an existing file; silently do nothing instead.
>     This option overrides a previous ‘-i’ option.  This option is
>     mutually exclusive with ‘-b’ or ‘--backup’ option.

that doesn't seem like failure to me.

So it kinda breaks the "promise" made by the documentation.


I like Pádraig's idea in message #26... but no idea whether it should
be more POSIX like (thus non-zero when nothing was copied) or more
backwards compatible.


Cheers,
Chris.

Message #35 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Alberto Salvia Novella <es20490446e <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Pádraig Brady <P <at> draigbrady.com>,
 Sven Joachim <svenjoac <at> gmx.de>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 03:36:48 +0200

[Message part 1 (text/plain, inline)]

I get the impression that right now --no-clover is optimized for the less
common scenarios, while making it less useful for the common ones.

Also --update isn't a substitute of --no-clover. As --no-clover is for
copying when the file is missing, not when it isn't updated.

For example imagine that I have a config template, and a script copies the
template only if it is missing using --no-clover.

If I did the same with --update it could happen the following: the package
that provides the template updates, then --update will override the config
even if it exists, just because the source file is now newer. No good.

So right now the only option that I have is to avoid both --no-clover and
--update all together, and to test for the file existence separately. So
totally useless.

On Sat, 1 Apr 2023 at 01:29, Paul Eggert <eggert <at> cs.ucla.edu> wrote:

> On 2023-03-31 14:32, Pádraig Brady wrote:
>
> > Perhaps we should support:
> >    --no-clobber[={skip, fail (default)}]
> >
> > so then users can at least easily change -n to --no-clobber=skip
> > to get the old behavior?
> >
> > An alternative would be to augment the --update option to support:
> >    --update[={none, older (default)}]
> > where --update=none would be the equivalent of the old -n behavior.
>
> The latter sounds a bit better but I suppose either would work. We could
> generalize it a bit further, e.g.:
>
>    --skip-diagnose[={yes,no}]
>       Whether to diagnose a copying action being skipped.
>    --skip-fail[={yes,no}]
>       Whether exit status should be 1 when skipping a copying action.
>
> Presumably similar options would apply to ln and mv.
>
> All these extra options might be overkill, though.
>
>
> > Perhaps we should also diagnose files skipped in the -n fail case,
> > to make it easier for users to see what the issue is.
>
> FreeBSD cp -n doesn't diagnose, and GNU cp -n has never diagnosed, so
> it's probably better to leave sleeping dogs lie.
>

[Message part 2 (text/html, inline)]

Message #38 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Alberto Salvia Novella <es20490446e <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Pádraig Brady <P <at> draigbrady.com>,
 Sven Joachim <svenjoac <at> gmx.de>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 03:41:10 +0200

[Message part 1 (text/plain, inline)]

Or use:
cp --no-clover $in $out || true

But again, surprising behavior. Just a new special case to memorize.

On Sat, 1 Apr 2023 at 03:36, Alberto Salvia Novella <es20490446e <at> gmail.com>
wrote:

> I get the impression that right now --no-clover is optimized for the less
> common scenarios, while making it less useful for the common ones.
>
> Also --update isn't a substitute of --no-clover. As --no-clover is for
> copying when the file is missing, not when it isn't updated.
>
> For example imagine that I have a config template, and a script copies the
> template only if it is missing using --no-clover.
>
> If I did the same with --update it could happen the following: the package
> that provides the template updates, then --update will override the config
> even if it exists, just because the source file is now newer. No good.
>
> So right now the only option that I have is to avoid both --no-clover and
> --update all together, and to test for the file existence separately. So
> totally useless.
>
> On Sat, 1 Apr 2023 at 01:29, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
>
>> On 2023-03-31 14:32, Pádraig Brady wrote:
>>
>> > Perhaps we should support:
>> >    --no-clobber[={skip, fail (default)}]
>> >
>> > so then users can at least easily change -n to --no-clobber=skip
>> > to get the old behavior?
>> >
>> > An alternative would be to augment the --update option to support:
>> >    --update[={none, older (default)}]
>> > where --update=none would be the equivalent of the old -n behavior.
>>
>> The latter sounds a bit better but I suppose either would work. We could
>> generalize it a bit further, e.g.:
>>
>>    --skip-diagnose[={yes,no}]
>>       Whether to diagnose a copying action being skipped.
>>    --skip-fail[={yes,no}]
>>       Whether exit status should be 1 when skipping a copying action.
>>
>> Presumably similar options would apply to ln and mv.
>>
>> All these extra options might be overkill, though.
>>
>>
>> > Perhaps we should also diagnose files skipped in the -n fail case,
>> > to make it easier for users to see what the issue is.
>>
>> FreeBSD cp -n doesn't diagnose, and GNU cp -n has never diagnosed, so
>> it's probably better to leave sleeping dogs lie.
>>
>

[Message part 2 (text/html, inline)]

Message #41 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Alberto Salvia Novella <es20490446e <at> gmail.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Pádraig Brady <P <at> draigbrady.com>,
 Sven Joachim <svenjoac <at> gmx.de>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 16:46:55 +0200

[Message part 1 (text/plain, inline)]

Also there's now a bigger problem: that you cannot tell when the copy
failed because the file exists, or because any other reason.

People will just use:
cp --no-clover $in $out || true

But if it fails for any other reason, cross your fingers.

Hence now the option, in practice, is useless. Nobody should be using it.


On Sat, 1 Apr 2023 at 03:41, Alberto Salvia Novella <es20490446e <at> gmail.com>
wrote:

> Or use:
> cp --no-clover $in $out || true
>
> But again, surprising behavior. Just a new special case to memorize.
>
> On Sat, 1 Apr 2023 at 03:36, Alberto Salvia Novella <es20490446e <at> gmail.com>
> wrote:
>
>> I get the impression that right now --no-clover is optimized for the less
>> common scenarios, while making it less useful for the common ones.
>>
>> Also --update isn't a substitute of --no-clover. As --no-clover is for
>> copying when the file is missing, not when it isn't updated.
>>
>> For example imagine that I have a config template, and a script copies
>> the template only if it is missing using --no-clover.
>>
>> If I did the same with --update it could happen the following: the
>> package that provides the template updates, then --update will override the
>> config even if it exists, just because the source file is now newer. No
>> good.
>>
>> So right now the only option that I have is to avoid both --no-clover and
>> --update all together, and to test for the file existence separately. So
>> totally useless.
>>
>> On Sat, 1 Apr 2023 at 01:29, Paul Eggert <eggert <at> cs.ucla.edu> wrote:
>>
>>> On 2023-03-31 14:32, Pádraig Brady wrote:
>>>
>>> > Perhaps we should support:
>>> >    --no-clobber[={skip, fail (default)}]
>>> >
>>> > so then users can at least easily change -n to --no-clobber=skip
>>> > to get the old behavior?
>>> >
>>> > An alternative would be to augment the --update option to support:
>>> >    --update[={none, older (default)}]
>>> > where --update=none would be the equivalent of the old -n behavior.
>>>
>>> The latter sounds a bit better but I suppose either would work. We could
>>> generalize it a bit further, e.g.:
>>>
>>>    --skip-diagnose[={yes,no}]
>>>       Whether to diagnose a copying action being skipped.
>>>    --skip-fail[={yes,no}]
>>>       Whether exit status should be 1 when skipping a copying action.
>>>
>>> Presumably similar options would apply to ln and mv.
>>>
>>> All these extra options might be overkill, though.
>>>
>>>
>>> > Perhaps we should also diagnose files skipped in the -n fail case,
>>> > to make it easier for users to see what the issue is.
>>>
>>> FreeBSD cp -n doesn't diagnose, and GNU cp -n has never diagnosed, so
>>> it's probably better to leave sleeping dogs lie.
>>>
>>

[Message part 2 (text/html, inline)]

Message #44 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Alberto Salvia Novella <es20490446e <at> gmail.com>,
 Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Sven Joachim <svenjoac <at> gmx.de>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 16:43:23 +0100

On 01/04/2023 15:46, Alberto Salvia Novella wrote:
> Also there's now a bigger problem: that you cannot tell when the copy
> failed because the file exists, or because any other reason.
> 
> People will just use:
> cp --no-clover $in $out || true
> 
> But if it fails for any other reason, cross your fingers.
> 
> Hence now the option, in practice, is useless. Nobody should be using it.

Well the current thinking is -n is useful to fail with unexpected existing files.

For the functionality you want of only updating non existent files,
one should use the proposed --update=none new option.

It's worth mentioning that this being a new option
impacts the portability of scripts that use it.

cheers,
Pádraig

Message #47 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 16:44:37 +0100

[Message part 1 (text/plain, inline)]

On 01/04/2023 00:29, Paul Eggert wrote:
> On 2023-03-31 14:32, Pádraig Brady wrote:
> 
>> Perhaps we should support:
>>     --no-clobber[={skip, fail (default)}]
>>
>> so then users can at least easily change -n to --no-clobber=skip
>> to get the old behavior?
>>
>> An alternative would be to augment the --update option to support:
>>     --update[={none, older (default)}]
>> where --update=none would be the equivalent of the old -n behavior.
> 
> The latter sounds a bit better but I suppose either would work. We could
> generalize it a bit further, e.g.:
> 
>     --skip-diagnose[={yes,no}]
>        Whether to diagnose a copying action being skipped.
>     --skip-fail[={yes,no}]
>        Whether exit status should be 1 when skipping a copying action.
> 
> Presumably similar options would apply to ln and mv.
> 
> All these extra options might be overkill, though.
> 
> 
>> Perhaps we should also diagnose files skipped in the -n fail case,
>> to make it easier for users to see what the issue is.
> 
> FreeBSD cp -n doesn't diagnose, and GNU cp -n has never diagnosed, so
> it's probably better to leave sleeping dogs lie.

OK first stab at --update=none support is attached.

cheers,
Pádraig

[copy--update-none-patch (text/plain, attachment)]

Message #50 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Alberto Salvia Novella <es20490446e <at> gmail.com>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>, Sven Joachim <svenjoac <at> gmx.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 20:14:07 +0200

[Message part 1 (text/plain, inline)]

Maybe simpler:

-m --missing
Only copy non existing files.

On Sat, 1 Apr 2023 at 17:44, Pádraig Brady <P <at> draigbrady.com> wrote:

> On 01/04/2023 00:29, Paul Eggert wrote:
> > On 2023-03-31 14:32, Pádraig Brady wrote:
> >
> >> Perhaps we should support:
> >>     --no-clobber[={skip, fail (default)}]
> >>
> >> so then users can at least easily change -n to --no-clobber=skip
> >> to get the old behavior?
> >>
> >> An alternative would be to augment the --update option to support:
> >>     --update[={none, older (default)}]
> >> where --update=none would be the equivalent of the old -n behavior.
> >
> > The latter sounds a bit better but I suppose either would work. We could
> > generalize it a bit further, e.g.:
> >
> >     --skip-diagnose[={yes,no}]
> >        Whether to diagnose a copying action being skipped.
> >     --skip-fail[={yes,no}]
> >        Whether exit status should be 1 when skipping a copying action.
> >
> > Presumably similar options would apply to ln and mv.
> >
> > All these extra options might be overkill, though.
> >
> >
> >> Perhaps we should also diagnose files skipped in the -n fail case,
> >> to make it easier for users to see what the issue is.
> >
> > FreeBSD cp -n doesn't diagnose, and GNU cp -n has never diagnosed, so
> > it's probably better to leave sleeping dogs lie.
>
> OK first stab at --update=none support is attached.
>
> cheers,
> Pádraig
>

[Message part 2 (text/html, inline)]

Message #53 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 1 Apr 2023 15:44:53 -0700

On 2023-04-01 08:44, Pádraig Brady wrote:

> OK first stab at --update=none support is attached.

Thanks, some comments:


> +  /* Always Overwrite.  */
> +  UPDATE_OVERWRITE,

Might be better to call this UPDATE_ALL as it doesn't overwrite if you 
use cp -l or -s or (in some cases) --preserve=links.

> +static char const *const update_type_string[] =
> +{
> +  "none", "older", NULL
> +};

Perhaps there should also be an --update=all, which is the same as no 
update option at all? It would presumably override previous --update 
suboptions.

>    -n, --no-clobber             do not overwrite an existing file (overrides\n\
> -                                 a previous -i option)\n\
> +                                 a -u or previous -i option)\n\

Shouldn't -n override only previous -u or --update options? I thought 
these options were processed left to right, with later overriding earlier.


> +  -u                           only update when the SOURCE file is newer\n\
>                                   than the destination file or when the\n\
>                                   destination file is missing\n\
> +  --update[=UPDATE_MODE]       like -u, but support skipping existing files;\n\
> +                                 UPDATE_MODE={none,older(default)}\n\

This might be clearer if we describe --update first, and then say that 
-u is equivalent to plain --update.


> +  if (make_backups && x.interactive == I_ALWAYS_SKIP)
> +    {
> +      error (0, 0,
> +             _("options --backup and --update=none are mutually exclusive"));
> +      usage (EXIT_FAILURE);
> +    }

Why is this needed? --backup and -u are not mutually exclusive now. 
Admittedly --backup is useless when -u is given, but it seems a bit 
harsh to error out.

Message #56 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sun, 2 Apr 2023 11:10:19 +0100

On 01/04/2023 23:44, Paul Eggert wrote:
> On 2023-04-01 08:44, Pádraig Brady wrote:
> 
>> OK first stab at --update=none support is attached.
> 
> Thanks, some comments:
> 
> 
>> +  /* Always Overwrite.  */
>> +  UPDATE_OVERWRITE,
> 
> Might be better to call this UPDATE_ALL as it doesn't overwrite if you
> use cp -l or -s or (in some cases) --preserve=links.

+1

>> +static char const *const update_type_string[] =
>> +{
>> +  "none", "older", NULL
>> +};
> 
> Perhaps there should also be an --update=all, which is the same as no
> update option at all? It would presumably override previous --update
> suboptions.

+1

>>     -n, --no-clobber             do not overwrite an existing file (overrides\n\
>> -                                 a previous -i option)\n\
>> +                                 a -u or previous -i option)\n\
> 
> Shouldn't -n override only previous -u or --update options? I thought
> these options were processed left to right, with later overriding earlier.


Well I was just keeping the existing global treatment of -n wrt -u, as per:
https://github.com/coreutils/coreutils/commit/7e244891b
-n does seem like more of a global "make sure you don't overwrite anything" setting
(which is analogous to the shell "noclobber" option BTW which also induces a failure when triggered).
I'll think a bit more about this.

>> +  -u                           only update when the SOURCE file is newer\n\
>>                                    than the destination file or when the\n\
>>                                    destination file is missing\n\
>> +  --update[=UPDATE_MODE]       like -u, but support skipping existing files;\n\
>> +                                 UPDATE_MODE={none,older(default)}\n\
> 
> This might be clearer if we describe --update first, and then say that
> -u is equivalent to plain --update.

+1

>> +  if (make_backups && x.interactive == I_ALWAYS_SKIP)
>> +    {
>> +      error (0, 0,
>> +             _("options --backup and --update=none are mutually exclusive"));
>> +      usage (EXIT_FAILURE);
>> +    }
> 
> Why is this needed? --backup and -u are not mutually exclusive now.
> Admittedly --backup is useless when -u is given, but it seems a bit
> harsh to error out.

Yes --backup is redundant with -n or --update=none.
We already disallowed --backup with -n,
so I was just being consistent with disallowing --backup with --update=none.
I'll remove this new restriction.

thanks for the review,
Pádraig

Message #59 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Sven Joachim <svenjoac <at> gmx.de>
Cc: Alberto Salvia Novella <es20490446e <at> gmail.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Thu, 6 Apr 2023 15:50:08 +0100

[Message part 1 (text/plain, inline)]

Take 2 attached.

cheers,
Pádraig

[copy--update-none-patch (text/plain, attachment)]

Message #62 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Thorsten Glaser <tg <at> mirbsd.de>
To: 62572 <at> debbugs.gnu.org
Subject: Make the errorlevel distinct perhaps?
Date: Sun, 12 Nov 2023 01:03:17 +0000 (UTC)

Hi,

from https://bugs.debian.org/1055694 where this broke things
where files were deliberately not overwritten (klibc installs
its utils but only those busybox (when used) does not provide).

In this case not copying the file is absolutely not an error.

Perhaps do it like diff(1) and use distinct errorlevels for
some files were not copied in -n mode vs. there was an error
in those that *were* copied. Then document that in the manpage
as extension and reach out to FreeBSD to follow.

bye,
//mirabilos
-- 
(gnutls can also be used, but if you are compiling lynx for your own use,
there is no reason to consider using that package)
	-- Thomas E. Dickey on the Lynx mailing list, about OpenSSL

Message #65 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Michael Stone <mstone <at> debian.org>
To: 62572 <at> debbugs.gnu.org
Cc: 1058752 <at> bugs.debian.org
Subject: Re: cp --no-clobber behavior has changed
Date: Fri, 15 Dec 2023 10:56:03 -0500

I tend to think this was a serious mistake: it breaks the behavior of 
existing scripts with no deprecation period. A stated advantage is 
better compatibility with freebsd, but I don't understand why that is 
more desirable than compatibility with all deployed gnu/linux systems? I 
also don't think it's sufficient to try to lawyer out by saying that the 
current behavior was undocumented: the previous documentation said that 
-n would "silently do nothing" and that the return code would be zero on 
success. Logically, unless cp fails to "do nothing", it should exit with 
a zero code.

Such a drastic change in behavior demands a new flag, not a radical 
repurposing of a widely used existing flag.

I was hoping to see more action on this bug, but that hasn't happened. 
I'm not sure I see a way forward for debian other than reverting to the 
old behavior. I am reluctant to do so as that will likely lead to 
divergent behavior between distributions, but breaking scripts without a 
compelling reason is also not good. I would encourage coreutils to 
reconsider the change and finding a non-breaking way forward.

Michael Stone

Message #68 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Michael Stone <mstone <at> debian.org>, 62572 <at> debbugs.gnu.org
Cc: 1058752 <at> bugs.debian.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 15 Dec 2023 18:33:00 +0000

On 15/12/2023 15:56, Michael Stone wrote:
> I tend to think this was a serious mistake: it breaks the behavior of
> existing scripts with no deprecation period. A stated advantage is
> better compatibility with freebsd, but I don't understand why that is
> more desirable than compatibility with all deployed gnu/linux systems? I
> also don't think it's sufficient to try to lawyer out by saying that the
> current behavior was undocumented: the previous documentation said that
> -n would "silently do nothing" and that the return code would be zero on
> success. Logically, unless cp fails to "do nothing", it should exit with
> a zero code.
> 
> Such a drastic change in behavior demands a new flag, not a radical
> repurposing of a widely used existing flag.
> 
> I was hoping to see more action on this bug, but that hasn't happened.
> I'm not sure I see a way forward for debian other than reverting to the
> old behavior. I am reluctant to do so as that will likely lead to
> divergent behavior between distributions, but breaking scripts without a
> compelling reason is also not good. I would encourage coreutils to
> reconsider the change and finding a non-breaking way forward.

Yes it's a fair point.
It's an awkward case, and worth discussing.

To summarise:

  coreutils >= 7.1 had -n skip existing in dest (2009)
  coreutils >= 9.2 has -n immediately fail if existing in dest
  coreutils >= 9.3 has --update=none to skip existing in dest

  FreeBSD >= 4.7/macos has -n immediately fail if existing in dest

  bash has noclobber as a file protection mechanism,
  and fails immediately upon trying to overwrite a file.
  This is more consistent with the new coreutils behavior.

I see a reasonable amount of cp -n usage across github:
https://github.com/search?q=/cp+.*+-n+.*/+path:*.sh&type=code

Now it's not clear which behavior these github usages expect,
and the original docs didn't make it clear which behavior to expect.
A quick scan of the github usages also seem mainly to expect
a protection rather than an update use case, so failing
immediately would be the most appropriate action there too.
Also the original coreutils bug report here expected the new behaviour.

So we probably all agree that failing immediately is the
most appropriate / consistent -n behavior,
but GNU had diverged from that so there are about 10 years
of scripts that may expect the silent skip behavior.

Two options I see are:

- Leave as is and fix -n usages that expected the skip behavior
- Deprecate -n entirely and prompt to use --update={fail,none}

Advantages of leaving as is:
We get consistency of "noclobber" behavior across systems / shells.
We fix cases where previously scripts could have proceeded with
stale old files in place.

Disadvantages of leaving as is:
Users expecting the skip behavior, have to change to --update=none.

There is no potential for data loss etc. so it just comes
down to how disruptive it is, or how often -n was used
with the "skip behavior" assumption.

We've not had much push back as of yet,
and my current thinking is it's not that disruptive a change.
So I'd be 55:45 if favor of keeping things as is.

thanks,
Pádraig.

Message #71 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Michael Stone <mstone <at> debian.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: 1058752 <at> bugs.debian.org, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 15 Dec 2023 13:49:59 -0500

On Fri, Dec 15, 2023 at 06:33:00PM +0000, Pádraig Brady wrote:
>Advantages of leaving as is:
>We get consistency of "noclobber" behavior across systems / shells.

You don't, unless you ignore the coreutils/linux installed base 
entirely. Essentially the current situation is that -n shouldn't be used 
if you expect a certain behavior for this case and you are writing a 
script for linux systems. Maybe in 10 years you'll be able to assume 
the new behavior. Better to just tell people to not use it at all, and 
leave the historic behavior alone until everyone has stopped using -n 
entirely.

>There is no potential for data loss etc.

There may not be, strictly speaking, if you look only at cp without 
context, but we have absolutely no idea what the impact is on the 
unknown number of existing scripts that depend on the historic behavior. 
This is causing breakages in practice.

>so it just comes
>down to how disruptive it is, or how often -n was used
>with the "skip behavior" assumption.

IMO, it should come down to trying to avoid breaking changes in core 
system utilities. There's no compelling reason to force this change, so 
why break anything that depended on the historic behavior--especially 
without any notice or transition period--regardless of arguments over 
whether the historic behavior was right?

>We've not had much push back as of yet,
>and my current thinking is it's not that disruptive a change.

I suspect that's because it has not yet been widely deployed, which 
makes now the time to fix it.

Michael Stone

Message #74 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Michael Stone <mstone <at> debian.org>, Pádraig Brady
 <P <at> draigbrady.com>
Cc: 1058752 <at> bugs.debian.org, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 15 Dec 2023 11:21:06 -0800

On 2023-12-15 10:49, Michael Stone wrote:
> There's no compelling reason to force this change

Well, certainly nobody compelled us at gunpoint....

Stlll, Pádraig gave a reasonable summary of why the change was made, 
despite its incompatibility with previous behavior. (One thing I'd add 
is that the FreeBSD behavior is inherently less race-prone.) It seemed 
like a good idea at the time all things considered, and to my mind still 
does.


> Essentially the current situation is that -n shouldn't be used if you expect a certain behavior for this case and you are writing a script for linux systems. Maybe in 10 years you'll be able to assume the new behavior. Better to just tell people to not use it at all, and leave the historic behavior alone until everyone has stopped using -n entirely.

Even if we tell people not to use -n at all, that doesn't mean we should 
revert to the coreutils 9.1 behavior.

The cat is to some extent out of the bag. Unless one insists on (FreeBSD 
| coreutils 9.2-9.4), or insist on coreutils 7.1-9.1, one should not 
rely on cp -n failing or silently succeeding when the destination 
already exists. This will remain true regardless of whether coreutils 
reverts to its 7.1-9.1 behavior.

Message #77 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Michael Stone <mstone <at> debian.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 1058752 <at> bugs.debian.org,
 Pádraig Brady <P <at> draigbrady.com>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Fri, 15 Dec 2023 15:13:16 -0500

On Fri, Dec 15, 2023 at 11:21:06AM -0800, Paul Eggert wrote:
>Stlll, Pádraig gave a reasonable summary of why the change was made, 
>despite its incompatibility with previous behavior. (One thing I'd add 
>is that the FreeBSD behavior is inherently less race-prone.) It seemed 
>like a good idea at the time all things considered, and to my mind 
>still does.

I think you underestimate the value of maintaining compatibity with 
deployed versions. In the abstract it may have been a nice cleanup, but 
there are a lot of dumb things in the posix utilities that have been 
dumb for so long it's not worth the pain of changing them. Since this 
change hasn't yet hit mainstream debian, ubuntu, rhel, or suse users, I 
strongly suspect that this is a case where the absence of complaints is 
simply a sign that most of the people who'd be impacted haven't 
experienced the change yet.

>Even if we tell people not to use -n at all, that doesn't mean we 
>should revert to the coreutils 9.1 behavior.

It does, IMO, as it would be less likely to break scripts written by 
existing coreutils users.

>The cat is to some extent out of the bag. Unless one insists on 
>(FreeBSD | coreutils 9.2-9.4), or insist on coreutils 7.1-9.1, one 
>should not rely on cp -n failing or silently succeeding when the 
>destination already exists. This will remain true regardless of 
>whether coreutils reverts to its 7.1-9.1 behavior.

Or you use a distribution that has to patch to maintain compatibility 
between versions. Ideally upstream would revert the behavior for now, 
deprecate as the long term fix, and all distributions would work the 
same. The other option is that each distribution decides whether to be 
compatible with upstream coreutils or their own previous release.

Message #80 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Bernhard Voelker <mail <at> bernhard-voelker.de>
To: Michael Stone <mstone <at> debian.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 1058752 <at> bugs.debian.org, Pádraig Brady <P <at> draigbrady.com>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sat, 16 Dec 2023 22:46:57 +0100

On 12/15/23 21:13, Michael Stone wrote:
> On Fri, Dec 15, 2023 at 11:21:06AM -0800, Paul Eggert wrote:
>> Stlll, Pádraig gave a reasonable summary of why the change was made,
>> despite its incompatibility with previous behavior. (One thing I'd add
>> is that the FreeBSD behavior is inherently less race-prone.)

Whether the implementation is race-prone or not is an internal thing.
I think we're currently discussing more on a user-perspective level.

IIUC then the question is whether `cp -n` should continue to behave like
the (new) `cp --update=none` which returns EXIT_SUCCESS.

Regardless what other implementations do, when reading the -n description
from a user's point of view:

  -n, --no-clobber             do not overwrite an existing file (overrides a
                                 -u or previous -i option). See also --update

then I'd expect the tool to just skip existing files like `rsync --ignore-existing`
does.  In that regard I would be surprised if skipping files would result in an error.
Well, I would understand if there'd be a '--no-clobber=fail' option.

As Kamil added the option in 2009, I'd assume that the same patch was already
active in RHEL versions for quite some longer time.
Now changing the exit code feels kind of rough.

Therefore, from a pure user's perspective and regarding many years of precedence,
I am 80:20 for reverting the exit code change.

Have a nice day,
Berny

Message #83 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Pádraig Brady <P <at> draigbrady.com>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sun, 17 Dec 2023 00:34:11 -0800

On 2023-12-16 13:46, Bernhard Voelker wrote:
> Whether the implementation is race-prone or not is an internal thing.

I wasn't referring to the internal implementation. I was referring to cp 
users. With the newer Coreutils (FreeBSD) behavior, you can reliably 
write a script to do something if cp -n didn't copy the file because the 
destination already existed. With the older Coreutils behavior you 
cannot do that reliably; there will always be a race condition.

Message #86 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Michael Stone <mstone <at> debian.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 1058752 <at> bugs.debian.org, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sun, 17 Dec 2023 14:46:03 +0000

On 16/12/2023 21:46, Bernhard Voelker wrote:
> On 12/15/23 21:13, Michael Stone wrote:
>> On Fri, Dec 15, 2023 at 11:21:06AM -0800, Paul Eggert wrote:
>>> Stlll, Pádraig gave a reasonable summary of why the change was made,

To clarify my summary a little, there I said that -n now _immediately_ fails.
I should have said _silently_ fails.  I.e. the complete copy operation
proceeds as before, and only the exit status is at issue here.

>>> despite its incompatibility with previous behavior. (One thing I'd add
>>> is that the FreeBSD behavior is inherently less race-prone.)
> 
> Whether the implementation is race-prone or not is an internal thing.
> I think we're currently discussing more on a user-perspective level.
> 
> IIUC then the question is whether `cp -n` should continue to behave like
> the (new) `cp --update=none` which returns EXIT_SUCCESS.
> 
> Regardless what other implementations do, when reading the -n description
> from a user's point of view:
> 
>     -n, --no-clobber             do not overwrite an existing file (overrides a
>                                    -u or previous -i option). See also --update
> 
> then I'd expect the tool to just skip existing files like `rsync --ignore-existing`
> does.  In that regard I would be surprised if skipping files would result in an error.
> Well, I would understand if there'd be a '--no-clobber=fail' option.

Agreed we should improve the docs a bit for this option.
I'll apply this at least:

diff --git a/doc/coreutils.texi b/doc/coreutils.texi
index 1f8b356d1..bf0f424d3 100644
--- a/doc/coreutils.texi
+++ b/doc/coreutils.texi
@@ -9057,6 +9057,8 @@ Do not overwrite an existing file; silently fail instead.
 This option overrides a previous
 @option{-i} option.  This option is mutually exclusive with @option{-b} or
 @option{--backup} option.
+See also the @option{--update=none} option which will
+skip existing files but not fail.

 @item -P
 @itemx --no-dereference
diff --git a/src/cp.c b/src/cp.c
index 04a5cbee3..3ccc4c4e6 100644
--- a/src/cp.c
+++ b/src/cp.c
@@ -192,8 +192,8 @@ Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY.\n\
   -L, --dereference            always follow symbolic links in SOURCE\n\
 "), stdout);
       fputs (_("\
-  -n, --no-clobber             do not overwrite an existing file (overrides a\n\
-                                 -u or previous -i option). See also --update\n\
+  -n, --no-clobber             ensure no existing files overwritten, and fail\n\
+                                 silently instead. See also --update\n\
 "), stdout);
       fputs (_("\
   -P, --no-dereference         never follow symbolic links in SOURCE\n\


> As Kamil added the option in 2009, I'd assume that the same patch was already
> active in RHEL versions for quite some longer time.
> Now changing the exit code feels kind of rough.

Well RHEL 6 came out a bit after (2010), and had the --no-clobber change,
while RHEL 5 before that did not.

Taking about distros, it's worth noting that the change is Fedora 39
which has been released for a month now.
We'll keep a close eye on issues, but haven't heard much as
of yet at least.

> Therefore, from a pure user's perspective and regarding many years of precedence,
> I am 80:20 for reverting the exit code change.

Thanks for your thoughts,
appreciated as always.

cheers,
Pádraig

Message #89 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Michael Stone <mstone <at> debian.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Pádraig Brady <P <at> draigbrady.com>, 62572 <at> debbugs.gnu.org,
 1058752 <at> bugs.debian.org
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sun, 17 Dec 2023 09:49:08 -0500

On Sun, Dec 17, 2023 at 12:34:11AM -0800, Paul Eggert wrote:
>On 2023-12-16 13:46, Bernhard Voelker wrote:
>>Whether the implementation is race-prone or not is an internal thing.
>
>I wasn't referring to the internal implementation. I was referring to 
>cp users. With the newer Coreutils (FreeBSD) behavior, you can 
>reliably write a script to do something if cp -n didn't copy the file 
>because the destination already existed. With the older Coreutils 
>behavior you cannot do that reliably; there will always be a race 
>condition.

You can now reliably write a script using the new long option. Changing 
the behavior of the short option helped nobody.

Message #92 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Dominique Martinet <asmadeus <at> codewreck.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 1058752 <at> bugs.debian.org, Pádraig Brady <P <at> draigbrady.com>,
 62572 <at> debbugs.gnu.org, Michael Stone <mstone <at> debian.org>
Subject: Re: bug#62572: cp --no-clobber behavior has changed
Date: Sun, 17 Dec 2023 21:18:33 +0900

Paul Eggert wrote on Fri, Dec 15, 2023 at 11:21:06AM -0800:
> The cat is to some extent out of the bag. Unless one insists on (FreeBSD |
> coreutils 9.2-9.4), or insist on coreutils 7.1-9.1, one should not rely on
> cp -n failing or silently succeeding when the destination already exists.
> This will remain true regardless of whether coreutils reverts to its 7.1-9.1
> behavior.

This. Scripts that want to be portable already can't assume cp -n will
do what they want, so at this point it doesn't really matter what
coreutils does in the grand scheme of things.

For distros like debian since even -testing hasn't seen coreutils 9.2,
there's still value in reverting locally (with a warning that it's not
reliable perhaps?), but in general coreutils 9.2 has been out for 9
months (2023 March 20), so many systems can already be considered
affected; but it's a disservice to users to just try to hide the problem
under the rug.


(To give a data point, this did bite us as well, and I was annoyed
enough that I went to look for the old bug report back in September, but
at that point 9.3 had already been out and I had given up without
reporting anything as nothing would change the fact that my scripts
would need updating. For the gory details I also need compatibility
with busybox cp (where -n silently ignores existing files), so
--update=none is not an option, but I for this particular usage I
settled for '-u' (--update=older, that busybox also support as short
option only...), and I since hurried to forget about it)

-- 
Dominique Martinet | Asmadeus

Message #95 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Michael Stone <mstone <at> debian.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 62572 <at> debbugs.gnu.org
Subject: Re: Bug#1058752: bug#62572: cp --no-clobber behavior has changed
Date: Sun, 28 Jan 2024 13:22:24 +0000

On 17/12/2023 14:46, Pádraig Brady wrote:
> On 16/12/2023 21:46, Bernhard Voelker wrote:
>> On 12/15/23 21:13, Michael Stone wrote:
>>> On Fri, Dec 15, 2023 at 11:21:06AM -0800, Paul Eggert wrote:
>>>> Stlll, Pádraig gave a reasonable summary of why the change was made,
> 
> To clarify my summary a little, there I said that -n now _immediately_ fails.
> I should have said _silently_ fails.  I.e. the complete copy operation
> proceeds as before, and only the exit status is at issue here.
> 
>>>> despite its incompatibility with previous behavior. (One thing I'd add
>>>> is that the FreeBSD behavior is inherently less race-prone.)
>>
>> Whether the implementation is race-prone or not is an internal thing.
>> I think we're currently discussing more on a user-perspective level.
>>
>> IIUC then the question is whether `cp -n` should continue to behave like
>> the (new) `cp --update=none` which returns EXIT_SUCCESS.
>>
>> Regardless what other implementations do, when reading the -n description
>> from a user's point of view:
>>
>>      -n, --no-clobber             do not overwrite an existing file (overrides a
>>                                     -u or previous -i option). See also --update
>>
>> then I'd expect the tool to just skip existing files like `rsync --ignore-existing`
>> does.  In that regard I would be surprised if skipping files would result in an error.
>> Well, I would understand if there'd be a '--no-clobber=fail' option.
> 
> Agreed we should improve the docs a bit for this option.
> I'll apply this at least:
> 
> diff --git a/doc/coreutils.texi b/doc/coreutils.texi
> index 1f8b356d1..bf0f424d3 100644
> --- a/doc/coreutils.texi
> +++ b/doc/coreutils.texi
> @@ -9057,6 +9057,8 @@ Do not overwrite an existing file; silently fail instead.
>    This option overrides a previous
>    @option{-i} option.  This option is mutually exclusive with @option{-b} or
>    @option{--backup} option.
> +See also the @option{--update=none} option which will
> +skip existing files but not fail.
> 
>    @item -P
>    @itemx --no-dereference
> diff --git a/src/cp.c b/src/cp.c
> index 04a5cbee3..3ccc4c4e6 100644
> --- a/src/cp.c
> +++ b/src/cp.c
> @@ -192,8 +192,8 @@ Copy SOURCE to DEST, or multiple SOURCE(s) to DIRECTORY.\n\
>      -L, --dereference            always follow symbolic links in SOURCE\n\
>    "), stdout);
>          fputs (_("\
> -  -n, --no-clobber             do not overwrite an existing file (overrides a\n\
> -                                 -u or previous -i option). See also --update\n\
> +  -n, --no-clobber             ensure no existing files overwritten, and fail\n\
> +                                 silently instead. See also --update\n\
>    "), stdout);
>          fputs (_("\
>      -P, --no-dereference         never follow symbolic links in SOURCE\n\
> 
> 
>> As Kamil added the option in 2009, I'd assume that the same patch was already
>> active in RHEL versions for quite some longer time.
>> Now changing the exit code feels kind of rough.
> 
> Well RHEL 6 came out a bit after (2010), and had the --no-clobber change,
> while RHEL 5 before that did not.
> 
> Taking about distros, it's worth noting that the change is Fedora 39
> which has been released for a month now.
> We'll keep a close eye on issues, but haven't heard much as
> of yet at least.
> 
>> Therefore, from a pure user's perspective and regarding many years of precedence,
>> I am 80:20 for reverting the exit code change.
> 
> Thanks for your thoughts,
> appreciated as always.

We were undecided how to proceed as of the above discussion,
with some hoping to consolidate -n behavior across all systems,
with others preferring to keep compat with original Linux behavior.

Things have changed in Debian I see, so that cp -n now aggressively warns like:
https://sources.debian.org/patches/coreutils/9.4-3/cp-n.diff/

  $ cp -n /bin/true tmp
  cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead

This is problematic as:

  - It's noisy
  - There is no way to get the behavior of indicating failure if existing files present
  - The --update=none advice is only portable to newer coreutils

To summarise existing use cases of -n:

  1. User expected -n to exit failure if existing files
  2. User didn't care about existing status
  3. User expected -n to exit success if existing files

Use case 3 is the problematic case we changed behavior for,
but also I think that's the minority of the 3 use cases
and could have been fixed forward.

The debian advice forces use case 2 to change,
and provides bad advice for use case 1,
and in fact there is no solution now for use case 1.

At this stage it seems best for us go back to the original Linux behiavor (use case 3),
and to silently deprecate -n in docs to document the portability issues with it.
We should also provide --update=noclobber for use case 1.
Having the control on the --update option, allows use to more clearly deprecate -n.

I do realise folks may expect use case 1 now with -n,
especially on releases like Fedora 39, but given the feedback
that would be the least bad option.

thanks,
Pádraig

Message #98 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 Michael Stone <mstone <at> debian.org>
Cc: 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Sun, 28 Jan 2024 23:14:14 -0800

On 2024-01-28 05:22, Pádraig Brady wrote:

> At this stage it seems best for us go back to the original Linux 
> behiavor (use case 3),
> and to silently deprecate -n in docs to document the portability issues 
> with it.

I'm not sure reverting would be best. It would introduce more confusion, 
and would make coreutils incompatible with FreeBSD again.

The recent Debian change indicates that their intent is to move to the 
FreeBSD behavior too. This would improve cross-platform portability and 
I don't think we should discourage that.


>   $ cp -n /bin/true tmp
>   cp: warning: behavior of -n is non-portable and may change in future; use --update=none instead
> 
> This is problematic as:
> 
>   - It's noisy

Yes that's a problem, and I doubt whether we should mimic Debian.

>   - There is no way to get the behavior of indicating failure if existing files present

Yes, it's not a good place to be. Surely current coreutils is better 
than what Debian is doing.

>   - The --update=none advice is only portable to newer coreutils

True, but that's not a deal-killer. No advice that we give can be 100% 
portable to all platforms.

> We should also provide --update=noclobber for use case 1.
> Having the control on the --update option, allows use to more clearly 
> deprecate -n.

Adding an --update=noclobber sounds like a good thing to do.

Another possibility is to add a warning that is emitted only at the end 
of 'cp'. The warning would occur only if the exit code differs because 
of this cp -n business. We could stretch things a bit and have a 
configure-time option --enable-compat-warnings that builders like Debian 
could use if they want such warnings.

Message #101 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Michael Stone <mstone <at> debian.org>
To: Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 1058752 <at> bugs.debian.org,
 Pádraig Brady <P <at> draigbrady.com>,
 Bernhard Voelker <mail <at> bernhard-voelker.de>, 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Mon, 29 Jan 2024 09:01:10 -0500

On Sun, Jan 28, 2024 at 11:14:14PM -0800, Paul Eggert wrote:
>I'm not sure reverting would be best. It would introduce more 
>confusion, and would make coreutils incompatible with FreeBSD again.

Reverting makes more sense than the current situation. I do not 
understand why you seem to value FreeBSD compatibility more than 
compatibility with the vast majority of installed coreutils/linux
systems.

>Yes, it's not a good place to be. Surely current coreutils is better 
>than what Debian is doing.

You've introduced a silent incompatibility and I'm trying to find some 
way to make that clear. If upstream would provide a better solution I 
would certainly use it. I have despaired of there being such since your 
attitude thus far seems to be entirely dismissive of compatibility 
concerns.

>Another possibility is to add a warning that is emitted only at the 
>end of 'cp'. The warning would occur only if the exit code differs 
>because of this cp -n business.

You'd only emit a notification of a change in behavior if some 
(potentially uncommon/rarely encountered) situation arises which would 
actually trigger breakage? So people can't prepare ahead of time and 
change their script to handle the necessary change in logic, they can 
only maybe figure out why something broke at 2am when the uncommon event 
occurred?

At the end of the day, -n is basically a useless option with unknowable 
semantics which should be avoided by everyone. In the past it was an 
option which wasn't portable between coreutils/linux and freebsd systems, 
and I guess you've "fixed" that (by making it an option everyone should 
avoid entirely), but let's be honest about how common that concern was.

Message #104 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Michael Stone <mstone <at> debian.org>, Paul Eggert <eggert <at> cs.ucla.edu>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Mon, 29 Jan 2024 16:11:05 +0000

On 29/01/2024 14:01, Michael Stone wrote:
> On Sun, Jan 28, 2024 at 11:14:14PM -0800, Paul Eggert wrote:
>> I'm not sure reverting would be best. It would introduce more
>> confusion, and would make coreutils incompatible with FreeBSD again.
> 
> Reverting makes more sense than the current situation. I do not
> understand why you seem to value FreeBSD compatibility more than
> compatibility with the vast majority of installed coreutils/linux
> systems.
> 
>> Yes, it's not a good place to be. Surely current coreutils is better
>> than what Debian is doing.
> 
> You've introduced a silent incompatibility and I'm trying to find some
> way to make that clear. If upstream would provide a better solution I
> would certainly use it. I have despaired of there being such since your
> attitude thus far seems to be entirely dismissive of compatibility
> concerns.

That's a bit unfair.  The current upstream -n behavior is with a view
to being _more_ compat across all systems.
Now I agree this may not be worth it in this case,
but it is a laudable goal.

>> Another possibility is to add a warning that is emitted only at the
>> end of 'cp'. The warning would occur only if the exit code differs
>> because of this cp -n business.
> 
> You'd only emit a notification of a change in behavior if some
> (potentially uncommon/rarely encountered) situation arises which would
> actually trigger breakage? So people can't prepare ahead of time and
> change their script to handle the necessary change in logic, they can
> only maybe figure out why something broke at 2am when the uncommon event
> occurred?

Yes I agree with this point, mostly.
Outputting a diagnostic would help users diagnose what's going on,
and help users to fix forward and avoid their problematic -n usage.
But yes, the crux of the issue with fixing issues as they occur,
is it depends on the state of the destination and so can become an issue
at any time.  Now I previously did an audit with github and debian code search
and noticed very few problematic uses of cp -n, but that does miss
the mountain of private code.

> At the end of the day, -n is basically a useless option with unknowable
> semantics which should be avoided by everyone. In the past it was an
> option which wasn't portable between coreutils/linux and freebsd systems,
> and I guess you've "fixed" that (by making it an option everyone should
> avoid entirely), but let's be honest about how common that concern was.

Right, that's why I'm still leaning towards my proposal in the last mail.
  - revert to previous exit success -n behavior
  - document -n as deprecated
  - provide --update=noclobber to give exit failure functionality
    - BTW, it probably makes sense to print a diagnostic for each skipped file here
      as it's exceptional behavior, for which we're exiting with failure for.
  - the existing --update=none provides the exit success functionality

With the above in place for the next coreutils release,
then debian could remove its noisy patch.

thanks,
Pádraig

Message #107 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Michael Stone <mstone <at> debian.org>
To: Pádraig Brady <P <at> draigbrady.com>
Cc: Paul Eggert <eggert <at> cs.ucla.edu>,
 Bernhard Voelker <mail <at> bernhard-voelker.de>, 62572 <at> debbugs.gnu.org,
 1058752 <at> bugs.debian.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Mon, 29 Jan 2024 11:44:57 -0500

On Mon, Jan 29, 2024 at 04:11:05PM +0000, Pádraig Brady wrote:
>>You've introduced a silent incompatibility and I'm trying to find some
>>way to make that clear. If upstream would provide a better solution I
>>would certainly use it. I have despaired of there being such since your
>>attitude thus far seems to be entirely dismissive of compatibility
>>concerns.
>
>That's a bit unfair.  The current upstream -n behavior is with a view
>to being _more_ compat across all systems.
>Now I agree this may not be worth it in this case,
>but it is a laudable goal.

You are saying that again without explicitly acknowledging that "being 
_more_ compat" in this case means "becoming _incompat_ with the vast 
majority of installed systems". IMO it could be reasonably phrased as 
"being more compatible across all systems in the long term when all 
existing legacy systems are gone", but the key here is that I read 
"_more_ compat across all systems" as dismissing the coreutils installed 
base as part of "all systems". I understand that may not be/have been 
the intent, but I also can't help feeling the way that I do when the 
benefits of compatability with freebsd are repeatedly emphasized while 
the costs of incompatibility with the coreutils installed base are 
dismissed with something along the lines of "we'll see what breaks". (If 
the costs of incompatibility are really that low in this case, why would 
compatability be a worthwhile goal in this case?)

I do wish that more users had noticed the change earlier and that we're 
fairly deep into a mess, but it's not always easy to see the impact of 
what seems like a relatively minor patch. I do appreciate that the new 
version printed some diagnostics when the change was triggered, as that 
certainly helped call attention to scripts which were impacted.

>With the above in place for the next coreutils release,
>then debian could remove its noisy patch.

I would certainly align with that, and the sooner the better to decrease 
the chances that different distributions handle this in different ways 
or we get to the point of having to release in an interim state. If you 
commit a final version I'll apply that patch if the next release isn't 
imminent.

Message #110 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Mon, 29 Jan 2024 13:44:16 -0800

On 1/29/24 08:11, Pádraig Brady wrote:

> Right, that's why I'm still leaning towards my proposal in the last mail.

Well, I won't insist on doing nothing; however, the proposal needs 
ironing out and now's a good time to do it before installing changes.


>    - revert to previous exit success -n behavior
>    - document -n as deprecated
>    - provide --update=noclobber to give exit failure functionality

So --update=noclobber would differ in meaning from the deprecated-in-9.5 
--no-clobber, but would agree in meaning with 9.4 --no-clobber? That 
sounds pretty confusing for future users. (And a nit: why should one 
spelling have a hyphen but the other doesn't?)


>      - BTW, it probably makes sense to print a diagnostic for each 
> skipped file here
>        as it's exceptional behavior, for which we're exiting with 
> failure for.

Coreutils 9.4 cp -n already does that, no? So I'm not sure what's being 
proposed here.

  $ touch a b
  $ cp -n a b; echo $?
  cp: not replacing 'b'
  1


>    - the existing --update=none provides the exit success functionality

It seems to me that this proposal conflates two questions:

* What rules should cp use to decide whether to update a destination?

* When cp decides not to update a destination, what should it do? Exit 
with nonzero status? Output a diagnostic? Both? Neither?

Aren't these independent axes? If so, shouldn't they have independent 
options? For example, since we have --update=older, shouldn't there be a 
way to say "I want to copy A to B only if B is older than A, and I want 
the exit status to be zero only if A was copied to B"?

Message #113 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Tue, 30 Jan 2024 11:18:12 +0000

On 29/01/2024 21:44, Paul Eggert wrote:
> On 1/29/24 08:11, Pádraig Brady wrote:
> 
>> Right, that's why I'm still leaning towards my proposal in the last mail.
> 
> Well, I won't insist on doing nothing; however, the proposal needs
> ironing out and now's a good time to do it before installing changes.
> 
> 
>>     - revert to previous exit success -n behavior
>>     - document -n as deprecated
>>     - provide --update=noclobber to give exit failure functionality
> 
> So --update=noclobber would differ in meaning from the deprecated-in-9.5
> --no-clobber, but would agree in meaning with 9.4 --no-clobber? That
> sounds pretty confusing for future users. (And a nit: why should one
> spelling have a hyphen but the other doesn't?)

That's a fair point; just avoid "no-clobber" entirely.
We could choose a new name so. How about --update=none-fail

>>       - BTW, it probably makes sense to print a diagnostic for each
>> skipped file here
>>         as it's exceptional behavior, for which we're exiting with
>> failure for.
> 
> Coreutils 9.4 cp -n already does that, no? So I'm not sure what's being
> proposed here.
> 
>     $ touch a b
>     $ cp -n a b; echo $?
>     cp: not replacing 'b'
>     1

Sorry I got confused between your suggestion of added diagnostics,
and FreeBSD's silent behavior here. Looks like we just keep the
existing behavior of diagnosing 'not replacing' iff exiting with failure.

>>     - the existing --update=none provides the exit success functionality
> 
> It seems to me that this proposal conflates two questions:
> 
> * What rules should cp use to decide whether to update a destination?
> 
> * When cp decides not to update a destination, what should it do? Exit
> with nonzero status? Output a diagnostic? Both? Neither?
> 
> Aren't these independent axes? If so, shouldn't they have independent
> options? For example, since we have --update=older, shouldn't there be a
> way to say "I want to copy A to B only if B is older than A, and I want
> the exit status to be zero only if A was copied to B"?

Well they're not entirely independent.
It's more appropriate to output a diagnostic if exiting failure,
and POSIX also advises along the same lines.
We also have --verbose to control diagnostics somewhat
for the non failure case.

So we now have the proposed change as:

  - revert -n to old silent success behavior
  - document -n as deprecated
  - Leave --update=none as is (will be synonymous with -n)
  - Provide --update=none-fail to diagnose and exit failure

thanks,
Pádraig.

Message #116 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Tue, 30 Jan 2024 10:31:32 -0800

On 2024-01-30 03:18, Pádraig Brady wrote:
> So we now have the proposed change as:
> 
>    - revert -n to old silent success behavior
>    - document -n as deprecated
>    - Leave --update=none as is (will be synonymous with -n)
>    - Provide --update=none-fail to diagnose and exit failure

Thanks, that's a better proposal, but I still see several opportunities 
for confusion.

If I understand things correctly, cp --update=none is not synonymous 
with the proposed (i.e., old-behavior) cp -n, because -n overrides 
previous -i options but --update=none does not. Also, -n overrides 
either previous or following --update=UPDATE options, but --update=none 
overrides only previous --update=UPDATE options. (For what it's worth, 
FreeBSD -n overrides

Some of this complication seems to be for consistency with how mv 
behaves with -f, -i, -n, and --update, and similarly with how rm behaves 
with -f, -i, -I, and --interactive. To be honest I don't quite 
understand the reason for all this complexity, which suggests it should 
be documented somewhere (the manual?) if it isn't already.

This raises more questions:

* If we deprecate cp -n, what about mv -n? FreeBSD mv -n behaves like 
Coreutils mv -n: it silently does nothing and exits successfully. So 
there's no compatibility argument for changing mv -n's behavior. 
However, whatever --update option we add to cp (to output a diagnostic 
and exit with failure) should surely also be added to mv, to aid 
consistency.

* Should cp --update=none be changed so that it really behaves like the 
old cp -n, in that it overrides other options in ways that differ from 
how the other --update=UPDATE options behave? I'm leaning toward "no" as 
this adds complexity that I don't see the use for.

* If we don't change cp --update=none's overriding behavior, is it still 
OK to tell users to substitute --update=none for -n even though the two 
options are not exactly equivalent? I'm leaning towards "yes" but would 
like other opinions.

Message #119 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Wed, 31 Jan 2024 14:06:08 +0000

On 30/01/2024 18:31, Paul Eggert wrote:
> On 2024-01-30 03:18, Pádraig Brady wrote:
>> So we now have the proposed change as:
>>
>>     - revert -n to old silent success behavior
>>     - document -n as deprecated
>>     - Leave --update=none as is (will be synonymous with -n)
>>     - Provide --update=none-fail to diagnose and exit failure
> 
> Thanks, that's a better proposal, but I still see several opportunities
> for confusion.
> 
> If I understand things correctly, cp --update=none is not synonymous
> with the proposed (i.e., old-behavior) cp -n, because -n overrides
> previous -i options but --update=none does not. Also, -n overrides
> either previous or following --update=UPDATE options, but --update=none
> overrides only previous --update=UPDATE options. (For what it's worth,
> FreeBSD -n overrides

Good point.
Well -n is a protection mechanism really, so should override.
Since --update now incorporates a protection mode, it should also override I think.

> Some of this complication seems to be for consistency with how mv
> behaves with -f, -i, -n, and --update, and similarly with how rm behaves
> with -f, -i, -I, and --interactive. To be honest I don't quite
> understand the reason for all this complexity, which suggests it should
> be documented somewhere (the manual?) if it isn't already.

To my mind the most protective option takes precedence.
So from least to most that would be -f -n --update=none --update=none-fail -i
So the main consideration there is that -n should not override --update=none-fail

> This raises more questions:
> 
> * If we deprecate cp -n, what about mv -n? FreeBSD mv -n behaves like
> Coreutils mv -n: it silently does nothing and exits successfully. So
> there's no compatibility argument for changing mv -n's behavior.
> However, whatever --update option we add to cp (to output a diagnostic
> and exit with failure) should surely also be added to mv, to aid
> consistency.

Yes I agree.

> * Should cp --update=none be changed so that it really behaves like the
> old cp -n, in that it overrides other options in ways that differ from
> how the other --update=UPDATE options behave? I'm leaning toward "no" as
> this adds complexity that I don't see the use for.
> 
> * If we don't change cp --update=none's overriding behavior, is it still
> OK to tell users to substitute --update=none for -n even though the two
> options are not exactly equivalent? I'm leaning towards "yes" but would
> like other opinions.

Yes I think we should still give that advice to deprecate -n,
if we ensure that -n does not override --update=none=fail.

thanks,
Pádraig

Message #122 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Paul Eggert <eggert <at> cs.ucla.edu>
To: Pádraig Brady <P <at> draigBrady.com>,
 Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Wed, 31 Jan 2024 16:36:16 -0800

On 1/31/24 06:06, Pádraig Brady wrote:
> To my mind the most protective option takes precedence.

That's not how POSIX works with mv -i and mv -f. The last flag wins. I 
assume this is so that people can have aliases or shell scripts that 
make -i the default, but you can override by specifying -f on the 
command line. E.g., in mymv:

   #!/bin/sh
   mv -i "$@"

then "mymv -f a b" works as expected.

Wouldn't a similar argument apply to cp's --update options?

Or perhaps we should play it safe, and reject any combination of 
--update etc. options that are incompatible. We can always change our 
mind later and say that later options override earlier ones, or do 
something else that's less conservative.

Message #125 received at 62572 <at> debbugs.gnu.org (full text, mbox):

From: Pádraig Brady <P <at> draigBrady.com>
To: Paul Eggert <eggert <at> cs.ucla.edu>, Michael Stone <mstone <at> debian.org>
Cc: 1058752 <at> bugs.debian.org, Bernhard Voelker <mail <at> bernhard-voelker.de>,
 62572 <at> debbugs.gnu.org
Subject: Re: bug#62572: Bug#1058752: bug#62572: cp --no-clobber behavior has
 changed
Date: Sat, 24 Feb 2024 22:13:42 +0000

[Message part 1 (text/plain, inline)]

On 01/02/2024 00:36, Paul Eggert wrote:
> On 1/31/24 06:06, Pádraig Brady wrote:
>> To my mind the most protective option takes precedence.
> 
> That's not how POSIX works with mv -i and mv -f. The last flag wins. I
> assume this is so that people can have aliases or shell scripts that
> make -i the default, but you can override by specifying -f on the
> command line. E.g., in mymv:
> 
>      #!/bin/sh
>      mv -i "$@"
> 
> then "mymv -f a b" works as expected.
> 
> Wouldn't a similar argument apply to cp's --update options?
> 
> Or perhaps we should play it safe, and reject any combination of
> --update etc. options that are incompatible. We can always change our
> mind later and say that later options override earlier ones, or do
> something else that's less conservative.

OK I err'd on the side of changing as little as possible wrt precedence.
-n still has precedence over any -u,--update option.
That's simplest to understand (while not changing existing precedence),
and shouldn't cause any practical issues.

I plan to push the 2 attached patches for this tomorrow.

thanks,
Pádraig

[0001-cp-mv-reinstate-that-n-exits-with-success-if-files-s.patch (text/x-patch, attachment)]

[0002-cp-mv-add-update-none-fail-to-fail-if-existing-files.patch (text/x-patch, attachment)]

Previous Next

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997,2003 nCipher Corporation Ltd, 1994-97 Ian Jackson.