GNU logs - #42034, boring messages

Message sent to bug-coreutils@HIDDEN:

Subject: bug#42034: option to truncate at end or should that be default?
Resent-From: L A Walsh <coreutils@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Wed, 24 Jun 2020 19:57:01 +0000
Resent-Message-ID: <handler.42034.B.159302862015862 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 42034 <at> debbugs.gnu.org
MIME-Version: 1.0
From: L A Walsh <coreutils@HIDDEN>
Date: Wed, 24 Jun 2020 12:56:41 -0700
Message-ID: <CALOnQv4E_Nv+drzXhCSmjQuP1xLKYK_hu5iv3hr1JCNvC2zOKw@HIDDEN>
Content-Type: multipart/alternative; boundary="00000000000098134805a8d9e0b4"
Received-SPF: softfail client-ip=209.85.166.42; envelope-from=gsuite@HIDDEN;
 helo=mail-io1-f42.google.com
X-Spam_score_int: -21
X-Spam_score: -2.2
X-Spam_bar: --
X-Spam_report: (-2.2 / 5.0 requ) BAYES_00=-1.9, HTML_MESSAGE=0.001,
 RCVD_IN_DNSWL_NONE=-0.0001, RCVD_IN_MSPIKE_H2=-1, SPF_HELO_NONE=0.001,
 SPF_SOFTFAIL=0.665 autolearn=_AUTOLEARN
X-Spam_action: no action
X-Spam-Score: -1.3 (-)
X-BeenThere: debbugs-submit <at> debbugs.gnu.org
X-Mailman-Version: 2.1.18
Precedence: list
List-Id: <debbugs-submit.debbugs.gnu.org>
List-Unsubscribe: <https://debbugs.gnu.org/cgi-bin/mailman/options/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=unsubscribe>
List-Archive: <https://debbugs.gnu.org/cgi-bin/mailman/private/debbugs-submit/>
List-Post: <mailto:debbugs-submit <at> debbugs.gnu.org>
List-Help: <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=help>
List-Subscribe: <https://debbugs.gnu.org/cgi-bin/mailman/listinfo/debbugs-submit>, 
 <mailto:debbugs-submit-request <at> debbugs.gnu.org?subject=subscribe>
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
X-Spam-Score: -2.3 (--)

--00000000000098134805a8d9e0b4
Content-Type: text/plain; charset="UTF-8"

I allocated a large file of contiguous space (~3.6T), the size of a disk
image I was going to copy into it with 'dd'.  I have the disk image
'overwrite' the existing file, in place using "conv=nocreat,notrunc" (among
other switches) and that works with the final file still using max-sized
8GB extents.

I realize that I _do_ want it to truncate the file to the actual size when
done.

The 'notrunc' switch doesn't work for this purpose as its meaning is
overloaded (it really specifies multiple behaviors) both the non-truncation
effect, as well as a directive to preserve any blocks not written during
that specific invocation of 'dd'.  A possible 3rd behavior arises from a
vague definition of a block.  Do they mean to preserve the data in the
block, or do they mean to preserve the position of the block on disk?  It
seems they mean to preserve data, but whether or not that also preserves a
place on disk isn't specified.

There really needs to be something to specify that writes occur "in-place"
such that no "_RE_-allocation" of blocks occurs (except to extend the file,
if needed)  A second option would be to truncate the file to the last
position written.

Maybe a oflag=overwrite, and a 'ftrunc' for 'final trunc' to the position
of the final byte written?

--00000000000098134805a8d9e0b4
Content-Type: text/html; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable

<div dir=3D"ltr"><div class=3D"gmail_default" style=3D"font-family:monospac=
e;font-size:large">I allocated a large file of contiguous space (~3.6T), th=
e size of a disk image I was going to copy into it with &#39;dd&#39;.=C2=A0=
 I have the disk image &#39;overwrite&#39; the existing file, in place usin=
g &quot;conv=3Dnocreat,notrunc&quot; (among other switches) and that works =
with the final file still using max-sized 8GB extents.<br><br></div><div cl=
ass=3D"gmail_default" style=3D"font-family:monospace;font-size:large">I rea=
lize that I _do_ want it to truncate the file to the actual size when done.=
<br><br></div><div class=3D"gmail_default" style=3D"font-family:monospace;f=
ont-size:large">The &#39;notrunc&#39; switch doesn&#39;t work for this purp=
ose as its meaning is overloaded (it really specifies multiple behaviors) b=
oth the non-truncation effect, as well as a directive to preserve any block=
s not written during that specific invocation of &#39;dd&#39;.=C2=A0 A poss=
ible 3rd behavior arises from a vague definition of a block.=C2=A0 Do they =
mean to preserve the data in the block, or do they mean to preserve the pos=
ition of the block on disk?=C2=A0 It seems they mean to preserve data, but =
whether or not that also preserves a place on disk isn&#39;t specified.<br>=
<br></div><div class=3D"gmail_default" style=3D"font-family:monospace;font-=
size:large">There really needs to be something to specify that writes occur=
 &quot;in-place&quot; such that no &quot;_RE_-allocation&quot; of blocks oc=
curs (except to extend the file, if needed)=C2=A0 A second option would be =
to truncate the file to the last position written.<br><br><br></div><div cl=
ass=3D"gmail_default" style=3D"font-family:monospace;font-size:large">Maybe=
 a oflag=3Doverwrite, and a &#39;ftrunc&#39; for &#39;final trunc&#39; to t=
he position of the final byte written?<br><br><br></div><div class=3D"gmail=
_default" style=3D"font-family:monospace;font-size:large"><br><br><br></div=
></div>

--00000000000098134805a8d9e0b4--

Message sent:

Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable
MIME-Version: 1.0
X-Mailer: MIME-tools 5.505 (Entity 5.505)
Content-Type: text/plain; charset=utf-8
X-Loop: help-debbugs@HIDDEN
From: help-debbugs@HIDDEN (GNU bug Tracking System)
To: L A Walsh <coreutils@HIDDEN>
Subject: bug#42034: Acknowledgement (option to truncate at end or should
 that be default?)
Message-ID: <handler.42034.B.159302862015862.ack <at> debbugs.gnu.org>
References: <CALOnQv4E_Nv+drzXhCSmjQuP1xLKYK_hu5iv3hr1JCNvC2zOKw@HIDDEN>
X-Gnu-PR-Message: ack 42034
X-Gnu-PR-Package: coreutils
Reply-To: 42034 <at> debbugs.gnu.org
Date: Wed, 24 Jun 2020 19:57:01 +0000

Thank you for filing a new bug report with debbugs.gnu.org.

This is an automatically generated reply to let you know your message
has been received.

Your message is being forwarded to the package maintainers and other
interested parties for their attention; they will reply in due course.

Your message has been sent to the package maintainer(s):
 bug-coreutils@HIDDEN

If you wish to submit further information on this problem, please
send it to 42034 <at> debbugs.gnu.org.

Please do not send mail to help-debbugs@HIDDEN unless you wish
to report a problem with the Bug-tracking system.

--=20
42034: http://debbugs.gnu.org/cgi/bugreport.cgi?bug=3D42034
GNU Bug Tracking System
Contact help-debbugs@HIDDEN with problems

Message sent to bug-coreutils@HIDDEN:

Subject: bug#42034: option to truncate at end or should that be default?
Resent-From: Andreas Schwab <schwab@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Wed, 24 Jun 2020 20:08:01 +0000
Resent-Message-ID: <handler.42034.B42034.159302925016954 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: L A Walsh <coreutils@HIDDEN>
Cc: 42034 <at> debbugs.gnu.org
From: Andreas Schwab <schwab@HIDDEN>
References: <CALOnQv4E_Nv+drzXhCSmjQuP1xLKYK_hu5iv3hr1JCNvC2zOKw@HIDDEN>
Date: Wed, 24 Jun 2020 22:07:23 +0200
In-Reply-To: <CALOnQv4E_Nv+drzXhCSmjQuP1xLKYK_hu5iv3hr1JCNvC2zOKw@HIDDEN>
 (L. A. Walsh's message of "Wed, 24 Jun 2020 12:56:41 -0700")
Message-ID: <87h7v0ut1g.fsf@HIDDEN>
User-Agent: Gnus/5.13 (Gnus v5.13) Emacs/27.0.91 (gnu/linux)
MIME-Version: 1.0
Content-Type: text/plain
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

On Jun 24 2020, L A Walsh wrote:

> A second option would be to truncate the file to the last position
> written.

$ truncate -r $src $dest

Andreas.

-- 
Andreas Schwab, schwab@HIDDEN
GPG Key fingerprint = 7578 EB47 D4E5 4D69 2510  2552 DF73 E780 A9DA AEC1
"And now for something completely different."

Message sent to bug-coreutils@HIDDEN:

Subject: bug#42034: option to truncate at end or should that be default?
Resent-From: Bob Proulx <bob@HIDDEN>
Original-Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>
Resent-CC: bug-coreutils@HIDDEN
Resent-Date: Wed, 24 Jun 2020 22:31:01 +0000
Resent-Message-ID: <handler.42034.B42034.15930378307740 <at> debbugs.gnu.org>
Resent-Sender: help-debbugs@HIDDEN
To: 42034 <at> debbugs.gnu.org
Date: Wed, 24 Jun 2020 16:30:18 -0600
From: Bob Proulx <bob@HIDDEN>
Message-ID: <20200624162015168087241@HIDDEN>
References: <CALOnQv4E_Nv+drzXhCSmjQuP1xLKYK_hu5iv3hr1JCNvC2zOKw@HIDDEN>
MIME-Version: 1.0
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
In-Reply-To: <CALOnQv4E_Nv+drzXhCSmjQuP1xLKYK_hu5iv3hr1JCNvC2zOKw@HIDDEN>
Precedence: list
Errors-To: debbugs-submit-bounces <at> debbugs.gnu.org
Sender: "Debbugs-submit" <debbugs-submit-bounces <at> debbugs.gnu.org>

L A Walsh wrote:
> I allocated a large file of contiguous space (~3.6T), the size of a disk
> image I was going to copy into it with 'dd'.  I have the disk image
> 'overwrite' the existing file, in place ...

It's possible that you might want to be rescuing data from a failing
disk or doing other surgery upon it.  Therefore I want to mention
ddrescue here.

  https://www.gnu.org/software/ddrescue/

Of course it all depends upon the use case but ddrescue is a good tool
to have in the toolbox.  It might be just the right tool.

Take for example a RAID1 image on two failing drives that should be
identical but both are reporting errors.  If the failures do not
overlap then ddrescue can be used to merge the successful reads from
those two images producing one fully correct image.

Bob

Last modified: Wed, 24 Jun 2020 22:30:01 UTC

GNU bug tracking system
Copyright (C) 1999 Darren O. Benham, 1997 nCipher Corporation Ltd, 1994-97 Ian Jackson.